Re: [HTCondor-users] collecting job statistics

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 6/10/2016 10:21 AM, Thomas Hartmann wrote:
> Hi all,

Hi Thomas, some quick thoughts below...

> Since this is a bit cumbersome, to go upstream to query job statistics,
> I wonder if there is a more direct way to get a job's ClassAds (run as
> on an exit hook, so in a job's context I suppose)?

So in the job's context, there will be an environment variable 
_CONDOR_JOB_AD that contains the path to a file in the job's scratch 
directory which contains the job ad for the currently running job. The 
job ad is current as of the start of the job, but is not updated during 
the running of the job. The job may read attributes and their values out 
of this file as it runs, but any changes will not be acted on in any way 
by HTCondor. The format is the same as the output of the condor_q -l 
command.  See the Manual at https://is.gd/A3GYLr

If you also want to see job attributes that HTCondor is updating while 
the job runs, such as RemoteUserCpu.... since it sounds like you are 
using a Job Exit hook, isn't the full job classad with all the runtime 
statistics included passed to your hook via stdin ?   At least that is 
what the Manual implies in section https://is.gd/KcVpRN. Can your Job 
Exit hook simply read in everything from stdin and call it good?

> ps: on the way I noticed, that the Local{User,Sys}Cpu actually stays
> zero for a running job at us - while the remote values change. Is there
> something missing at us, that the statistics are not recorded/updated?

I think the Local* values are only updated for Standard Universe jobs, 
since with Standard Universe most system calls invoked by the job are 
actually sent over the network to be performed by the condor_shadow - 
thus the shadow could accumulate some pretty significant CPU usage. With 
vanilla universe and all other job universes, the shadows CPU usage 
should be very minimal / insignificant, thus HTCondor is not bothering 
to track it.

Hope the above helps
Todd