Re: [HTCondor-users] Weirdness with cpususage

Mailing List Archives Authenticated access	UW Madison Computer Sciences Department Computer Systems Lab

On Oct 25, 2024, at 11:08âAM, Jaime Frey via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

There are a few confounding factors with using these job attributes. RemoteWallClockTime records the time across all execution attempts for the job, whereas RemoteUserCpu records the time from just the last execution attempt. To get the wall clock time for just the last execution, you should use LastRemoteWallClockTime. Also, RemoteWallClockTime and LastRemoteWallClockTime include time spent doing file transfer.
CpusUsage is computed from the cpu usage and wall clock time of just the last execution of the job (no file transfer time included).

- Jaime

On Oct 25, 2024, at 3:34âAM, Jeff Templon <templon@xxxxxxxxx> wrote:

Hi,

I was trying to use Cpususage in a meaningful way, but itâs not working. See below (done-cpususage.cpf included at end of mail):

condor_history -completedsince $(date -d "7 days ago" +"%s") -constraint 'RemoteUserCpu > 10*RemoteWallClockTime' -print-format ~templon/done-cpususage.cpf -wide:132 | head -20

JOB_ID Username CMD Finished CPUS CPUS used RAM MEM ST CPU_TIME WALL_TIME WorkerNode

703590.0 admayol run_63.sh 10/25 09:35 32 0.435 1.9 GB 2.9 GB C 2+02:02:22 3:47:09 wn-lot-054

703595.0 admayol run_68.sh 10/25 09:16 32 0.594 2.0 GB 3.1 GB C 1+01:30:54 1:57:07 wn-lot-044

703593.0 admayol run_66.sh 10/25 08:42 32 14.76 2.0 GB 2.9 GB C 1+00:22:37 1:54:23 wn-lot-048

703592.0 admayol run_65.sh 10/25 08:18 32 12.52 1.9 GB 2.9 GB C 1+00:09:40 1:51:30 wn-lot-052

696992.0 roystege pineko theory -c / 10/25 07:18 32 0.997 11.5 GB 11.9 GB C 22+11:17:47 23:34:58 wn-lot-044

696632.0 roystege pineko theory -c / 10/25 06:51 32 0.997 11.8 GB 14.3 GB C 31+12:35:56 1+09:11:12 wn-lot-056

703589.0 admayol run_62.sh 10/25 06:26 32 13.76 1.9 GB 2.6 GB C 20:39:48 1:37:30 wn-lot-052

696917.0 roystege pineko theory -c / 10/25 06:00 32 0.999 11.7 GB 11.9 GB C 22+21:33:26 23:55:36 wn-lot-047

703586.0 admayol run_59.sh 10/25 04:48 32 13.22 1.8 GB 2.6 GB C 19:34:31 1:31:57 wn-lot-052

696842.0 roystege pineko theory -c / 10/25 04:33 32 0.997 11.6 GB 11.9 GB C 23+01:42:46 1+00:12:05 wn-lot-048

703585.0 admayol run_58.sh 10/25 04:01 32 14.85 2.2 GB 2.9 GB C 1+01:50:47 1:55:44 wn-lot-054

696767.0 roystege pineko theory -c / 10/25 03:41 32 1.000 12.5 GB 14.3 GB C 24+22:23:19 1+01:27:43 wn-knek-002

703584.0 admayol run_57.sh 10/25 03:16 32 0.610 2.1 GB 3.1 GB C 1+05:00:05 2:14:22 wn-lot-052

The CPUS used column is Cpususage, and should be according to the docs, CPU_TIME divided by WALL_TIME. In the output, sometimes it is, and sometimes it isnât.

How is this possible?

JT

done-cpususage.cpf:

SELECT NOSUMMARY

ClusterId AS JOB_ID PRINTAS JOB_ID

Owner AS ' Username'

join(" ",split(Cmd,"/")[size(split(Cmd,"/"))-1], Args) AS ' CMD' WIDTH -18

CompletionDate AS ' Finished ' PRINTAS DATE

CpusProvisioned AS ' CPUS ' WIDTH 3

CpusUsage AS ' CPUS used ' WIDTH 5

# MemoryProvisioned AS ' MEMREQ' PRINTAS READABLE_MB

ResidentSetSize_RAW AS ' RAM' PRINTAS READABLE_KB WIDTH 8

ImageSize AS ' MEM' PRINTAS READABLE_KB WIDTH 10

JobStatus AS " ST" PRINTAS JOB_STATUS

interval(RemoteUserCpu) AS " CPU_TIME" WIDTH 12

interval(RemoteWallClockTime) AS " WALL_TIME" WIDTH 12

split(splitSlotName(LastRemoteHost)[1], ".")[0] AS "WorkerNode" WIDTH -12

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

Mailing List Archives

Authenticated access

Re: [HTCondor-users] Weirdness with cpususage