Re: [HTCondor-users] headers on condor_status command

[Stefano Dal Pra <stefano.dalpra@xxxxxxxxxxxx>]

On 14/12/22 11:08, Jeff Templon wrote:

Hi

The same thing with condor_history yields âinterestingâ results:

â  ~  condor_history -pr han1.cpf -limit 5
JOB_ID    Username     CMD           CPUS      MEMREQ  ST    RUN_TIME WorkerNode
49726.0   apuecher    /data/gravwav/a 32       31.2 GB C   3+18:32:11 wn-lot-043
49717.255 templon     /user/templon/g 1       128.0 MB C   17+11:49:5 wn-lot-056
49717.236 templon     /user/templon/g 1       128.0 MB C   17+06:08:4 wn-lot-057
49725.0   apuecher    /data/gravwav/a 16       31.2 GB C           25 wn-lot-040
49724.0   apuecher    /data/gravwav/a 16       31.2 GB C           25 wn-lot-040
â  ~  condor_history -pr han1.cpf -limit 5 | cat
JOB_ID    Username     CMD           CPUS      MEMREQ  ST    RUN_TIME WorkerNod
49726.0   apuecher    /data/gravwav/a 32       31.2 GB C   3+18:32:11 wn-lot-04
49717.255 templon     /user/templon/g 1       128.0 MB C   17+11:49:5 wn-lot-05
49717.236 templon     /user/templon/g 1       128.0 MB C   17+06:08:4 wn-lot-05
49725.0   apuecher    /data/gravwav/a 16       31.2 GB C           25 wn-lot-04
49724.0   apuecher    /data/gravwav/a 16       31.2 GB C           25 wn-lot-04

condor_history appears to detect whether the output goes to a pipe, and if so, it truncates the output at 79 characters. condor_q does not do this. How can I get condor_history to give the entire line to the pipe?

    You could possibly cherrypick from the following:
    
    [root@ce05-htc
        ~]# condor_history -lim 10 -af 'strcat(ClusterId,".",ProcId)'
        owner Cmd CpusProvisioned MemoryProvisioned '{"X","C"}[job
      status-3]' 'interval(EnteredCurrentStatus-JobStartDate)'
      'split(lastremotehost,".@")[1]'
      
    
    5077389.471
          faked_username
/storage/gpfs_data/some/long_path/simoutput/protons_100TeV_1PeV_Geant11/FTFP_BERT/flight_geometry/protons_1PeV_2600_3
        599_221212-131146/job.sh 1 128 C 1+18:22:18 wn-205-13-27-07-a
      5256230.0 pillhcb012 DIRAC_5g9watsq_pilotwrapper.py 1 2048
      C 1+10:49:33 cn-313-06-05
      
      5273081.0 belleprd DIRAC__byLP2_pilotwrapper.py 1 2048 C 2:14:34
      cn-316-01-02
      
      5274371.0 atlasprd005 runpilot2-wrapper.sh 1 2048 C 2:54:21
      cn-316-03-02
      
      5272259.0 atlasprd005 runpilot2-wrapper.sh 1 3072 C 7:18:41
      cn-314-04-04
      
      5272006.0 atlasprd005 runpilot2-wrapper.sh 1 2048 C 7:28:37
      wn-204-11-13-01-mtu
      
      5267541.0 atlasprd005 runpilot2-wrapper.sh 1 2048 C 16:49:49
      cn-316-01-09
      
      5273653.0 atlasprd005 runpilot2-wrapper.sh 1 2048 C 4:42:36
      cn-315-03-04
      
      5272308.0 atlasprd005 runpilot2-wrapper.sh 1 3072 C 7:12:43
      wn-205-13-35-05-a
      
      5272430.0 atlasprd005 runpilot2-wrapper.sh 1 3072 C 7:05:31
      wn-205-11-05-02-a
      
      5261559.0 pillhcb012 DIRAC_ghi0jsh1_pilotwrapper.py 1 2048 C
      1+00:39:25 wn-204-11-35-02-mtu
      
    
    Stefano

Thanks,

J âtruncate THIâ T

On 14 Dec 2022, at 9:31, Stefano Dal Pra wrote:

On 14/12/22 08:59, Jeff Templon wrote:

Hi,

Coming back to this, ÂI have the following cpf file:

SELECT NOSUMMARY

 ÂClusterId ÂAS ÂJOB_ID PRINTAS JOB_ID

 ÂOwner AS Username

 ÂCmd        AS CMD WIDTH 15 PRINTAS JOB_DESCRIPTION

 ÂCpusProvisioned  AS CPUS

 ÂMemoryProvisioned AS MEMREQ PRINTAS READABLE_MB

 ÂJobStatus     AS ST PRINTAS JOB_STATUS

 ÂRemoteUserCpu   AS RUN_TIME ÂWIDTH 12  ÂPRINTAS CPU_TIME

 ÂRemoteHost    ÂAS WorkerNode PRINTAS REMOTE_HOST

That last one is bugging me, because I really want a remote HOST, but what I get is a remote SLOT.

49718.224 templon Âferm.condor 274 1 Â Â128.0 MB R Â 19+21:43:30 slot1_24@xxxxxxxxxxxxxxxxxxxx

49718.226 templon Âferm.condor 276 1 Â Â128.0 MB R Â 19+21:43:30 slot1_24@xxxxxxxxxxxxxxxxxxxx

49718.243 templon Âferm.condor 293 1 Â Â128.0 MB R Â 19+21:43:30 slot1_24@xxxxxxxxxxxxxxxxxxxx

Is there some way I can get just the HOST (ie wn-lot-054.nikhef.nlÂor even better wn-lot-054)? ÂIâve tried several permutations (also with WIDTH and TRUNCATE) but canât make it work.

I would try substituting RemoteHost with
split(RemoteHost,".@")[1]

Example:
[root@htc-2 ~]# condor_q -name sn-02 -run -lim 3 -af 'split(remotehost,".@")[1]'
cn-313-02-03
cn-610-01-09
wn-204-13-39-02-a


Stefano

Thanks,

JT

On 22 Nov 2022, at 17:26, John M Knoeller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

You canât control the headers using -af, but you can if you use a print format file with the -pr option.

Custom print formats can fully control the classad print engine that -af and -format use, including the headings and specialized data formatters.

Â

https://htcondor.readthedocs.io/en/latest/classads/print-formats.html

Â

For example, if you create file named templon.cpf with this content:

Â

SELECT NOSUMMARY

ÂÂ Cmd

 ÂArguments

ÂÂÂCpusProvisioned AS CPUs

ÂÂÂJobIdÂ

ÂÂÂOwner

 ÂformatTime(Qdate) AS Qdate

ÂÂformatTime(JobCurrentStartDate) AS StartDateÂ

ÂÂÂMachine

 Âinterval(time()-JobCurrentStartDate) AS RunTime

 ÂCpusUsage

 ÂNumJobStarts

GROUP BY

ÂÂJobCurrentStartDate DECENDING

Â

Â

then run thisÂ

Â

condor_status -claimed -pr templon.cpf -constraint 'Owner=="templon"'

Â

Â

To duplicate some of the time formatting of the regular condor_status output, try using FORMATAS QDATE

Â

-tj

Â

Â

From:ÂHTCondor-users <htcondor-users-bounces@xxxxxxxxxxx>ÂOn Behalf OfÂJeff Templon
Sent:ÂTuesday, November 22, 2022 7:10 AM
To:ÂHTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject:Â[HTCondor-users] headers on condor_status command

Â

Hi,

I have commands like this:

â gofact_extendrange_ganymede git:(master) â condor_status -claimed -af:h Cmd Arguments CpusProvisioned JobId Owner 'formatTime(Qdate)' 'formatTime(JobCurrentStartDate)' Machine 'interval(time()-JobCurrentStartDate)' CpusUsage NumJobStarts -sort "-JobCurrentStartDate" -constraint 'Owner=="templon"'

CmdÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ÂÂÂÂÂÂArguments CpusProvisioned JobIdÂÂÂÂ OwnerÂÂ formatTime(Qdate)ÂÂÂÂÂÂÂ formatTime(JobCurrentStartDate) MachineÂÂÂÂÂÂÂÂÂÂÂÂÂ interval(time()-JobCurrentStartDate) CpusUsageÂÂÂÂÂÂÂÂÂÂÂÂ NumJobStarts

/user/templon/gofact_extendrange_ganymede/mers.condor 214ÂÂÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 49710.152 templon Fri Nov 18 10:21:07 2022 Fri Nov 18 10:22:33 2022ÂÂÂÂÂÂÂ wn-lot-048.nikhef.nl 4+03:08:05ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 0.9994810870997662ÂÂÂ 0

/user/templon/gofact_extendrange_ganymede/mers.condor 242ÂÂÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 49710.180 templon Fri Nov 18 10:21:07 2022 Fri Nov 18 10:23:56 2022ÂÂÂÂÂÂÂ wn-lot-047.nikhef.nl 4+03:06:42ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 0.9995679102645538ÂÂÂ 0

/user/templon/gofact_extendrange_ganymede/mers.condor 254ÂÂÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 49710.192 templon Fri Nov 18 10:21:07 2022 Fri Nov 18 10:32:35 2022ÂÂÂÂÂÂÂ wn-lot-050.nikhef.nl 4+02:58:03ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 0.9994999861801459ÂÂÂ 0

/user/templon/gofact_extendrange_ganymede/mers.condor 257ÂÂÂÂÂÂ 1ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 49710.195 templon Fri Nov 18 10:21:07 2022 Fri Nov 18 10:39:16 2022ÂÂÂÂÂÂÂ wn-lot-050.nikhef.nl 4+02:51:22ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ 0.9995049555423193ÂÂÂ 0

The output gets really bulky, partially because I have headers in the output with names likeÂinterval(time()-JobCurrentStartDate). Is there a way (aside from wrapping stuff in a script that does the substitution) to give the headers different names?

JT

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message toÂhtcondor-users-request@xxxxxxxxxxxÂwith a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/