Mailing List Archives
Authenticated access
|
|
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif)
|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] condor_status -compact
- Date: Thu, 5 Dec 2019 17:58:01 +0000
- From: John M Knoeller <johnkn@xxxxxxxxxxx>
- Subject: Re: [HTCondor-users] condor_status -compact
I have added this as a bug. the ticket is here
https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=7415
-tj
-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of John M Knoeller
Sent: Friday, November 29, 2019 4:46 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] condor_status -compact
Ah, so those machines all have both static slots and p-slots. And the static slots are *before* the p-slot (when sorted by name)
The code for summing all of the slots on a machine knows how to summarize static slots to a p-slot, but not how to summarize
a p-slot into an existing summary of static slots, so it doesn't know the actual slot total, and just prints _ instead.
This has nothing to do with SL6 vs. Centos however. The problem is that condor_status -compact can only handle a mixture
of static slots and p-slots if the p-slots come first when sorted by name.
You can work around this problem by using a SLOT_TYPE_N_NAME_PREFIX to cause either the static slots or the p-slot to be named something
other than slotN, and choose names that make the p-slot show up first in the sort order.
-tj
-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Beyer, Christoph
Sent: Friday, November 29, 2019 2:47 PM
To: htcondor-users <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] condor_status -compact
Hi Tj,
thanks for taking care about this :)
batch1051 has 10 static slots for jupyter notebooks (which should be just 1 slot but I reconfigured it while testing, have to revert that) and 1 partitionable slot with the rest cores (38):
[root@htc-it11 config.d]# condor_status batch1051
Name OpSys Arch State Activity LoadAv Mem ActvtyTime
slot1@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:44
slot2@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot3@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot4@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot5@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot6@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot7@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot8@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot9@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot10@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 4096 14+22:38:54
slot11@xxxxxxxxxxxxxxxxx LINUX X86_64 Unclaimed Idle 0.000 140941 14+22:39:45
slot11_1@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.830 2048 0+00:03:34
slot11_2@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.920 2048 0+00:03:31
slot11_3@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.650 2048 0+00:21:20
slot11_4@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.790 2048 0+00:19:39
slot11_5@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.050 2048 0+00:01:16
slot11_6@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Idle 0.000 2048 0+00:02:04
slot11_8@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.850 2048 0+00:17:14
slot11_9@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.890 2048 0+00:15:12
slot11_10@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.910 2048 0+00:15:03
slot11_11@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.890 2048 0+00:15:02
slot11_12@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.900 2048 0+00:15:00
slot11_13@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.860 2048 0+00:14:59
slot11_14@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.890 2048 0+00:14:57
slot11_15@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.910 2048 0+00:14:56
slot11_16@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.890 2048 0+00:14:56
slot11_17@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.870 2048 0+00:14:49
slot11_18@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.880 2048 0+00:14:49
slot11_19@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.820 2048 0+00:14:48
slot11_20@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.850 2048 0+00:14:48
slot11_21@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.910 2048 0+00:14:48
slot11_22@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.870 2048 0+00:14:47
slot11_23@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.730 2048 0+00:12:47
slot11_24@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.890 2048 0+00:12:47
slot11_25@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.870 2048 0+00:12:47
slot11_26@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.850 2048 0+00:12:46
slot11_27@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.900 2048 0+00:12:46
slot11_28@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.850 2048 0+00:12:46
slot11_29@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.820 2048 0+00:12:45
slot11_30@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.770 2048 0+00:12:45
slot11_31@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.910 2048 0+00:12:44
slot11_32@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.870 2048 0+00:12:44
slot11_33@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.870 2048 0+00:12:44
slot11_34@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.870 2048 0+00:12:43
slot11_35@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.930 2048 0+00:12:43
slot11_36@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.890 2048 0+00:12:42
slot11_37@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.830 2048 0+00:12:42
slot11_38@xxxxxxxxxxxxxxxxx LINUX X86_64 Claimed Busy 0.860 2048 0+00:12:42
Machines Owner Claimed Unclaimed Matched Preempting Drain
X86_64/LINUX 48 0 37 11 0 0 0
Total 48 0 37 11 0 0 0
[root@htc-it11 config.d]# condor_status batch1051.desy.de -af:h Name PartitionableSlot DynamicSlot NumDynamicSlots
Name PartitionableSlot DynamicSlot NumDynamicSlots
slot1@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot2@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot3@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot4@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot5@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot6@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot7@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot8@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot9@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot10@xxxxxxxxxxxxxxxxx undefined undefined undefined
slot11@xxxxxxxxxxxxxxxxx true undefined 34
slot11_1@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_2@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_3@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_4@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_5@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_6@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_7@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_9@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_11@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_12@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_13@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_14@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_15@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_16@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_17@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_18@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_19@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_20@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_22@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_23@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_24@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_25@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_26@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_27@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_28@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_29@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_30@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_31@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_32@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_34@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_35@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_36@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_37@xxxxxxxxxxxxxxxxx undefined true undefined
slot11_38@xxxxxxxxxxxxxxxxx undefined true undefined
All the centos7 nodes show the hyphen in status -compact ...
Best
Christoph
--
Christoph Beyer
DESY Hamburg
IT-Department
Notkestr. 85
Building 02b, Room 009
22607 Hamburg
phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx
----- UrsprÃngliche Mail -----
Von: "johnkn" <johnkn@xxxxxxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Freitag, 29. November 2019 19:45:07
Betreff: Re: [HTCondor-users] condor_status -compact
compact mode fetches only p-slots and static slots, it adds the constraint
&& (PartitionableSlot =?= true || DynamicSlot =!= true)
So that it doesn't fetch dynamic slots at all, and the "Slots" column is the value of the NumDynamicSlots field
a _ indicates that there is no NumDynamicSlots field, is batch1051.desy.de a single huge static slot?
what does
condor_status batch1051.desy.de -af:h Name PartitionableSlot DynamicSlot NumDynamicSlots
show?
-tj
-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Beyer, Christoph
Sent: Friday, November 29, 2019 3:40 AM
To: htcondor-users <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] condor_status -compact
Hi,
I do have two questions concerning condor_status compact.
The output seems to be somehow different for SL6 and CEntOS7:
[root@bird-htc-sched14 ~]# condor_status -compact
Machine Platform Slots Cpus Gpus TotalGb FreCpu FreeGb CpuLoad ST Jobs/Min MaxSlotGb
batch1051.desy.de x64/CentOS7 _ 48 251.64 1 4.00 0.45 Ui 0.00 *
<snip>
bird-cfel01.desy.de x64/SL6 11 12 252.37 1 234.37 0.85 ** 3.50 2.00
The number of slots is not displayed but a '-' instead ?
Also the total at this moment gives me:
Machines Owner Claimed Unclaimed Matched Preempting Drain
x64/CentOS7 3302 0 3081 215 0 0 6
x64/SL6 2883 0 2759 111 0 9 4
Total 6185 0 5840 326 0 9 10
While adding up the total of partitionable slot-cpus gives:
[root@bird-htc-sched14 ~]# condor_status -constraint 'OpSysAndVer == "CentOS7"' -af NAME TotalSlotCpus SlotType | awk '$3=="Partitionable"{s+=$2}END{print s}'
3826
(I know this could be done more professional but it happens to be the way we process it for some plots)
I went through most of the documentation (at least I think so) but could not figure out where the considerable difference between the two numbers comes from ?
As always thanks for every hint ! ;)
Best
Christoph
--
Christoph Beyer
DESY Hamburg
IT-Department
Notkestr. 85
Building 02b, Room 009
22607 Hamburg
phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/