Re: [HTCondor-users] Is anyone using the condor_stats command-line tool? We may remove it

["Hitchen, Greg (IM&T, Kensington WA)" <Greg.Hitchen@xxxxxxxx>]

Hi Todd

We use condor_stats and have KEEP_POOL_HISTORY=True on all our central managers.

We have our 9 pools forwarding to our condorview server. We don't use the default condorview graphs
or usage data any more. For graphs similar to what condorview used to do we use cron jobs to send info
to a ganglia server. We use condorview as more of a central repository that we use to track execute machine usage.

We have cron jobs that generate daily/weekly/monthly pool usage information from the viewhist files
 i.e. for each execute node we produce data for the time period in question like the following:

Name				%online %offline %claimed %unclaimed %owner
slot1@execute-node-01.*.*.au     100.00   0.00  24.99  70.28   4.72
slot2@execute-node-01.*.*.au     100.00   0.00  24.99  70.84   4.17
slot3@execute-node-01.*.*.au     100.00   0.00  24.99  70.56   4.45
slot4@execute-node-01.*.*.au     100.00   0.00  24.99  70.00   5.00
slot5@execute-node-01.*.*.au     100.00   0.00  24.99  70.56   4.45
slot6@execute-node-01.*.*.au     100.00   0.00  24.99  69.73   5.28
slot1@execute-node-02.*.*.au       33.62  66.38   0.00  10.56  23.06
slot2@execute-node-02.*.*.au       33.62  66.38   0.00  10.56  23.06
slot3@execute-node-02.*.*.au       33.62  66.38   0.00  10.56  23.06
slot4@execute-node-02.*.*.au       33.62  66.38   0.00  10.56  23.06
slot1@execute-node-03.*.*.au        100.00   0.00  24.99  39.44  35.56
slot2@execute-node-03.*.*.au        100.00   0.00  24.99  39.44  35.56
slot3@execute-node-03.*.*.au        100.00   0.00  24.99  39.44  35.56
slot4@execute-node-03.*.*.au        100.00   0.00  24.99  38.89  36.11
slot5@execute-node-03.*.*.au        100.00   0.00  24.99  39.44  35.56
slot6@execute-node-03.*.*.au        100.00   0.00  24.99  38.61  36.40
slot7@execute-node-03.*.*.au        100.00   0.00  24.99  38.61  36.40
slot8@execute-node-03.*.*.au        100.00   0.00  24.99  39.44  35.56
slot1@execute-node-04.*.*.au    100.00   0.00  24.99  72.51   2.50
slot2@execute-node-04.*.*.au    100.00   0.00  24.99  71.67   3.33
slot3@execute-node-04.*.*.au    100.00   0.00  24.99  70.56   4.45
slot4@execute-node-04.*.*.au    100.00   0.00  24.99  71.67   3.33
etc.
etc.

We then use automated scripts to generate daily overall execute machine statistics/usage for
monthly reporting. Separate to this we also generate daily job statistics from all the submit
nodes history files.

We can also run manual scripts to produce usage data for individual machines, groups of
machines and/or pools.

Whether any/all of this is of any use is anyone's guess. We mainly use it to update central
mssql databases that have these types of statistics for all our HPC and HTC clusters. It keeps
managers happy when we can then generate pretty coloured graphs and pie charts. ð

It has also proven useful on a few occasions when someone up the management chain
starts querying the "utilization" of our scientific computing resources.

Sorry to be a potential spanner in the works re condor_stats and condorview. Maybe there is
another way for us to generate this type of data/information?

Cheers

Greg


-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Todd Tannenbaum
Sent: Tuesday, 5 May 2020 4:12 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Is anyone using the condor_stats command-line tool? We may remove it

Hi,

Is anyone using the "condor_stats" [1] command line tool?  Or the ancient Java applet based CondorView? (yes, you read 
that correct... Java applet, not JavaScript!)

These tools only work if your pool's collector config has explicitly configured KEEP_POOL_HISTORY=True (the default is 
False).

I am asking because we would like to remove the KEEP_POOL_HISTORY collector option, and also the condor_stats command 
line tool, from the HTCondor v8.9 series.  Would anyone object?

Thanks
Todd

[1] https://htcondor.readthedocs.io/en/latest/man-pages/condor_stats.html

-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) xxx-xxxx                  Madison, WI 53706-1685
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/