Re: [HTCondor-users] how to get current 'fair share' quota/weight of a group?

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi Todd,

thanks for the info.
Unfortunately I had not checked the release notes but only had a look in
the manual's appendix on the ad types
  http://research.cs.wisc.edu/htcondor/manual/latest/12_Appendix_A.html
I guess the accounting type is related to the daemoncore's statistics

Sorry for the noise,
  Thomas

On 2016-09-26 18:01, Todd Tannenbaum wrote:
> On 9/26/2016 10:33 AM, Thomas Hartmann wrote:
>> Hi Iain,
>>
>> thanks for the info.
>>
>> But while we are running on 8.5.1, I do not find any ads of an
>> 'Accounting' type?
>>
> 
> Short answer:
> 
> You need v8.5.3+ to see the accounting info in the collector, and
> starting with v8.5.4+ the condor_userprio tool queries the collector
> instead of the negotiator.
> 
> Longer answer:
> 
> We try hard to make the Version History section of the Manual pretty
> comprehensive.  I find it useful to go to Manuals page at
>   http://htcondor.org/manual/
> then click on "Version History" for the series I care about (v8.5.x,
> v8.4.x, whatever) and see what changed.  Doing this followed by Ctrl-F
> (find in my browser) for the word "accounting", I found
> 
> Version 8.5.4
> 
> - condor_user_prio now queries the condor_collector for accounting
> information by default, when appropriate. This should be much faster
> than the older way of querying the condor_negotiator. The old path is
> still available by passing the -negotiator option to the tool. (Ticket
> #5508).
> 
> 
> Version 8.5.3
> 
> - The negotiator now forwards accounting information to the collector,
> where it can be easily queried and monitored. (Ticket #5491).
> 
> 
> Clicking on the "Ticket XXX" entries next to items in the Version
> History takes you to the associated development ticket in the HTCondor
> Wiki.  In the ticket you can see details incl discussion/remarks on the
> feature, design documents (for more complex changes), changes made to
> the HTCondor Manual, code changes, etc.
> 
> Hope the above helps
> Todd
> 
> 
> 
>> Skimming through all(?) ads, followingly I do not see neither related
>> ads, e.g.,
>>   > condor_status -pool condor01 -any -long  | grep Weighted | cut -d "
>> " -f 1 | sort | uniq
>> WeightedIdleJobs
>> WeightedRunningJobs
>>   > condor_status -pool condor01 -any -long  | grep Accumulated | cut -d
>> " " -f 1 | sort | uniq
>>  -
>>
>> Are we missing maybe something in our setup?
>>
>> Cheers and thanks,
>>   Thomas
>>
>> On 2016-09-26 16:53, Iain Bradford Steers wrote:
>>> Hi Thomas,
>>>
>>> Which version of condor are you running on your central manager?
>>>
>>> In the 8.5 series the negotiator now sends it's accounting
>>> information to the collector as an ad per item in the tree.
>>>
>>> Alternatively using the python bindings you can instantiate a
>>> Negotiator object and use .getPriorities().
>>>
>>> The Collector is obviously much faster.
>>>
>>> Cheers, Iain
>>>
>>> You can see the list with condor_status  -any -const 'MyType =?=
>>> "Accounting"'
>>> For example:
>>>
>>> Parts of the tree also have a couple of others like ConfigQuota set.
>>>
>>> UnchargedTime = -6
>>> LastUsageTime = 1474901308
>>> AuthenticatedIdentity = "condor@fsauth"
>>> IsAccountingGroup = false
>>> PriorityFactor = 100000.0
>>> MyType = "Accounting"
>>> AccountingGroup = "group_u_X"
>>> Name = "group_u_x"
>>> WeightedAccumulatedUsage = 128064692224.0
>>> LastUpdate = 1474901308
>>> WeightedResourcesUsed = 7677.0
>>> ResourcesUsed = 7677
>>> TargetType = "none"
>>> AccumulatedUsage = 127910363136.0
>>> LastHeardFrom = 1474901320
>>> BeginUsageTime = 1447772083
>>> DaemonStartTime = 1473746649
>>> UpdateSequenceNumber = 0
>>> Priority = 937499392.0
>>> WeightedUnchargedTime = -6.0
>>> ________________________________________
>>> From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf
>>> of Thomas Hartmann [thomas.hartmann@xxxxxxx]
>>> Sent: 26 September 2016 16:45
>>> To: HTCondor-Users Mail List
>>> Subject: [HTCondor-users] how to get current 'fair share'
>>> quota/weight of a     group?
>>>
>>> Hi all,
>>>
>>> on a user's question [1], I am currently looking for a ClassAd with the
>>> current weighting/fair share quota of a given group?
>>>
>>> I.e.,
>>> we have several dynamic quota groups. If I remember correctly, by
>>> default the resource usage over the last three days is taken into
>>> account for weighting quotas.
>>> So, I have been skimming, so far unsuccessfully, through
>>>   -state
>>>   -collector
>>> looking for an ad with a flat value of the current quota.
>>> But is there actually such a 'scalar' ad?
>>>
>>> If not, I assume, that the quota is calculated by the collector each
>>> negotiation cycle anew for any queued job with its resource requests
>>> (including ConcurrencyLimits, ChildAccountingGroup,...)?
>>> And if so, is there an 'easy' algorithm to calculate oneself the
>>> collector's hypothetical current quota for a job against everybody else
>>> being queued?
>>>
>>> Cheers and thanks,
>>>   Thomas
>>>
>>>
>>> [1]
>>> who would like to try to 'safe up' its fair share quota for some more
>>> higher priority work load - which is from my point of view really *not*
>>> a good approach, i.e., folding some oscillating behaviour into the
>>> system,...
>>> _______________________________________________
>>> HTCondor-users mailing list
>>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>>> with a
>>> subject: Unsubscribe
>>> You can also unsubscribe by visiting
>>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>>
>>> The archives can be found at:
>>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>>
>>
>>
>>
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>> with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>
> 
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature