Re: [HTCondor-users] Programmatic access to match analysis?

[Ian McEwen <mian@xxxxxxxxxxx>]

This is a decent start! We don't have C++ expertise in-house, so ideally
this would be accessible some other fairly sane way -- python bindings
or similar -- rather than making any attempt at parsing condor_q output
or reimplementing the analysis code ourselves in one language or
another.

Similarly to what Michael said, it'd also be nice if there's a way to
query the negotiator for concurrency limits, rather than having
out-of-band knowledge of what the limits are set to and extracting it
from the running jobs. But in both of these cases I do at least realize
the officially-blessed way of doing this would be to use condor_q
interactively and our system's way of doing things is out of the norm.

Thanks for the response and the pointer to the condor_q code for this!

On Tue, Jan 29, 2019 at 05:43:45PM +0000, John M Knoeller wrote:
> If you are writing C++ code,  the better-analyze code is in HTCondor source at src\condor_q.V6\jobmatch.cpp. look at the doJobRunAnalysis function. 
> 
> Were you looking for programmatic access in a different language?
> 
> By the way this code does matchmaking analysis only.  There is currently no code in HTCondor outside of the Negotiator itself that checks concurrency limits -  only the Negotiator knows what limits have been handed out already.   
> 
> -tj
> 
> -----Original Message-----
> From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Ian McEwen
> Sent: Monday, January 28, 2019 6:34 PM
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Subject: [HTCondor-users] Programmatic access to match analysis?
> 
> Hi;
> 
> This isn't something I'm expecting to have time to do soon, so I'm just
> gathering information for now, but I'm wondering if there's an
> appropriate way to get programmatic access to what `condor_q -analyze`
> and `-better-analyze` output, especially for jobs that have not yet been
> submitted.
> 
> That is, given a job ad (constructed however is appropriate), is there a
> good way to test, non-interactively and without submitting the ad,
> whether there is a slot that meets the requirements (or, perhaps, if
> there could be one with partitionable slot defragmentation?), whether
> concurrency limits are met, etc.?
> 
> As many of you probably know, we run a system where the direct
> interaction with HTCondor (including building job ads and submitting
> them) is handled by us, so users are using our web interface without
> needing to know HTCondor itself, but lately we've been adding more
> features that allow users to select their requirements more
> specifically, which may increase the possibility that jobs will be
> unable to match (or wait a long time to do so). I'd like to build
> something that can give users an idea of whether their job is likely to
> run soon, and why or why not.
> 
> Thanks in advance for any help!
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/