Re: [HTCondor-users] why condor_q returns very slow with SOAP API?

[Greg Thain <gthain@xxxxxxxxxxx>]

On 05/28/2018 02:48 PM, Beyer, Christoph wrote:

---

Von: "çå" <aswywy@xxxxxxx>
An: "htcondor-users" <htcondor-users@xxxxxxxxxxx>
Gesendet: Montag, 28. Mai 2018 02:30:14
Betreff: [HTCondor-users] why condor_q returns very slow with SOAP API?

HI:

  I have build a 1000 node system with htcondor 8.4.9.The system use SOAP API for submitting and querying the jobs.It works well,except that condor_q function returns very slow.The condor_q function returns About 40000 jobs(completed jobs and running jobs) infomation,uses about 240 seconds.The Center Node runs Collector and Negotiator is a server which has 64GB DRAM and 500GB hard disk, Xeon 2605v4 CPU.

 ÂI wondor whyÂÂcondor_q return so slow with SOAP API?And Is there any solution speed up the function?

Thanks very much.

First, note that future versions of HTCondor will be removing SOAP support.

Because SOAP encodes everything in XML, it can be very expensive to parse large numbers of jobs, each of which is a large object.

If you can use the Python API to query the schedd, it supports the concept of a projection, which means "only send me these attributes from the job". If you don't need every attribute from every job, using a projection can often speed up query time by a factor of 10 or more.

-greg