Re: [HTCondor-users] slow submission rate

[Dan Bradley <dan@xxxxxxxxxxxx>]

Be aware that turning off fsync in the condor_schedd can lead to loss of job state in the event of power loss or other sudden death of the schedd.  This could result in jobs that were submitted shortly before the outage disappearing from the queue without being run.  It could also result in jobs being run twice.

If that is acceptable for your purposes, then your problem is solved.  If it is not acceptable, then focus on improving the performance of the filesystem containing $(SPOOL).

--Dan

On 8/2/13 8:14 AM, Pek Daniel wrote:

Thanks, the FSYNC trick solved the issue! :)

2013/8/1 Dan Bradley <dan@xxxxxxxxxxxx>

Are you timing just condor_submit, or are you also timing job run/completion rates?

Job submissions cause the schedd to commit a transaction to $(SPOOL)/job_queue.log.  If the disk containing that is slow, submissions will be slow.  One way to verify if this is the limiting factor is to add the following to your configuration:

CONDOR_FSYNC = FALSE

Another thing to keep in mind is that if you can batch submissions of many jobs into a single submit file, there will be fewer transactions.

--Dan

On 8/1/13 10:17 AM, Pek Daniel wrote:

Hi!

I'm experimenting with condor: I'm trying to submit a lot of dummy
jobs with condor_submit from multiple submission hosts simultaneously.
I have only a single schedd. I'm trying to stresstest this schedd.
These jobs are in the vanilla universe.

The problem is that I couldn't reach better result than 4-6
submission/sec, which seems a little low. I can't see any real
bottleneck on the machine, so I suspect that it's because of some
default value of a configuration option which throttles down the
submission requests.

Any idea how to solve this?

Thanks,
Daniel

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/