Long negotiation cycles can be caused by poor auto-clustering of jobs.
hmm, yes, this may be the cause. I've changed (even if it's not exactly
recommended) the SIGNIFICANT_ATTRIBUTES option to be like this:
SIGNIFICANT_ATTRIBUTES =
JobUniverse,LastCheckpointPlatform,NumCkpts,GreedyDebug,User,Requirements,NiceUser,ConcurrencyLimit
i.e. I've removed the DiskUsage attribute, which was the only differing
attribute in otherwise identical jobs. This sped up significantly the
negotiation phase (from ~2 minutes to 1 second).
Now about the side-effects of this modification: as far as I can see, the only
problem we may incur is if the first job in the auto-cluster has a completely
outlandish DiskUsage, while the others are more reasonable, those other job
will not get negotiated/executed, since the negotiator won't see them, being
hidden by the big DiskUsage of the first job.
Is that correct? Did I miss some other unwanted side-effect?