Re: [HTCondor-users] Filling pool breadth-first with partitionable slots (Beyer, Christoph)

[Tom Downes <downes@xxxxxxx>]

I like depth-first owing as it is an anti-fragmentation tool, but breadth-first is, in my experience, better for input file transfers in the real-world because my storage can easily swamps the 1GbE network interface on an N-core machine run N jobs.

--

Tom Downes
Senior Scientist and Data CenterÂManager
Center for Gravitation, Cosmology and Astrophysics
University of Wisconsin-Milwaukee
414.229.2678

On Mon, Oct 31, 2016 at 11:09 AM, Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx> wrote:

I whipped this up a couple of years ago:

STARTD_ATTRS = $(STARTD_ATTRS) DynamicSlotID
DynamicSlotID = ifThenElse(DynamicSlot, regexps("slot\d+_(\d+).*", Name, "\1", "i"), UNDEFINED)

However I don't think this will help much with ranking.

There's also a NumDynamicSlots attribute which may be what you're looking for - you'd rank the partitionable slots with the lowest number of dynamic slots highest. I'm not sure which version that attribute appeared in, though.

Personally, I prefer depth-first fill because the Linux buffer cache coupled with a 12-hour automount timeout helps limit the amount of NFS traffic generated by jobs starting up, as most of our jobs pull inputs from NFS filesystems rather than an input transfer.

    -Michael Pelletier.

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Jason Liu
Sent: Monday, October 31, 2016 3:39 AM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] Filling pool breadth-first with partitionable slots (Beyer, Christoph)

Hi Christoph,


Thank you for your email.

While the solution you suggested works for static slots, it doesnât work for p-slots. SlotID is always 1 for p-slots since p-slot names are slot1_1, slot1_2 etc instead of slot1, slot2.

Unless I am missing something?

Kind Regards
Jason

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/