Re: [Condor-users] quick condor_rooster question

[Dan Bradley <dan@xxxxxxxxxxxx>]

Ian,

Good point.  I am thinking that the following two things should be added 
to rooster to address this concern:

1. as you suggested, have a maximum number of machines to wake up per cycle

2. verify that the UNHIBERNATE expression is still true immediately 
before trying to wake up a machine.  Currently, it builds a list of all 
machines to wake up and then it iterates through that list and wakes 
them up.  If there is a significant time delay during the iteration, it 
could be that the UNHIBERNATE expression will expire.  Example of an 
expiring expression:

UNHIBERNATE = MachineLastMatchTime > CurrentTime - 600

--Dan

Smith, Ian wrote:
> Thanks for  the quick reply. I had thought of this as it's
> a quick fix but the only problem I see is if someone submits
> a large number of short jobs. By the time condor_power
> has finished waking up all the machines it may be a couple
> of hours later and they may  have completed anyway. I'm
> thinking of creating something that will just wakeup the
> first n machines on each cycle and forget about the
> rest (which will be woken on a later cycle in necessary).
> Bit like a badly behaved FIFO.
>
> Would be nice if a limit could be included in Condor - hint hint.
>
> cheers.
>
> -ian.
>
>   
> > -----Original Message-----
> > From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-
> > bounces@xxxxxxxxxxx] On Behalf Of Dan Bradley
> > Sent: 02 March 2010 16:03
> > To: Condor-Users Mail List
> > Subject: Re: [Condor-users] quick condor_rooster question
> >
> > Hello Ian,
> >
> > There is no configuration setting to limit the number of machines woken
> > in a cycle.  As you suggested, you could limit the rate by wrapping
> > condor_power with a script that sleeps.  Here's an example that limits
> > the rate to 1 wakeup per 30s:
> >
> > ROOSTER_WAKEUP_CMD = "/path/to/sleep_exec 30 $(BIN)/condor_power -d -i"
> >
> > And then create sleep_exec:
> >
> > #!/bin/sh
> > sleep $1
> > shift
> > exec "$@"
> >
> > Hacky workaround or elegant plug-in.  Call it what you wish!
> >
> > --Dan
> >
> >
> > Smith, Ian wrote:
> >     
> > > Hello All,
> > >
> > > Very quick question - is there an easy way of limiting the number of machines
> > >       
> > woken up
> >     
> > > on each condor_rooster cycle. ( The reason I ask is that if a user submits a large
> > > number of jobs then it appears that condor_rooster will try to wake up everything
> > > and the manager will get swamped. ) .I can think of a hacky workaround by
> > > substituting condor_power for my own script but something more elegant
> > > would be nice.
> > >
> > > cheers,
> > >
> > > -ian,
> > >
> > > --------------------------------------------
> > > Dr Ian C. Smith,
> > > e-Science Team,
> > > The University of Liverpool,
> > > Computing Services Department
> > >
> > > _______________________________________________
> > > Condor-users mailing list
> > > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > > subject: Unsubscribe
> > > You can also unsubscribe by visiting
> > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > >
> > > The archives can be found at:
> > > https://lists.cs.wisc.edu/archive/condor-users/
> > >
> > >       
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >     
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
>