Re: [Condor-users] Condor + EC2

[Lukas Slebodnik <slebodnik@xxxxxxxx>]

I know, that you solved your problem with shutdown, but I am replying only for
record in mailing list. If default hibernation plugin do not recognize shutdown
state (file "/sys/power/disk" did not contain string "shutdown"), then it is
possible to write own plugin. I attached an example bash plugin.

But if you would like to consider near an hour boundary shutdown, than cron
version will be more similar than condor versiona.

Regards,
Lukas

> $ /usr/lib/condor/libexec/condor_power_state -d -m pm-utils ad
> 11/18/11 00:01:02 LinuxHibernator: Trying method 'pm-utils'
> 11/18/11 00:01:02 hibernator: 'pm-utils' detected
> HibernationSupportedStates = ""
> HibernationMethod = "pm-utils"
> HibernationRawMask = 0
> CurrentTime = time()
> 
> $ /usr/lib/condor/libexec/condor_power_state -d -m /sys ad
> 11/17/11 23:59:00 LinuxHibernator: Trying method '/sys'
> 11/17/11 23:59:00 hibernator: skipping 'pm-utils'
> 11/17/11 23:59:00 hibernator: '/sys' detected
> HibernationSupportedStates = ""
> HibernationMethod = "/sys"
> HibernationRawMask = 0
> 
> $ /usr/lib/condor/libexec/condor_power_state -d -m /proc ad
> 11/17/11 23:59:10 LinuxHibernator: Trying method '/proc'
> 11/17/11 23:59:10 hibernator: skipping 'pm-utils'
> 11/17/11 23:59:10 hibernator: skipping '/sys'
> 11/17/11 23:59:10 hibernator: '/proc' not detected; hibernation disabled
> condor_power_state: Initialization error.
> 
> Since I need to get this done I came out with a cron:
> */1 * * * * if grep -Fxq 'CONDOR_HOST = $(FULL_HOSTNAME)'
> /etc/condor/condor_config.local; then loadavg=`cat /proc/loadavg | tr " "
> "\n" | awk 'NR==2'`; loadavgpoweroff=`echo "${loadavg} < 0.01" | bc`; if [
> ${loadavgpoweroff} -ne "0" ]; then /sbin/poweroff; fi; fi;
> 
> Expanded:
> 
> if grep -Fxq 'CONDOR_HOST = $(FULL_HOSTNAME)'
> /etc/condor/condor_config.local
>     then
>         loadavg=`cat /proc/loadavg | tr " " "\n" | awk 'NR==2'`
>         loadavgpoweroff=`echo "${loadavg} < 0.01" | bc`
>         if [ ${loadavgpoweroff} -ne "0" ]
>             then
>                 /sbin/poweroff
>         fi
> fi
> 
> The first if is specific to my set-up, but you can get the idea.
> Using it everything works fine, but would be good to figure why condor
> dosent work...
> Cheers
> 
> On Thu, Nov 17, 2011 at 3:41 PM, Lukas Slebodnik <slebodnik@xxxxxxxx> wrote:
> 
> > Althought hibernation will not be possible, shutdown "S5" should be
> > detected
> > with "/sys" method. It works on our machines.
> >
> > /usr/lib/condor/libexec/condor_power_state -d -m /sys ad
> >
> > Details:
> >
> > http://research.cs.wisc.edu/condor/manual/v7.7/3_16Power_Management.html#SECTION004164000000000000000
> >
> > Regards,
> > Lukas
> >
> > On Thu, Nov 17, 2011 at 03:31:24PM -0800, Tiago Macarios wrote:
> > > Hey Lukas thanks for the help. That's the output:
> > >
> > > Using STARTD_DEBUG = D_FULLDEBUG:
> > > 11/17/11 23:25:00 HibernationSupportedStates invalid '' in ad from
> > > hibernation plugin /usr/lib/condor/libexec/condor_power_state
> > > 11/17/11 23:25:00 Detected hibernation states:
> > >
> > > $ /usr/lib/condor/libexec/condor_power_state -d ad
> > > 11/17/11 23:29:06 LinuxHibernator: Trying all methods
> > > 11/17/11 23:29:06 hibernator: 'pm-utils' detected
> > > HibernationSupportedStates = ""
> > > HibernationMethod = "pm-utils"
> > > HibernationRawMask = 0
> > > CurrentTime = time()
> > >
> > > On Thu, Nov 17, 2011 at 2:59 PM, Lukas Slebodnik <slebodnik@xxxxxxxx>
> > wrote:
> > >
> > > > Enable verbose logging for startd daemon and then restart startd
> > daemon.
> > > > Startd
> > > > daemon determine power setting only in startup process. I think that
> > there
> > > > will be something interesting.
> > > >
> > > > STARTD_DEBUG = D_FULLDEBUG
> > > >
> > > > By default condor use power_state utility for detection, so this could
> > be
> > > > another point of failure. Try to run it by hand. power_state should be
> > > > installed
> > > > in directory /usr/libexec/condor/
> > > >
> > > > /usr/libexec/condor/power_state -d ad
> > >
> > >
> > > > Regards,
> > > > Lukas
> > > >
> > > > On Thu, Nov 17, 2011 at 02:01:40PM -0800, Tiago Macarios wrote:
> > > > > Your command returns blank:
> > > > > condor_status -l | grep HibernationSupportedStates
> > > > > HibernationSupportedStates = ""
> > > > >
> > > > > does it mean that my Condor configuration can not
> > hibernate/poweroff? If
> > > > > yes, how can I enable it? What could be missing?
> > > > >
> > > > > On Thu, Nov 17, 2011 at 1:16 PM, Lukas Slebodnik <slebodnik@xxxxxxxx
> > >
> > > > wrote:
> > > > >
> > > > > > You can test with next command, whether shutdown state is
> > supported:
> > > > > > condor_status -f "%s " Name -f "%s" HibernationSupportedStates -f
> > "\n"
> > > > > > NewLine
> > > > > >
> > > > > > Daemon condor_startd initially invokes hibernation plug-in to
> > determine
> > > > > > what
> > > > > > low power setting to use on further invocations of the plug-in.
> > With
> > > > > > attribute
> > > > > > HibernationMethod you can specifies a string which describes the
> > > > mechanism
> > > > > > used
> > > > > > by the plug-in. Default order is 1st "pm-utils", 2nd "/sys", 3rd
> > > > "/proc"
> > > > > >
> > > > > > I tried default setting and only "S3" and "S4" states was detected.
> > > > > > "S5" was detected only with "/sys" method and not with default
> > > > "pm-utils"
> > > > > > I tested it manualy with command  power_state -m /sys ad
> > > > > >
> > > > > > Regards,
> > > > > > Lukas
> > > > > >
> > > > > > On Thu, Nov 17, 2011 at 09:38:22AM -0500, Tim St Clair wrote:
> > > > > > > You can set the hibernation level:
> > > > > > >
> > > > > > > "S5", "5", "SHUTDOWN", "OFF": Shutdown (soft-off)
> > > > > > >
> > > > > > > http ://research.cs. wisc . edu
> > /condor/manual/v7.7/3_3Configuration.
> > > > > > html #18332
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Tim
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > >
> > > > > > > > From: " Tiago Macarios " < tiagomacarios @ gmail .com>
> > > > > > > > To: "Condor-Users Mail List" <condor-users@cs. wisc . edu >
> > > > > > > > Sent: Wednesday, November 16, 2011 7:40:16 PM
> > > > > > > > Subject: Re: [Condor-users] Condor + EC2
> > > > > > >
> > > > > > > > Finally got everything working an will post it here when it is
> > > > > > > > cleaned up.
> > > > > > > > I was just wondering is there is a way in condor to power off
> > the
> > > > > > > > machines if they idle for a while.
> > > > > > >
> > > > > > > > I saw in the power management section of the documentation
> > that you
> > > > > > > > can hibernate using " ShouldHibernate ", but I could not find
> > a "
> > > > > > > > ShouldPoweroff ". Any non distro specific ideas?
> > > > > > >
> > > > > > > > On Tue, Nov 15, 2011 at 7:56 AM, Tiago Macarios <
> > tiagomacarios @
> > > > > > > > gmail .com > wrote:
> > > > > > >
> > > > > > > > > Hi Matthew,
> > > > > > > >
> > > > > > >
> > > > > > > > > I saw your blog, i just did not had the time to test your
> > steps.
> > > > I
> > > > > > > > > am
> > > > > > > > > downloading the debian package and will try that using
> > ubuntu .
> > > > > > > >
> > > > > > > > > If I dont get it running and I have to go for fedora then...
> > > > > > > >
> > > > > > >
> > > > > > > > > Is it really necessary to use condor_shared_port? I saw here
> > (
> > > > http
> > > > > > > > > :// www . isi . edu /~ gideon /condor-ec2/ ) that you can
> > just
> > > > > > > > > specify a port range.
> > > > > > > >
> > > > > > > > > Anyway you gave me an idea, would it be possible to use
> > > > > > > > > condor_shared_port and tunnel all the communication trough
> > ssh?
> > > > how
> > > > > > > > > difficult would that be?
> > > > > > > >
> > > > > > >
> > > > > > > > > Thanks for your help
> > > > > > > >
> > > > > > >
> > > > > > > > > On Tue, Nov 15, 2011 at 3:36 AM, Matthew Farrellee < matt @
> > > > redhat
> > > > > > > > > .com > wrote:
> > > > > > > >
> > > > > > >
> > > > > > > > > > On 11/15/2011 01:27 AM, Tiago Macarios wrote:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > > Hi all,
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > > I am trying to get condor to run on amazon ec2. I am
> > trying
> > > > to
> > > > > > > > > > > use
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > ubuntu and the standard apt-get ubuntu (7.2.4), but I
> > saw at
> > > > > > > > > > > some
> > > > > > > > > > > places
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > that this version may be a bit old for that. Did any of
> > you
> > > > > > > > > > > guys
> > > > > > > > > > > made
> > > > > > > > > > > it
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > work using this configuration?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > > Would you have any suggestion of distros I could try? I
> > found
> > > > > > > > > > > this
> > > > > > > > > > > link
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > that seems to be what I wanted to do, but I am not
> > familiar
> > > > > > > > > > > with
> > > > > > > > > > > Fedora
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > and wanted to try ubuntu first:
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > http :// spinningmatt . wordpress .
> > com/2011/11/10/getting-
> > > > > > > > > > > started-condor-and-ec2-ec2- execute-node/
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > > To be honest I am new to condor, have been playing
> > around for
> > > > > > > > > > > about
> > > > > > > > > > > 2
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > weeks on a couple of virtual machines. I tried
> > installing the
> > > > > > > > > > > last
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > stable release of condor in ubuntu , but I failed is
> > there a
> > > > > > > > > > > tutorial
> > > > > > > > > > > for
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > installing it on ubuntu ?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > I saw this version here: http :// neuro . debian .net/
> > pkgs /
> > > > > > > > > > > condor.
> > > > > > > > > > > html did
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > anyone tried it out?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > > Please help me out. I am more than willing to write a
> > > > > > > > > > > documentation
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > about it if I get things running.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > > > > Tiago
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > If you want to run your whole pool in EC2 then you should
> > try
> > > > > > > > > > following,
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > http :// spinningmatt . wordpress . com/2010/07/26/getting-
> > > > > > > > > > started-installing-a-single- node-condor-pool/
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > then,
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > https :// spinningmatt . wordpress .com/2011/06/21/
> > > > > > > > > > getting-started-multiple-node- condor-pool-with-firewalls/
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > The primary difference for the instructions in EC2 is that
> > you
> > > > > > > > > > need
> > > > > > > > > > to authorize ports in the instance's security group as
> > well as
> > > > > > > > > > iptables .
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > The instructions talk about using the condor_shared_port
> > > > feature,
> > > > > > > > > > which isn't available in 7.2.4 ( IIRC ).
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > If you're itching for Ubuntu , you could try UW's debian
> > > > > > > > > > repository
> > > > > > > > > > from,
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > http :// www .cs. wisc . edu /condor/ debian /
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > Otherwise, I'd suggest just trying the Fedora AMIs ,
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > http :// fedoraproject .org/ wiki / Cloud_images
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > A gotcha[0] is resolved in the updates-testing repository.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > Best,
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > matt
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > > [0] https :// bugzilla . redhat .com/ show_bug. cgi
> > ?id=656562
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > > > --
> > > > > > > >
> > > > > > >
> > > > > > > > --
> > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Condor-users mailing list
> > > > > > > > To unsubscribe , send a message to condor-users-request@xxxxxxx .
> > > > > > > > edu with a
> > > > > > > > subject: Unsubscribe
> > > > > > > > You can also unsubscribe by visiting
> > > > > > > > https ://lists.cs. wisc . edu /mailman/ listinfo /condor-users
> > > > > > >
> > > > > > > > The archives can be found at:
> > > > > > > > https ://lists.cs. wisc . edu /archive/condor-users/
> > > > > >
> > > > > > > _______________________________________________
> > > > > > > Condor-users mailing list
> > > > > > > To unsubscribe, send a message to
> > condor-users-request@xxxxxxxxxxxxxxx
> > > > > > a
> > > > > > > subject: Unsubscribe
> > > > > > > You can also unsubscribe by visiting
> > > > > > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > > > > > >
> > > > > > > The archives can be found at:
> > > > > > > https://lists.cs.wisc.edu/archive/condor-users/
> > > > > >
> > > > > > _______________________________________________
> > > > > > Condor-users mailing list
> > > > > > To unsubscribe, send a message to
> > condor-users-request@xxxxxxxxxxxxxxx a
> > > > > > subject: Unsubscribe
> > > > > > You can also unsubscribe by visiting
> > > > > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > > > > >
> > > > > > The archives can be found at:
> > > > > > https://lists.cs.wisc.edu/archive/condor-users/
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > >
> > > > > _______________________________________________
> > > > > Condor-users mailing list
> > > > > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxxxxxx
> > > > a
> > > > > subject: Unsubscribe
> > > > > You can also unsubscribe by visiting
> > > > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > > > >
> > > > > The archives can be found at:
> > > > > https://lists.cs.wisc.edu/archive/condor-users/
> > > >
> > > > _______________________________________________
> > > > Condor-users mailing list
> > > > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxxxxxx a
> > > > subject: Unsubscribe
> > > > You can also unsubscribe by visiting
> > > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > > >
> > > > The archives can be found at:
> > > > https://lists.cs.wisc.edu/archive/condor-users/
> > > >
> > >
> > >
> > >
> > > --
> >
> > > _______________________________________________
> > > Condor-users mailing list
> > > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
> > a
> > > subject: Unsubscribe
> > > You can also unsubscribe by visiting
> > > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> > >
> > > The archives can be found at:
> > > https://lists.cs.wisc.edu/archive/condor-users/
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >

> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

Attachment: power_state.sh
Description: Bourne shell script