Re: [HTCondor-users] condor and AWS

Mailing List Archives Authenticated access	UW Madison Computer Sciences Department Computer Systems Lab

Yes, Fermilab has done this as part of the HEPCloud project. See Burt Holzman's talk at last year's Condor Week for details. http://research.cs.wisc.edu/htcondor//HTCondorWeek2016/presentations/ThuHolzman_HepCloud.pdf There are others who have done it as well. We have the machines call back to the pool at Fermilab. There are other people who have set up a separate pool and flock to it. See my talk at Condor Week this year for details on how we did the same in Google Compute Engine.

Steve

Hi,

Has anyone implemented a condor pool, with bursting into AWS to eliminate peaks of jobs?

So the idea is to spin up machines in AWS when the job Q internally gets above a certain amount.. till the Q drops below some threshold.

Would it make sense to setup a separate condor pool in the cloud and then flock to it? Or just spin up AWS instances and have them join the existing pool whenever you determine?

Thanx

-tv

Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorized and regulated by the Financial Conduct Authority.

Mailing List Archives

Authenticated access

Re: [HTCondor-users] condor and AWS