Re: [Condor-users] VMware universe

[Dave STREET <davey_street@xxxxxxxxxxx>]

To be honest I haven't yet tested how to get the start up files in to the vmware machine, but I thought there was a command in the submit file you could use? vm_cdrom_files = a.txt,b.txt,c.txt where this would create a CD image taht is loaded on to the VMware machine when it runs? 

so I could have a sumit file such as 

universe                     = vm
executable                   = MATJOBS  (this is simple a name isn't it is has no effect on the job running?)
log                          = simple.vm.log.txt
vm_type                      = vmware
vm_memory                    = 64
vmware_dir                   = C:\condor-test
vm_cdrom_files = a$(PROCESS).txt,b$(PROCESS).txt 
vmware_should_transfer_files = True
queue =20

so each run of the job uses the same vmmware image but different inputs as specified by the cdrom files? or is this not how it works?


As for the pre-staging its nice to know its in the pipe line, my idea was that the first time a client recives a VM job it coopies over the VMware images. Condor then tags this image with the queue number(unique to that specifice queue) and master server ID.

when the job has completed and the results are returned/schedule updated on the master, the client/master checks if there are still any more jobs in that queue, if so the image is kept, if not it is removed from the client. The client could then take on a new job as normal, after ever run it cheks if the vm que still has jobs and once it is empty deleted the image. 

because you may want to have other jobs run in-between VM ware jobs (some one submits a high priority task), idealy you would want 

wipe vm-ware ware image on job completion,
wipe vm-ware image if job queue changes (so if it dosn't follow on with a second job from the same queue, the image is deleted and re-downloaded if needed later on)
wipe vm-ware image if queue empty
never wipe VM-ware image (ie if it is a image you are going to come back to again and again but with possible long delays between jobs)

Just my ideas but look forward to seeing how this will work, Wish I had a bit more developmment background and not just networking as I would love to be able to get stuck in and help out on this project more. 

> From: ahoward@xxxxxxxxxx
> Date: Tue, 8 Jun 2010 16:51:52 -0400
> To: condor-users@xxxxxxxxxxx
> Subject: Re: [Condor-users] VMware universe
>
> Aaron,
> I'm not sure that it's possible with the current way the VM universe
> works to get the behavior you're wanting, although you might be able
> to create a script inside the VM that copies the ISOs down to the VM
> itself and mounts them via loopback.
>
> As for prestaging the VM image on all of the machines, this isn't
> supported by the current version of Condor, but is in development for
> one of the next releases (we've requested this functionality at
> Purdue).
>
> --
> Andrew Howard
> ahoward@xxxxxxxxxx
> Assistant Research Programmer
> Rosen Center for Advanced Computing, Purdue University
>
>
> On Mon, Jun 7, 2010 at 9:43 AM, Dave STREET <davey_street@xxxxxxxxxxx> wrote:
> > I am interested in using the VMware universe but I have a question.
> > My current VMX is about 600mb, and I am wanting to use a iso image file as
> > the input for the jobs. I have about 200 iso files and the job will run on
> > about 15 clients.
> > From what I can tell each run of the job will require either the VMX getting
> > copied from the master server, or from a shared file system. Is there any
> > way that once the client has copied the VMX over the first time, when it has
> > finished a job, it checks for more ISO files. and if there are any only
> > copies over this for its next run. If there are no iso files (ie the run has
> > finished) CONDOR then cleans up and deletes the local copy of the VMX of the
> > client?
> > I hope you can see what I am getting at, I just don't want the whole VMX
> > copied each time as that will kill that bandwith and slow down the run as it
> > will take time to copy it. The only other way I can see to do it is to copy
> > the VMX to all the clients before the run starts and use a local path in the
> > submite file.
> > Any ideas or pointers to more documentation for the VMware universe?
> > Cheers
> > Aaron
> > ________________________________
> > Get a new e-mail account with Hotmail - Free. Sign-up now.
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> >
> >
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

---

Get a new e-mail account with Hotmail - Free. Sign-up now.