Re: [Condor-users] condor and co-linux

[Craig Struble <craig.struble@xxxxxxxxxxxxx>]

Hi Vladimir,

On Dec 16, 2009, at 9:51 AM, Vladimir Konrad wrote:
> > With VirtualBox, we're trying to integrate with the Condor virtual
> > machine universe, which should provide some advantages for  
> > controlling
> > the process of starting and stopping on demand. In practice though,
> > having VirtualBox running a Linux image all the time hasn't caused
> > problems, even in very busy student labs.
>
> As I understand it, a job would still get evicted from condor under
> virtual-box, when windows started to be used by a user again (so all
> of the cpu would be available to user when needed).
>
> How does this work with memory usage? Under virtual-box + linux  
> combination,
> does it balloon or it is always pre-allocated? (my experience is -  
> pre-allocated,
> but it could have changed recently)

It's pre-allocated, but from reading the VirtualBox forums, the plans  
are to support memory ballooning at some point. Our philosophy has  
been to let VirtualBox be responsible for "fixing" the memory issue,  
so we haven't put much effort into working around it. Since VirtualBox  
is open source, we could contribute an implementation when it becomes  
a high priority. It just hasn't for us in our environment.

> Main reason for all this is, that hopefully some time 1st half of  
> next year,
> we will be doing pilot of condor in one of the better labs, and I want
> to make it's effect invisible to the students using that lab.  
> Sometimes
> they have large memory requirements for their research, that is the  
> main
> reason why I cannot have pre-allocated big chunk of ram to a vm all  
> the time.

That makes a lot of sense. We're not quite ready to implement the on- 
demand feature, but your input is making the priority go up. :-)

> > P.S. coLinux should be detecting user presence. We were using this
> > functionality before switching to VirtualBox (and it is implemented  
> > in
> > our current releases, but is getting increasingly difficult to
> > implement with each new version of Windows).
>
> Oops, maybe this escaped my attention, I only noticed this when I had
> a co-linux console open and using it - so co-linux would detect  
> presence
> of host windows user, even if co-linux runs as service (i.e. the user
> is using windows, but co-linux "knows")?

Right. The University of Oklahoma and University of Nebraska--Lincoln  
work to use Condor within coLinux included services that communicated  
the host (Windows) usage to the coLinux environment. A Hawkeye job was  
run to monitor the file containing this information and update the  
idle time of the Windows host. If the Windows host was in use, Condor  
would evict jobs from the coLinux environment.

We use a similar approach in POVB, since we based our implementation  
on coLinux. Services monitor Windows activity and write the data into  
a VirtualBox shared folder. The guest Linux environment picks this up  
from the shared folder and evicts jobs if there is activity or if the  
Windows load gets high (usually during Windows updates).

One approach that might not be too difficult to implement in the  
interim is to have our services launch POVB when the Windows side is  
idle (since we're monitoring that anyway and we already launch POVB  
from a service), and to have POVB shut down when someone comes to use  
the Windows machine. I'll see what we can do.

    Craig

P.S. Let's take this offline for now.

> Kind regards,
>
> Vladimir Konrad
>
> Please access the attached hyperlink for an important electronic  
> communications disclaimer: http://www.lse.ac.uk/collections/planningAndCorporatePolicy/legalandComplianceTeam/legal/disclaimer.htm
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx  
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

--
Craig A. Struble, Ph.D. | 369 Cudahy Hall  | Marquette University
Associate Professor of Computer Science    | (414)288-3783
Director, Master of Bioinformatics Program | (414)288-5472 (fax)
http://www.mscs.mu.edu/~cstruble | craig.struble@xxxxxxxxxxxxx