Re: [condor-users] globus -> condor matching problem

[James Frey <jfrey@xxxxxxxxxxx>]

On Wed, 29 Oct 2003, Alain Roy wrote:

>
> > > If it is problem #1, then it's a bit trickier. You may need to mess
> > > with Globus to adjust the submit file that it creates. (See
> > > condor.pm.)
> >
> >OK, you've given me a good pointer where to start looking. I'll scour
> >the net for any relevant clues, though I'd appreciate any ready hints
> >that you may have!
>
> Assuming you're using Globus 2.2 or 2.4, look in
> $GLOBUS_LOCATION/lib/perl/Globus/GRAM/JobManager/condor.pm.
>
> In my copy, around line 234, it writes the submit file. It looks like this:
>
> print FH "Environment = $environment_string\n";
> print FH "Arguments = $argument_string\n";
> print FH "InitialDir = " . $description->directory() . "\n";
> print FH "Input = " . $description->stdin() . "\n";
> ...
>
> You can add your own statements to transfer files.
>
> I haven't actually done it before, but it should work. The trick is knowing
> which files to transfer. :)
>
> Certainly you should specify:
>
>       should_transfer_files = YES
>       when_to_transfer_output = ON_EXIT
>
> If you need to transfer more than the default files though, I'm not sure
> what to specify, or how to get the list of files from the jobmanager.

The jobmanager doesn't tell the scheduler script (condor.pm) what files
the client (Condor-G) requested to be staged in and out. There's also no
way for the client to specify a list of files that need to be present on
the execute machine. This isn't much of a problem for output files, as
Condor can be told transfer all output files from the job (those created
or modified after the job began running).

Input files are a little trickier. You could modify condor.pm to scan
IntialDir (specified by the directory RSL attribute) and list all files
found in "transfer_input_files". This should work well if each job has a
unique IntialDir on the gatekeeper machine. If the gatekeeper is running
Globus 2.2 or 2.4 and remote_initialdir isn't given in the Condor-G submit
file, each job will automatically get it's own InitialDir. But if a user
submits a job via globus-job-run, the default InitialDir is their home
directory, which would lead to their entire home directory being
transferred for each job.

Another option is to use the "condorsubmit" RSL attribute. This lets the
Globus GRAM client specify lines that should appear in the submit file
condor.pm generates to submit the job to Condor. You can use it through
Condor-G by putting something like the following into your Condor-G submit
file:

GlobusRSL = (condorsubmit=(should_transfer_files IF_NEEDED)(when_to_transfer_output ON_EXIT)(transfer_input_files foo,bar))

Make sure you don't have any spaces after the commas in your file list
(otherwise you need to put single-quotes around the entire list).

I haven't tried either of these ideas personally, but they should work.

-- Jaime
Condor Support Information:
http://www.cs.wisc.edu/condor/condor-support/
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>