HTCondor Project List Archives



[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-devel] Transferring files from/to gsiftp servers with gt4 universe jobs



Hi Everyone.

I'm working on using Condor as a Globus 4 MetaScheduler, and I've decided that the only practical way to be able to use Condor to do this is to be able to transfer input files from Globus gridftp sites, and transfer the output files to gridftp sites.

What I want to be able to do is have gsiftp urls in my job classad, and have the globus resource that condor submits the job to stage them itself. So have something like the following:

...
Transfer_input_files = gsiftp://globus-host:2811/home/user/somefile, gsiftp://globus-host:2811/home/user/someotherfile
Transfer_output_files = gsiftp://globus-host:2811/home/user/someoutput

This seems to be pretty easy to do with Condor, since it just creates a GT4 xml job description, then submits it with GRAM to a GT4 resource. I modified condor_gridmanager/gt4job.C to check if files in the stage_in_list starts with the gsiftp scheme name, and if so, insert the url directly into the <transfer> element of the fileStageIn/ Out.

I've included a patch for how I did this, but I really can't find any information on how to get this into Condor, or whether this is even something you would like to include. The patch was against the condor 7.1.0 source.

Let me know what you think.

--patrick

Attachment: gt4job.patch
Description: Binary data