Re: [Condor-users] Different arguments file for each job

[Alain Roy <roy@xxxxxxxxxxx>]

> I just finish installing condor and I'm trying to figure out a way to have 
> submit a job with different file as argument for each job.
> I have a directory with 100s of files, for each file, I run the following 
> program (simplified call):
> blast -i <one_file>

Your submit file is correct and it's the only way to do it--Condor will not 
look through your directory at each file.

Are you on a Unix system? If so, all you need to do is write simple script 
that will generate a submit file for you. I've written such a script and 
you can try it out. It will generate one big submit file for each .dat file 
in a directory. It looks like this:

------------------------------
#!/bin/sh

echo Creating submit file...
cat > submit_all_jobs <<EOF
# Here we have the information shared between all jobs
executable              = blastall
Universe                = vanilla
Error                   = err.\$(Process)
Output                  = out.\$(Process)
Log                     = blast.log
should_transfer_files   = YES
when_to_transfer_output = ON_EXIT
Requirements            = Memory > 200

# Here is each job
EOF

for dat in `ls *.dat`
do
cat >> submit_all_jobs <<EOF
arguments               = -i ${date}
transfer_input_files    = ${dat}
queue 1

EOF
done

echo Done. Submit file is  called submit_all_jobs.
------------------------------

Remove the lines with hyphens before placing this into a file in your 
directory with all of your .dat files. Call the file something like 
make-submit. Then do "chmod +x make-submit". Then just run make-submit, and 
you will have a submit file that will submit a job for each dat file. It 
will generate a cluster of jobs. That is, each job will have the same 
cluster number and a different process number. For example, your jobs may 
all be in cluster 10, so you'll see your jobs listed as 10.0, 10.1, 10.2, 
and so on.

I hope this helps.

-alain