Re: [HTCondor-users] Newbie Question

[Greg Thain <gthain@xxxxxxxxxxx>]

On 02/24/2017 08:55 AM, Justin Fisher wrote:

I have a bunch of bash shell scripts that convert one graphics format to another. Is there a way to run them in Condor 8.6? In a shell I'd do:

source /path/to/script/script.sh

I created a file with all the scripts in a list and my submit file look like this

transfer_input_files = $(filename)

Arguments = $(filename)

Queue filename from /path/to/file/fconvert.in

I'm not sure how to handle the sourcing, though. Source isn't a program, rather it's part of Bash, so I guessed the input file should look like

source /path/to/script/script1.sh

source /path/to/script/script2.sh

source /path/to/script/script3.sh

and the executable should be set to  /bin/bash

You'll probably want to set

executable = /path/to/script/script1.sh

so that the shell script is the command.  If you need to run more than one shell script in sequence, perhaps you need to wrap them up into a wrapper shell script. 

One of the HTCondor developers gave a great talk at HTCondor week last year about using condor_submit trickery to manage large number of file conversions -- perhaps looking at those slides will be helpful:

http://research.cs.wisc.edu/htcondor//HTCondorWeek2016/presentations/TueKnoeller_AdvancedSubmit.pdf

-greg