Re: [Condor-users] Running multiple batch files under one description file or running multiple description files under the same condor_submit command

["Alas, Alex [FEDI]" <aalas@xxxxxxxxxxxxx>]

Matt,
Thank you very much for your answer... That was exactly what I was
looking for.
Alex

-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx
[mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matthew Farrellee
Sent: Saturday, January 17, 2009 12:44 PM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Running multiple batch files under one
description file or running multiple description files under the same
condor_submit command

condor_submit is well suited for this. you can have multiple queue 
commands in a single submit file...

universe = vanilla
executable = /bin/true
queue 1
executable = /bin/false
queue 2

...will give you two false jobs and 1 true. you can expand this too...

(below, purely illustrative)

universe = vanilla
executable = /bin/true
run_as_user = true
queue 1
run_as_user = false
queue 1

...will give you 1 job run as the user and one not.

hope that gives you a taste for what you can do.

best,


matt

p.s. if you want to do case #2 you can just concatenate all the files...

Alas, Alex [FEDI] wrote:
> Hello to all Condor users (Gurus, Experts, not so experts and the rest
> of us) again,
> 
> I have a very important question, I have a condor pool of 23 Windows
> nodes (CPU's). The way we are running batching jobs today is creating
a
> batch file that will encompass 500 process into one file. The way I
> understand condor when a submitting machine sends a job to the Central
> Manager Systems, it will allocate that batch file (submitting job) to
> only one CPU or the first CPU that matches its requirements, then the
> second, the third and so on until it matches the amount of time you
> stipulated to run the job in your queue. For a better understanding of
> my question I am adding a sample of the submitting job and a sample of
> the batch file we are running. As I said before some time we run some
> jobs with up to 500 process on one batch file. To clarify my question
I
> will create two different scenarios, case 1 and 2; 
> 
> Case 1: I need to know if there is a way I can run multiple batch
files
> under the same description file for example adding the another
> testjob1.bat right after \\Sharename\Subdir1\Subdir2\testjob.bat
> <file:///\\Sharename\Subdir1\Subdir2\testjob.bat> ;
> \\Sharename\Subdir1\Subdir2\testjob1.bat
> <file:///\\Sharename\Subdir1\Subdir2\testjob1.bat> , in some cases up
to
> 500 different batch files, and can those multiple and different files
> being executed on different CPU's. 
> 
>  
> 
>
Xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> x
> 
> Case 1:
> 
> Description File sample:
> 
> UNIVERSE = VANILLA
> 
> REQUIREMENTS = (Arch == "INTEL" && OpSys == "WINNT52" || Arch ==
"INTEL"
> && OpSys == "WINNT51")
> 
> NOTIFY_USER = user1@xxxxxxxxxx,user2@xxxxxxxxxx
> 
> INITIALDIR = c:\condor\execute
> 
> SHOULD_TRANSFER_FILES = YES
> 
> WHEN_TO_TRANSFER_OUTPUT = ON_EXIT
> 
> TRANSFER_INPUT_FILES =
>
\\Sharename\Subdir1\BatchProcessingPackage\Subdir3\Subdir4\lasEnvelop.ex
> e
> 
> RUN_AS_OWNER = TRUE
> 
> EXECUTABLE =  \\Sharename\Subdir1\Subdir2\testjob.bat
> <file:///\\Sharename\Subdir1\Subdir2\testjob.bat> ,  ||
> 
>   \\Sharename\Subdir1\Subdir2\testjob1.bat
> <file:///\\Sharename\Subdir1\Subdir2\testjob1.bat>  ||
> 
>   \\Sharename\Subdir1\Subdir2\testjob2.bat
> <file:///\\Sharename\Subdir1\Subdir2\testjob2.bat>  ||
> 
>   \\Sharename\Subdir1\Subdir2\testjob3.bat
> <file:///\\Sharename\Subdir1\Subdir2\testjob3.bat> 
> 
> OUTPUT = testjob.out.$(Process)
> 
> ERROR = testjob.err.$(Process)
> 
> LOG = testjob.log
> 
> QUEUE 1
> 
>
Xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> x
> 
> Case 2: 
> 
> If that is not possible is there a way to submit multiple jobs under
one
> condor_submit command.
> 
> Condor_submit Test_job.txt, Test_job1.txt, Test_job2.txt... up to n
> jobs...
> 
>  
> 
>
Xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> x
> 
> Samples:
> 
> Description File sample; Test_job.txt:
> 
> UNIVERSE = VANILLA
> 
> REQUIREMENTS = (Arch == "INTEL" && OpSys == "WINNT52" || Arch ==
"INTEL"
> && OpSys == "WINNT51")
> 
> NOTIFY_USER = user1@xxxxxxxxxx,user2@xxxxxxxxxx
> 
> INITIALDIR = c:\condor\execute
> 
> SHOULD_TRANSFER_FILES = YES
> 
> WHEN_TO_TRANSFER_OUTPUT = ON_EXIT
> 
> TRANSFER_INPUT_FILES =
>
\\Sharename\Subdir1\BatchProcessingPackage\Subdir3\Subdir4\lasEnvelop.ex
> e
> 
> RUN_AS_OWNER = TRUE
> 
> EXECUTABLE =  \\Sharename\Subdir1\Subdir2\testjob.bat
> 
> OUTPUT = testjob.out.$(Process)
> 
> ERROR = testjob.err.$(Process)
> 
> LOG = testjob.log
> 
> QUEUE 1
> 
>  
> 
> Testjob.bat:
> 
> echo off
> 
>
\\Sharename\Subdir1\BatchProcessingPackage_beta\Subdir3\Subdir4\Envelop.
> exe 3 -1 2 \\Sharename\Subdir1\Subdir2\Envelop.txt
> \\Sharename\Subdir1\Subdir2 a.LAS
> <file:///\\Sharename\Subdir1\Subdir2%20a.LAS>  (Process 1)
> 
>
\\Sharename\Subdir1\BatchProcessingPackage_beta\Subdir3\Subdir4\Envelop.
> exe 3 -1 2  \\Sharename\Subdir1\Subdir2\Envelop.txt
> \\Sharename\Subdir1\Subdir2 b.LAS
> <file:///\\Sharename\Subdir1\Subdir2%20b.LAS>  (Process 2)
> 
>  
> 
>  
> 
> Respectfully,
> 
> Alex Alas
> 
> Systems Administrator
> Fugro EarthData Inc.
> 
> Tel. 301-948-8550 x219 Fax 301-963-2064 E-mail: aalas@xxxxxxxxxxxxx
> <mailto:aalas@xxxxxxxxxxxxx>  
> 
> 7320 Executive Way, Frederick, MD  21704
> 
> Website: http://www.fugroearthdata.com <http://www.earthdata.com/> 
> 
>  
> 
> 
> 
> 
>
------------------------------------------------------------------------
> 
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at: 
> https://lists.cs.wisc.edu/archive/condor-users/

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with
a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at: 
https://lists.cs.wisc.edu/archive/condor-users/