I have a dag that consists of 2 general kinds of jobs.
1) Get data
2) Insert data into database.
I'd like to be able to limit the number of "get data" jobs to 20 and the
number of "insert data into database" jobs two 5. However, I do not want
to make artificial parent-child relationships in the dag. When we create a
big dag we can have thousands of each kind of job. the --maxjobs switch to
condor_submit_dag doesn't do what we need. Anyone else have a situation
like this? How did you handle it?
Michael J. Remijan
Research Programmer
http://www.ncsa.uiuc.edu
(217) 244-7069
remijan@xxxxxxxxxxxxx
mikeremijan (AIM)
mjremijan (Yahoo)
|