[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] data scheduling in DAG jobs



On Tue, 20 Jan 2015, John Bent wrote:

Can anyone kindly point me to documentation about how to submit DAG jobs
that contain data scheduling?  For example, the input data set is in one
location and I need it prestaged into a different location before running my
jobs and I need to poststage the output data.  

Especially if by "locations" you mean "grid sites", you might want to take a look at Pegasus (http://pegasus.isi.edu/). Pegasus is a layer on top of DAGMan that can automatically add jobs to move your data to and from the execute sites, register it in a catalog, etc.

Kent Wenger
CHTC Team