Re: [HTCondor-users] How to use Dagman (python bindings) in condor 8.6 version.

[Htcondor user <htcondor.user@xxxxxxxxx>]

HiÂJosh Karpel,

Thanks for yourÂreply :)

I am building a library where user can submit multiple jobs by passing a text file with 'n' no of jobs.But currently i am getting different clusterids for each job.

But as per our requirement 'n' no of jobs specified in a single text file must be running in same cluster & also handle dependency jobs.

To acheive my requirement i have found Dagman. but unfortunately i am using condor 8.6.12 version and as of now i can't upgrade to latestÂversion.

Now how do i achieveÂ:Â

       Â1) Same clusterid for multiple jobs without dagman?

       Â2) How do i handle dependency between those jobs?

If I can't use submit API with cmd condor_submit_dag my.dag how can i acheive my requirement? any other alternative solution?

Thanks inÂadvance.

On Mon, Jan 27, 2020 at 7:59 PM Josh Karpel via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

Hi,

In its current form, DAGMan requires real condor_submit submit files on disk to work, so there's no way to use the Python bindings to directly provide submit descriptions for DAGMan at runtime. This is definitely an area we're trying to improve, so I'd love to hear more about your specific use case, especially if the below information isn't as helpful as you'd like...

We're working on improving integration between the bindings and DAGMan. As mentioned in a previous thread, we now haveÂhttps://htcondor.readthedocs.io/en/v8_9_5/apis/python-bindings/api/htcondor.html#htcondor.Submit.from_dagÂin the 8.9 series, which lets you do the equivalent of condor_submit_dag from Python (but the submit files the DAG references still need to exist on disk).

We also have a prototypeÂ"DAG builder" API for Python atÂhttps://github.com/htcondor/htcondor-dagsÂ(examples atÂhttps://github.com/htcondor/htcondor-dags/tree/master/examples). It requires the 8.8 bindings, but you can install those manually from pip if you're using a linux system of some kind. It lets you pass submit descriptions to it in the form of Python bindings Submit objects.

Josh Karpel

karpel@xxxxxxxx

On Sat, Jan 25, 2020 at 11:10 AM Htcondor user <htcondor.user@xxxxxxxxx> wrote:

Hi all,

I want to use dagman(Python bindings) so that user can specify dependency between jobs.

Unfortunately, i am using 8.6 version and dagman is been exposed 8.9 version. due to some reasons i can't upgrade my version.

so do i have any other alternative?

To submit a job I already created a structure as below:

sub = htcondor.Submit({
  "executable": "/bin/sleep",
  "arguments": "5m",
  "request_cpus": "1",
})

If i call dagman cmd (condor_submit_dag my.dag) how should i pass this to my above mentioned submit method?

Can anyone please help in providing the alternative solution?

ThanksÂin advance :)

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/