Re: [HTCondor-users] How to transfer output file back into submission dir.

["Dr. Harinder Singh Bawa" <harinder.singh.bawa@xxxxxxxxx>]

Hi Walid

Is $(Process) can be used in SmallD3PD.sh? I thought it can only be used in SmallD3PD.sub.

Harinder

On Mon, Sep 15, 2014 at 7:53 AM, Walid Saad <walid.saadd@xxxxxxxxx> wrote:

Hi,

In theÂSmallD3PD.sh, try to modify the test.root file to test.root.$1

transfer_output_files = /tmp/test.root.$(Process)

Or, you can useÂtransfer_output_remaps.

Cordialement

Walid SAADÂ

LaTICE Laboratory University of Tunis,Â

ESSTT 5, Av. Taha Hussein, B.P. 56,Â

Bab Mnara, Tunis, Tunisia.Â

TÃl : (+216) 97 86 13 75Â

Email : walid.saad@xxxxxxxxxxx, walid.saad@xxxxxxxxxxxxxxxxxxxx

2014-09-15 16:45 GMT+02:00 Dr. Harinder Singh Bawa <harinder.singh.bawa@xxxxxxxxx>:

Hi Walid,

I already had Argument=$(Process) but I get output as test.root since my output from exe is test.root

Jobs = 100

getenv     = false

executable   = SmallD3PD.sh

output     = output/SmallD3PD.out.$(Process)

error     Â= error/SmallD3PD.error.$(Process)

log      Â= log/SmallD3PD.log.$(Process)

arguments = $(Process) $(Jobs)

should_transfer_files = YES

when_to_transfer_output = ON_EXIT

transfer_input_files = dijet.tgz,/tmp/x509up_u55261,inputFileListLarge

transfer_output_files = /tmp/test.root

+ProjectName = "atlas-org-fresnostate"

queue $(Jobs)

Harinder

On Mon, Sep 15, 2014 at 7:36 AM, Walid Saad <walid.saadd@xxxxxxxxx> wrote:

You can use the ÂArguments attribute:

ÂÂArgumentsÂÂ Â =Â$(Process)

Automatically, the $(Process) will be the first parameter of your executable. But you should first recompile your application code after adding this parameter.

Cordialement

Walid SAADÂ

LaTICE Laboratory University of Tunis,Â

ESSTT 5, Av. Taha Hussein, B.P. 56,Â

Bab Mnara, Tunis, Tunisia.Â

TÃl : (+216) 97 86 13 75Â

Email : walid.saad@xxxxxxxxxxx, walid.saad@xxxxxxxxxxxxxxxxxxxx

2014-09-15 16:24 GMT+02:00 Dr. Harinder Singh Bawa <harinder.singh.bawa@xxxxxxxxx>:

Hi Walid,

But my exe is producing test.root in working node. If I putÂ

transfer_output_filesÂ=Â/tmp/test.root.$(Process)

It would complain of not finding e.g test.root.1.??Â

I am trying to see how can I change the name of output dir. test.root into adding job parameter but not sure how to do?

Harinder

On Mon, Sep 15, 2014 at 3:38 AM, Walid Saad <walid.saadd@xxxxxxxxx> wrote:

Hi Harinder,

You can set for each job different file name: test.root.$(Process)

$(Process) is the number of the current submitted job.

For example if you have 10 jobs, you will obtain 10 output files: from test.root.0 to test.root.9

You should addÂ$(Process) as parameters of your applicationÂExecutableÂ

Universe    Â= xxx

Executable   Â= xxx

Error      =Â$(Process).err

Output     Â=$(Process).out

Log       = xxx.log

Arguments    =Â$(Process)

transfer_input_files = xxxx

should_transfer_files = yes

when_to_transfer_output = ON_EXIT

transfer_output_filesÂ= /tmp/test.root.$(Process)

Queue 10

Best Regards

Cordialement

Walid SAADÂ

LaTICE Laboratory University of Tunis,Â

ESSTT 5, Av. Taha Hussein, B.P. 56,Â

Bab Mnara, Tunis, Tunisia.Â

TÃl : (+216) 97 86 13 75Â

Email : walid.saad@xxxxxxxxxxx, walid.saad@xxxxxxxxxxxxxxxxxxxx

2014-09-15 2:27 GMT+02:00 Dr. Harinder Singh Bawa <harinder.singh.bawa@xxxxxxxxx>:

Hi Walid,

Yes, it can copy my test.root into /tmp/ dir on worker node and then transfer back to submission node. But If I have 10 files, all would be having name and would be overwritten. I should have seperate dir to put on atleast like

1/ 2/ 3/ 4/....10/ so that test.root should go to all dir. seperately

How to do this?

Harinder

On Sat, Sep 13, 2014 at 3:20 PM, Walid Saad <walid.saadd@xxxxxxxxx> wrote:

Hi,

May be you should put the Âpath Âof test.root file in the job description file as follows:

Universe    Â= xxx

Executable   Â= xxx

Error      = xxx.err

Output     Â=xxx.out

Log       = xxx.log

Arguments    = xxx

transfer_input_files = xxxx

should_transfer_files = yes

when_to_transfer_output = ON_EXIT

transfer_output_files = /tmp/test.root

Queue

Don't forget to copy/move the test.root file into /tmp directory.

I hope this will help you.

Cordialement

Walid SAADÂ

Laboratoire LaTICE Università de Tunis,Â

ESSTT 5, Av. Taha Hussein, B.P. 56,Â

Bab Mnara, Tunis, Tunisie.Â

TÃl : 97 86 13 75Â

Email : walid.saad@xxxxxxxxxxx, walid.saad@xxxxxxxxxxxxxxxxxxxx

2014-09-13 22:37 GMT+02:00 Dr. Harinder Singh Bawa <harinder.singh.bawa@xxxxxxxxx>:

Hello,

Could anybody suggest how I transfer an output file I got from running my executable back to my submission Âdir.Â

The thing is I am sending a package .tar with the job. The job runs as

It untar the file in working node....

go to dir. test/resonance/ i.e cd test/resonance

run the exe. and there I get output file as test.root.Â

1. How to transfer back test.root into my submission dir. ?Â

2. All my jobs gives test.root as output. I suppose I need to transfer into dir. named 1/, 2/, 3/ etc otherwise they will overwrite.

I triedÂ

transfer_output_file= test.root. But it complains of not getting test.root.

Then I thought since I ran in sub. test/resonance, May be I need to transfer my test.root into working node main dir. so I put in the end where it runs exe,Â

I put cp test.root ../../

Still it is not finding test.root.

Any further idea?

--

Dr. Harinder Singh Bawa

ÂÂÂÂÂÂÂÂÂÂ Â Â ÂÂÂ Â Â Â Â Â Â Â Â Â Â Â Â
[web][facebook][youtube][twitter]

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

Dr. Harinder Singh Bawa

ÂÂÂÂÂÂÂÂÂÂ Â Â ÂÂÂ Â Â Â Â Â Â Â Â Â Â Â Â
[web][facebook][youtube][twitter]

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

Dr. Harinder Singh Bawa

ÂÂÂÂÂÂÂÂÂÂ Â Â ÂÂÂ Â Â Â Â Â Â Â Â Â Â Â Â
[web][facebook][youtube][twitter]

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

Dr. Harinder Singh Bawa

ÂÂÂÂÂÂÂÂÂÂ Â Â ÂÂÂ Â Â Â Â Â Â Â Â Â Â Â Â
[web][facebook][youtube][twitter]

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

Dr. Harinder Singh Bawa

ÂÂÂÂÂÂÂÂÂÂ Â Â ÂÂÂ Â Â Â Â Â Â Â Â Â Â Â Â
[web][facebook][youtube][twitter]