Re: [HTCondor-users] specify scratch directory on computation machine

[Jiande Wang <jiande.wang@xxxxxxxx>]

Zach:
  My purpose is take advantage of fast disk on computer node, but they 
are not on shared filesystem, nor they are the path as configured as 
"condor_scratch_dir"
  I made a simple test code for testing, following your suggestion, I 
added "remote_initialdir", so my condor script looks like

Universe = vanilla
Executable = 1.exe
should_transfer_files = YES
when_to_transfer_output = on_exit
transfer_executable = True
Requirements = (Machine == "xxxxxxx")
remote_initialdir = /home/zzz       <--- I created this directory on 
computation machine in advance
transfer_output_files = /home/zzz/
Queue

When this job is submitted, I can see output files are generated on 
remote_initialdir, and they are transferred back to the machine and 
directory where the job was submitted. The problem I am having is

(1) the output files generated on remote_initialdir were still there 
after job was done, they were not deleted automatically.
(2) If I put some other files on the remote_initialdir before I submit 
the job, all the old and newly generated files will be transferred back. 
I only need newly generated files related to this job. This job is just 
a test case, my real job will involve hundreds of output file, so I can 
only specify directory instead of list each individual output file names.

Thanks

Jiande






On 6/4/2014 7:50 AM, Zachary Miller wrote:
> On Tue, Jun 03, 2014 at 10:54:04PM -0400, Jiande Wang wrote:
> > Then I further add "environment = <CONDOR_SCRATCH_DIR>=</tmp/zzz/>", I
> > still get output file. However, when I login to computation node during
> > the job running time, check file and disk usage on "/tmp/zzz", I can't
> > see anything. So how do I confirm that the directory/disk I specified
> > here is really being used by condor as scratch? Also how do I check what
> > is the default scratch directory?
> The user does not get to control the scratch dir through the submit file.
> Rather, the admin of the execute machine sets it up using the EXECUTE macro
> in the condor_config file, and Condor then sets the _CONDOR_SCRATCH_DIR in
> the environment of the job.
>
> I'm not sure exactly what you are trying to do.  You might also be interested
> in setting "Initial_Dir" in the submit file to control the current working
> directory when the job starts.  More info can be found here:
>    http://research.cs.wisc.edu/htcondor/manual/current/condor_submit.html#80219
>
>
> Cheers,
> -zach
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/