Re: [HTCondor-users] alternative scratch directory?

[RAUL H C LOPES <rlopes@xxxxxxx>]

Hi Todd,

condor_tail works fine, but it has the limitations of stdout. When I asked for an alternative scratch dir, it was because of my
users might need to inspect big binary files  while they are being generated. Big here means many MB.
Also there is the requirement that the users would like to inspect their big  files after the job runs, even the job is killed by condor_rm or whatever.
And they might want to inspect them while the job is running using tools that may not be available in the worker nodes.

BTW, the nodes where these jobs run already mount a shared disk space that users can see. If the users can see their files while the job is running, they
can use their tools, even Windows specific ones.

I have tried with 'remote_initialdir'. However, the job is still running in the node scratch dir. In my tests, I've seen
 - if job is writing one file in addition to stdout, stdout was going in the node scratch dir and the additional file was going into my declared remote_initialdir;
 - if the job was writing more than one output file, one file was going in my remote dir, and the other seemd lost.


I used something like

executable              = sleep0.sh
remote_initialdir       = /users/myhome/remotedir.test/scratch
log                     = job.log
output                  = outfile0.txt
error                   = errors0.txt
should_transfer_files   = IF_NEEDED
#when_to_transfer_output = ON_EXIT_OR_EVICT
transfer_input_files    = sleep0.sh
#transfer_output_files  = s.out, s1.out
queue

I tried a few variations on the commented lines, when came to no conclusion.
 
Sorry, if confusing.


Thanks, raul

On 05/09/16 13:44, Todd Tannenbaum wrote:

On Sep 5, 2016, at 2:23 AM, Raul Lopes <rlopes@xxxxxxx> wrote:

Hi Todd,

remote_initaldir should be what I need. I hadn't noticed it. I am
assuming that it will change to the specified directory and use
it in the same way as the scratch dir.

No, it will only change the current working directory. It will not work the same way as the scratched directory. 

Some of my users would
like to follow the output of their MPI and abaqus jobs in real time
without logging into worker nodes.

Ah. Perhaps the condor_tail command line tool will serve your purpose? See "man condor_tail". 

Usually when jobs opt not to use the scratch directory, it is because they are using some shared file system directory...

Hope the above helps
Todd

I'll test  it.

Thanks, raul

Todd Tannenbaum wrote:

On Sep 4, 2016, at 11:12 AM, RAUL H C LOPES <rlopes@xxxxxxx> wrote:

Hi,

I wonder if it is possible to specify an alternative scratch directory for certain jobs.  I thought, for example,  that I could use +PreCmd to change to a directory different than the default scratch dir, before the job started. However,

I would like to be able to use this for the parallel universe.


Thanks, raul

You can certainly specify the initial working directory for the job to be something other than the scratch directory via the submit file macro
  remote_initialdir = /some/path

See the man condor_submit. 

Or did you have something else in mind ?

Regards
Todd

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/