Re: [HTCondor-users] Transferring files while a job is still running

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 6/2/2017 9:26 AM, Greg Thain wrote:
> On 06/02/2017 09:18 AM, Koschmieder, Lukas wrote:
> > Hi,
> >
> >
> > You can tell Condor to automatically transfer all files when a job is 
> > finished. Does Condor provide a feature to access files *while a job 
> > is still running*? Or are Condor administrators supposed to use an 
> > external solution such as NFS? Are you aware of any alternative solutions?
>
> The condor_chirp program can be run from the job, and can transfer files 
> from the job's sandbox back to the submit machine (or vice versa).
>
> The condor_tail command can be run from the execute machine, and return 
> parts of the job's standard output file, or other output file while the 
> job is running.
>
> -greg

In addition to the mechanisms Greg mentions above, you can also tell 
HTCondor to stream a job's stdout and/or stderr files back to the submit 
machine in real-time while the job is running.  See entries 
"stream_output" and "stream_error" in the condor_submit man page -

 http://htcondor.org/manual/v8.7/condor_submit.html

Hope the above helps
Todd