Re: [HTCondor-devel] [HTCondor-users] Condor Jobs Logging for recurring jobs


Date: Mon, 21 Mar 2016 13:33:04 -0500
From: Todd Tannenbaum <tannenba@xxxxxxxxxxx>
Subject: Re: [HTCondor-devel] [HTCondor-users] Condor Jobs Logging for recurring jobs
On 3/14/2016 3:57 AM, Vamshikrishna Akundi wrote:
Hi,

I have to submit a job which runs every 1 hour.

As of now, job is configured in such a way that the output of the job is
written to a file which is mentioned in OUT.

But, the logs in the file are overwritten whenever job runs.

So I am looking for an alternative solution which covers either of
following points:

A.*Output file to append to the Main OUT file. (OR)*

B.*Way to create dynamic output files, On basis of time or number.*


Hi Vamshi,

One way you can achieve the above is via the $$([ classad-expression ]) macro in your submit file. See the "Macros" section of the condor_submit man page at
  http://htcondor.org/manual/v8.4/condor_submit.html
for details.

So, for example, if you have a job run multiple times here is how to include a number that will increment with each run by using the "NumJobStarts" attribute in the job classad so that you will get foo.1.out, foo.2.out, etc.

  executable = foo.exe
  # Run every 15 minutes
  cron_minute = 0,15,30,45
  on_exit_remove = false
  # Have stdout file include numjobstarts value
  skip_filechecks = true
  output = foo.$$([ NumJobStarts ]).out
  queue

Similarly, you could use classad functions like formattime() to put the time into the file name:

  executable = foo.exe
  # Run every 15 minutes
  cron_minute = 0,15,30,45
  on_exit_remove = false
  # Have stdout file include numjobstarts value
  skip_filechecks = true
  output = foo.$$([formattime(time(),\"%H_%M\")]).out
  queue

Docs for the classad functions appear here: http://is.gd/JjH2h2

Hope the above helps
Todd

--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257

[← Prev in Thread] Current Thread [Next in Thread→]
  • Re: [HTCondor-devel] [HTCondor-users] Condor Jobs Logging for recurring jobs, Todd Tannenbaum <=