Re: [Condor-devel] Is it possible to change the log file for a running job?

["João Abecasis" <joao@xxxxxxxxxxxxx>]

Hi Todd,

Thanks for your comments.

Todd Tannenbaum wrote:
> João Abecasis wrote:
>> The Condor adaptor in SAGA is currently using XML Logfiles to track the
>> progress of running jobs. To this end, and as needed, a randomly named
>> (temporary) file is created for each pool and writes to the
>> file are tracked and processed in the background. When a log file is
>> no longer being used, it is removed.
>>
>> A drawback of this approach, if the SAGA application exits before the
>>  Condor job terminates, the log file will be, once again, created by
>> Condor on the next event.
>
> Some ideas/comments off the top of my head:
>
> Curious comment: what do you do if the SAGA application exits
> abnormally?  is the log file ever cleaned up?

Ha, ha! SAGA applications don't exit abnormally... :-p

Seriously, it depends on how abnormal the interruption is. Anyway,
leaving a temporary log behind in case of a fatal error doesn't sound
as bad, but accumulating them in the course of normal operation is
another matter.

> Naive idea:  instead of having SAGA delete the log file when no longer
> needed, how about it remove the job and delete the log file?

The SAGA application might be providing something like condor_submit.
Some other SAGA application could potentially pick up where the first
one left.

> Alternative naive idea:  Use Condor's schedd-wide event log instead of
> the per job event log since this log rotates.  From the Condor Manual:
>>
>> The following macros control where and what is written to the event
>> log, a file that receives job user log events, but across all users
>> and user's jobs.
>>
>> EVENT_LOG The full path and file name of the event log. There is no
>> default value for this variable, so no event log will be written, if
>> not defined.
>>
>> MAX_EVENT_LOG Controls the maximum length in bytes to which the event
>> log will be allowed to grow. The log file will grow to the specified
>> length, then be saved to a file with the suffix .old. The .old files
>> are overwritten each time the log is saved. A value of 0 specifies
>> that the file may grow without bounds. The default is 1 Mbyte.
>>
>> EVENT_LOG_USE_XML A boolean value that defaults to False. When True,
>> events are logged in XML format.
>>
>> EVENT_LOG_JOB_AD_INFORMATION_ATTRS A comma-separated list of job
>> ClassAd attributes, whose evaluated values form a new event, the
>> JobAdInformationEvent. This new event is placed in the event log in
>> addition to each logged event.
>
>
> Course, now you'd have to figure out what to do if the log rotates
> before SAGA has had a chance to read whatever events it needs...

:-) That sounds like the Wrong Way to go about it.

So, does that mean there is no easy way?

Thanks,


João