Re: [Condor-users] how to kill job when output dir removed ?

[Dan Bradley <dan@xxxxxxxxxxxx>]

Dr Ian C. Smith wrote:
> Unfortunately the DAGMan bug in 6.8 (mentioned
> again here recently) is a show stopper for us.

Just to be clear, there are two manifestations of this bug.  In 6.8, it 
is causing DAGMan to go on hold when it exits abnormally.  In 6.9.1, 
this bug causes the schedd to crash :(

The problem will be fixed in 6.8.4 and 6.9.2.

> You say that Condor 6.8 will continue to attempt to write to the submit host
> if the filesystem is full. Does Condor actually detect this error
> separately ? In other words - if the directory is missing (or unwriteable)
> will it put the job on hold or just keep trying ?
>   

In 6.8, Condor detects the error writing output, but it doesn't put the 
job on hold.  The job goes back to the idle state and will try to run 
again.  In the common case where the failure to write the output was 
because the initial working directory had been deleted, the job will go 
on hold when it tries to run again.  However, if you are writing your 
output to some other directory, and there are no problems fetching input 
files, then the job will run again, and possibly fail again when it 
tries to write output.

In 6.9.1, the job goes on hold when it fails to write output.

Just to make matters more confusing (sorry), standard, pvm, and local 
universes have not yet been incorporated into the new hold-on-error 
regime, so jobs in these universes currently exhibit the traditional 
behavior of going back to the idle state and trying to run again when 
they hit errors caused by missing input files/directories and/or output 
failures.

--Dan