Re: [Condor-users] Is setting "POOL_HISTORY_DIR = $(HISTORY)" bad for condor_history?

[Alain Roy <roy@xxxxxxxxxxx>]

On Apr 3, 2012, at 8:25 AM, Rob wrote:
> My condor_history command is producing lots of error messages:
> 
> 
> $ condor_version
> $CondorVersion: 7.7.5 Mar 07 2012 $
> $CondorPlatform: X86_64-Fedora_16 $
> $ condor_history
>  ID      OWNER            SUBMITTED     RUN_TIME ST   COMPLETED CMD            
> *** Warning: Bad history file; skipping malformed ad(s)
> *** Warning: Bad history file; skipping malformed ad(s)
> *** Warning: Bad history file; skipping malformed ad(s)
> [...zip....a zillion more of these lines!!!!....]
> 
> I thought I let the view_client module share the history folder by setting:
> POOL_HISTORY_DIR = $(HISTORY)
> 
> 
> Is that the reason for condor_history going bezerk?

Yes. The POOL_HISTORY_DIR should be separate from the HISTORY file. 

The HISTORY file is a single file (rotated periodically) that keeps track of all completed jobs. The default in our configuration is to put it in the spool directory, i.e.:

  HISTORY = $(SPOOL)/HISTORY

The PER_JOB_HISTORY_DIR is a directory (not a file) where a single file per job is created. It's an alternative to the HISTORY file. Unlike the HISTORY file which is rotated, the PER_JOB_HISTORY_DIR is not cleaned up because Condor assumes that some external process will remove and process those files. If you process those files, they will accumulate until you run out of disk space.

Most people don't need the PER_JOB_HISTORY_DIR. It was created for use by an external accounting system and is heavily used in the Open Science Grid by one such accounting system. I haven't heard of it being used elsewhere, though I suppose it could be. 

-alain
------------------------------
Alain Roy
Condor Project
roy@xxxxxxxxxxx
http://www.cs.wisc.edu/condor