Re: [Condor-users] understanding condor history file

[Ian Chesal <ichesal@xxxxxxxxxxxxxxxxxx>]

On Fri, Oct 29, 2010 at 6:18 AM, Santanu Das <santanu@xxxxxxxxxxxxxxxxx> wrote:

Thanks Ian. I do understand that batch system log files should be read using the batch system command but I think there some problem with that at the moment.
Does, history file logs every single jobs those were submitted?

No. It logs every job that completes. So when your job finds its way to a terminal state it will be logged to the history file.

 

We have a separate files which logs the jobID (along with some other info) as soon as the job is submitted to the cluster. There is a huge difference in the number of jobs in history file and our own log file. Any idea what am I missing?

You need to compare jobs that have finished only. Jobs still in the queue won't show up in the history file (unless leave_in_queue=true when you submit).

Also note that the history file rotates. It's not ever increasing. So it may have rolled over and you may have lost some history. If you want to save all history for all time consider using Quill or setting the history file to save several backups and then have an external process copy those backups off to some other place where Condor won't delete the oldest history file.

- Ian