Re: [HTCondor-users] job queue log file rotate

[Cheng-Jui Yu <cheng-jui.yu@xxxxxxxxxxxxxx>]

Thanks for the reply. This data is used to keep us updated on what's happening in each job queue.ÂÂ

We may try using "job router" or "global event log" if needed, but would like to see if it can be solved byÂ

not skipping all the lines until the first 105 after a log rotation. Currently, the process already ignores lines withÂ103 code and -1 procId.ÂÂ

Here is one example: "103 0571636.-1 NiceUser false".Â

Would we get any duplicate events or miss any events If we just ignore theÂ103 and -1ÂprocId lines and don't skip to 105?

And please point me to documentations on the file format and definitions. Thanks.

On Tue, Apr 14, 2020 at 3:45 PM Gregory Thain <gthain@xxxxxxxxxxx> wrote:

On 4/13/20 4:55 PM, Cheng-Jui Yu wrote:
> Hi,
>
> We have a process that reads job-queue-log-file. Currently It must
> skip lines at the beginning of the new log file after a log-rotate to
> avoid duplicate transaction lines carried over from the previous
> file. The process skips all the lines until it sees a 105. We are
> looking for a better way to do this. Is there a way to tell schedd not
> to carry overÂtransaction lines after a rotate? Or is there a way to
> definitively know what lines or how many lines to skip. Thanks.


HI:

There's now way to tell the schedd not write write these lines before
the 105 -- that's all the state of the schedd that it needs in case it
restarts and needs to recover. I don't know what you need this data
for, but the condor_job_router tails and ingests the job_queue.log,
perhaps you could use the job router? Or, the global event log can
write some job events to a separate file that is rotated, but not
compressed like the job_queue.log -- would this be a useful source of data?


-greg

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/