Re: [HTCondor-users] DAGMan write jobs log into *.nodes.log even if log file specified for all jobs

[Tom Downes <tpdownes@xxxxxxxxxxxxxxxxxxxx>]

Hi Kent:

I expressed some concern a while back about these logs. I can't remember the details, but the underlying issue was that was large DAGs can generate tens of thousands of separate open/close operations on these logs over the course of a day. The way things ran here, the default location of these logs ended up on NFS.

The way I read the manual at the time, I wasn't expecting the log files to show up at all given the Log-ging settings in the submit file. I am busy with other matters right now, but you might take a 2nd look at Condor 8.x era e-mails from me (perhaps on the LIGO list).

--

Tom Downes

Associate Scientist and Data Center Manager

Center for Gravitation, Cosmology and Astrophysics

University of Wisconsin-Milwaukee

414.229.2678

On Tue, Nov 12, 2013 at 8:56 AM, R. Kent Wenger <wenger@xxxxxxxxxxx> wrote:

On Tue, 12 Nov 2013, éææ wrote:

DAGMan write jobs log into *.nodes.log file even if log file specified for
all jobs in their submit file.

Their content are similar, except that ImageUpdate event is missing in
*.nodes.log file.

Can I avoid this? I am using 8.0.1.

First of all, is there any particular reason you don't want to have the two log files?

There are two ways you can avoid having duplicate user logs for your jobs:

1) Don't specify log files in your submit files.
2) Set DAGMAN_ALWAYS_USE_NODE_LOG to false in a per-DAG configuration
 Âfile (or some other configuration setting).

There are issues with both of these alternatives -- if you do #1, you won't get log files if you ever run your jobs outside of a DAG. ÂIf you do #2, you'll potentially increase DAGMan's file descriptor usage, and decrease performance a bit. Â(Also, the #2 option may go away in the future.)

In general, unless you have some reason that the duplicate logs are really a problem, we recommend leaving things the way they are.

Kent Wenger
CHTC Team
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/