Re: [Condor-devel] changes to condor_submit

[Peter Keller <psilord@xxxxxxxxxxx>]

If I recall, we changed the defaults so that the executable is never spooled
except if your stduniv (and there might have been a revision or two where even
that wasn't spooled). We did this because people would spool 1e5 jobs and
fill up the disk. 

However, now that we have the md5 sum on the executable fix which uses
a reference counting like solution for executables, we could change the
default behavior to be spooling the executable again? I always thought
it was a bit shakey that we got rid of that default since it causes
exactly Doug's problem and he isn't the first to have been hit by it.

-pete

On Thu, Aug 05, 2010 at 12:41:57PM -0500, Zachary Miller wrote:
> i heard on another list that the behavior of condor_submit changed between 7.4
> and 7.5, and inquired about how since i was curious.  below are the details
> from Doug Thain.
> 
> cheers,
> -zach
> 
> 
> 
> Short story: Something changed in the behavior of copy_to_spool
> between 7.4 and 7.5.
> 
> Long story:  Makeflow generates Condor jobs on the fly, and then    
> re-use the files, so as not to fill up the filesystem with lots of
> junk.  Makeflow creates a condor.submit and a condor.sh as the
> executable, and submits it with copy_to_spool enabled, and then
> repeats that process using the same file names.
> 
> It worked fine on 7.4, then on 7.5 we discovered that the executable
> condor.sh was *not* being spooled, and chaos resulted.  Our workaround
> was to submit jobs with /bin/sh as the executable and -c "commandline"
> as the arguments.
> 
> Hope that helps...
> Doug
> 
> 
> 
> _______________________________________________
> Condor-devel mailing list
> Condor-devel@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/condor-devel