Re: [HTCondor-users] Condor execute directory on Windows keeps pilling up

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 9/29/2017 9:31 AM, Dan, Bowen wrote:
> Hi,
> 
> We have been using condor to submit jobs to windows to do some data 
> processing.  However, the scratch directories on dir_XXXX under the 
> condor execute directory keeps piling up.   We thought that condor will 
> clean these up as these jobs finish successfully 
> (condor/logs/StarterLog.slot1 suggests that the jobs finished 
> succcessfully).  Note, the condor configs are default other than specify 
> the directory and num_slots open on the machine.
> 
> Appreciate your help!
> 

Hi,

Thanks for reporting the above.  To help us fix it, please answer the following:
+ Which version of Windows (Win 7, 10, Server 2016, etc) are you using? 
+ And which version of HTCondor?  
+ Are the leaked directories cleaned up when the HTCondor service is restarted?
+ Do you get a leaked directory every time a job completes, or only occasionally?

I happen to have a personal htcondor v8.7.1 installation on my Windows 7 laptop, and I reproduced the problem above.  The tell-tale sign of the problem can be found in the StartLog file:

  10/03/17 10:11:45 Starter pid 7376 exited with status 0
  10/03/17 10:11:45 rmdirAttempt using command: C:\condor\bin\condor_rmdir.exe /s  /c "C:\condor\execute\dir_7376"
  10/03/17 10:11:45 rmdirAttempt using command: C:\condor\bin\condor_rmdir.exe /s  /c "C:\condor\execute\dir_7376"
  10/03/17 10:11:45 ERROR: C:\condor\execute\dir_7376 still exists after trying to add Full control to ACLs for PRIV_UNKNOWN

and I used this simple submit file to trigger the problem:

 executable = c:\utils\sleep.exe
 should_transfer_files = YES
 transfer_input_files = c:\home\tannenba\test_file.txt
 arguments = 10
 queue

Unfortunately, I could only reproduce it once. :(  I tried several more times, and the dir_xxx directory
went away as expected.  Apparently some sort of race condition.


Thanks
Todd

> Best,
> Bowen Dan
> 
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> 


-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685