Re: [HTCondor-users] sandbox too small

["Beyer, Christoph" <christoph.beyer@xxxxxxx>]

Ah,

OK so I see - this is actually from the  good news department and the job bypassed the transfer queue :)

Thanks for letting me know - one red hering less in my logs ;)

best

christoph

--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx

---

Von: "Cole Bollig via HTCondor-users" <htcondor-users@xxxxxxxxxxx>
An: "HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx>
CC: "Cole Bollig" <cabollig@xxxxxxxx>
Gesendet: Mittwoch, 13. November 2024 20:18:24
Betreff: Re: [HTCondor-users] sandbox too small

Hi Christoph,

No, as the following line states file transfer was successful. Traditionally when the shadow and starter are ready to begin transferring files (input or output) the shadow enters the schedd's transfer queue. This queue is to do a user fair share ordering of file transfers for jobs, so the AP doesn't get bogged down due to many active file transfers at once. The drawback to this queueing of transfers is jobs with very little/minimal bytes of data to transfer may get stuck waiting in the queue to transfer, so we added functionality so that jobs with a sandbox to transfer under a specified size bypass the transfer queue and just begin transferring data. This default value is 100MB. This can be overridden by the currently undocumented configuration knob BYTES_REQUIRED_TO_QUEUE_FOR_TRANSFER.

Hope this helps,

Cole Bollig

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Beyer, Christoph <christoph.beyer@xxxxxxx>
Sent: Wednesday, November 13, 2024 12:54 PM
To: htcondor-users <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] sandbox too small

 

Hi,

I see a lot of these in the logs lately:


11/13/24 19:43:57 Initializing a VANILLA shadow for job 6276104.24
11/13/24 19:43:57 (6276104.24) (763436): LIMIT_DIRECTORY_ACCESS = <unset>
11/13/24 19:43:57 (6276104.24) (763436): Request to run on slot2_1@xxxxxxxxxxxxxxxxx <131.169.165.129:37923?addrs=131.169.165.129-37923+[2001-638-700-10a0--1-581]-37923&alias=batch1451.desy.de> was ACCEPTED
11/13/24 19:43:57 (6276104.24) (763436): Not entering transfer queue because sandbox (39229) is too small (<= 104857600).
11/13/24 19:43:58 (6276104.24) (763436): File transfer completed successfully.

Does that mean the input file transfer failed ?

best
christoph


--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at: https://www-auth.cs.wisc.edu/lists/htcondor-users/