Re: [HTCondor-users] Condor held jobs should retry/release after certain configured timeout automatically

[Sridhar Thumma <deadman.den@xxxxxxxxx>]

Hi,

After usingÂNumSystemHolds, It is working as expected. Then, I added some more setting to filter out jobs from list. Basically, I want to release jobs which are held in state withÂHoldReason asÂInvalidInstanceID.NotFound orÂUnavailable orÂInvalidAMIID.NotFoundÂ. I am using following regular _expression_. It is not evaluating. Any thing wrong inÂbelowÂsettings ?Â

I tried below two. didn't work.Â

SYSTEM_PERIODIC_RELEASE=((NumSystemHolds < 5 && (time() - EnteredCurrentStatus) > 30) && (HoldReason.substr("InvalidAMIID.NotFound",0)!=""))

ÂSYSTEM_PERIODIC_RELEASE=((NumSystemHolds < 5 && (time() - EnteredCurrentStatus) > 30) && regexp("^.+InvalidAMIID.+$",HoldReason))

condor_q -hold 2878 output:

ÂID Â Â ÂOWNER Â Â Â Â Â HELD_SINCE HOLD_REASON

2878.0  cgw_dev     4/8 Â07:12 <?xml version="1.0" encoding="UTF-8"?>\n<Response><Errors><Error><Code>InvalidAMIID.NotFound</Code><Message>The image id 'ami-c4f2d0ac' does not exist</Message></Error></Errors><RequestID>d38b989a-40b6-4594-bf69-4824b201786d</RequestID></Response>

On Tue, Apr 7, 2015 at 10:26 PM, Sridhar Thumma <deadman.den@xxxxxxxxx> wrote:

Hey..Â

Thanks alot. I used NumJobStarts.. It worked like a charm.. :)Â

On Tue, Apr 7, 2015 at 8:53 PM, Ben Cotton <ben.cotton@xxxxxxxxxxxxxxxxxx> wrote:

On Tue, Apr 7, 2015 at 11:16 AM, Sridhar Thumma <deadman.den@xxxxxxxxx> wrote:

> Attached wrong image?
>
Hah, oops. I need more coffee. What I meant to copy was:

04/07/15 10:38:13 (pid:12519) Evaluated periodic expressions in
0.000s, scheduling next run in 61s

Looking at your job's classad, I notice that the JobRunCount attribute
isn't in there, which may explain why your _expression_ isn't evaluating
as expected. According to the docs, that attribute is on the way out
the door:

JobRunCount: This attribute is retained for backwards compatibility.
It may go away in the future. It is equivalent to NumShadowStarts for
all universes except scheduler. For the scheduler universe, this
attribute is equivalent to NumJobStarts.
http://research.cs.wisc.edu/htcondor/manual/v8.2/12_Appendix_A.html

You might try using NumJobStarts or NumSystemHolds instead.

Thanks,
BC

--
Ben Cotton
main: 888.292.5320

Cycle Computing
Better Answers. Faster.

http://www.cyclecomputing.com
twitter: @cyclecomputing
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/