If I understand it correctly this procedure transfers a host classadd value into the job classadd ?
Von: "John M Knoeller via HTCondor-users" <htcondor-users@xxxxxxxxxxx>
An: "alessandro pascolini" <alessandro.pascolini@xxxxxxxxxxxx>, "HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx>
CC: "John M Knoeller" <johnkn@xxxxxxxxxxx>
Gesendet: Montag, 7. Oktober 2024 18:24:07
Betreff: Re: [HTCondor-users] Issues moving to new job route syntax
This doesn't have anything to do with SET or EVALSET.
This is the way that "$$()" works.
in a job, if there is any string literal that has a "$$(yyy)" in it, then the value of the slot attribute yyy is substituted for $$(yyy) in the job when the job is matched to a slot. To keep track of what value was substituted, the attribute match_yyy is
inserted into the job at that time.
-tj
From: Alessandro Pascolini
Sent: Monday, October 7, 2024 10:40 AM
To: HTCondor-Users Mail List
Cc: Cole Bollig; John M Knoeller
Subject: Re: [HTCondor-users] Issues moving to new job route syntax
Hi John,
the SET macro seems to be the one helping us. Thanks for the help!
I managed to transfer the machine attribute to the job.
JOB_TRANSFORM_HEPScore @=end
SET t1_HEPSCORE "$$(t1_wn_hepscore)"
@end
even if it works, it does in a misleading way.
For a job that is transformed with this transform I see:
$ condor_q -af:h jobstatus match_t1_wn_hepscore t1_HEPSCORE
jobstatus match_t1_wn_hepscore t1_HEPSCORE
4 172 "$$(t1_wn_hepscore)"
it seems that condor is writing the value to the job classad into the
MATCH_<machine_attr> attribute, but I specified another attribute name where to copy it.
The "target" attribute is then written as the string
"$$(<machine_attr>)" so if I look for the desired job attribute I just see that string.
It is not a big deal for us since we managed to get that number, but is it really the way the SET macro should work?
Cheers,
Alessandro
On 7 Oct 2024, at 16:21, John M Knoeller via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:
Alessandro, EVALSET does behave the same way in old vs. new syntax, but there is a difference in the order of statement evaluation. I don't think that will matter in this case. but in the new syntax. statements are evalated top to bottom in the transform,
while in the old syntax they are not.
$$() expansions are a bit tricky, since they aren't evaluated in the transform at all. They are evaluated when the job is matched, so when you do the condor_q query, what you get back will depend on whether the job id idle or running.
try running
condor_q -long
to see the whole classad. and look for all attributes that have HepScore, etc in their name.
Also, I'm not sure why these statements were ever eval_set. The expressions are all strings, so there is nothing to evaluate in the transform.
This should just be
JOB_ROUTER_ROUTE_AddHS06 @=jre
SET HepScore "$$(t1_wn_hepscore:0)"
SET HostFactor "$$(t1_wn_hs06:0)"
SET numcpus "$$(TotalCpus:16)"
@jre
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Alessandro Pascolini <alessandro.pascolini@xxxxxxxxxxxx>
Sent: Thursday, October 3, 2024 5:14 AM
To: Cole Bollig <cabollig@xxxxxxxx>
Cc: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Issues moving to new job route syntax
Hi Cole,
yes the AP has many other job transforms that don't use the EVALSET macro and they work fine with the new syntax.
The issue is with the old eval_set macro in this particular router/transform that doesn't seem to translate to EVALSET in the new syntax.
Cheers,
Alessandro
On 2 Oct 2024, at 17:11, Cole Bollig <cabollig@xxxxxxxx> wrote:
Hi Alessandro,
I will dig into this some more but being on V23.0.0 should not matter as the 'new' job router syntax is actually quite old/has been around a while.
Does this AP have other routes or just this one?
-Cole Bollig
From: Alessandro Pascolini
Sent: Wednesday, October 2, 2024 10:07 AM
To: Cole Bollig
Cc: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] Issues moving to new job route syntax
Hi cole,
thanks for your help.
Unfortunately i tired with this solution and it didn't work:
$ condor_q 720 -af jobstatus HepScore HostFactor numcpus
2 undefined undefined undefined
This syntax was also one of the many tries i had before writing to you.
Just for context, the AP has HTCondor 23.0.0 installed on it (I don't know if it may be the issue).
Thanks,
Alessandro
On 2 Oct 2024, at 16:33, Cole Bollig <cabollig@xxxxxxxx> wrote:
Hi Alessandro,
Indeed, the new route should be output to stdout. When I tried the exact route you passed all I got returned was 'JOB_ROUTER_ROUTE_NAMES = ' which is not helpful at all so I messed around with things to get the following:
$cat old-syntax
JOB_ROUTER_ENTRIES @=end
[
name = "AddHS06";
eval_set_HostFactor = "$$(t1_wn_hs06:0)";
eval_set_numcpus = "$$(TotalCpus:16)";
eval_set_HepScore = "$$(t1_wn_hepscore:0)";
]
@end
$ condor_transform_ads -convert:file old-syntax
##### Route 1
JOB_ROUTER_ROUTE_AddHS06 @=jre
# autoconversion of route 'AddHS06' from old route syntax
# eval_set_* rules
EVALSET HepScore "$$(t1_wn_hepscore:0)"
EVALSET HostFactor "$$(t1_wn_hs06:0)"
EVALSET numcpus "$$(TotalCpus:16)"
@jre
JOB_ROUTER_ROUTE_NAMES = AddHS06
-Cole Bollig
From: Alessandro Pascolini
Sent: Wednesday, October 2, 2024 9:16 AM
To: HTCondor-Users Mail List
Cc: Cole Bollig
Subject: Re: [HTCondor-users] Issues moving to new job route syntax
Hi Cole,
indeed I tired the condor_transform_ads as well, I forgot to write it in the previous mail (ops..).
I don't get any output when running the comand:
# cat old-syntax-file
JOB_TRANSFORM_AddHS06 = [
eval_set_HostFactor = "$$(t1_wn_hs06:0)";
eval_set_numcpus = "$$(TotalCpus:16)";
eval_set_HepScore = "$$(t1_wn_hepscore:0)";
]
# condor_transform_ads -convert:file old-syntax-file
>From what I read on the documentation it should print the new syntax on stdout, am I wrong?
Thanks,
Alessandro
On 2 Oct 2024, at 16:04, Cole Bollig via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:
Hi Alessandro,
Condor_upgrade_check only checks for use of the old job router syntax in the configuration. If the old syntax was already removed, then the tool will say everything is successful. To help assist with the transition we updated condor_transform_ads to convert
old job router syntax to new job router syntax. You can write the old syntax to a file `foo` and then run the following to see the new syntax:
condor_transform_ads -convert:file foo
If this doesn't work let us know,
Cole Bollig
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Alessandro Pascolini <alessandro.pascolini@xxxxxxxxxxxx>
Sent: Wednesday, October 2, 2024 3:22 AM
To: htcondor-users@xxxxxxxxxxx <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Issues moving to new job route syntax
Hi everyone,
since HTCondor24 is dropping support to the old job routing syntax I've been working on updating our job_routers/transforms.
I faced an issue since I couldn't find a replacement for this transform:
JOB_TRANSFORM_AddHS06 = [
eval_set_HostFactor = "$$(t1_wn_hs06:0)";
eval_set_numcpus = "$$(TotalCpus:16)";
eval_set_HepScore = "$$(t1_wn_hepscore:0)";
]
It basically allowed us to copy these custom machine attributes as job attributes.
>From what I read on the documentation the eval_set_* macro will no longer be available so I am looking for an equivalent macro in the new syntax.
I have already tried with the EVALSET macro but with no luck, maybe I am doing it wrong.
Here is my attempt:
JOB_TRANSFORM_HEPScore @=end
EVALSET t1_HEPSCORE $(t1_wn_hepscore)
@end
Speaking with Todd during the HTC Wokrshop he suggested to use the tool provided in new HTCondor versions.
I used the condor_upgrade_check tool but it told me:
"No issues found with installed HTCondor for known incompatibilities."
Do you have any suggestion on how to implement this in the new syntax?
Cheers,
Alessandro
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}