Re: [HTCondor-users] select multiple elements from lists, eval order in job transforms

[John M Knoeller <johnkn@xxxxxxxxxxx>]

yes, the 'random' evaluation order of  the old job-router style transform language leaves a lot to be desired.
This is the main reason why the Schedd supports a new optional transform language base  on the config/submit language

Steps in the new transform language are evaluated in order, and temporary variables are permitted.  The schedd
can tell which language you use by looking for the enclosing [] around the transform staements.  leave them off 
for the new language like this.

JOB_TRANSFORM_TagJob @=end
  EVALSET AcctGroup  toLower(AcctGroup)

  # these two statements set temporary variables, they do not alter the job
  EVALMACRO lead = split(AcctGroup,".")[0]
  EVALMACRO prio = split(AcctGroup,".")[1]

  # set AccountingGroup using the first two fields from AcctGroup that we stored in temp variables
  EVALSET AccountingGroup join(".", "$(lead)","$(prio)",AcctGroupUser)
@end

The keywords and rules for the new transform language are in the latest 8.9.7 manual, but the language 
itself actually works in 8.8 for Schedd transforms,  this was the internal language that your job-router style
transform was converted into.  You can see the conversion happen by looking at the SchedLog on startup.

-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Carsten Aulbert
Sent: Tuesday, June 9, 2020 8:54 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] select multiple elements from lists, eval order in job transforms

Dear condor wizards, sorcerers, conjurers and initiates,

for accounting purposes we let our users put something like

accounting_group = a.b.c.d

into their submit files and let condor_submit/schedd complain via submit
requirements if it does not conform to something expected.

Now, we want to modify the given value to only retain the leading two
bits and but it into the AccountingGroup job ad along with the user. For
a single piece of the value this can be easily done via

JOB_TRANSFORM_TagJob @=end
  Eval_set_AccountingGroup = join(".", split(toLower(AcctGroup),
".")[1], AcctGroupUser);
  Eval_set_AcctGroup = toLower(AcctGroup);
]
@end

however, I have no idea how to select say the first and second or the
first and third from the result of "split" - is there a direct way?

As a workaround I started creating a number of intermediate results:
JOB_TRANSFORM_TagJob @=end
[
Eval_set_AccountingLead = split(toLower(AcctGroup), ".")[0];
Eval_set_AccountingPrio = split(toLower(AcctGroup), ".")[1];
Eval_set_AccountingGroup = join(".", AccountingLead, AccountingPrio,
AcctGroupUser);
Eval_set_AcctGroup = toLower(AcctGroup);
]
@end

however, condor does re-order the eval_sets by alphabet (as can be seen
in the SchedLog:

06/09/20 13:41:39 (pid:3239) JOB_TRANSFORM_TagJob setup as transform
rule #1 :
NAME TagJob
EVALSET AccountingGroup
join(".",AccountingLead,AccountingPrio,AcctGroupUser)
EVALSET AccountingLead split(toLower(AcctGroup),".")[0]
EVALSET AccountingPrio split(toLower(AcctGroup),".")[1]
EVALSET AcctGroup toLower(AcctGroup)

And, quite correctly: AccountingGroup = undefined

Changing the job transform slightly:

JOB_TRANSFORM_TagJob @=end
[
Eval_set_AccountingALead = split(toLower(AcctGroup), ".")[0];
Eval_set_AccountingAPrio = split(toLower(AcctGroup), ".")[1];
Eval_set_AccountingGroup = join(".", AccountingALead, AccountingAPrio,
AcctGroupUser);
Eval_set_AcctGroup = toLower(AcctGroup);
]
@end

yiekds the wanted result:

AcctGroup = "admin.test.parallel1"

is transformed into

AccountingGroup = "admin.test.carsten"

Now, my two questions:

(1) Is there a way to get multiple elements from a list?

(2) Why is the ordering of job transforms so broken^Wsurprising for the
uninitiated?

Cheers

Carsten
-- 
Dr. Carsten Aulbert, Max Planck Institute for Gravitational Physics,
CallinstraÃe 38, 30167 Hannover, Germany
Phone: +49 511 762 17185