[HTCondor-users] materialized jobs and transforms

[Ben Jones <ben.dylan.jones@xxxxxxx>]

Hi,

Noticed something on our 23.0.3 schedds that doesnât seem to happen on our 9.0.17 schedds. Do transforms get handled differently for materialized jobs now?

Among other things, we use transforms to manipulate job requirements, and we tend to compound them, ie we might have a chain of âDefaultToEl9â, âCvmfsRequirementâ and âAFSJobâ that might end up with a set of requirements that look like:

Requirements = OpSysAndVer =?= âAlmaLinux9â && PreVerRequirements 
PreVerRequirements = CVMFS_HEALTHY =?= True && PreCvmfsRequirements
PreCvmfsRequirements = NODE_MOUNTS_AFS && PreAfsReqjuirements 
PreAfsRequirements = <standard list of vanilla requirements>

Itâs probably evil, but thatâs what we do. 

Now on 23.0.3, weâre seeing some of these become circular, _only_ for materialized jobs, which is obviously not good. Is there some way that transforms are handled differently? I note in the logs, thereâs a -1 step that I donât think I noticed, ie:

05/21/24 15:24:15 (pid:24247) job_transforms for 6867177.-1: 20 considered, 12 applied (DefaultCERNTransform,DefaultToEl9,DockerNetwork,RebalanceMemCpu,AccountingGroupCheck,ChargeGroupCheck,ChargeRoleCheck,CvmfsRequirements,AFSJob,DefaultToLocal,AutoWrapSingularity,AddHome)

Then for the jobs:

05/21/24 15:24:15 (pid:24247) job_transforms for 6867177.0: 20 considered, 9 applied (DefaultCERNTransform,RebalanceMemCpu,AccountingGroupCheck,ChargeGroupCheck,ChargeRoleCheck,CvmfsRequirements,AFSJob,DefaultToLocal,AddHome)
05/21/24 15:24:15 (pid:24247) job_transforms for 6867177.1: 20 considered, 9 applied (DefaultCERNTransform,RebalanceMemCpu,AccountingGroupCheck,ChargeGroupCheck,ChargeRoleCheck,CvmfsRequirements,AFSJob,DefaultToLocal,AddHome)
[âetcâ]

Do transforms happen more than once, or is there something special for requirement for these jobs now?

cheers,
Ben