|
The AuthToken* job attributes are added to the job ad by the condor_schedd based on the token used to authenticate during job submission.
For ARC CE submitting into a local HTCondor pool, itâs using the FILESYSTEM authentication method, so the AuthToken* attributes wonât be set.
- Jaime
On May 5, 2023, at 4:02 AM, Jose Caballero <jcaballero.hep@xxxxxxxxx> wrote:
Hi Thomas,
I don't see classAd AuthTokenIssuer in any of our jobs. Is it being injected by HTCondor-CE? Note we use ARC.
It seems we will have to write a JOB_TRANSFORM to keep backwards compatibility.
Cheers,
Jose
Hi Jose,
AFAIS the token issuer ad should be somewhat a good identifier for the
VO. I.e., you could maybe write a transform matching & selecting on
AuthTokenIssuer
and inject the corresponding VO into x509UserProxyVOName, if it does not
exists.
As us, we see a mixed back of GSI and token authz jobs [1], so that one
might need to be careful not accidentially miss a case. (being a bit
worried about occasional token only jobs and the still large number of
pure GSI jobs)
Cheers,
Thomas
[1]
> condor_ce_q -af AuthTokenIssuer x509UserProxyVOName | sort | uniq -c
548
https://atlas-auth.web.cern.ch/ atlas
4
https://atlas-auth.web.cern.ch/ undefined
474
https://cms-auth.web.cern.ch/ cms
3
https://cms-auth.web.cern.ch/ undefined
4497 undefined belle
6 undefined desy
51 undefined ilc
280 undefined lhcb
672 undefined ops
On 05/05/2023 10.18, Jose Caballero wrote:
> Hi Maarten,
>
> Thanks a lot for the explanation.
> Would it be possible to replicate the old functionality with a
> JOB_TRANSFORM ?
>
> Cheers,
> Jose
>
> El vie, 5 may 2023 a las 9:00, Maarten Litmaath
> (<Maarten.Litmaath@xxxxxxx <mailto:Maarten.Litmaath@xxxxxxx>>) escribiÃ:
>
> Hi JosÃ,
> in 10.x there is no code that looks into the VOMS extensions that an
> X509 proxy may have
> and hence there are no variables defined anymore for the VO and the
> FQANs.
>
> We will need to decide on sustainable ways for the accounting to
> keep working...
>
>
> ------------------------------------------------------------------------
> *From:* HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx
> <mailto:htcondor-users-bounces@xxxxxxxxxxx>> on behalf of Jose
> Caballero <jcaballero.hep@xxxxxxxxx <mailto:jcaballero.hep@xxxxxxxxx>>
> *Sent:* Friday, May 5, 2023 9:33 AM
> *To:* HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx
> <mailto:htcondor-users@xxxxxxxxxxx>>
> *Cc:* condor-users@xxxxxxxxxxx <mailto:condor-users@xxxxxxxxxxx>
> <condor-users@xxxxxxxxxxx <mailto:condor-users@xxxxxxxxxxx>>
> *Subject:* Re: [HTCondor-users] Missing `x509UserProxyVOName`
> ClassAd in Condor 10.0.3
> Hi,
>
> if I understand correctly what I see, the classAd mentioned by Tom,
> "x509UserProxyVOName", is added to the jobs at the schedd level.
> Picking one random job on a schedd 9.0.5, this is the submit file
> from the CE middleware (ARC) [1] and these are the classad of the
> submitted job [2].
> So clearly the classAds x509* have been added by our local Schedd.
>
> However, on a schedd 10.0.3, some of those classAds are missing [3].
>
> I have downloaded the code from GITHUB, and a simple grep gives me
> the same results for the main branch and tag V9_0_5.
> Also, the classAd x509UserProxyVOName is still mentioned in the
> documentation.
> So I am quite lost as well. Why suddenly the jobs submitted from
> schedd 10.x are missing these classads?
>
> Any comment/question is more than welcome.
>
> Cheers,
> Jose
>
> [1]
> [root@arc-ce04
> PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm]# cat
> condorjob.jdl
> # HTCondor job description built by arex
> Executable = condorjob.sh
> Input = /dev/null
> Log =
> /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm/log
> Output =
> /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm.comment
> Error =
> /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm.comment
> +NordugridQueue = "EL7"
> Description = gridjob
> Universe = vanilla
> Notification = Never
> Requirements = (NumJobStarts == 0) && ( (OpSys == "LINUX" &&
> OpSysMajorVer >= 7) )
> Priority = 0
> x509userproxy =
> /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm/user.proxy
> request_cpus = 1
> request_memory=4000
> +JobMemoryLimit = 4096000
> should_transfer_files = YES
> When_to_transfer_output = ON_EXIT_OR_EVICT
> Transfer_input_files =
> /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm
> Periodic_remove = (JobStatus == 1 && NumJobStarts > 0) ||
> ((ResidentSetSize isnt undefined ? ResidentSetSize : 0) >
> JobMemoryLimit)
> Queue
>
> [2]
> [root@arc-ce04
> PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm]# condor_q -l
> 2479042 | grep ^x509
> x509userproxy =
> "/var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm/user.proxy"
> x509UserProxyEmail = "lb.pilot@xxxxxxx <mailto:lb.pilot@xxxxxxx>"
> x509UserProxyExpiration = 1683605339
> x509UserProxyFirstFQAN = "/lhcb/Role=pilot/Capability=NULL"
> x509UserProxyFQAN = "/DC=ch/DC=cern/OU=Organic
> Units/OU=Users/CN=lbpilot/CN=693025/CN=Robot: LHCb
> pilot,/lhcb/Role=pilot/Capability=NULL,/lhcb/Role=NULL/Capability=NULL"
> x509userproxysubject = "/DC=ch/DC=cern/OU=Organic
> Units/OU=Users/CN=lbpilot/CN=693025/CN=Robot: LHCb pilot"
> x509UserProxyVOName = "lhcb"
>
> [3]
> [root@arc-ce-test01 ~]# condor_history -l 605625.0 | grep ^x509
> x509UserProxyEmail = "Andrea.Sciaba@xxxxxxx
> <mailto:Andrea.Sciaba@xxxxxxx>"
> x509UserProxyExpiration = 1682927827
> x509userproxy =
> "/var/spool/arc/grid05/ZPsKDmZFHD3n61QDjqWNiMpoABFKDmABFKDmAaFKDmAEFKDmDzgJen/user.proxy"
> x509userproxysubject = "/DC=ch/DC=cern/OU=Organic
> Units/OU=Users/CN=sciaba/CN=430796/CN=Andrea Sciaba"
>
>
>
>
> El mar, 2 may 2023 a las 10:33, Thomas Hartmann
> (<thomas.hartmann@xxxxxxx <mailto:thomas.hartmann@xxxxxxx>>) escribiÃ:
>
> Hi Thomas,
>
> from Condor 10 on GSI is not supported anymore but only token authz.
> Also IIRC has ATLAS recently switched Harvester submission to
> Condor 10
> as well, so that their jobs do not get submitted anymore with
> X509 ads.
>
> Probably the only option on the midterm run would be to add
> cases for
> routes, that evaluate the Auth* ads similar as for X509 ads.
>
> Cheers,
> Thomas
>
> On 02/05/2023 10.07, Thomas Birkett - STFC UKRI via
> HTCondor-users wrote:
> > Hi Condor community,
> >
> > I hope you are all keeping well, hopefully a simple fix but Iâve
> > recently upgraded our test Condor pool from 9.0.15 to 10.0.3
> (LTS) and I
> > notice that jobs no longer show the ClassAd
> âx509UserProxyVONameâ. The
> > following x509 classads are present when running a `condor_q
> -l *jobid*`
> >
> > x509UserProxyEmail
> >
> > x509UserProxyExpiration
> >
> > x509userproxy
> >
> > x509userproxysubject
> >
> > however, ` x509UserProxyVOName` is missing.
> >
> > This is a problem for us as a large proportion of our Job
> Transforms use
> > this missing ClassAd `x509UserProxyVOName`. Downgrading to
> Condor
> > 9.0.15, the ClassAd is then applied to new incoming jobs. Any
> help in
> > debugging this issue would be gratefully received.
> >
> > Many thanks,
> >
> > *Thomas Birkett*
> >
> > Senior Systems Administrator
> >
> > Scientific Computing Department
> >
> > Science and Technology Facilities Council (STFC)
> >
> > Rutherford Appleton Laboratory, Chilton, Didcot
> > OX11 0QX
> >
> > signature_609518872
> >
> >
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to
> htcondor-users-request@xxxxxxxxxxx
> <mailto:htcondor-users-request@xxxxxxxxxxx> with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> >
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>
> >
> > The archives can be found at:
> >
https://lists.cs.wisc.edu/archive/htcondor-users/
> <https://lists.cs.wisc.edu/archive/htcondor-users/>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to
> htcondor-users-request@xxxxxxxxxxx
> <mailto:htcondor-users-request@xxxxxxxxxxx> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> <https://lists.cs.wisc.edu/archive/htcondor-users/>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to
htcondor-users-request@xxxxxxxxxxx
> <mailto:htcondor-users-request@xxxxxxxxxxx> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> <https://lists.cs.wisc.edu/archive/htcondor-users/>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to
htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
>
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
>
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
|
![[Computer Systems Lab]](http://www.cs.wisc.edu/pics/csl_logo.gif){kind=logo}