Re: [HTCondor-users] Migrating to htcondor2 -> pypi LTS supported version?

[Todd L Miller <tlmiller@xxxxxxxxxxx>]

> The bug seems to be related to the fact we have 2 jobs, one with 
> DESIRED_ExtraMatchRequirements, and another without it.

	The documentation* states that when itemdata is supplied as an 
iterator over dictionaries, only the first dictionary's keys will be used. 
That's because itemdata is intended to a Pythonic way of duplicating the 
QUEUE FROM (et alia) submit-language commands, and this was felt to be 
less user-hostile than refusing the submit because you were handing us 
dictionaries with extra data in them.

	Thanks for the test case though: it revealed a bug where the extra 
attribute in `job2` should have been silently ignored based on the 
`htcondor2.Schedd.submit()` documentation.  I have opened a ticket, 
HTCONDOR-3351, to fix this.)

	Unless there's a good reason for cramming both jobs into a single 
cluster, I'd recommend just passing the dictionaries involved to the 
Submit object directly:

sub1 = htcondor.Submit("""
        universe = vanilla
        executable = test.sh
        output = out.$(Cluster)-$(Process)
        log = log.$(Cluster).log
        """,
        ** job1
)

sub2 = htcondor.Submit("""
        universe = vanilla
        executable = test.sh
        output = out.$(Cluster)-$(Process)
        log = log.$(Cluster).log
        """,
	# If job2 defines every key in job1, you don't need this line.
        ** job1,
	** job2,
)

	If there is, you just need to have the keys all be in the first 
dictionary, which can you ensure programatically:

missing_keys = {x: None for x in job2.keys() - job1.keys()}
job1.update(missing_keys)

or in this specific case by adding

'My.DESIRED_ExtraMatchRequirements': '',

to the `job1` dictionary.

	If you could share more about what you're trying to accomplish by 
having the two procs be in the same cluster, we might be able to help you 
better.

-- ToddM

*: https://htcondor.readthedocs.io/en/latest/apis/python-bindings/api/version2/htcondor2/schedd.html#htcondor2.Schedd.submit