Re: [HTCondor-users] Python bindings and per job history files

[Brian Bockelman <bbockelm@xxxxxxxxxxx>]

Hi,

This definitely looks like a bug to me â I wonder if some objectâs lifetime gets inappropriately timed, causing a destructor to fire as part of the attribute lookup?

I can at least replicate the bug locally.  Looks like the workaround Ian suggested functions.

Brian

On Mar 7, 2016, at 12:24 PM, Iain Bradford Steers <iain.steers@xxxxxxx> wrote:

Also in reference to your original issue.

classad.parseAds expects a stream of classads e.g. (âcondor_history -l -match 4â) whereas youâre only giving it one file.

IIRC the function you need is parseOne(f).

Cheers, Iain

On Mar 7, 2016, at 19:21, Iain Steers <iain.steers@xxxxxxx> wrote:

Hi Luke,

Ah the htcondor-ce history files.

Unless something has changed in this latest release the advise is to parse the batch schedd history files instead.

IIRC thereâs some missing fields that are added in job routing and by the batch schedd that donât yet make it back to the CE schedd.

LastRemoteHost being one of the missing fields makes it difficult to apply per machine scaling factors to the jobs.

Cheers, Iain

On Mar 7, 2016, at 19:12, L Kreczko <L.Kreczko@xxxxxxxxxxxxx> wrote:

Dear experts,

I am currently trying to parse our per job history files. The example I got together so far is
import classad;
f = open("/var/lib/gratia/condorce_data/history.10.0");
ads = classad.parseAds(f);
for c in ads:
    print c

where the printout looks similar to that of a classAd (indeed the type is a class ad).

However,
print c["JobStatus"]
fails with
Traceback (most recent call last):
  File "test_history.py", line 7, in <module>
    print c["JobStatus"]
StopIteration

From the examples I've seen, this should be possible.

Fun fact: if I add
print dir(c )

to the for-loop, it works.

Equivalently, if I change the iterator to a list
for c in ads: -> for c in list(ads):

it works as well.

Is this a bug in the way the iterator is constructed or am I missing something?

The history file can be seen here:
https://gist.github.com/kreczko/9056874bbb54a1b4a21b#file-gistfile1-txt

and the python example:
https://gist.github.com/kreczko/9056874bbb54a1b4a21b#file-python

working python:
https://gist.github.com/kreczko/9056874bbb54a1b4a21b#file-working-python

I am running htcondor 8.4.3 and python 2.6.6.

Cheers,

Luke

--

*********************************************************
  Dr Lukasz Kreczko           
  Research Associate

  Department of Physics

  Particle Physics Group

  University of Bristol
  HH Wills Physics Lab
  University of Bristol
  Tyndall Avenue
  Bristol
  BS8 1TL

  +44 (0)117 928 8724 

  L.Kreczko@xxxxxxxxxxxxx

  

  A top 5 UK university with leading employers (2015)

  A top 5 UK university for research (2014 REF)

  A world top 40 university (QS Ranking 2015)

*********************************************************

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/