Re: [HTCondor-users] python api about history

Mailing List Archives Authenticated access	UW Madison Computer Sciences Department Computer Systems Lab

Ah I was thinking of the since and stop constraints.

https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5642,56

Looks like they haven't been added to the python bindings yet.

________________________________________

From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of jiangxw@xxxxxxxxxxxxxxx [jiangxw@xxxxxxxxxxxxxxx]

Sent: 15 November 2016 10:53

To: htcondor-users

Subject: Re: [HTCondor-users] python api about history

Hi Iain,

Thanks for your advice.

I can catch your meaning.

I implemented that function like yours.

And the space of history files in our environments is 132G totally.

When I execute the script with the lines

requirements = "EnteredCurrentStatus <= %s && EnteredCurrentStatus > %s" % (current_time,pre_time)

job_history = self.schedd.history(requirements, projection, 0),

the script will take too long times. I think self.schedd.history() will look over all 132G history files.

How can I optimize it?

Thanks.

Cheers,

Jiang Xiaowei

________________________________

NAME：Jiang Xiaowei

MAIL：jiangxw@xxxxxxxxxxxxxxx

TEL：010 8823 6024

DEPARTMENT：Computing Center of IHEP

From: Iain Steers<mailto:iain.steers@xxxxxxx>

Date: 2016-11-15 17:16

To: HTCondor-Users Mail List<mailto:htcondor-users@xxxxxxxxxxx>

Subject: Re: [HTCondor-users] python api about history

Hi,

history() can take a Requirement constraint as one of its arguments.

In order to do this efficiently i would suggest something like the following.

start_dt = datetime.datetime.now() - datetime.timedelta(minutes=30)

start_stamp = calendar.timegm(d.timetuple())

schedd = htcondor.Schedd()

jobs = schedd.history('EnteredCurrentStatus >= {0}'.format(start_stamp))

This should return an iterator of all the jobs which were added to the history in the last half hour.

Cheers, Iain

________________________________________

From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of jiangxw@xxxxxxxxxxxxxxx [jiangxw@xxxxxxxxxxxxxxx]

Sent: 15 November 2016 10:08

To: htcondor-users

Subject: [HTCondor-users] python api about history

Hi all,

I used python api to get job history information then insert job information into database.

The function is htcondor.Schedd.history().

But when the amount of the history files in "/var/lib/condor/spool/" became large,

htcondor.Schedd.history() will be executed slowly.

I want to gain the job history information every half hour. So I just need to look over the history file generated last half hour.

But the htcondor.Schedd.history() may look over all history files in "/var/lib/condor/spool" every time.

I can not find the parameter from htcondor.Schedd.history() like "condor_history -file".

Are there some ways to solve it?

Thanks for help.

Best regards,

Jiang XiaoWei

________________________________

NAME：Jiang Xiaowei

MAIL：jiangxw@xxxxxxxxxxxxxxx

TEL：010 8823 6024

DEPARTMENT：Computing Center of IHEP

_______________________________________________

HTCondor-users mailing list

To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a

subject: Unsubscribe

You can also unsubscribe by visiting

https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:

https://lists.cs.wisc.edu/archive/htcondor-users/