Re: [HTCondor-users] Python condor_chirp client available for testing

[Jason Patton <jpatton@xxxxxxxxxxx>]

Brian,

Thanks for review! I'll try to address all of these before we get to a
"v1.0", but to follow up on a few...

On Mon, Feb 5, 2018 at 3:30 PM, Brian Bockelman <bbockelm@xxxxxxxxxxx> wrote:
> 1.  This seems to have copied the condor_chirp behavior of having no
> timeouts and/or hanging indefinitely on remote error.  Probably worth
> changing... might I suggest 20 second timeouts?

The socket object should be created with a default timeout of 10
seconds. Did I miss setting that somewhere?

> 2.  There's actually a few chirp commands that can be done without setting
> IO proxy -- sending delayed updates comes to mind.

Cool! I thought that was necessary for $_CONDOR_CHIRP_CONFIG to exist,
but indeed that's not the case. I'll have to think about how to handle
this case.

(Test on init? Leave up to user? The proxy returns INVALID_REQUEST if
using an unsupported command.)

> 3.  Can we add documentation to http://htcondor-python.readthedocs.io?

This would be ideal! Definitely can't leave the docs in their current
state of shuffling users back and forth between help() and the
condor_chirp manual.

> 4.  You probably want the object to behave as a context manager to avoid
> tons of connects / disconnects:
>
> import htchirp
> with htchip.HTChirp() as chirp:
>     chirp.ulog("I'm using HTChirp!")
>     chirpt.set_job_attr("UsingHTChirp", "True")
>
> It'd be straightforward to avoid this disconnects, and it'll also make the
> code exception safe.  For example, it's possible to have a dangling
> connection here:
>
> https://github.com/htcondor/htchirp/blob/master/htchirp/htchirp.py#L654

Yes, in retrospect, this is how I should have done it. And will do
it... I'll make this change a priority.

> 5.  Do we need to worry about sending the server responses greater than 1024
> bytes?  I seem to recall that the starter code in that area seemed a bit ...
> crashy ... with large responses.  Even if it doesn't cause a crash, it might
> be worthwhile to have a reasonably good error message if too-long filenames
> can cause protocol violations.
> 6.  What Unicode encoding issues should we think about?  That's always been
> a bit mysterious to me in HTCondor land -- and potentially problematic as
> Python 3 marches forward.

Definitely would be good to know, I have neglected at this point to
shovel much "garbage" at the chirp proxy to see what happens.

> 7.  Would love to see some appveyor / Travis-CI style integration tests!

I will need some guidance in setting this up, but for sure this would be useful.

Thanks again!

Jason

> Hope this is helpful,
>
> Brian
>
> On Feb 2, 2018, at 10:30 AM, Jason Patton <jpatton@xxxxxxxxxxx> wrote:
>
> Attention Chirp users and/or Python users! We are welcoming any
> feedback you can provide on another tool, this time a native Python
> (2/3) implementation of the condor_chirp client, "HTChirp."
>
> pip install htchirp
>
> For example, within an interactive job:
>
> import htchirp
> chirp = htchirp.HTChirp()
> chirp.ulog("I'm using HTChirp!")
> chirp.write("I'm using HTChirp!\n", "/home/jpatton/my-chirp-log", "cwa")
> chirp.set_job_attr("UsingHTChirp", "True")
>
>
> As with condor_chirp, using HTChirp requires that your submit file (or
> object) include:
> +WantIOProxy = true
>
> If you need a refresher on condor_chirp:
> http://research.cs.wisc.edu/htcondor/manual/current/condor_chirp.html
>
> Feel free to direct feedback to this thread or over at GitHub:
> https://github.com/htcondor/htchirp
>
> Jason Patton
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/