Re: [HTCondor-users] change to condor_submit - user feedback desired! (was Re: multiple condor_submit's - one cluster)

[David Champion <dgc@xxxxxxxxxxxx>]

Thanks for opening this.  It looks really good to me.  My only
suggestion on it (if comments are still open) concerns this notation:

	Queue METHOD,ITEM from ...

It could be useful to adjust the syntax a bit to allow specifying the
separation symbol (allowing not just whitespace-delimited lists but also
comma, colon, etc).  That could be as simple as:

	Queue METHOD,ITEM from (
		Broil, Chicken
		Fry, Fish
		Bake, Bread
		Bake, Raspberry Pie
		Spit Roast, Wild Javelina
	)

	Queue METHOD:ITEM from (
		Broil:Chicken
		Fry:Fish
		Bake:Bread
		Bake:Raspberry Pie
		Spit Roast:Wild Javelina
	)

	Queue METHOD ITEM from (
		Broil Chicken
		Fry Fish
		Bake Bread
		Bake Raspberry Pie
		Spit Roast Wild Javelina	# TOTALLY DIFFERENT, EWW
	)

... with left and right whitespace stripping for each value.


* On 02 Mar 2015, John (TJ) Knoeller wrote: 
> There is now a design document here
> https://docs.google.com/document/d/10UKaiu1BkirXP6qUmx9DyEOPPSNqKqmS-2foztKi_zU/edit#
> That describes our proposed improvements to the Queue statement for
> condor_submit.
> 
> We plan to have (most?) of this working for the 8.3.5 release.
> 
> -tj
> 
> On 2/6/2015 10:00 AM, Todd Tannenbaum wrote:
> >On 2/6/2015 9:04 AM, Krieger, Donald N. wrote:
> >>Hi Todd,
> >>
> >>Thanks for posting back with the answers. It's very helpful.
> >>
> >>My problem is that the names I'm using for the log files step through
> >>a sequence of strings rather than a sequence of numbers. And I have a
> >>downstream management routine that uses those names which I would
> >>have to alter if I changed the naming sequence.
> >
> >Hi Don,
> >
> >Thanks for the above explanation.  Indeed, $(Process) is great in your
> >submit files, but only if your data files are numerically sequenced!
> >
> >Perhaps we could create a general solution in condor_submit that addresses
> >the above and yet would still a) allow all the submits to happen at once
> >into one cluster and b) not require the user to know how to write scripts.
> >
> >I listed a couple brainstorm ideas below that would be relatively easy to
> >implement.  Do folks think that either of the below be helpful? Would love
> >to hear any feedback, alternative ideas.
> >
> >regards,
> >Todd
> >
> >Some brainstorm ideas:
> >
> >1. A "queue foreach <filepattern>" command.  Folks could then have submit
> >files that look like this:
> >
> >    input = $(file)
> >    output = $(file).output
> >    queue foreach data/*.csv
> >
> >So for each file in subdir data that ends in .csv, a job would be
> >submitted and $(file) would expand to the path to the file.
> >
> >and/or
> >
> >2. A command line option to condor_submit that tells it to read stdin, and
> >to do a submit for each stdin line, substituting each line from stdin with
> >$(input_line).  Folks could then have submit files that look like this:
> >
> >   input = $(input_line)
> >   output = $(input_line)
> >   queue
> >
> >and invoke condor_submit via lines like:
> >
> >   ls data/*.csv | grep foo | condor_submit -submit_per_line
> >
> >Comments? Other ideas?
> >_______________________________________________
> >HTCondor-users mailing list
> >To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
> >a
> >subject: Unsubscribe
> >You can also unsubscribe by visiting
> >https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >
> >The archives can be found at:
> >https://lists.cs.wisc.edu/archive/htcondor-users/
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

-- 
       David Champion â dgc@xxxxxxxxxxxx â University of Chicago