Re: [HTCondor-users] Preferred htcss installation method

[Matthew T West <m.t.west@xxxxxxxxxxxx>]

On 24/06/2024 15:58, Greg Thain via HTCondor-users wrote:
> CAUTION: This email originated from outside of the organisation. Do 
> not click links or open attachments unless you recognise the sender 
> and know the content is safe.
>
>
> On 6/24/24 05:08, Matthew T West via HTCondor-users wrote:
> > Hi All,
> >
> > Why is the projects preferred installation method via compiled
> > binaries rather than building from source? Not saying I disapprove, as
> > working with an .rpm repo makes my life easier but so many other
> > research computing projects expect one to build from source. "I am
> > able to pick up bug fixes faster if I recompile source" is a common
> > response.
>
> Hi Matt:
>
> I think you answered your own question, there, "because it makes your
> life easier" :-)


Hi Greg,

I guess I am so used to middleware developers not caring much about 
ease-of-install/setup that "it makes my life easier" seem implausible as 
a driving force. I appreciate the user-centric choice that was made, 
given that it does load a bunch more work onto your collective shoulders.

> But it is an interesting question. HTCondor is certainly open source,
> everything we ship is on github, and anyone can see all our development
> and changes in real time as they get pushed to the repository. We know
> of several large users who do compile from source, for one reason or
> another, but it is the minority.


As most(all?) of HTCondor's tuning is done via config files & classad 
tricks rather than compile flags, I can see why it makes more sense to 
offer binaries beyond the "ease of management" argument. But I have to 
wonder, what improvement to the overall throughput of the pool would you 
get by using a tuned set of compile flags for that microarchitecture 
rather than a provided binary? Any at all?

> Wrt bug fixes, we try to push bug fix releases out as soon as we can.
> For critical bug fixes to the stable release, we can get a new release
> out often in a week or so.
>
> We do take some pride in trying to ship a reliable system, which means
> that we build and run our regression test suite on 17 different
> platforms on every commit. (More platforms than we ship). We then ship
> these same binaries, which gives us somewhat more confidence that they
> will work out of the box on your system. When we ship source, we don't
> know which compiler version, which compiler flags, what versions of what
> library, what build configuration, etc. you may have used. While we
> hope minor changes like that wouldn't break the software, everyone on
> this list should have a good understanding of the nature of software,
> and how seemingly unrelated changes can cause unexpected problems.
>
> Now, building and shipping and maintaining the native packages is (for
> me) a surprising amount of work. Not just learning the ins and outs of
> rpmbuild and similar systems, but also it forces us to work with the
> distro people, to ensure that our decisions are in harmony with the rest
> of the distribution. The distro folks have a different view and scope
> than we do, so we appreciate their perspective, and I think this results
> in better HTCondor code in the long run.

You have before stressed the importance of the "Unix way" as a design 
philosophy. I feel like sometimes in projects folks may forget that 
while "we do this one thing well" is admirable, it is also essential to 
make sure that your code plays nice with others. Glad to see this is 
built into the working practice of the HTCSS.
> > I am just curious about the htcondor teams mindset about this.
>
> But, at the end of the day, we respond to what the users need, and
> native packages are what we get the most requests for. If, by some
> miracle, all large-scale computing users mystically switched all of
> their systems to running on RISC-V using the VMS operating system, well,
> we'd follow there.
>
> > Again not complaining, just curious.
>
>
> Always like talking shop,
>
>
> -greg
Cheers,
Matt
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx 
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users 
>
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/