Re: [HTCondor-users] bug in schedd_negotiate.cpp - 144-core startd

[Michael V Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

"HTCondor-users" <htcondor-users-bounces@xxxxxxxxxxx>
wrote on 11/20/2015 04:08:17 AM:

> From: Thomas Hartmann <thomas.hartmann@xxxxxxxx>
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Date: 11/20/2015 04:10 AM
> Subject: Re: [HTCondor-users] bug in schedd_negotiate.cpp
- 144-core startd
> Sent by: "HTCondor-users" <htcondor-users-bounces@xxxxxxxxxxx>
> 
> hi,
> i just looked it up. lscpu says we have 8 xeon e7-8895 v3 @ 2.6ghz.
i do 
> not count the hyperthreading cpus. its some machine by oracle, if
i 
> remember correctly. i could ask the IT department if you need specifics.
> just let me know.
> 
> the machine was not bought or configured by me. our IT department
did and 
> according to them the information engineering department needs a machine
> that provides a massive amount of cores and RAM (2T in this case)
in one 
> machine. apparently, they are not using it at the moment so the IT
> department asked me if i wanted to use it as a HTCondor execute machine.
i
> gladly accepted.
> 
> however, you can see that the machine was not designed for the purpose
of 
> being an execute machine in an HTCondor pool. if i would configure
a setup
> for it, i would also go for cheaper CPUs and more machines.

Thanks for the info! That's a really good HTCondor
story, too - you should put it on your annual accomplishments list.

Looks like it must be an Oracle Server X5-8, and those
suckers cost so much Oracle won't even give you a price online. The processors
alone are probably in the neighborhood of $70-$80,000, and 2TB of memory
in 32GB LRDIMMs is another $64,000 list price, so you're already pushing
$150,000 before you even start installing all that in the chassis.

Suppose that one machine cost $200,000 - over a two
year period that comes to $273 (255 euro) per day, meaning that every day
that server was not in use, and thus not contributing to creating business
value, your company was just bleeding that out. Your ability to add it
to the HTCondor pool with a few keystrokes means that you recovered that
machine's value for the company, or at least 69.4% of it until this 1%-minimum-disk
thing gets straightened out. :D ("Nobody will ever need more than
100 processors in a single system" - reminds me of that mythical Bill
Gates quote...)

So, well done!

And just imagine, if you turned on Hyperthreading
you'd have 288 slots with 7GB of memory each. Wow! And to think I started
learning computers in grade school on a Commodore PET with a whopping 8
kilobytes of memory and a 0.001GHz 6502 processor. Amazing...

[I've found for our usual HTCondor workloads modern
Hyperthreading only imposes at the most a 20-30% penalty in per-job speed,
so doubling the number of slots increases the overall throughput (jobs-per-day)
substantially.]

        -Michael Pelletier.