Re: [HTCondor-users] Negative kflops benchmarks (-2147483648) in HTCondor 25.0 LTS

[Valerio Bellizzomi <valerio@xxxxxxxxxx>]

On Tue, 2026-01-13 at 16:58 +0100, Stefan Mohn via HTCondor-users
wrote:
> Hi Greg,
> 
> Am Mon, Jan 12, 2026 at 05:08:24PM -0600, schrieb Greg Thain via
> HTCondor-users:
> > On 1/9/26 12:48, Stefan Mohn via HTCondor-users wrote:
> > > Hello,
> > > 
> > > I would like to report an issue of our institutes HTCondor pool.
> > > 
> > > Some nodes are not receiving any jobs and report unexpectedly
> > > negative benchmark values, with kflops = -2147483648. Multiple
> > > restarts of the condor service on the ten affected nodes only
> > > resolved the issue on 20 % of them. As a workaround, I generated
> > > artificial load on the nodes during the benchmarking process,
> > > which temporarily resolved the issue. However, the problem
> > > reappears whenever the condor service is restarted without
> > > artificial load present.
> > 
> > This looks like a different (though perhaps related?) problem than
> > the
> > earlier one. When the startd computes the kflops, it just runs the
> > condor_kflops program. Does the problem ever happen on a
> > standalone run of
> > "condor_kflops" ?

Why I can't find the condor_kflops command ?

> 
> yes, condor_kflops actually shows me the -2147483648 as result, when
> running standalone. Sometimes. 
> 
> Although I haven't seen it before, it gave me these results when
> running
> condor_kflops in a loop.
> 
> I hope it's not to much to post my results here for this issue:
> 
> root@boa:~# i=1; while true; do echo $i:
> `/usr/libexec/condor/condor_kflops | grep KFlops` `date` ;
> i=$(($i+1)); done
> 1: KFlops = -2147483648 Tue Jan 13 16:18:40 CET 2026
> 2: KFlops = 2492250 Tue Jan 13 16:19:06 CET 2026
> 3: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 4: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 5: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 6: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 7: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 8: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 9: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 10: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 11: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 12: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 13: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 14: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 15: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 16: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 17: KFlops = -2147483648 Tue Jan 13 16:19:06 CET 2026
> 18: KFlops = -2147483648 Tue Jan 13 16:19:07 CET 2026
> 19: KFlops = -2147483648 Tue Jan 13 16:19:07 CET 2026
> 20: KFlops = -2147483648 Tue Jan 13 16:19:07 CET 2026
> 21: KFlops = 2641825 Tue Jan 13 16:19:31 CET 2026
> 22: KFlops = -2147483648 Tue Jan 13 16:19:31 CET 2026
> 23: KFlops = 2623610 Tue Jan 13 16:19:56 CET 2026
> 24: KFlops = 2636171 Tue Jan 13 16:20:21 CET 2026
> 25: KFlops = 2632379 Tue Jan 13 16:20:47 CET 2026
> 26: KFlops = 2638033 Tue Jan 13 16:21:12 CET 2026
> 27: KFlops = 2632940 Tue Jan 13 16:21:37 CET 2026
> 28: KFlops = 2631667 Tue Jan 13 16:22:02 CET 2026
> 29: KFlops = 2635359 Tue Jan 13 16:22:27 CET 2026
> 30: KFlops = 2633526 Tue Jan 13 16:22:52 CET 2026
> 31: KFlops = 2633656 Tue Jan 13 16:23:18 CET 2026
> 32: KFlops = 2631147 Tue Jan 13 16:23:43 CET 2026
> 33: KFlops = 2634787 Tue Jan 13 16:24:08 CET 2026
> 34: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 35: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 36: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 37: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 38: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 39: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 40: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 41: KFlops = -2147483648 Tue Jan 13 16:24:08 CET 2026
> 42: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 43: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 44: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 45: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 46: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 47: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 48: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 49: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 50: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 51: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 52: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 53: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 54: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 55: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 56: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 57: KFlops = -2147483648 Tue Jan 13 16:24:09 CET 2026
> 58: KFlops = 2636001 Tue Jan 13 16:24:34 CET 2026
> 59: KFlops = -2147483648 Tue Jan 13 16:24:34 CET 2026
> 60: KFlops = -2147483648 Tue Jan 13 16:24:34 CET 2026
> 61: KFlops = 2628210 Tue Jan 13 16:24:59 CET 2026
> 62: KFlops = 2633669 Tue Jan 13 16:25:24 CET 2026
> 63: KFlops = 2633985 Tue Jan 13 16:25:49 CET 2026
> 64: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 65: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 66: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 67: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 68: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 69: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 70: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 71: KFlops = -2147483648 Tue Jan 13 16:25:49 CET 2026
> 72: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 73: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 74: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 75: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 76: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 77: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 78: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 79: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 80: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 81: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 82: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 83: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 84: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 85: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 86: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 87: KFlops = -2147483648 Tue Jan 13 16:25:50 CET 2026
> 88: KFlops = 2427306 Tue Jan 13 16:26:16 CET 2026
> 89: KFlops = 2633159 Tue Jan 13 16:26:41 CET 2026
> 90: KFlops = -2147483648 Tue Jan 13 16:26:41 CET 2026
> 91: KFlops = -2147483648 Tue Jan 13 16:26:41 CET 2026
> 92: KFlops = 2636277 Tue Jan 13 16:27:06 CET 2026
> 93: KFlops = 2635542 Tue Jan 13 16:27:31 CET 2026
> 94: KFlops = 2638784 Tue Jan 13 16:27:56 CET 2026
> 95: KFlops = 2638253 Tue Jan 13 16:28:21 CET 2026
> 96: KFlops = 2636385 Tue Jan 13 16:28:46 CET 2026
> 97: KFlops = 2635581 Tue Jan 13 16:29:11 CET 2026
> 98: KFlops = 2640928 Tue Jan 13 16:29:36 CET 2026
> 99: KFlops = 2636307 Tue Jan 13 16:30:01 CET 2026
> 100: KFlops = 2636285 Tue Jan 13 16:30:26 CET 2026
> 101: KFlops = -2147483648 Tue Jan 13 16:30:26 CET 2026
> 
> As you can see, condor_kflops gives me mixed results. A normal run
> with
> a reasonable result takes around 25 seconds. However when it shows
> -2147483648, the result comes immediately.
> 
> > For an immediate workaround, you can hardcode the kflops
> > value in the startd config file.
> Â
> Thank you very much, i will consider it.
> 
> Please let me know, if I can provide additional information about
> this
> issue.
> 
> All the best
> Stefan