Re: [HTCondor-users] condor commands taking long time

[John M Knoeller <johnkn@xxxxxxxxxxx>]

One possibility is that configuration parsing is taking a long time.

 

try timing 

 

condor_config_val -summary

 

or

 

condor_config_val -config

 

-tj

 

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Nagaraj Panyam
Sent: Tuesday, April 26, 2022 7:39 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] condor commands taking long time

 

Hi,

After a recent reboot of submit node, htcondor commands are taking very long time. On Master and Exec nodes, it is fine.Even the --version switch takes nearly 1.5min

[pn@sim01 ~]$ time condor_status --version
$CondorVersion: 8.8.15 Jul 29 2021 BuildID: 552034 PackageID: 8.8.15-1 $
$CondorPlatform: x86_64_CentOS7 $

real    1m28.053s
user    0m0.010s
sys     0m0.013s

Other commands are ok. For example,

[pn@sim01 condor]$ time wc /var/log/condor/*
     120     1060    10829 /var/log/condor/KernelTuning.log
    1235     9679    82668 /var/log/condor/MasterLog
   <snip>

   26892  1514609 10485984 /var/log/condor/XferStatsLog.old
  740503  7210435 65991899 total

real    0m0.695s
user    0m0.662s
sys     0m0.028s

I notice that the process goes to sleep:

top - 17:52:59 up 40 min,  2 users,  load average: 0.06, 0.03, 0.05
Tasks: 437 total,   1 running, 435 sleeping,   1 stopped,   0 zombie
%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  0.0 us, 16.7 sy,  0.0 ni, 83.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu7  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  8172968 total,  7132596 free,   562416 used,   477956 buff/cache
KiB Swap:  1019900 total,  1019900 free,        0 used.  7362288 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ P COMMAND
 1561 pn        20   0  161768   2744   1308 S   0.0  0.0   0:00.19 1 sshd: pn@pts/0
 1562 pn        20   0  120064   2420   1776 S   0.0  0.0   0:00.13 0 -bash
 8667 pn        20   0  161768   2728   1300 S   0.0  0.0   0:00.14 3 sshd: pn@pts/1
 8668 pn        20   0  119928   2372   1792 S   0.0  0.0   0:00.07 0 -bash
 9536 pn        20   0  166776   2760   1704 T   0.0  0.0   0:00.15 0 top
 9564 pn        20   0   45752   4408   3732 S   0.0  0.1   0:00.00 2 condor_status --version
 9579 pn        20   0  166624   2644   1696 R   0.0  0.0   0:00.02 2 top -c -b -n 1 -u pn

 

What could be the cause?

- Nagaraj