Mailing List Archives

Authenticated access

 		[Computer Systems Lab]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Submit requirements & ignoring significant attributes

Hi all,
I'm facing some strange issues in my configuration file regarding auto-clustering/significant attributes and adding requirements upon job submission (in our setup we have a job route defined - we append requirements in order to make sure the router acts before the jobs starts) 


APPEND_REQUIREMENTS = ( (INPUT_FILES =?= UNDEFINED) || (HPDA_Route =?= TRUE) || (( CurrentTime - QDate ) > ( $(JOB_ROUTER_POLLING_PERIOD) * 3 )) )


The following configuration values (should) handle significant attributes:


# Round attributes (up) for better AutoClustering.
# 25%: 112 => 125, 1133 => 1250
# 3  : 112 => 1000, 1212 => 2000
SCHEDD_ROUND_ATTR_RequestWalltime = 3
SCHEDD_ROUND_ATTR_RequestMemory = 20%
SCHEDD_ROUND_ATTR_RequestDisk = 25%

SIGNIFICANT_ATTRIBUTES = JobUniverse,WantDocker,\
                         RequestWalltime,\
                         RequestCpus,RequestMemory,RequestDisk,\
                         Requirements,\
                         RemoteJob,ExperimentalJob,\
                         HPDA_Route
REMOVE_SIGNIFICANT_ATTRIBUTES = DiskUsage, QDate


So far so good.

Now here's an example of condor_q -autocluster -long:


ServerTime = 1477408979
AutoClusterId = 2836
JobCount = 7
Requirements = ( ( TARGET.CLOUDSITE == "BWFORCLUSTER" ) ) && ( ( ( INPUT_FILES =?= undefined ) || ( HPDA_Route =?= true ) || ( ( CurrentTime - QDate ) > ( 10 * 3 ) ) ) ) && ( TARGET.Arch == "X86_64" ) && ( TARGET.OpSys == "LINUX" ) && ( TARGET.Disk >= RequestDisk ) && ( TARGET.Memory >= RequestMemory ) && ( TARGET.HasFileTransfer )
RequestDisk = DiskUsage
JobIds = "175098.1 ... 175098.16"
JobUniverse = 5
RequestCpus = 1
DiskUsage = 750000
RequestWalltime = 69000
QDate = 1477393245
RemoteJob = true
RequestMemory = 4000
Apparently QDate and DiskUsage are NOT removed from significant attributes, although I explicitly told HTCondor to do so. 

Am I missing something or do you see an error in my though process?
I'm out of ideas, what I could try to increase the number of clustered jobs.

Thanks & regards
Frank