[HTCondor-users] Troubleshooting HTCondor on Windows

["Sam.Dana@xxxxxxxxxxx" <Sam.Dana@xxxxxxxxxxx>]

I'm sure I'm missing some subtle, and not-so-subtle, nuances for configuring HTCondor 23.0.1 on a fresh pool of Windows Server 2019 and Windows 10 Ent computers, under an Active Directory domain and Infiniband networking.

I'm looking for the latest "best practices" for this multi-machine environment, perhaps that means using IDTokens - I simply don't know, but security is desired. 

[Accounts]

I'm a bit confused on when to use which "accounts".
Is the "condor" account strictly for use with IDTokens?

Is the "condor_pool" account strictly for use with "store_cred" and Password handling?
   Does the POOL signing key need to be on every machine that uses the condor_pool account?

Other than running "store_cred", when is the elevated "SYSTEM" account appropriate to use?

How do I verify the desired and necessary accounts, passwords, signing keys, idtokens, etc are properly configured on all computers in the pool?

Other than to stop/start the condor service, when should, and shouldn't, the "Run as Administrator" Command Prompt be used?

What is significant about the contents of "admintoken.log" ?
   What is the purpose of the LOCAL signing key?
   What is the purpose of the "admin" token?
   Does the "admin" token pertain to the INSTALL_USER?
   What is the purpose of the identity and Windows ID?

[Domain Accounts]

How do I limit condor users to a specific group of domain users?

[condor_ping] 

I have been using condor_ping to troubleshoot, yet I wonder what other troubleshooting tools or tips should I try?

from AP to CM and EP to CM, I get:

from CM to AP or EP, I get:  - is this expcted?

is this expected?

To get his far, the CM config was set to:

   ALLOW_NEGOTIATOR = *.$(UID_DOMAIN)

   ALLOW_DAEMON = *.$(UID_DOMAIN)

Although "condor_ping ... -verbose" shows the user is htadmin@iipac, why does setting
   ALLOW_NEGOTIATOR = htadmin@iipac    fail?

What specific log settings and logs will help track this down?

[condor_config]

Which settings MUST be placed in the "base" condor_config file, instead of being accessed through the "config" directory; and why is the directory inadequate? Is the trend moving toward having a single dedicated file for each individual machine?

[condor_credd]

Specifically, in looking at "condor_config.local.credd", is this the correct breakdown by type
-- which need to be condor_config and which can be under \config?

credd_server          (CM, if hosting CredD) 

Credd_Log

Credd_Debug

Max_Credd_Log

Daemon_List

CredD

Sec_Credd_Session_Timeout

Credd.Sec_Default_Authentication

Credd.Sec_Default_Encryption

Credd.Sec_Default_Integrity

Credd.Sec_Default_Negotiation

Credd.Sec_Daemon_Authentication_Methods

Credd.Allow_Daemon

Credd.Sec_Default_Authentication_Methods

-- if CredD runs on a standalone host (not a CM, AP, or EP), do the non-credd_server settings, below, apply to it?

credd_global          CM, AP, EP   

## Note: The following settings will need to be present in your global config file: Does this mean condor_config?

Credd_Host

Starter_Allow_Runas_Owner

Credd_Cache_Locally

Must the following groups of Cred settings be in the "global config file"?

credd_all                 CM, AP, EP

## you'll need to enable CONFIG-level access for all machines in the pool so that the pool password can be stored:

Allow_Config

Sec_Config_Negotiation

Sec_Config_Authentication

Sec_Config_Encryption

Sec_Config_Integrity

credd_ep                EP   (although "readthedocs" says, "These variables must be set for all machines in the pool." (including AP and CM?)

## ensure that clients are configured to use PASSWORD authentication on any machine that can *run jobs as the submitting user*. For example,

Sec_Client_Authentication_Methods = NTSSPI, PASSWORD

What other methods might be appropriate; can this be IDTOKEN, for example?
"readthedocs" says:

The example above can be modified to meet the needs of your pool, providing the following conditions 
    { but which settings ensure these } are met:

The requesting client must use an authenticated connection { SEC_CONFIG_AUTHENTICATION = REQUIRED ?}

The requesting client must have an encrypted connection      { SEC_CONFIG_ENCRYPTION = REQUIRED ?}

The requesting client must be authorized for DAEMON level access.   
                                          { per credd-server: CREDD.ALLOW_DAEMON = condor_pool@$(UID_DOMAIN)  ??? }

[ALLOW_ settings]

Specifically, from condor_config.submit.generic, can these be in \config\... ?

condor_ap              AP

Allow_Administrator

Allow_Negotiator

Does the EP role have similar requirements, for Administrator and Negotiator, or others?

Thanks,

Sam

NOTICE: This email message and all attachments transmitted with it may contain privileged and confidential information, and information that is protected by, and proprietary to, Parsons Corporation, and is intended solely for the use of the addressee for the specific purpose set forth in this communication. If the reader of this message is not the intended recipient, you are hereby notified that any reading, dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited, and you should delete this message and all copies and backups thereof. The recipient may not further distribute or use any of the information contained herein without the express written authorization of the sender. If you have received this message in error, or if you have any questions regarding the use of the proprietary information contained therein, please contact the sender of this message immediately, and the sender will provide you with further instructions.