Re: [HTCondor-users] How do I have my interactive job and my submission job in condor match 100%?

[brando.hpcs@xxxxxxxxx]

Thanks everyone, to my surprise everything seems to be working. I still get the memory SIGKILL issue but I get that in interactive mode. This seems to have been a sufficient file for it to work:

#!/bin/bash

echo JOB STARTED

# a submission job is usually empty and has the root of the submission so you probably need your HOME env var
export HOME=/home/miranda9
# to have modules work and the conda command work
source /etc/bashrc
source /etc/profile
source /etc/profile.d/modules.sh
source ~/.bashrc
source ~/.bash_profile

conda activate metalearningpy1.7.1c10.2
#conda activate metalearning11.1

module load cuda-toolkit/10.2
#module load cuda-toolkit/11.1

#nvidia-smi
nvcc --version
#conda list
echo hostname
echo $PATH
which python

# - run script
mprof run ~/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py --serial
# python ~/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py
# python ~/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py --serial
# python -m memory_profiler ~/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py --serial
#python -m memory_profiler ~/ML4Coq/ml4coq-proj/embeddings_zoo/tree_nns/main_brando.py

echo JOB ENDED

however, the memory issue is likely a problem with my script so I don't think condor has anything to do with it anymore.

Thanks for all the help everyone!Â

Sincerely, Brando

PS: lucky I didn't need to do docker/singularity! :P yes I've done that before and it's rarely been worth the effort. But it was a very good idea for sure - lucky I didn't need it.

Â

On Mon, Mar 29, 2021 at 4:27 AM Thomas Hartmann <thomas.hartmann@xxxxxxx> wrote:

Hi Brando,

maybe a sledgehammer to crack a nut, but if you cluster has Singularity
installed, you could build a Singularity container yourself with the
necessary packages and environments, so that you have a well defined
environment (that does not really depends on the 'ordinary' environments).
However, that might be *a lot of work* :-/

Cheers,
 ÂThomas