Re: [Condor-users] Condor and Python

[Mick Beaver <mick@xxxxxxxxxxx>]

If for some reason, you might want a newer version of Python, you could 
always try compiling it yourself and submitting a tarball. Here's an 
example that resulted in a 23 MB tarball.

1. Build Python and tar it up

export PATH=/usr/bin:/usr/sbin:/bin:/sbin
export INSTALL_DIR=/scratch/mick/src/python-3.1.2-x86_64-rhel5
mkdir -p $INSTALL_DIR
wget http://python.org/ftp/python/3.1.2/Python-3.1.2.tar.bz2
tar xjf Python-3.1.2.tar.bz2
cd Python-3.1.2
./configure --prefix=$INSTALL_DIR
make install
cd $INSTALL_DIR/lib/python3.1/lib-dynload
ldd *.so | \
grep '/usr/' | \
gawk '{print $3}' | \
sort -u | \
while read n; do cp $n $INSTALL_DIR/lib; done;
cd `dirname $INSTALL_DIR`
tar czf python-3.1.2-x86_64-rhel5.tar.gz python-3.1.2-x86_64-rhel5

2. Create a wrapper to extract the tarball and run python. I called my 
wrapper 'python3'

#!/bin/bash

VERSION=python-3.1.2-x86_64-rhel5
export LD_LIBRARY_PATH=./$VERSION/lib

/bin/tar xzf ./$VERSION.tar.gz
exec ./$VERSION/bin/python3 "$@"

3. Submit your job.

universe                = vanilla
executable              = python3
arguments               = hello_world.py
should_transfer_files   = yes
transfer_input_files = hello_world.py, python-3.1.2-x86_64-rhel5.tar.gz
when_to_transfer_output = on_exit
output                  = condor.out.$(CLUSTER).$(PROCESS)
log                     = condor.log.$(CLUSTER).$(PROCESS)
error                   = condor.err.$(CLUSTER).$(PROCESS)
queue


-Mick