Re: [HTCondor-users] Job Submission in Parallel Universe

Mailing List Archives Authenticated access	UW Madison Computer Sciences Department Computer Systems Lab

I have entered the following commands in condor_config.local but this is still not working the job keeps on idle for a long time and is not working at all.

DedicatedScheduler = "DedicatedSchedular@xxxxxxxxxxxxxxxxxxxxxxxxxxx"
STARTD_EXPRS = $(START_EXPRS), DedicatedScheduler

And I confirm this change with this command

condor_status -const 'DedicatedScheduler =!= Null'
But it dont show anything.

I refer this the presentation, here is the link

http://www.docstoc.com/docs/43721750/Condor-Parallel-Universe-European-Condor-Week-2008-Barcelona

Date: Tue, 9 Apr 2013 08:33:11 -0400
From: dhentchel@xxxxxxxxx
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] Job Submission in Parallel Universe

M.,

There are many config options that are not included in the "default" configuration files created in the installation process.

Also, Condor lets you set up many config files some shared across all hosts others local to a certain host.

I was referencing Andrew's configuration, earlier in this thread. The goal is to identify a unique, dedicated scheduler so that there's no risk of some other scheduler interfering with your Parallel Universe runs. You declare this dedicated scheduler on the machine running the dedicated scheduler by adding to any of its config files:

Scheduler = "DedicatedScheduler@Server.Name.Cleaned"

Then, in the condor_config.local file for every host that will be used to execute parallel jobs, you associate the start daemon with that named scheduler:

DedicatedScheduler = "DedicatedScheduler@xxxxxxxxxxxxxxxxxxx"

STARTD_ATTRS = $(STARTD_ATTRS), DedicatedScheduler

(but note my remark above: if you are running Personal Condor you need to include the user name in the host string). Andrew also had some other configuration settings in his example that ensure parallel jobs get properly scheduled. You should look at his code, and also the example provided in your condor installation (mentioned in my earlier post on this thread): $INSTALL_DIR/etc/examples/condor_config.local.dedicated.resource.

This would be a good discussion, along with a nice description of how Parallel Universe jobs compare to other modes, to include in the documentation. Us Condor users could even provide draft versions of this kind of information if we had a developer's Wiki we could update.

On Tue, Apr 9, 2013 at 3:45 AM, Muak rules <muakrules@xxxxxxxx> wrote:

Hello

Will you plz tell me in which file I should add this

Scheduler = "DedicatedScheduler@MY_USERNAME@Server.Name.Cleaned"

as I'm not finding this entry in "condor_config" and "condor_config.local"
Date: Mon, 8 Apr 2013 16:29:05 -0400
From: dhentchel@xxxxxxxxx
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] Job Submission in Parallel Universe

I don't know whether there may be other limitations that could get in the way, but I do know that if using Personal Condor you need to qualify the hostname strings with your user name, e.g.:

Scheduler = "DedicatedScheduler@MY_USERNAME@Server.Name.Cleaned"
On Mon, Apr 8, 2013 at 3:49 PM, Usman Khan <muakrules@xxxxxxxx> wrote:
Hello all thankx for your great help..
I think I forgot to mentioned that I'm running jobs on personal condor.
till now I hadn't made my own cluster so I jst wanna know is this job is executable on personal condor.

Greetings.

On 04/09/2013 12:13 AM, Andrew Kuelbs wrote:
I am running into an issue with my parallel universe jobs as well. I have just installed condor as these instructions mentioned, with a few other alterations to customize my environment.
I am running RHEL 6.3 Server 64-bit on my master and compute nodes (about 50). I am running Condor 7.8.7. I have tried a few parallel mpi scripts including the one mentioned here in this thread.

I come up with the following error in the log for that script in /var/log/codor/ShadowLog:

04/08/13 14:06:26 Initializing a PARALLEL shadow for job 50.0
04/08/13 14:06:26 (50.0) (24348): condor_write(): Socket closed when trying to write 37 bytes to daemon at <Server.IP.Cleaned:46094>, fd is 5
04/08/13 14:06:26 (50.0) (24348): Buf::write(): condor_write() failed
04/08/13 14:06:26 (50.0) (24348): ChildAliveMsg: failed to send DC_CHILDALIVE to parent daemon at <Server.IP.Cleaned:46094> (try 1 of 3): CEDAR:6002:failed to send EOM
04/08/13 14:06:27 (50.0) (24348): ERROR "Failed to get number of procs" at line 241 in file /slots/01/dir_65060/userdir/src/condor_shadow.V6.1/parallelshadow.cpp

Dedicated Server’s /etc/condor/condor_config.local

## What machine is your central manager?
CONDOR_HOST = Server.Name.Cleaned
## Pool's short description
COLLECTOR_NAME = "Server.Name.Cleaned"

#FOR MPI and other Parallel Universe runs
Scheduler = "DedicatedScheduler@xxxxxxxxxxxxxxxxxxx"

## When is this machine willing to start a job?
START = TRUE
## When to suspend a job?
SUSPEND = FALSE
## When to nicely stop a job?
## (as opposed to killing it instantaneously)
PREEMPT = FALSE
## When to instantaneously kill a preempting job
## (e.g. if a job is in the pre-empting stage for too long)
KILL = FALSE
## This macro determines what daemons the condor_master will start and keep its watchful eyes on.
## The list is a comma or space separated list of subsystem names

DAEMON_LIST = COLLECTOR, MASTER, NEGOTIATOR, SCHEDD

- - - - - - - - - - - - - - - - - - - - -
All Compute Node’s /etc/condor/condor_config.local

## What machine is your central manager?
CONDOR_HOST = Server.Name.Cleaned

## Pool's short description
COLLECTOR_NAME = "Server.Name.Cleaned"

#FOR Parallel MPI files to run
DedicatedScheduler = "DedicatedScheduler@xxxxxxxxxxxxxxxxxxx"
STARTD_ATTRS = $(STARTD_ATTRS), DedicatedScheduler

CONTINUE = True
WANT_SUSPEND = False
WANT_VACATE = False
RANK = Scheduler =?= $(DedicatedScheduler)

## When is this machine willing to start a job?
START = TRUE

## When to suspend a job?
SUSPEND = FALSE

## When to nicely stop a job?
## (as opposed to killing it instantaneously)
PREEMPT = FALSE

## When to instantaneously kill a preempting job
## (e.g. if a job is in the pre-empting stage for too long)
KILL = FALSE

## This macro determines what daemons the condor_master will start and keep its watchful eyes on.
## The list is a comma or space separated list of subsystem names
DAEMON_LIST = MASTER, STARTD
STARTD_EXPRS = $(STARTD_EXPRS), DedicatedScheduler

----------

Note the STARTD_EXPRS and the STARTD_ATTRS the online manual referenced has STARTD_ATTRS but the example file for Condor 7.8.7 says STARTD_EXPRS so I put both in. When I run a script it just hangs with the following in the job’s log:

007 (049.000.000) 04/08 13:39:56 Shadow exception!
                Failed to get number of procs
                0 - Run Bytes Sent By Job
                0 - Run Bytes Received By Job
...
007 (049.000.000) 04/08 13:40:00 Shadow exception!
                Failed to get number of procs
                0 - Run Bytes Sent By Job
                0 - Run Bytes Received By Job
...
009 (049.000.000) 04/08 13:40:00 Job was aborted by the user.
                via condor_rm (by user condor)

I tried just tossing in the example script as it was with the server name changed and simply get the same results. A condor_status lists all the Compute Nodes in an unclaimed Idle state. When I specify machine_count=X . X number of nodes become claimed but the job just sits idle. Is there anyone who has any thoughts on this?

From: htcondor-users-bounces@xxxxxxxxxxx [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of David Hentchel
Sent: Monday, April 08, 2013 11:39 AM
To: jerome.leconte@xxxxxxxxxxxxxxx; HTCondor-Users Mail List
Subject: Re: [HTCondor-users] Job Submission in Parallel Universe

I just went through this as a beginner. The key is to set up every host running a Start daemon to reference a "dedicated scheduler" in the pool, according the the instructions in the manual section labelled "3.12.8 HTCondor's Dedicated Scheduling". You can also merge the examples in the condor install (see $INSTALL_DIR/etc/examples/condor_config.local.dedicated.resource). All I had to change from that example was the hostname for the machine I wanted to use for the dedicated scheduler.

Hope this helps.

On Mon, Apr 8, 2013 at 8:45 AM, leconte <jerome.leconte@xxxxxxxxxxxxxxx> wrote:
Le 08/04/2013 09:53, Muak rules a écrit :

I'm submitting my first parallel universe job. which is basically a simple hello world problem.
I'm using CentOs 6.3 and had installed HTCondor using "yum install condor".
I'm not sure about about version of HTCondor. When I submit it the job goes on idle state.
Please help me out through this......Following is my submit description file.

universe=parallel
executable=mpi_hello_world
machine_count=1
log=hello.log
out=hello.out
queue

The following attachment contain my job.

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
hello Muak,

I have testing your prog and submit file on my own test cluster. it work fine.

I suppose it is not your prog nor submit file but your configuration that cause the problem.

I've corrected only the line
out=hello.out
by
output=hello.out
my condor complains about it.

I don't know if I can correct your problem but can you post your cluster config ?

Greetings

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--
David Hentchel
Performance Engineer
www.nuodb.com
(617) 803 - 1193
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
--
David Hentchel
Performance Engineer

www.nuodb.com
(617) 803 - 1193
_______________________________________________ HTCondor-users mailing list To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users The archives can be found at: https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

David Hentchel

Performance Engineer

www.nuodb.com

(617) 803 - 1193

_______________________________________________ HTCondor-users mailing list To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users The archives can be found at: https://lists.cs.wisc.edu/archive/htcondor-users/

Mailing List Archives

Authenticated access

Re: [HTCondor-users] Job Submission in Parallel Universe