Re: [HTCondor-users] Cluster/Job not setting up via SOAP calls

[Mukherjee Arijit <mukherjee.arijit@xxxxxxx>]

Thanx Jordan.

> And as for
the second question, does this line in your code not work with submit files?
ClassAdStructAndStatus jobAd
= buildJobAd(owner, jobFileLocation, stub, clusterId, jobId);

I used an example code from an older
blog which showed how to invoke the DAGMAN via SOAP (http://spinningmatt.wordpress.com/condorsubmitdag-java/).
But those calls are obsolete for Axis 2. I rewrote the entire code based
on Axis 2 stubs and I added that parameter to the call so that I can build
the classAd by reading the file in my program. I haven't found any API
through which I can submit a classAd as a file. While reading the file,
I wasn't able to use the createJobTemplate() API to add the variables to
the classAd. This API has setters only for variables like owner, Cmd, type,
args etc.

But stubs also contain a structure called
ClassAdStructAttr - which was used by the blog I mentioned. I think I can
use that to create individual entries in the classAd, but that may not
be generic enough. An API to submit a classAd directly would have been
very useful. The submitFile API which is mentioned in the manual probably
is for data files - but I am not sure.

Regards
Arijit

Arijit Mukherjee
Senior Scientist R&D; Innovation Lab
Tata Consultancy Services
Ph:- 913366367137
Cell:- 9903705285
Mailto: mukherjee.arijit@xxxxxxx
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                
       Business Solutions
                
       Consulting
____________________________________________

From:	Jordan Williamson <jordan.williamson@xxxxxxxxxxx>
To:	HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>,
Date:	05/10/2013 11:00 PM
Subject:	Re: [HTCondor-users] Cluster/Job not setting up via SOAP calls
Sent by:	htcondor-users-bounces@xxxxxxxxxxx

---

Hi Mukherjee,

Do all hosts in the cluster need to be SOAP
enabled?

No, just the ones that need it.

And as for the second question, does this line in your
code not work with submit files?
ClassAdStructAndStatus jobAd
= buildJobAd(owner, jobFileLocation, stub, clusterId, jobId);
 
That would be my first guess.

-Jordan

On Fri, May 10, 2013 at 7:10 AM, Mukherjee Arijit <mukherjee.arijit@xxxxxxx>
wrote:
Made some progress. The WEB_ROOT variable
on my host was set incorrectly which I have corrected. The jobs are now
being submitted, but there is an error while executing it, which seemingly
comes from another host. 

Do all hosts in the cluster need to be SOAP enabled?

And the second question is - the APIs for JobTemplate class is a limited
set. If I want to submit a job with a submit file, how do I create the
JobAdd? 

Regards 
Arijit 



Arijit Mukherjee
Senior Scientist R&D; Innovation Lab
Tata Consultancy Services
Ph:- 913366367137
Cell:- 9903705285
Mailto: mukherjee.arijit@xxxxxxx
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                     
 Business Solutions
                     
 Consulting
____________________________________________

From:	Mukherjee Arijit <mukherjee.arijit@xxxxxxx>
To:	HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>,
Cc:	condor-users@xxxxxxxxxxx
Date:	05/10/2013 03:38 PM
Subject:	Re: [HTCondor-users] Cluster/Job not setting up via SOAP calls
Sent by:	htcondor-users-bounces@xxxxxxxxxxx

---

Just checked few more things - 

For the "createCluster()" call, I'm getting an error status:
"Could not create new cluster" 

I checked the mailing list archive and found a suggestion about setting
"QUEUE_ALL_USERS_TRUSTED = True" - is it in condor_config or
condor_config.local? I've tried both, but it didn't change anything. I'm
sure I'm missing something. 

Regards 
Arijit 

Arijit Mukherjee
Senior Scientist R&D; Innovation Lab
Tata Consultancy Services
Ph:- 913366367137
Cell:- 9903705285
Mailto: mukherjee.arijit@xxxxxxx
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                     
Business Solutions
                     
Consulting
____________________________________________

From:	Mukherjee Arijit <mukherjee.arijit@xxxxxxx>
To:	condor-users@xxxxxxxxxxx,
Date:	05/10/2013 02:42 PM
Subject:	[HTCondor-users] Cluster/Job not setting up via SOAP calls
Sent by:	htcondor-users-bounces@xxxxxxxxxxx

---

Hi All 

I'm trying to use the Condor SOAP interface by enabling the SOAP parameters
in condor_config (as per Section 4.5.1 of Condor manual). I've also followed
the sequence of beginTransaction(), newCluster(), newJob() followed by
createJobTemplate() and submit(). But the job doesn't seem to be submitted
- I've checked it using commands from the command line. I am using Condor
7.8, Axis2 (which makes the code little bit different from what is available
on the net on various blogs as they used Axis 1) and Java 1.7 on a Ubuntu
11.10 host. Here's the code snippet: 

        String scheddLocation = "http://192.168.161.166:34317";
      String owner = "<my_user_id>";
      String jobFileLocation = "/home/<my user id>/work/dev/CondorWSClient/sample/example.sub";

      try { 
          // Create the service stub
          CondorScheddStub stub = new CondorScheddStub(scheddLocation);

          // Begin a transaction, allow for 60
seconds between calls 
          BeginTransaction bg = new BeginTransaction();
          bg.setDuration(60);
          BeginTransactionResponse bgr = stub.beginTransaction(bg);
          TransactionAndStatus status = bgr.getResponse();
          logger.info("status
code: " + status.getStatus().getCode().getValue());
          Transaction trx = status.getTransaction();
          logger.info("Transaction:
" + trx.getId()); 

          // Get a new cluster for the DAG job.
          NewCluster nc = new NewCluster();
          nc.setTransaction(trx);
          NewClusterResponse ncr = stub.newCluster(nc);
          int clusterId = ncr.getResponse().getInteger();
          logger.info("Cluster
ID: " + clusterId); 

          // Get a new Job ID (aka a ProcId) for
the Condor Job 
          NewJob nj = new NewJob();
          nj.setTransaction(trx);
          nj.setClusterId(clusterId);
          NewJobResponse njr = stub.newJob(nj);
          int jobId = njr.getResponse().getInteger();
          logger.info("Job
ID: " + jobId); 

          // Build the job's ClassAd.
          ClassAdStructAndStatus jobAd = buildJobAd(owner,
jobFileLocation, stub, clusterId, jobId); 
          
          // Now submit the JOB's ClassAd.
          Submit submit = new Submit();
          submit.setClusterId(clusterId);
          submit.setJobId(jobId);
          submit.setTransaction(trx);
          submit.setJobAd(jobAd.getClassAd());
          stub.submit(submit);

          // Commit the transaction.
          CommitTransaction comm = new CommitTransaction();
          comm.setTransaction(trx);
          stub.commitTransaction(comm);

          // Ask the Schedd to kick off the JOB
immediately. 
          RequestReschedule resched = new RequestReschedule();
          stub.requestReschedule(resched);
      } catch (AxisFault ax) { 
          ax.printStackTrace();
      } catch (RemoteException rx) {
          rx.printStackTrace();
      } 

The transaction ID returned by the beginTransaction() call is a negative
value, and changes at every invocation. But the clusterID and jobID are
always 0. There are no exceptions anywhere. 

I've seen people in the mailing list mentioning Aviary - but unfortunately,
we cannot use Aviary right now. In the short term, we need to set up this
SOAP interface, and move to Aviary at a later date.

Can anyone please let me know what am I doing wrong?

Regards 
Arijit 


Arijit Mukherjee
Senior Scientist R&D; Innovation Lab
Tata Consultancy Services
Ph:- 913366367137
Cell:- 9903705285
Mailto: mukherjee.arijit@xxxxxxx
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                     Business
Solutions
                     Consulting
____________________________________________

=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/ _______________________________________________

HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/