Re: [HTCondor-users] Condor Win!

[Mukherjee Arijit <mukherjee.arijit@xxxxxxx>]

Hi George

That's exactly the same thing we did,
but we assumed that RScript is already installed on the nodes at /usr/bin.
Thanx for confirming :-)

Regards
Arijit

Arijit Mukherjee
Senior Scientist R&D; Innovation Lab
Tata Consultancy Services
Ph:- 913366367137
Cell:- 9903705285
Mailto: mukherjee.arijit@xxxxxxx
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                
       Business Solutions
                
       Consulting
____________________________________________



From:      
 "Dunn, George
Jr" <dunng@xxxxxxxx>
To:      
 HTCondor-Users Mail
List <htcondor-users@xxxxxxxxxxx>, 
Date:      
 06/26/2013 09:51 PM
Subject:    
   Re: [HTCondor-users]
Condor Win!
Sent by:    
   htcondor-users-bounces@xxxxxxxxxxx

---

Hi Arijit,
 
The lab machines that
we are marshaling are windows machines whose primary functions are instruction
during the day.
 
I have read where folks
were running the R binaries from a network share. For this I just created
a silent install script to install R and the necessary packages on each
machine.
 
I created a simple .bat
file:
(note not sure if the
TMP, TEMP, USERPROFILE settings are necessary but I saw it on some matlab
examples and figured it could not hurt)
 
set R_SCRIPT="C:\Program
Files\R\R-2.15.1\bin\RScript.exe"
set TMP=%CD%
set TEMP=%CD%
set USERPROFILE=%CD%
%R_SCRIPT%  %1 %2
%3 %4  > %2_%3-%4.txt 2>&1
 
The first parameter is
the R script itself, the second is the data file the script was using.
The third and fourth were parameters associated with the partitioning used
to get the runtime down to 12 hours or so.
 
The R script generated
another .RData with the output for the job with a similar naming convention
as the above std out std err redirection file.
This was done so that
there would be uniqueness in the files as 1000’s were being returned to
a single directory.
 
Another script was written
to generate the appropriate lines in the form of a condor_submit file to
queue them all up!
 
Hope this helps!
 
Eddie
 
From: htcondor-users-bounces@xxxxxxxxxxx
[mailto:htcondor-users-bounces@xxxxxxxxxxx]
On Behalf Of Mukherjee Arijit
Sent: Wednesday, June 26, 2013 2:44 AM
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] Condor Win!
 
That's great.

Just one quick question - how did you submit the R jobs to Condor? We have
done a similar thing (but not in the scale you have mentioned) using /usr/bin/rscript
on the cluster nodes. 

Regards 
Arijit 

Arijit Mukherjee
Senior Scientist R&D; Innovation Lab
Tata Consultancy Services
Ph:- 913366367137
Cell:- 9903705285
Mailto: mukherjee.arijit@xxxxxxx
Website: http://www.tcs.com
____________________________________________
Experience certainty.        IT Services
                    
  Business Solutions
                    
  Consulting
____________________________________________



From:        "Dunn,
George Jr" <dunng@xxxxxxxx>
To:        "htcondor-users@xxxxxxxxxxx"
<htcondor-users@xxxxxxxxxxx>,
Date:        06/25/2013
08:31 PM 
Subject:        [HTCondor-users]
Condor Win! 
Sent by:        htcondor-users-bounces@xxxxxxxxxxx

---

Hi All, 
  
Just wanted to share a little success story.
  
We are a small public university that has been hit hard in both human and
budget resources in recent years (as many schools have). We do have a research
computing group but overall our IT is stretched so thin that most all of
those folks are on fire duty. A group in our Statistics department was
up against a runtime wall trying to run some R jobs in VM’s. There were
a total of 8000 jobs that each took 12 hours  to complete (with unfettered
access to a single core). By marshaling a few of our labs we were able
to run over 1100 simultaneous jobs on 175 ish machines (mostly i7’s a
few i5’s) even through several electrical storms that rebooted all the
machines and caused all manner of networking errors IN TIME! She is going
to add me to the Author list for the paper and we will put a blurb about
Condor. 
  
I hope this will be a proof of concept for a larger roll-out. I also plan
to replace our torque scheduler for ROCKS with this for our mpi jobs.
  
Thank you for such a polished and stable product. I am a fan for life!!!
  
Thanks, 
Eddie Dunn 
Systems Administrator 
Department of Computer Science
University of North Carolina Wilmington_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/