[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Limiting user stdout and stderr in condor jobs
- Date: Tue, 31 Jan 2006 14:29:12 -0500
- From: Bruno Goncalves <bgoncalves@xxxxxxxxx>
- Subject: Re: [Condor-users] Limiting user stdout and stderr in condor jobs
There doesn't seem to be any problem with kernel support for large files...
[bgoncalves@underdark potts]$ cat potts-WL-2.7800000000.tr.17.0 Potts-2.7800000000.tr.17.9 > temp.d
[bgoncalves@underdark potts]$ du -hs
temp.d
7.6G temp.d
[bgoncalves@underdark potts]$
as far as I can tell this is a condor issue... Any ideas anyone?
On 1/31/06, David A. Kotz
<dkotz@xxxxxxxxxxxxx> wrote:If I remember correctly, you have to turn on large file support in your
kernel config, in addition to using a filesystem that supports large
files. It's been a while since I looked at it, though.
- dave
Bruno Goncalves wrote:
> I am using 6.7...
>
> [
bgoncalves@underdark sergei]$ condor_version
> $CondorVersion: 6.7.13 Nov 7 2005 $
> $CondorPlatform: I386-LINUX_RH9 $
> [bgoncalves@underdark sergei]$
>
> could this be an NFS thing?
>
> On 1/30/06, *Matt Hope* <matthew.hope@xxxxxxxxx
> <mailto:matthew.hope@xxxxxxxxx>> wrote:
>
> On 1/30/06, Bruno Goncalves <
bgoncalves@xxxxxxxxx
> <mailto:bgoncalves@xxxxxxxxx>> wrote:
> > Is there any preset limit for this? I've been having problems
> with jobs that
> > write more than 2Gb to stdout/stderr. Is this a Condor problem?
> or some
> > filesystem limit (I was under the impression ext3 was good up to
> 4Gb/file)?
>
> are you running 6.6.x?
>
> if so then you should know that files (or sets of files) > 2Gb are
> totally broken.
>
> workarounds:-
> 1) move to 6.7
> 2) make sure you make 4 Gb* of data in separate files of no more than
> 1.9Gb each :)
> 3) windows vanilla only - ask the condor team nicely they might be
> able to help.
> 4) transfer the output separately pre condor write back
>
> Matt
>
> * I kid you not, 0-2 4-6 8-10 etc. you get the idea
>
> _______________________________________________
> Condor-users mailing list
>
Condor-users@xxxxxxxxxxx <mailto:Condor-users@xxxxxxxxxxx>
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
>
>
>
> --
> *******************************************
> Bruno Miguel Tavares Goncalves, MS
> PhD Candidate
> Emory University
> Department of Physics
> Office No. N117-C
> 400 Dowman Drive
> Atlanta, Georgia 30322
> Homepage: www.bgoncalves.com <http://www.bgoncalves.com
>
> Email: bgoncalves@xxxxxxxxx <mailto:bgoncalves@xxxxxxxxx>
> Phone: (404) 712-2441
> Fax: (404) 727-0873
> *******************************************
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-users
--
*******************************************
Bruno Miguel Tavares Goncalves, MS
PhD Candidate
Emory University
Department of Physics
Office No. N117-C
400 Dowman Drive
Atlanta, Georgia 30322
Homepage: www.bgoncalves.com
Email: bgoncalves@xxxxxxxxx
Phone: (404) 712-2441
Fax: (404) 727-0873
*******************************************