Re: [Condor-users] tuning a file server

[Steffen Grunewald <steffen.grunewald@xxxxxxxxxx>]

On Tue, Aug 02, 2005 at 12:27:50AM +0100, David McBride wrote:
> Steffen Grunewald wrote:
> 
> Hi,
> 
> >some weeks ago, I had to move our file systems (/home and some data) to 
> >another server, equipped with 3ware 9000 cards, and using 8 400GB disks
> >in RAID5 with hot spare (2.3TB net capacity). To manage the disk space,
> >I had to switch to Linux kernel 2.6 - and there are indications that this
> >move caused some problems.
> 
> 3ware tend to make excellent Linux-compatible RAID solutions, so it is 
> unlikely to be a RAID-level problem, as your basic read benchmark 
> suggested.
> 
> I'd suggest also running the 'bonnie++' benchmark on your filesystem to 
> get a better idea of real-world performance (file creates/erasures, 
> small loads, etc.)

Did that before (still under 2.4.xx), and it didn't show any problems.

> >The file server has to serve 360 VMs in 180 dual-CPU machines, and if 
> >there are lots of small I/O operations, the iowait percentage goes up to
> >more than 80%! I have never seen that before (the previous server was
> >almost identical, kernel 2.4, 1.4TB each).
> 
> Some important missing information:
> 
> * How (if at all) are the fileserver contents exported? NFS?
NFS.

> * What filesystem(s) are you using?
XFS.

> > Now I suspect that the i/o strategies introduced with 2.6 kernels are
> > badly configured... are there any suggestions?
> 
> The 2.6 IO schedulers tend to _better_ than the 2.4 one.  The default is 
> to use the 'anticipatory' scheduler which tends to be excellent for most 
> needs.

I suppose it's anticipating the wrong stuff. If 100+ client tasks start
to send off 24 nfsd threads to work, probably the scheduler gets a bit
confused. I guess that some elevator type would be the better choice?
(Can you point me to some docs?)

Cheers,
 Steffen

-- 
Steffen Grunewald * MPI fuer Gravitationsphysik (Albert-Einstein-Institut)
SciencePark Golm, Am Mühlenberg 1, D-14476 Potsdam * http://www.aei.mpg.de
* e-mail: steffen.grunewald(*)aei.mpg.de * +49-331-567-{fon:7233,fax:7298}
No Word/PPT mails - http://www.gnu.org/philosophy/no-word-attachments.html