Mailing List Archives
Authenticated access
|
|
|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Blast job crash !
- Date: Fri, 19 Apr 2013 10:03:25 -0500
- From: Dimitri Maziuk <dmaziuk@xxxxxxxxxxxxx>
- Subject: Re: [HTCondor-users] Blast job crash !
On 4/18/2013 6:45 AM, Romain wrote:
Cotton, Benjamin J <bcotton@...> writes:
It does sound like the NAS can't handle the load.
You're right, it's a problem with the NAS or more precisely with transfer to
the NAS I guess.
...
What can I do to solve the problem ? can I use NFS with others options mount
for better performances ?
The $15 question is how many slots are hitting the share at once.
We blast about 8,000 sequences weekly on a ~40-core cluster. The search
database is custom, > 10GB (I forget exactly how many).
Placing input and output files on a basic linux nfs server (opteron
supermicro w/ decent amount of ram & desktop-level sata drives in
software raid) works fine here. Placing the database on nfs doesn't
work, we rsync the database on each host at the start of the batch.
If your problem is with the transfer of blast output files and you have
<100 jobs running at once, you definitely need a better a nas. If your
problem is with blast reading search database, that is not unexpected,
you need to cache the database locally on each node.
HTH,
Dimitri