Re: [HTCondor-users] Issues Running BLAST through HTCondor

[dmitri.maziuk@xxxxxxxxx]

On 2022-03-08 6:44 PM, sameh.abedin@xxxxxxxxx wrote:
> I have 3 hosts that can execute jobs. 1 host has 16 slots, and the other 
> 2 have 8 slots. So, 32 slots total.

How much RAM?

The best way to run blast is to put the databases on a RAM disk and run 
multi-threaded on all available cores, with all of your query sequences 
in the single query file ("all against all"). I.e. not really the way 
condor does it.

I would try setting up whole-machine slots and only running one blast 
instance per machine. It shouldn't be too hard to give it the correct 
"numThreads" argument but you could start with the lowest common 
denominator: 8. Your search DB seems small enough that RAM shouldn't 
matter, but you really need enough RAM to hold it -- or 2GB, whichever's 
smaller -- *per slot* just for blast.

That said, we only ran protein blast and when they switched to the new 
and improved "blast+" it started taking 10 times longer which 
effectively killed our pipeline. So I'm not sure what's happening with 
your setup, exactly. Not enough RAM would be my first suspect if you 
were searching nr or nt, but with your database I don't know.

Dima