[Condor-users] Simultaneous writes to post-script in DAGMan

[Kshitij Sanghi <kshitij_sanghi@xxxxxxxxx>]

Hi,

I am using DAGMan to submit 300,000 jobs. The CONDOR
uses an NFS. I was planning to run a PERL script to
delete the unnecessary files after each node has
completed execution as I have limited space on my NFS.
I can do so for the log, err, etc files but there is
this one result file from which I want to extract
relevant output data.

What I wanted to know is that whether it is possible
to include in the post script a piece of code to
extract information from these files and write it to a
single file. In such a situation I would not have to
maintain 300,000 of these result files but just a
single file with relevant information. Basically I
wanted to know whether such a thing would be possible
without any chances of conflicts due to simultaneous
writes to the same file on the NFS.

In case such a thing is not possible would it be
possible to extract the relevant information from
these files and populate a MySQL database in the post
script itself?

Thanks and Regards,
Kshitij

=====
I've never met a man so ignorant that I couldn't learn something from him.
                                                                                         -Galileo Galilie


	
	
		
___________________________________________________________ 
ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com