[Chtc-users] HTC Gluster and MPI support available again; Read the updated guide for new policies

[chtc-users@xxxxxxxxxxx]

Greetings,

The new HTC Gluster is now available and support for MPI jobs in the HTC system should be working as before. (Please let us know otherwise by writing to chtc@xxxxxxxxxxx)

PLEASE READ OVER THE UPDATED ONLINE GUIDE FOR GLUSTER, TAKING SPECIAL NOTE OF THE FOLLOWING POLICIES:

(really, #1 and #2 have always been true, but we are providing a strong reminder)

1. Users should keep backups of ALL Gluster data (even software) in a non-CHTC location, as the new Gluster is better optimized for near-term performance and will be more susceptible to data loss/corruption, over time.

2. You must immediately remove data from Gluster once a batch of jobs has completed.

3. Transfers for data staging (prior to running jobs) and cleanup (after jobs complete) should ONLY and ALWAYS be performed via the CHTC's new transfer server, transfer00.chtc.wisc.edu (the same goes for the SQUID web proxy).

4. Initial quotas of 10GB and 100 files have been implemented for all user directories in /mnt/gluster. You will need to set up and run up to a few test jobs before requesting more space (via email to chtc@xxxxxxxxxxx) so that a Research Computing Facilitator can confirm your need for Gluster and that you are setting up jobs without violating submission policies.

5. If a single job will need multiple files and/or a directory of files that are large enough to require Gluster, you will need to create a single 'tar' file of these many files, with the aim of having as few files per job as possible in Gluster at any time - perhaps one job-specific tar file plus one job-shared file, per job. (Numerous files in Gluster from any user are the largest contributor to slower performance for all users.)

As a reminder, only individual, job-specific files that are individually larger than 100 MB -OR- job-shared files or output files that are individually greater than 1 GB should ever require Gluster. We will always help users to avoid using Gluster at all, since it's capabilities are more limited than other in-job file delivery mechanisms.

Please let us know if you have any questions as you begin to use the new HTC Gluster by emailing chtc@xxxxxxxxxxx (rather than replying to this email). We are excited to help you.

Cheers,

Your CHTC Team

Lauren Michael -ÂResearch Computing Facilitator,ÂCenter for High Throughput Computing,ÂUniversity of Wisconsin - Madison

Other Affiliations:ÂAdvanced Computing Initiative;ÂWisconsin Institute for Discovery;ÂACI-REF

lmichael@xxxxxxxx,Âwww.tinyurl.com/LMichaelCalendar,ÂDiscovery 2262, 608.316.4430