Ongoing CHTC Gluster semi-outage

[chtc-users@xxxxxxxxxxx]

Greetings CHTC users,

This message is for users of the high throughput (HTC) system who also use the Gluster file system to stage their data.

We've been unable to completely fix the issues that brought down the Gluster file system over the weekend, and it will take some time to fully resolve. In the meantime, we want to give CHTC users the option to try (if they want) to continue running work, rather than shutting down Gluster completely while we work on a long-term solution. Therefore, Gluster was brought back up on Monday and jobs that use Gluster have been allowed to run. Note that some jobs that use Gluster may still fail to access files there. A typical error when this happens looks like:

 "Transport endpoint is not connected"

Thanks for your patience as we work out a long-term solution. If you have longer jobs or will lose significant progress by not having consistent access to Gluster, feel free to reach out to us at chtc@xxxxxxxxxxx to discuss potential solutions.

Best,

Your CHTC team

---------- Forwarded message ---------
From: <chtc-users@xxxxxxxxxxx>
Date: Fri, May 31, 2019 at 5:18 PM
Subject: CHTC Gluster Outage
To: chtc-users@xxxxxxxxxxx <chtc-users@xxxxxxxxxxx>

Hi Everyone,

This message is for users of the high throughput (HTC) system who also use Gluster to stage their data.

The Gluster file system is currently down and will likely remain down for at least the remainder of the weekend. We have no indication that any data has been lost at this point, but jobs that depend on reading from, writing to, or running executables from Gluster will likely see failures due to the outage.

Jobs that do not depend on Gluster should not be impacted by this outage.

Please continue to direct questions to chtc@xxxxxxxxxxx, and have a great weekend!

Cheers,
Your CHTC Team

_______________________________________________
CHTC-users mailing list
CHTC-users@xxxxxxxxxxx
To unsubscribe send an email to:
chtc@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/chtc-users

--

Christina Koch -ÂResearch Computing Facilitator,Â

Center for High Throughput Computing; Open Science Grid + OSG Connect
University of Wisconsin - Madison, Wisconsin Institute for Discovery, (the International Space Station by name, if not in person)

email: ckoch5@xxxxxxxx // phone: (608) 316 - 4041 // calendar: tinyurl.com/ChristinaCHTC