Incident Summary:

At approximately 9:45 a.m. on Saturday, March 4th, 2017, the GeorgiaVIEW Online Learning XPROD environment experienced a disruption of service due to disk space. This prevented the following USG Institutions from accessing their instances of D2L hosted on the server:

-Abraham Baldwin Agricultural College
-Armstrong State University
-Bainbridge State College
-Clayton State University
-College of Coastal Georgia
-Columbus State University
-Competency Based Education
-Dalton State College
-East Georgia State College
-Fort Valley State University
-Georgia Online Virtual Instruction Enterprise Wide (GoVIEW)
-Georgia Southwestern State University
-Georgia State University
-Middle Georgia State University
-University of North Georgia
-University of West Georgia
-USG Human Resources Office of Organizational Development
-USG Strategic Initiatives (SREB, RESA, AMAC, OUTREACH Best Practices)
-Valdosta State University

These issues persisted until approximately 10:10 a.m., when full functionality of the service was restored.

Because we recognize that interruptions of service impact institutions across the state, we are communicating this post-outage analysis of what occurred and the measures being taken to address the factors resulting in this incident.

Incident Cause:

Upon investigation, ITS technical staff determined there was an increase in the rate at which our logs were writing to the XPROD database. Although there are monitoring alerts in place, this rapid increase caused the database to run out of space before steps could be taken to prevent the outage.

Incident Response Measures:

ITS staff continue to investigate the root cause of the rate increase. In the meantime, we will be adjusting our monitoring alerts to notify us sooner in order to reduce the risk of future outages.