Incident Summary:
At approximately 9:02 a.m. on Monday, September 18th, 2017, the GeorgiaVIEW Online Learning QPROD and XPROD environments experienced a disruption of service due to low CPU. This negatively impacted performance for USG Institutions accessing their instances of D2L hosted on our servers. These issues persisted until approximately 1:53 p.m., when full functionality of the service was restored.

Because we recognize that interruptions of GeorgiaVIEW service impact institutions across the state, we are communicating this post-outage analysis of what occurred and the measures being taken to address the factors resulting in this incident.

  Incident Cause:
During scheduled GeorgiaVIEW maintenance on Friday, September 15, ITS technical staff implemented a new CPU layout recommended by our virtualization vendor for the purpose of optimizing and improving performance of the D2L production servers.  Unfortunately, ITS technical staff made a configuration error during this process that resulted in the allocation of fewer virtual CPU's to QPROD and XPROD than usual, which significantly decreased system performance, the opposite of the intended effect.  

Incident Response Measures:
ITS technical staff corrected the configuration error, leaving the optimized CPU layout in place, which has improved performance as originally planned.  We are also reviewing internal ITS processes to improve testing, improve communications, and reduce incident response times.