Affected
Major outage from 5:28 AM to 6:09 AM
Major outage from 5:28 AM to 6:09 AM
Major outage from 5:28 AM to 6:09 AM
- ResolvedResolved
Root Cause Analysis
On 11/18/26 at 11:05 PM CST, the KC1 datacenter experienced a loss of utility power due to an upstream outage. UPS systems immediately carried the full datacenter load. Generator startup was initiated but did not complete before UPS battery capacity was exhausted, resulting in a facility-wide power loss. Utility power was restored at 11:43 PM CST.
The generators successfully started and operated during routine testing and annual maintenance the prior week. During that maintenance, a cold-weather component (block heater) was identified for replacement, and repairs were already in progress. At the time of the outage, extreme cold temperatures and high winds prevented the generators from reaching operating conditions quickly enough to assume the load.
Because the outage resulted in a hard shutdown of datacenter systems, additional time was required after power restoration to safely bring servers and supporting services back online, validate system integrity, and address any startup issues.
Block heater replacement has been expedited, cold-start performance is being re-verified, and additional winter readiness checks are being implemented. - MonitoringMonitoring
KC1 experienced a power outage due to a loss of utility power beginning at 11:05:19 PM CST. UPS systems carried the load until approximately 11:28 PM CST, at which time the UPS failed and on-site generators did not start as expected.
Utility power was restored at 11:43:38 PM CST.
Most servers are back online at this time. We are actively working to bring any remaining offline servers back into service. If your server is still offline, please submit a support ticket at https://tickets.patmos.tech so we can prioritize restoration.
An investigation is underway, and a root cause analysis (RCA) will be provided within the next 24–48 hours.