#incident-management

#incident-management

An observability control plane isn't just a dashboard. It's the operational authority system. It defines alert rules, routing, ownership, escalation policy, and notification endpoints. When that layer is wrong, the impact is immediate. The wrong team gets paged. The right team never hears about the incident. Your service level indicators look clean while production burns.

PagerDuty, the incident management platform used by thousands of organisations to alert them to problems on their systems, suffered a major outage itself on 28th August 2025. The incident disrupted or delayed the processing of incoming events to customers in PagerDuty's US service region. Significant service degradations affected PagerDuty for more than nine hours. At its peak, approximately 95% of events were rejected over a 38-minute period, and 18% of create requests generated errors for 130 minutes.

"Causely’s integration of Grafana dashboards enhances root cause analysis for DevOps teams, providing better visibility and actionable intelligence in IT workflows."

Most businesses still treat crisis response like it's 2015. A ransomware alert goes out. Emails fly. Group chats explode. Someone digs out the playbook.

As part of our proactive management of the incident, we took a decision to take some of our systems temporarily offline.

“As part of our proactive management of a cyber incident, we have made the decision to pause taking orders via our M&S.com websites and apps.”