Alerts are delayed

Incident Report for Splunk Observability Cloud US1

Resolved

This incident has been resolved.
Posted 1 month ago. May 09, 2025 - 01:23 PDT

Monitoring

The system has recovered and we are continuing to monitor
Posted 1 month ago. May 09, 2025 - 01:00 PDT

Update

We are actively implementing backend improvements to address delays in event availability. This multi-step process is expected to take several hours. Engineering teams across regions are coordinating to monitor progress and ensure full functionality is restored. Our next update will be provided by 10:00 AM UTC on May 9, 2025
Posted 1 month ago. May 08, 2025 - 21:35 PDT

Update

We are continuing to work on the issue. Alert notifications are being sent out on time. A small percentage of the events behind those notifications are delayed being created and are not available to see in the user interface.
Posted 1 month ago. May 08, 2025 - 20:39 PDT

Update

Efforts to resolve the issue are ongoing, and we’ll continue to share updates along the way
Posted 1 month ago. May 08, 2025 - 19:51 PDT

Update

We're making steady progress on the fix and will keep you informed as more details emerge
Posted 1 month ago. May 08, 2025 - 18:53 PDT

Update

We are continuing to work on a fix and will provide updates as more information becomes available
Posted 1 month ago. May 08, 2025 - 17:53 PDT

Update

We are continuing to work on a fix for this issue
Posted 1 month ago. May 08, 2025 - 16:41 PDT

Identified

Starting at 10:30a PT some small percentage of alerts may have been delayed by up to 2-3 hours. The root cause is known and a fix is being worked on.
Posted 1 month ago. May 08, 2025 - 15:34 PDT
This incident affected: Alerting.