Splunk APM Trace Monitoring, Troubleshooting and Trace Ingestion MetricSets Delayed
Incident Report for Splunk Observability Cloud US1
Resolved
This incident has been resolved.
Posted Sep 30, 2024 - 12:04 PDT
Investigating
A degradation in the performance of the Splunk APM trace processing pipeline is causing Troubleshooting MetricSets to be delayed by more than five minutes. As a result, the APM Troubleshooting experience, service maps and Tag Spotlight do not have access to the most recent data.

A degradation in the performance of the Splunk APM metrics processing pipeline is causing Monitoring MetricSets to be delayed by more than five minutes. Service, endpoint, and workflow dashboards, and other charts and detectors built from Monitoring MetricSets are impacted.

A degradation in the performance of the Splunk APM data ingestion pipeline is causing the processing and storage of raw trace data to be delayed by more than five minutes.

The processing of metrics for Business Workflows, which also depends on this pipeline, are equally delayed. Trace data ingest is not impacted at this time; service-level and endpoint-level Monitoring MetricSets and the detectors built from them are also not impacted.
Posted Sep 30, 2024 - 11:28 PDT
This incident affected: Splunk APM (Splunk APM Monitoring MetricSets, Splunk APM Troubleshooting MetricSets, Splunk APM Trace Data, Splunk APM Tag Spotlight, Splunk APM Business Workflows).