This incident has been resolved. All organizations are ingesting and indexing logs.
There is a gap of the data on the organization affected from 5pm PDT 6/15 till 1:30am PDT 6/17. The team will be working on backfilling those. Current estimation is by 6/25.
Posted Jun 17, 2021 - 01:38 PDT
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jun 17, 2021 - 01:32 PDT
Update
We are continuing to work on a fix for this issue.
Posted Jun 16, 2021 - 14:30 PDT
Update
We are continuing to work on a fix for this issue.
Posted Jun 16, 2021 - 12:58 PDT
Update
We are working testing the fix and will be deploying once done. Subset customers are experiencing logs delayed by more than five minutes. No data is being lost at this time, but the most recent data may not be available.
Posted Jun 16, 2021 - 10:36 PDT
Update
We are continuing to work on a fix for this issue.
Posted Jun 16, 2021 - 08:34 PDT
Update
We are continuing to work on a fix for this issue.
Posted Jun 15, 2021 - 21:08 PDT
Update
We are continuing to work on a fix for this issue.
Posted Jun 15, 2021 - 19:37 PDT
Identified
A degradation in the performance of the Splunk Log Observer data ingestion pipeline is causing the processing and storage of logs to be delayed by more than five minutes. No data is being lost at this time, but the most recent data may not be available.