We have replayed all the data. All Experimentation metrics should be accurate at this point.
Posted Dec 16, 2019 - 17:20 PST
Identified
The change to resolve the issue with Experimentation was not effective. We are continuing to investigate a remediation to the issue.
Expected Impact: Customers utilizing the Experimentation functionality will see abnormal metrics starting from 9:45 am PST this morning. These metrics will return to normal as we replay the dataset. We will advise as the situation progresses.
Posted Dec 16, 2019 - 16:34 PST
Monitoring
We have identified the issue and have pushed a fix. New data from 1320 PST forward will appear in experimentation metrics. We are currently replaying the dataset between 0945 and 1320 PST and expect to recover shortly.
Posted Dec 16, 2019 - 13:50 PST
Investigating
We’ve identified errors while processing experimentation events. Starting at 9:45am PST, experiment results will not reflect new impression data.
We are working to resolve now.
Experiment event data is still being collected and experiment results will catch up once these errors are corrected.