All systems have recovered and impacted is fully mitigated.
Posted Aug 30, 2023 - 13:45 PDT
Monitoring
Our systems are actively recovering.
- the LaunchDarkly web application and REST API health has returned to normal - the Flag Delivery Network has a 2% error rate connecting for new server-side streaming connections, however, our SDKs are configured to retry and retried connections are likely to succeed. - the Flag Delivery Network's client-side streaming services and polling services have recovered - the event pipeline has recovered
The team is continuing to monitor, investigate, and respond.
Posted Aug 30, 2023 - 12:40 PDT
Update
We are continuing to work on a fix for this issue.
Posted Aug 30, 2023 - 12:17 PDT
Identified
We’re investigating a major outage to LaunchDarkly systems. Currently identified impact includes:
- the LaunchDarkly web application and REST APIs have elevated error rates - the Flag Delivery Network has elevated error rates for streaming and polling SDK requests - the event pipeline is dropping a small portion of inbound events
The system is showing possible signs of recovering.
Posted Aug 30, 2023 - 12:16 PDT
Investigating
We are investigating a potential service interruption to the LaunchDarkly web app and Flag Delivery Network. Please continue to monitor this page for updates.
Posted Aug 30, 2023 - 11:34 PDT
This incident affected: Flag Delivery Network (core functionality) (Server-side streaming API, Client-side streaming API, Polling API, Edge API), Feature management (core functionality) (Feature workflows, Flag targeting), and Global (core functionality) (Web app (app.launchdarkly.com)).