Elevated web app and API error rate and increased latency

Incident Report for LaunchDarkly

Resolved

The issue with the LaunchDarkly service has been resolved. Impacted services have returned to normal operation.
Posted Feb 21, 2024 - 19:01 PST

Monitoring

The LaunchDarkly web application and the Flag Delivery Network have recovered and are operating normally.

We will continue to monitor the service.
Posted Feb 21, 2024 - 17:20 PST

Update

We have made progress in addressing the problem with a production database node that is causing the increased error rate and connection errors, but the problem is not yet mitigated.

The LaunchDarkly Web Application remains unavailable and Client and Serverside SDKS experiencing high error rates when attempting to initialize new connections.

Next update in 30m.
Posted Feb 21, 2024 - 16:58 PST

Identified

We have identified a problem with a production database node that is causing the increased error rate and connection errors. The LaunchDarkly Web Application remains unavailable and Client and Serverside SDKS experiencing high error rates when attempting to initialize new connections.

We are attempting to mitigate.

Next update in 30m.
Posted Feb 21, 2024 - 16:24 PST

Update

We are seeing high error rates in both the LaunchDarkly web application and in the Flag Delivery Network.

Both client-side and server-side SDKS will experiencing errors when initializing new connections.
Posted Feb 21, 2024 - 15:50 PST

Update

We see escalating problems with customers access the LaunchDarkly web application.

Flag stream initialization is seeing elevated error rates as well. LaunchDarkly SDKs will continue to retry until they can connect.
Posted Feb 21, 2024 - 15:08 PST

Update

We are continuing to investigate this issue.
Posted Feb 21, 2024 - 14:45 PST

Investigating

We are investigating elevated web app and API errors and increased latency. Please continue to monitor this page for updates.
Posted Feb 21, 2024 - 13:57 PST
This incident affected: Flag Delivery Network (core functionality) (Server-side streaming API, Client-side streaming API, Polling API), Feature management (core functionality) (Feature workflows, Flag targeting, Segment management), Experimentation (core functionality) (Experiment management), and Global (core functionality) (Authentication, Metrics).