User search is failing intermittently
Incident Report for LaunchDarkly
Resolved
The user search cluster has fully recovered to a stable healthy state. We re-enabled persisting user updates more than 8 hours ago, and the search requests remain performant. If you suspect any issues with your account's user data in LaunchDarkly, please contact LaunchDarkly Support.
Posted Aug 25, 2017 - 10:54 PDT
Monitoring
We have restored user search functionality, which initially started failing due to hardware failures. The search cluster is still recovering and rebalancing, so search query performance will suffer during this time.

We have also disabled user saving/updates to allow the search cluster to recover in a swift and stable manner. User updates received by our service during this period will not be available.
Posted Aug 24, 2017 - 15:22 PDT
Update
User search is now consistently failing across the search cluster. We are working to restore user search functionality.
Posted Aug 24, 2017 - 12:59 PDT
Identified
User search is occasionally timing out due to a system failure in our search cluster. We are building a new system to replace the failed server.
Posted Aug 24, 2017 - 09:34 PDT