Messaging API Subscription Failures
Incident Report for 75F
Postmortem

A sudden burst of reconnections in the Messaging API exposed a thread lock management issue, which spread to all active connections and prevented them from reconnecting and receiving messages. This temporarily prevented all Messaging clients from receiving messages.

A simple app restart kicked the service back into a healthy state, but the team is still closely monitoring it while the root cause is addressed.

Posted Jul 26, 2023 - 10:36 CDT

Resolved
The Messaging API has entered an unhealthy state and is failing to connect and deliver messages to subscribers
Posted Jul 26, 2023 - 10:00 CDT