Degraded performance in chat messaging

Incident Report for Ada

Resolved

This incident has been resolved.
Posted 9 months ago. Jul 15, 2024 - 13:10 EDT

Monitoring

A fix has been implemented and we are monitoring the results.
Posted 9 months ago. Jul 15, 2024 - 12:42 EDT

Identified

The issue has been identified and a fix is being implemented.
Posted 9 months ago. Jul 15, 2024 - 12:41 EDT

Update

We are continuing to investigate this issue.
Posted 9 months ago. Jul 15, 2024 - 12:21 EDT

Update

Some users may experience further degradation in bot performance due to the queue spike timing. Our team is continuing to work on the issue.
Posted 9 months ago. Jul 15, 2024 - 11:51 EDT

Update

We're seeing an improvement in bot performance and positive trends in our queue traffic. Chatters may see responses from their respective chatbot again.
Posted 9 months ago. Jul 15, 2024 - 11:27 EDT

Update

We are continuing to investigate this issue.
Posted 9 months ago. Jul 15, 2024 - 11:17 EDT

Update

We are continuing to investigate the issue. Chatters may experience that they are not receiving responses from the chatbot (this can include the initial greeting response).
Posted 9 months ago. Jul 15, 2024 - 11:16 EDT

Investigating

Issue: spikes in our backend queues leading to degraded performance in messaging performance of chats
Who it affects: chatters may experience slower responses from the bot and some messages may not be sent.
Posted 9 months ago. Jul 15, 2024 - 11:06 EDT
This incident affected: APIs, AI Agent, and Scripted Bot.