A fix has been deployed and the Ada team will continue to monitor.
Posted Sep 06, 2023 - 16:15 EDT
Update
We are continuing to monitor for any further issues.
Posted Sep 06, 2023 - 16:02 EDT
Update
Some users have reported issues loading the admin dashboard (eg. Reports tab, build tab). The team is investigating the cause of these reports. Front-end chatters seem unaffected at the moment.
Posted Sep 06, 2023 - 15:06 EDT
Update
The push to switchover the pod has been released. Chats will return to normal as users pull the latest version in their session.
Posted Sep 06, 2023 - 14:09 EDT
Update
Our team continues to work on the required changes.
Posted Sep 06, 2023 - 13:56 EDT
Update
The affected service has reported a fix for the affected bot. Our team continues to work on the switchover to ensure a more stable pod is used while we confirm the affected pod is fully fixed.
Posted Sep 06, 2023 - 13:27 EDT
Update
The team is continuing work on the service pod switch.
Posted Sep 06, 2023 - 12:57 EDT
Update
The team has identified a drop in performance. Additional work is being done to direct traffic to more stable pods.
Posted Sep 06, 2023 - 12:26 EDT
Monitoring
Connection rates have returned to acceptable parameters. Customers may experience some latency as the affected services are working through the current workload and the team continues to monitor the issue.
Posted Sep 06, 2023 - 11:58 EDT
Update
We are continuing to work on a fix for this issue.
Posted Sep 06, 2023 - 11:55 EDT
Update
We are continuing to work on the fix for the issue. There are improvements in the incident rate so some customers may see fewer instances of the issue with the chat widget.
Posted Sep 06, 2023 - 11:41 EDT
Identified
Some customers are experiencing issues with our chat services. This is affecting some chatters interacting with Ada bots. The issue has been identified as the result of a third party websocket failure and our technical team is actively working with their representatives to resolve this and monitor their implemented solution.
Posted Sep 06, 2023 - 11:10 EDT
This incident affected: Ada Dashboard and Ada Web Chat.