Ada's System Status
All Systems Operational

About This Site

This is Ada's status page. This is where you can get updates on how our systems are doing. If there are interruptions to service, we will post about it here.

Don't hesitate to get in touch with us at help@ada.support.

Website ? Operational
90 days ago
100.0 % uptime
Today
Core API ? Operational
90 days ago
99.86 % uptime
Today
Dashboard ? Operational
90 days ago
99.99 % uptime
Today
Chat Client ? Operational
90 days ago
99.78 % uptime
Today
Static Uploads ? Operational
90 days ago
100.0 % uptime
Today
Link Previews ? Operational
90 days ago
100.0 % uptime
Today
Zendesk Integration ? Operational
90 days ago
100.0 % uptime
Today
LivePerson Integration ? Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Past Incidents
May 13, 2021

No incidents reported today.

May 12, 2021

No incidents reported.

May 11, 2021

No incidents reported.

May 10, 2021
Resolved - The issue is currently resolved and we will continue to monitor the situation.
May 10, 17:15 EDT
Monitoring - A fix has been implemented and we are monitoring the results.
May 10, 17:01 EDT
Update - Chats are currently loading at normal speed as normally would. The team will continue to monitor.
May 10, 15:53 EDT
Update - We are continuing to investigate this issue.
May 10, 15:43 EDT
Investigating - Hello Everyone ,

All bots are currently taking longer to load than they normally would. The team is aware of this issue and we are actively working on it. We will update on progress as we resolve this issue.
May 10, 15:43 EDT
May 9, 2021

No incidents reported.

May 8, 2021

No incidents reported.

May 7, 2021

No incidents reported.

May 6, 2021

No incidents reported.

May 5, 2021

No incidents reported.

May 4, 2021

No incidents reported.

May 3, 2021

No incidents reported.

May 2, 2021
Completed - The scheduled maintenance has been completed.
May 2, 04:00 EDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 2, 03:03 EDT
Scheduled - We are completing scheduled maintenance to our Analytics infrastructure. Redshift clusters will be updated by AWS for database maintenance and clusters won't be accessible. This may result in the Analytics page of the Ada dashboard being unavailable during the maintenance window.
Apr 30, 16:22 EDT
May 1, 2021

No incidents reported.

Apr 30, 2021
Resolved - CHATTER EXPERIENCE SUMMARY:

- From 7:20 a.m. to 9:14 a.m. EDT and from 10:40 a.m. to 11:10 a.m. EDT, the bot was unresponsive.

DASHBOARD EXPERIENCE SUMMARY:

- The bot builder dashboard was disabled from 9:50 a.m. and 11:15 a.m. EDT.
- Your analytics may show increased volumes due to duplication of some conversations.
- Transcripts in the Conversations View may not be in chronological order between 9:14 a.m. and 11:10 a.m. EDT.

EVENT SUMMARY:

Starting at around 7:20 a.m. EDT on April 30, 2021 chatters interacting with our chat service began experiencing errors and significant latency.

We identified that a vendor managed database update was under-resourced for the capacity needs of our service. The vendor's support team joined our incident response at 7:38 a.m. EDT to begin increasing database resources. The vendor reported that due to a system limitation on their side, the resources could not be added directly to the existing database. Instead a time consuming database copy would need to be performed by the vendor.

In the interest of bringing chat back online as soon as possible, we decided to begin using a backup database from the previous day. Chat service was restored using our backup database at 9:14 a.m. EDT.

Between 9:50 a.m. and 11:15 a.m. EDT we intentionally limited access to the Ada Dashboard to minimize lost work.

At 10:40 a.m. EDT we paused chat services again to switch back to our primary database.

Chat service was fully functional at 11:10 a.m. EDT.

We are currently working to restore full conversation data to our production database of chats that occured on our backup database.

A visual timeline of the incident is available here https://static.ada.support/outage-timeline-2021-04-30.png .

A full post-event summary including actions taken will be provided next week through your ACX Manager.
Apr 30, 18:19 EDT
Update - We continue to monitor chat services and perform our data sync.

An initial post-event summary will be shared by end of day Eastern Time.
Apr 30, 17:14 EDT
Update - Chat services remain fully functional.

We have reduced the data gap in chat logs to less than two hours. We are continuing efforts to sync this data from our backup database to our primary.

We will post another update by 5:00 PM EDT.
Apr 30, 13:32 EDT
Update - Chat service is fully functional.

We continue to sync data from the past 24 hours. We will post an update within two hours.
Apr 30, 11:20 EDT
Monitoring - Chat service is restored. The database changeover has been completed successfully.

Bot builders may notice that chat logs more recent than 5:00 PM EDT April 29, 2021 are currently not available in their dashboard. We are working to copy this data now from backup to our main database.
Apr 30, 09:21 EDT
Update - The database reprovisioning has been completed and we are performing testing before restoring service. We expect service to be restored shortly.

Another update will be posted within 30 minutes.
Apr 30, 09:04 EDT
Update - We are implementing a database reprovisioning and are in the process of cloning data volumes.

Another update will be posted within 30 minutes.
Apr 30, 08:28 EDT
Identified - The issue has been identified and a fix is being implemented.
Apr 30, 07:54 EDT
Investigating - We are currently investigating this issue.
Apr 30, 07:34 EDT
Completed - The scheduled maintenance has been completed.
Apr 30, 11:15 EDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 30, 10:00 EDT
Scheduled - The Bot Builder dashboard is temporarily disabled as we undergo database maintenance.
Apr 30, 09:50 EDT
Completed - The scheduled maintenance has been completed.
Apr 30, 11:13 EDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 30, 10:40 EDT
Scheduled - We are performing a brief maintenance on our chat service as we sync chat logs from the past few hours.
Apr 30, 10:37 EDT
Apr 29, 2021

No incidents reported.