System latency and app outage

Incident Report for Autopilot Journeys

Resolved

We have finished processing the backlog of tasks. Thank you for your patience.
Posted Apr 04, 2019 - 12:42 UTC

Update

We are continuing to make good progress working through the backlog of tasks.
Posted Apr 04, 2019 - 00:43 UTC

Update

We are continuing to work through the backlog of tasks.
Posted Apr 03, 2019 - 21:40 UTC

Update

Performance continues to be stable and we are continuing to work through the backlog of tasks.
Posted Apr 03, 2019 - 19:13 UTC

Monitoring

All users can now login and we are continuing to work through the backlog of journey and API tasks.
Posted Apr 03, 2019 - 18:15 UTC

Update

Most users are able to log in again, however performance may still be degraded. We are starting to work our way through the backlog of tasks. Our database provider is continuing to work to improve performance.
Posted Apr 03, 2019 - 17:23 UTC

Update

Until this is resolved we are queueing journeys tasks (e.g. emails) but not yet processing them. They will be processed as a priority once this incident is resolved.
Posted Apr 03, 2019 - 16:08 UTC

Update

We are sorry that we don't have any further information at this time regarding this incident. It remains our database provider's top priority to resolve this.
Posted Apr 03, 2019 - 15:37 UTC

Update

Our database provider is continuing to investigate this as a top priority.
Posted Apr 03, 2019 - 14:48 UTC

Update

Our database provider is continuing to investigate this as a top priority. Unfortunately we still don't have any new information to share, but please rest assured that we are doing absolutely everything in our power to ensure this situation is resolved.
Posted Apr 03, 2019 - 14:02 UTC

Identified

Our provider needs to take some further action to resolve this which unfortunately means some users may experience app unavailability while they do so. We are continuing to queue all tasks in the meantime and will process them as soon as possible. We are again very sorry for this experience.
Posted Apr 03, 2019 - 13:19 UTC

Monitoring

Action has been taken to address the root cause and performance is improving. We are working through the backlog of queued tasks as a priority.
Posted Apr 03, 2019 - 12:58 UTC

Update

Our database provider is continuing to investigate this as a top priority and we will share new information the moment we have it.
Posted Apr 03, 2019 - 11:22 UTC

Update

Our database provider is continuing to investigate this as a top priority and we will share new information the moment we have it. We are very sorry for the continued disruption. We are continuing to queue all journey tasks and will process them as soon as possible.
Posted Apr 03, 2019 - 09:46 UTC

Update

Our database provider is continuing to investigate this as a top priority and we will share new information the moment we have it. We are very sorry for the continued disruption.
Posted Apr 03, 2019 - 08:43 UTC

Update

We are continuing to work with our database provider to restore system performance. We again apologize for the inconvenience this is causing.
Posted Apr 03, 2019 - 07:09 UTC

Update

Our database provider is continuing to work on resolving the latency as a priority.
Posted Apr 03, 2019 - 06:17 UTC

Update

Our database provider is continuing to work on resolving the latency as a priority.
Posted Apr 03, 2019 - 05:51 UTC

Update

We sincerely apologize for the inconvenience caused. Our database provider is continuing to work on resolving the latency as a priority. All journey tasks are being queued and will be processed once this incident is resolved.
Posted Apr 03, 2019 - 05:24 UTC

Update

Our database provider is continuing to work on resolving the latency as a priority.
Posted Apr 03, 2019 - 04:57 UTC

Update

Our database provider is continuing to work on resolving the latency as a priority. Some accounts are temporarily unavailable.
Posted Apr 03, 2019 - 04:41 UTC

Identified

We are experiencing database latency and will provide more details shortly.
Posted Apr 03, 2019 - 04:28 UTC
This incident affected: Application, API, and Journey task processing.