All systems are now functioning correctly. There may be some activity feed items slow to show up. Please log a support ticket if you see any issues from here. We are sorry for this issue and will be demanding a full explanation from IBM which we can share with you.
Posted Jun 10, 2020 - 01:25 UTC
Monitoring
Requests to IBM Cloud are now completing correctly and our system is recovering. There may still be issues with API calls queueing the requests (but they will be completed) and journey processing is still catching up on queued tasks. We are continuing to monitor the connection quality and will monitor journey processing catching up. We will update again once things are fully up to date.
Posted Jun 10, 2020 - 00:54 UTC
Update
We are starting to see our requests go through again and task processing is resuming. We are now reviewing all of our services to make sure they are working and will provide a further update soon once we can confirm when things are normal again. Since we've had no update from IBM Cloud, we can't be certain that things won't relapse, so are not declaring that things are fixed just yet.
Posted Jun 10, 2020 - 00:26 UTC
Update
We have received an update IBM Cloud. Unfortunately it provides no new information at all:
"Network Specialists were informed of multiple accounts of users not being able to access VPN and other IBM Cloud services beginning at approximately 22:00 UTC 09 JUN 2020. Network Specialist are actively investigating the cause at this time." As of now, we do not have an update regarding an ETA on resolution, however we will continue to monitor this situation and relay updates as they become available.
Posted Jun 10, 2020 - 00:09 UTC
Update
I'm sorry that we can't provide further updates at this time. IBM is yet to comment publicly in any way on their entire cloud being down. Our regular contacts assure us it is being worked on, but it is unlikely they have any direct influence on the entire IBM Cloud and we await their updates.
Posted Jun 09, 2020 - 23:49 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Jun 09, 2020 - 23:20 UTC
Update
Our database provider, IBM Cloud, told us the following privately "This is what appears to be a networking issue affecting a large portion of IBM Cloud services."
Posted Jun 09, 2020 - 23:18 UTC
Update
IBM Cloud is experiencing a worldwide cloud outage. They have not commented publicly despite being down for nearly an hour. Additionally, their status page is offline due to being inexplicably hosted in the same Cloud.
Our actual database provider has said they are "all hands on deck", but have not provided an ETA for when things will be back online.
Posted Jun 09, 2020 - 23:03 UTC
Update
The whole of IBM Cloud is not responding. We await updates from IBM as to how they will fix it.
Our provider's initial response is "There appears to be a networking issue with our hardware provider. We expect to have more information shortly."
Posted Jun 09, 2020 - 22:17 UTC
Update
We are continuing to investigate this issue.
Posted Jun 09, 2020 - 22:08 UTC
Investigating
We are in touch with our database provider to fix an outage on their end. Will update as soon as we have more information.
Posted Jun 09, 2020 - 22:08 UTC
This incident affected: Application, API, and Journey task processing.