This incident has been resolved. Load is back to normal levels and all systems have been running well.
Posted Jan 23, 2018 - 10:05 UTC
Monitoring
We've successfully finished the migration of high load web boxes. Load dropped already and we will continue to monitor the situation very closely until tomorrow morning.
Posted Jan 22, 2018 - 17:29 UTC
Update
We identified a few boxes that are causing the high load and are in the process of moving them off the host to a separate host. This already reduced load and we are seeing improvement of the situation.
Posted Jan 22, 2018 - 15:36 UTC
Identified
We identified high read load as the cause of the issue. We are in the process of narrowing it down at the moment.
We also moved off the first web boxes and the situation has been quiet so far.
We are going to update this report again in two hours, or earlier if we have new information.
Posted Jan 22, 2018 - 12:57 UTC
Investigating
We're experiencing load spikes on one of our servers, resulting in impaired application performance for a few of our customers.
We've noticed this first on Friday and the phenomenon kept repeating over the weekend.
We've decided to restore reliable operation by moving applications off the machine, starting with those of Enterprise SLA customers. After stabilising operation, we'll investigate the underlying causes.
We're going to update this report in two hours, or earlier if we have new information.