Opalstack - Notice history

At the present time mail is not being sent by cron jobs on our web servers. We're looking into the cause.

Opal4 is not responding. We're looking into the cause.

Apache was once again not responding on Opal4 after hitting its configured ServerLimit. The underlying cause appears to have been a high number of PHP-FPM processes being re-spawned for one customer site. We've adjusted the PHP-FPM configuration for that site to avoid the problem in the future.

The Apache service on Opal4 was not responding for approximately 1 hour today beginning 18:23 UTC.

The problem was that the Apache service had hit its configured ServerLimit which meant that no further Apache processes could spawn to handle requests. That has been corrected and the service is online now.

We're now investigating the root cause and also looking into why our own monitoring systems did not alert us to this condition.

Several servers in our Frankfurt DE datacenter have experienced network connectivity issues.

The data center team reports that the problem is due to high temperatures in the DC. Their investigation into the root cause is ongoing.

At this time all servers are online. We'll update this post when more information is available.

Opal4 experienced a brief spike in system load between 23:40 and 23:50 UTC on Friday 10 April 2020.

The cause appears to have been high CPU usage due to a SYN flood attack.

The server is operating normally at this time. We'll continue to monitor.