Servers: opal7.opalstack.com
Fixed - The MariaDB recovery is complete and by all indications customer sites should be working normally at this time. If you have a site hosted on opal7 and notice any problems then please contact our support team for further assistance. (15:13 UTC — Dec 7)
Identified - At this time we are rebooting opal7. The recovery is nearly complete. (14:29 UTC — Dec 7)
Identified - The restoration process is 80% complete. (13:37 UTC — Dec 7)
Identified - Databases are still restoring. (13:15 UTC — Dec 7)
Identified - Databases are continuing to be restored. We do not have an ETA on when the full restore will be finished but we’ll keep this post updated. (12:11 UTC — Dec 7)
Identified - The recovery of MariaDB has been unsuccessful through all of the recovery methods. We will begin restoring the databases from latest backup.
The latest backup was taken ~4 hours before the MariaDB server went offline. (11:13 UTC — Dec 7)
Identified - The InnoDB transaction log has been corrupted. We’re working to restore service by running in recovery mode. (10:54 UTC — Dec 7)
The MariaDB server on OPAL7 has stopped responding to queries. We’re investigating this failure now. (10:16 UTC — Dec 7)