{
  "is": "issue",
  "title": "OPAL4 MariaDB Outage",
  "body": "\u003cp\u003e\u003cem\u003eFixed\u003c/em\u003e - We\u0026rsquo;ve seen no further problems since our last update. \n  \u003cspan class=\"faded\"\u003e(19:11 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eWatching\u003c/em\u003e - The issue with database permissions is resolved and the MariaDB service is back online and operating normally. We will continue to monitor. \n  \u003cspan class=\"faded\"\u003e(14:22 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eIdentified\u003c/em\u003e - We\u0026rsquo;ve discovered that there is still a problem with database permissions. We\u0026rsquo;re working to resolve that at this time. \n  \u003cspan class=\"faded\"\u003e(13:29 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFixed\u003c/em\u003e - The data restoration has completed and we are continuing to monitor the server as usage increases. \n  \u003cspan class=\"faded\"\u003e(12:29 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eIdentified\u003c/em\u003e - The restoration is approximately 75% complete. \n  \u003cspan class=\"faded\"\u003e(11:24 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eIdentified\u003c/em\u003e - The data restoration is approximately 50% complete. \n  \u003cspan class=\"faded\"\u003e(10:32 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eIdentified\u003c/em\u003e - The MariaDB restore process is still running. \n  \u003cspan class=\"faded\"\u003e(09:41 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eIdentified\u003c/em\u003e - The database corruption is unrecoverable so we will proceed with restoring from backups. The most recent valid backups were from 1.5 hours before the server crashed so we\u0026rsquo;ll proceed with the restore from there.\u003c/p\u003e\n\u003cp\u003eThe restoration process varies in completion time. It may take up to 6 hours or more. We\u0026rsquo;ll keep this post updated. \n  \u003cspan class=\"faded\"\u003e(08:37 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003eMariaDB on OPAL4 has crashed, we\u0026rsquo;re working to identify the cause and solution now. \n  \u003cspan class=\"faded\"\u003e(08:12 UTC — Mar 9)\u003c/span\u003e\n\n\u003c/p\u003e\n",
  "createdAt": "2025-03-09 08:12:00 +0000 UTC",
  "lastMod": "2025-03-09 08:12:00 +0000 UTC",
  "permalink": "https://opalstackstatus.com/issues/2025-03-09-opal4-mariadb-outage/",
  "severity": "down",
  "resolved": true,
  "informational": false,
  "resolvedAt": "2025-03-09 19:11:43",
  "affected": ["Shared: Dallas (USA)"],
  "filename": "2025-03-09-opal4-mariadb-outage.md"
}