{
  "is": "issue",
  "title": "Dashboard control loop issue",
  "body": "\u003cp\u003e\u003cem\u003eFixed\u003c/em\u003e - The dashboard and API are back online.\u003c/p\u003e\n\u003cp\u003eThe problem was caused by human error by one of our team while using our admin shell to make a mass update to a single customer\u0026rsquo;s websites during a migration. This kicked off a massive cascade of events which overloaded the control loop.\u003c/p\u003e\n\u003cp\u003eThis was strictly an admin error - the WebFaction migration tool that many customers are using is safe to use.\u003c/p\u003e\n\u003cp\u003eWe\u0026rsquo;ve rolled back the updates and have cleared the pending updates from the control loop and all systems are operating normally at this time.\u003c/p\u003e\n\u003cp\u003eThis issue affected Apache/PHP-CGI applications in the dashboard. We believe there should not be any lingering issues but if you notice a problem with any of your Apache/PHP-CGI applications then please try an edit+save cycle on the application and wait a minute for the changes to take effect. If the problem persists then please contact Opalstack Support for further assistance. \n  \u003cspan class=\"faded\"\u003e(02:12 UTC — Aug 8)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eIdentified\u003c/em\u003e - We\u0026rsquo;ve taken the dashboard offline while we work to resolve this. \n  \u003cspan class=\"faded\"\u003e(01:05 UTC — Aug 8)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003eAt this time there is an issue with our control loop which deploys changes from the dashboard to our servers. Dashboard changes will remain pending until the issue is resolved. We\u0026rsquo;ll update this post as soon as possible. \n  \u003cspan class=\"faded\"\u003e(00:57 UTC — Aug 8)\u003c/span\u003e\n\n\u003c/p\u003e\n",
  "createdAt": "2021-08-08 00:57:00 +0000 UTC",
  "lastMod": "2021-08-08 00:57:00 +0000 UTC",
  "permalink": "https://opalstackstatus.com/issues/2021-08-08-dashboard-control-loop-issue/",
  "severity": "down",
  "resolved": true,
  "informational": false,
  "resolvedAt": "2021-08-08 02:12:46",
  "affected": ["API", "Control panel"],
  "filename": "2021-08-08-dashboard-control-loop-issue.md"
}