{
  "is": "issue",
  "title": "Ongoing DNS DDOS Attack",
  "body": "\u003cp\u003e\u003cem\u003eFixed\u003c/em\u003e - Post-mortem: this was a DNS flood attack which targeted all 9 of our regional name servers. The attack originated from several cloud provider subnets, including Google and Cloudflare.\u003c/p\u003e\n\u003cp\u003eTo mitigate the attack we had to temporarily block the provider subnets. This had some unintended and undesirable side effects:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe subnets included public DNS resolver infrastructure run by Google and Cloudflare, so blocking them prevented those resolvers from looking up DNS records on Opalstack name servers. As a result, people who use those resolvers were temporarily unable to access opalstack.com (including this status page) and customer domains that use our name servers.\u003c/li\u003e\n\u003cli\u003eThe block also prevented Google from looking up DNS records related to email such as SPF and DKIM. As a result, mail sent (or forwarded) to Gmail from Opalstack\u0026rsquo;s mail system was rejected by Google while the block was in place.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe block was lifted over 24 hours ago and there have been no further issues since that time.\u003c/p\u003e\n\u003cp\u003eGoing forward, we\u0026rsquo;ll refine our mitigation techniques to avoid blocking major providers when possible. We\u0026rsquo;ll also move this status page to an externally-hosted domain to ensure that system status updates will be available regardless of the state of our infrastructure. \n  \u003cspan class=\"faded\"\u003e(16:40 UTC — Dec 30)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eFixed\u003c/em\u003e - We\u0026rsquo;ve seen no further issues in the past several hours and consider this to be resolved. \n  \u003cspan class=\"faded\"\u003e(21:18 UTC — Dec 29)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eWatching\u003c/em\u003e - The attack has subsided, we\u0026rsquo;ll continue to monitor. \n  \u003cspan class=\"faded\"\u003e(14:27 UTC — Dec 29)\u003c/span\u003e\n\n\u003c/p\u003e\n\u003cp\u003eOver the last few hours we\u0026rsquo;ve identified and mitigated an ongoing DDOS attack against our DNS infrastructure. This attack has also effected our Singapore and German DNS servers.\u003c/p\u003e\n\u003cp\u003eWe\u0026rsquo;re continuing to monitor the attack. \n  \u003cspan class=\"faded\"\u003e(12:00 UTC — Dec 29)\u003c/span\u003e\n\n\u003c/p\u003e\n",
  "createdAt": "2022-12-29 12:00:00 +0000 UTC",
  "lastMod": "2022-12-29 12:00:00 +0000 UTC",
  "permalink": "https://opalstackstatus.com/issues/2022-12-29-ongoing-dns-ddos-attack/",
  "severity": "down",
  "resolved": true,
  "informational": false,
  "resolvedAt": "2022-12-30 16:40:56",
  "affected": ["USA"],
  "filename": "2022-12-29-ongoing-dns-ddos-attack.md"
}