Opalstack - Notice history

All systems operational

Notice history

May 2021

Unsecured outgoing SMTP connections from web servers to be blocked beginning on 29 May 2021
  • Completed
    May 29, 2021 at 12:20 AM
    Completed
    May 29, 2021 at 12:20 AM
    The maintenance is complete.
  • In progress
    May 29, 2021 at 12:00 AM
    In progress
    May 29, 2021 at 12:00 AM

    Maintenance is now in progress

  • Planned
    May 29, 2021 at 12:00 AM
    Planned
    May 29, 2021 at 12:00 AM

    At 00:00 UTC on 29 May 2021 we will begin blocking unsecured outgoing SMTP connections from Opalstack web servers by blocking outgoing port 25 in our network firewalls.

    If you are running an application on an Opalstack web server that makes SMTP connections to external SMTP servers then please ensure that your application is configured to connect securely over port 465 or port 587.

    Applications that use the native PHP mail function and other apps that use sendmail directly on the server will not be affected by this change.

    2021-05-29 00:20 UTC The new firewall rules are now in effect on all servers.

Network outage affecting domains that use Opalstack DE name servers
  • Resolved
    Resolved

    All DE services are back online.

  • Monitoring
    Monitoring

    Service has been partially restored. The remaining affected servers are ns2.de.opalstack.com and 5 private customer VPS.

    We'll continue to monitor and update as the service is restored.

  • Investigating
    Investigating

    Our DE name servers, SMTP, and several customer VPS located in Frankfurt, Germany are currently offline due to a connectivity issue with one of our regional providers there.

    They're working to restore service. In the interim, we are relocating our DE nameservers to new locations.

    Since the DE name servers are affected, this is unfortunately affecting all domains that use Opalstack's DE name servers. We're hopeful that service will be restored quickly but if it is not then you may want to switch the NS for your domain to one of our other regional sets of name servers. Our regional NS are listed at: What are your name servers?

    We'll update this post as soon as more information is available.

opal1 not responding
  • Resolved
    Resolved

    The problem was caused by a spike in system load due to a problem with a customer site. Conditions have returned to normal and we'll work with that customer to prevent further issues.

  • Investigating
    Investigating

    opal1.opalstack.com (WDC shared hosting) is currently responding to ping but not to SSH and HTTP requests. We're investigating and will update this post when more info is available.

Temporary SSL problem in US mail system
  • Resolved
    Resolved

    At approximately 06:00 UTC on 17 May 2021 we inadvertently pushed the wrong set of SSL certificates to our US mail system.

    Customers who attempted to access the mail system during that period may have seen certificate warning messages and would not have been able to access their mail.

    The problem was fixed approximately 30 minutes later and the mail system has been working normally since that time. No incoming or outgoing mail was lost during the incident.

    If you are still seeing a certificate error while attempting to use the US mail system then please restart your mail client and/or reboot your computer. If the problem persists then please contact Opalstack support for assistance.

German SMTP IP range on MS block list
  • Resolved
    Resolved

    Microsoft is now accepting mail from smtp.de.opalstack.com.

  • Identified
    Identified

    We've identified and corrected the cause of the block, and have passed our findings along to Microsoft with a request to delist the German SMTP address.

    We'll update this post when Microsoft have confirmed that the server is no longer on the block list.

  • Investigating
    Investigating

    We've received some reports that outlook.com and other Microsoft mail services are blocking smtp.de.opalstack.com.

    We're investigating now and will follow up when we have more info.

Apr 2021

MariaDB down on opal9
  • Resolved
    Resolved

    The MariaDB service on Opal9 is back online. The problem was due to an error in a monitoring check that has been corrected.

  • Investigating
    Investigating

    The MariaDB (mysql) service on Opal9 is currently offline, we're working to restore service at this time.

Control panel and API down for scheduled maintenance.
  • Resolved
    Resolved

    The control panel and API are back online.

  • Monitoring
    Monitoring

    We expect the service will be back online in ~15 minutes.

  • Investigating
    Investigating

    The control panel and API are down for scheduled maintenance. We'll update this post when the service is back online.

Opal1 not responding
  • Resolved
    Resolved

    Opal1 is back online. The outage appears to have been due to high load due to abnormally heavy memory usage by a customer app. We're working with the customer to prevent this from happening again.

  • Update
    Update

    We're rebooting Opal1 at this time.

  • Investigating
    Investigating

    Opal1 is not responding to SSH and HTTP requests, we're looking into it.

Dashboard & API maintenance downtime
  • Completed
    April 17, 2021 at 5:07 AM
    Completed
    April 17, 2021 at 5:07 AM
    The maintenance is complete.
  • In progress
    April 17, 2021 at 4:00 AM
    In progress
    April 17, 2021 at 4:00 AM

    Maintenance is now in progress

  • Planned
    April 17, 2021 at 4:00 AM
    Planned
    April 17, 2021 at 4:00 AM

    The Opalstack dashboard at https://my.opalstack.com/ and the related API will be taken offline for maintenance at 04:00 UTC on 17 April 2021. The expected downtime is 1 hour or less.

Opal7 server relocation
  • Resolved
    Resolved

    The maintenance is now complete and opal7 is online.

  • Monitoring
    Monitoring

    Opal7 is back online. We'll continue to monitor.

  • Investigating
    Investigating

    Opal7 is offline again, the maintenance may be ongoing. Awaiting confirmation from the data center.

  • Resolved
    Resolved

    The migration is complete and Opal7 is back online.

  • Monitoring
    Monitoring

    Opal7 is offline for the migration at this time.

  • Investigating
    Investigating

    Our data center team will be moving opal7.opalstack.com to a new facility within the next 6 hours. During the move the server will be offline.

Mar 2021

Apache was down on Opal6
  • Resolved
    Resolved

    The Apache service on Opal6 was offline for approximately 2 hours after the server ran out of available semaphores. The problem has been corrected as of around 13:30 UTC (~1 hour ago) and we'll be adjusting our monitoring system to detect and correct this condition when it occurs.

IMAP1US Offline
  • Resolved
    Resolved

    The US IMAP server has been back online since 2021-03-24 10:30 UTC.

  • Investigating
    Investigating

    The IMAP1US server is currently down and we are working to restore access.

DE SMTP on Proofpoint blacklist
  • Resolved
    Resolved

    The DE SMTP IP is no longer on the Proofpoint blacklist.

  • Monitoring
    Monitoring

    We're still waiting on Proofpoint to delist the DE SMTP IP address.

  • Investigating
    Investigating

    One of the Opalstack DE SMTP IP addresses is currently on the Proofpoint blacklist. We're working to have the IP de-listed and expect this to be complete within the next 24-48 hours.

Mar 2021 to May 2021

Next