Webmail (USA) on Opalstack Status

[Resolved] Delays in US incoming mail

Tue, 02 Apr 2024 18:04:00 +0000

Fixed - All pending incoming US mail has been delivered. (20:30 UTC — Apr 2)

Watching - The high load condition is resolved and the backlog of pending mail is being delivered. (18:25 UTC — Apr 2)

At this time there is a high load condition on one of our US IMAP servers which is a variety of issues for some US mail customers including:

Delays in the receipt of incoming mail
Slow performance in IMAP mail clients
Slow performance in the Opalstack US webmail system

We’re working to mitigate the problem at this time and will update this post when more information is available. (18:04 UTC — Apr 2)

[Resolved] Change to default SPF and DMARC policies

Wed, 24 Jan 2024 20:03:12 +0000

On 23 January 2024 we began publishing default SPF and DMARC DNS records for domains that use Opalstack’s DNS and mail systems.

The default policies that we published were somewhat restrictive and were causing mail sent from some customer domains to go to spam so we’ve made the following changes to the policies:

The policy for SPF has been changed from hard fail “-all” to soft fail “~all”.
The policy for DMARC has been changed from “quarantine” to “none”.

The net result of these changes is that customer mail is still compliant with SPF and DMARC but with a less-restrictive policy to help legitimate mail arrive in your inbox. (20:03 UTC — Jan 24)

[Resolved] US webmail performance issues

Tue, 07 Mar 2023 18:20:00 +0000

Fixed - We’ve seen no further performance issues on the US webmail system since implementing the new proxy, so we consider this issue to be resolved. (19:18 UTC — Mar 17)

Watching - We’ve deployed a new IMAP proxy on the US webmail server and the performance seems much better at this time. We’ll continue to monitor. (01:20 UTC — Mar 17)

We’re aware that the US webmail server has been having intermittent timeout issues, particularly during US business hours.

We’re working to resolve this but we expect that it will take at least a couple of weeks. Our recommended workaround in the interim is to use a desktop mail client or to run your own webmail server separately from our managed webmail instance.

Desktop email client configuration info is available in our user guide at: Email Client Configuration

If you’d like to run your own webmail server setup instructions are available at: HOWTO run your own Roundcube webmail server on Opalstack (18:20 UTC — Mar 7)

[Resolved] Slow connections or timeouts on US IMAP/POP3/Webmail

Thu, 06 Oct 2022 15:46:25 +0000

There was a ~30 minute period between 14:50 UTC and 15:20 UTC during which system load on our US IMAP server was high. This may have resulted in slow IMAP performance or connection timeouts, including connections from the US webmail system.

The condition has passed and we’re seeing no further issues at this time. (15:46 UTC — Oct 6)

[Resolved] Mail server certificates expired

Thu, 30 Sep 2021 14:43:09 +0000

Fixed - The issue is now resolved. The root cause was an obsolete root certificate on our servers. The certificate has been updated. (15:10 UTC — Sep 30)

The SSL certificates used by Opalstack’s mail servers are expired. We’re working to correct that at this time and should have it resolved shortly. (14:43 UTC — Sep 30)

[Resolved] Intermittent errors on webmail

Tue, 02 Feb 2021 17:38:00 +0000

Fixed - Webmail has been stable since our last update. (20:59 UTC — Feb 3)

Watching - We’ve adjusted the webmail server configuration to accommodate higher traffic and the server is now stable. We’ll continue to monitor. (18:55 UTC — Feb 2)

At this time we’re seeing intermittent database errors and 50x responses on https://webmail.opalstacked.com/ - we’re investigating and will update this post when more info is available. (17:38 UTC — Feb 2)

[Resolved] US IMAP service performance degraded

Tue, 19 Jan 2021 20:35:00 +0000

Fixed - The US IMAP server is back online and is operating normally. We’ll be upgrading the server in February to avoid future problems. (21:22 UTC — Jan 19)

Watching - We’ve rebooted the US IMAP server and are monitoring. (20:57 UTC — Jan 19)

The IMAP service on our US mail system is currently running very slowly. We’re working to correct that at this time.

This is also causing timeouts and errors for users accessing Opalstack webmail, including users of our German mail system. (20:35 UTC — Jan 19)

[Resolved] Slow IMAP performance

Mon, 14 Dec 2020 21:14:00 +0000

Identified - The US IMAP service is currently under heavy load while importing mail for new customers. (02:15 UTC — Dec 23)

Fixed - All pending mail imports have been completed and the IMAP servers are operating normally. (22:17 UTC — Dec 19)

At this time we’re importing hundreds of gigabytes of mail for new customers migrating from WebFaction. This is causing heavy load on our IMAP servers so you may see slow performance in our webmail system, IMAP clients, and POP3 downloads over the next few days.

We’ll update this post when the imports are complete. (21:14 UTC — Dec 14)

[Resolved] Service disruption in San Francisco

Sun, 16 Aug 2020 13:00:00 +0000

Fixed - All SFO servers are now back online. (16:17 UTC — Aug 16)

Identified - vps15 and mx2.us are back online. (15:54 UTC — Aug 16)

Identified - The webmail server is back online and is operational with the exception of sending mail. We’re still waiting on the remaining servers to come back online. (15:26 UTC — Aug 16)

Identified - The data center team reports:

The issue has been identified as a utility power issue. Power is transferred to generators and is currently running stable and network connectivity is restored for the vast majority of the services.

We’re starting to see the remaining machines come online at this time. (15:08 UTC — Aug 16)

Investigating - opal3.opalstack.com is back online at this time. Other servers are still offline and we’re awaiting a report on the initial investigation from the data center team. (14:19 UTC — Aug 16)

Services in our San Francisco data center are currently disrupted. The following servers are affected:

community.opalstack.com
mx2.us.opalstack.com
opal3.opalstack.com
smtp2.us.opalstack.com
vps4.opalstack.com
vps5.opalstack.com
vps6.opalstack.com
vps13.opalstack.com
vps15.opalstack.com
vps16.opalstack.com
vps17.opalstack.com
vps19.opalstack.com
vps21.opalstack.com
vps24.opalstack.com
vps28.opalstack.com
webmail.us.opalstack.com

We’ll update this post as soon as more information is available. (13:00 UTC — Aug 16)

[Resolved] Cron mail not working

Sun, 14 Jun 2020 22:24:11 +0000

Fixed - Mail from cron jobs should be working normally now. (20:58 UTC — Jun 16)

At the present time mail is not being sent by cron jobs on our web servers. We’re looking into the cause. (22:24 UTC — Jun 14)