Summary and Impact to Customers

On Wednesday 13th October 2021 from 12:03 to 14:06, SYNAQ Cloud Mail experienced a mail authentication incident.

The resultant impact of the event was that users received an authentication pop-up message when trying to login and experienced slow access to mail.

Root Cause and Solution

The root cause of this event was due to an abnormal amount of modification requests being received from the SYNAQ API. The increased writes and load on the master LDAP server prevented the LDAP replicas from being able to replicate the LDAP changes from the master. Since LDAP replicas are responsible for servicing requests for authentication and mail delivery, and were unable to sync correctly from the master LDAP server in this instance, authentication and mail delivery requests were not processed.

To resolve this issue, the primary server and all replicas had to be restarted to re-establish the syncing between LDAP servers. Once all the LDAP servers were in sync, authentication requests were once again being processed and users could access their mail.

Remediation Actions

• More stringent limits on the API are to be put in place to limit the number of requests that can come in from a single IP address to prevent increased load on the LDAP servers.

• The SYNAQ infrastructure teams are working in collaboration with the 3rd Party vendor to improve and harden our LDAP configuration.