Email and HTTP Notification Failures [Resolved]

On August 22nd from 10:10 AM Eastern to approximately 2:30 PM Eastern one of the servers which serves store pages failed to deliver email or HTTP notifications. These notifications include order email notifications to you, HTTP notifications to your server, subscription activation notifications, and fulfillment emails to the customer.

Additionally the server failed to recognize that the failures were actually occurring. Under normal circumstances some failures are expected and the system is designed to gracefully recover from those. However in this case the server wasn't aware of the failures and couldn't take automatic action.

This is the first time we've seen something like this occur. From our logs we can see an original issue which led to the problem and we are using this as a starting point to determine how we can prevent something like this in the future. At a minimum we should have been able to catch the issue much sooner.

Due to the nature of the problem we are unable to identify which notifications haven't gone out. We also do not know which orders you may have already tried to resend for that time range. Therefore we recommend taking issues case by case and using the resend notification functionality found on the order detail UI.

Please contact us with any specific questions.

Have more questions? Submit a request

Comments

Powered by Zendesk