Possible Webhook POST Delays and Marketing Campaigns Issues

Sendgrid Outgoing Mail Gateway Original incident notification from SendGrid
These maintenances affect outgoing email on the Zenith Media Canada Network of servers, all outgoing email unless configured by the client to do otherwise rely on the smart host gateway of SendGrid.

Nov 26, 12:28 PST
Resolved - We have monitored the fix and everything is in working order. We apologize for the inconvenience this issue has been.

Nov 26, 06:17 PST
Monitoring - The fix is currently being monitored to ensure no further issues come up. We apologize for the inconvenience. We will post an update as soon as one is available.

Nov 26, 00:34 PST
Update - We are currently monitoring the fix that has been implemented. Single Sends and Automations have been turned back on and adding or updating contacts should no longer be delayed. We apologize for the inconvenience. We will post an update as soon as one is available.

Nov 25, 23:16 PST
Update - We are currently monitoring the fix our third party has implemented. We apologize for the inconvenience. We will post an update as soon as one is available.

Nov 25, 21:14 PST
Update - We are continuing to monitor as our third party works towards implementing a fix. We apologize for the inconvenience. Please expect an update within two hours.

Nov 25, 19:12 PST
Update - We will continue to monitor as our third party works towards implementing a fix. We apologize for the inconvenience. Please expect another update within two hours.

Nov 25, 17:15 PST
Update - We are continuing to monitor as our third party works towards implementing a fix. We apologize for the inconvenience. Please expect an update within two hours.

Nov 25, 16:06 PST
Update - We continue to work on the third party incident. To clarify the impact to the Event Webhook: events were delayed between 6:11am MT - 8:11am MT, we took mitigation steps and no webhook posts since 8:11am MT have been impacted. Email API was not impacted at any time.

New Marketing Campaigns Single Sends and Automations are disabled and adding or updating Contacts is delayed.

Legacy Marketing Campaigns is not affected. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 15:05 PST
Update - We will continue to monitor as our third party works towards implementing a fix. We apologize for the inconvenience. Please expect another update within two hours.

Nov 25, 14:12 PST
Update - We are continuing to monitor as our third party works towards implementing a fix. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 13:05 PST
Update - The third party incident is still ongoing. Impact has not changed - there are still delayed Webhook POSTs and New Marketing Campaigns issues. Single Sends and Automations are disabled and adding or updating Contacts is delayed. Legacy Marketing Campaigns is not affected. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 12:05 PST
Update - We are still working on the third party incident. Current impact has not changed - there are still delayed Webhook POSTs and New Marketing Campaigns issues. Single Sends and Automations are disabled and adding or updating Contacts is delayed. Legacy Marketing Campaigns is not affected. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 11:06 PST
Update - We continue to work on the third party incident. Current impact is unchanged - there are still delayed Webhook POSTs and New Marketing Campaigns issues. Single Sends and Automations are disabled and adding or updating Contacts is delayed. Legacy Marketing Campaigns is not affected. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 10:14 PST
Update - We continue to work on the third party incident. Current impact is still delayed Webhook POSTs and New Marketing Campaigns issues. Currently we have temporarily disabled Single Sends and Automations during this issue. It is also important to note that adding or updating contacts is delayed. Legacy Marketing Campaigns is not affected. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 09:05 PST
Update - We are still working on the third party incident. Current impact is delayed Webhook POSTs and Marketing Campaigns issues including delays in sends. We apologize for the inconvenience. Please expect an update within 60 minutes.

Nov 25, 08:07 PST
Update - We are continuing to work on a fix for this issue.

Nov 25, 08:05 PST
Update - Further investigation shows that there is also a delay in Marketing Campaigns Single Sends and Automatons due to the third party incident. We are investigating the issue and will provide updates every 60 minutes or sooner if information becomes available. We apologize for any inconvenience this may cause.

Nov 25, 07:15 PST
Identified - Due to a third party incident occurring this morning starting around 6:11AM MT, customers may experience delays in stats and event webhoook POSTs to their endpoints. We apologize for the inconvenience. Please expect an update within 60 minutes.

Interested in using SendGrid as your outgoing email strategy? Take advantage of our Cloud Website Hosting and our WordPress Website Hosting that are already fully integrated and configured to send using their global network. Interested in cPanel Anti-Spam Email Protection? if you're an Email administrator, that solution is for you.

Have an issue thats being affected by this maintenance? Open a Ticket and we can help you find a solution.

Leave a Reply