Engage: Slowness in API reported

Incident Report for Voyado

Postmortem

Summary

On 2024-05-27, we experienced two brief service interruptions due to an issue related to enabling BI exports for new tenants, which caused database overloads due to a flaw in the activation method.

Customer Impact

15:11 - 15:14 Services were unavailable for 3 minutes due to this issue.
15:19 - 15:22 Services were unavailable again for 3 minutes due to this issue.

Root Cause and Mitigation

The interruptions were caused by a flaw in the BI export activation method for new tenants, leading to temporary database overloads. A fix has been deployed to resolve this issue and prevent it from recurring.

Next Steps

No further steps required.

Posted May 30, 2024 - 11:04 CEST

Resolved

Monitoring has shown no signs of deterioration and we deem it safe to declare this incident resolved.

Investigations on cause and subsequent improvements are still ongoing. A post mortem will be provided as soon as possible.

Posted May 27, 2024 - 17:04 CEST

Monitoring

A fix has been implemented and we are monitoring the platform to make sure it had the intended effect.

Posted May 27, 2024 - 15:55 CEST

Investigating

We have been noticing slowness in the API response times and timeouts since around 15:20 CEST. We are investigating.

Posted May 27, 2024 - 15:29 CEST

This incident affected: Engage (API).