Datacake - Data processing delays – Incident details

All systems operational

Show current status

Data processing delays

ResolvedDegraded performance7 hours 45 minutes

Affected

Web Application

Degraded performance from 6:14 AM to 11:59 AM, Operational from 11:59 AM to 1:59 PM

MQTT Server

Degraded performance from 6:14 AM to 1:59 PM

Reports

Degraded performance from 6:14 AM to 11:59 AM, Operational from 11:59 AM to 1:59 PM

Outgoing Webhooks

Operational from 6:14 AM to 9:03 AM, Degraded performance from 9:03 AM to 11:59 AM, Operational from 11:59 AM to 1:59 PM

  • ResolvedUTC

    All queues have now finished processing and are operating in real-time. Thank you for your patience.

  • MonitoringUTC

    We are pleased to report that almost all important queues have caught up and are now processing data in real-time. The MQTT publish queue is still slightly behind, and we are working to resolve this.

  • UpdateUTC

    The main affected queue has been successfully processed. However, due to the high volume of data, some secondary queues, such as MQTT and database insertion, are experiencing high load. Data delays are still occurring, and we are actively managing this to minimize impact.

  • IdentifiedUTC

    We have identified delays in data processing due to an overloaded queue and have already scaled up our processing power for faster performance. Our team is closely monitoring the queue to ensure it is cleared efficiently.

  • InvestigatingUTC

    Our system is experiencing processing delays. No data has been lost, and the system should be caught up shortly.