Datacake - Data processing delays – Incident details

Data processing delays

Resolved
Degraded performance
Started 7 months agoLasted about 22 hours

Affected

Web Application

Degraded performance from 10:31 AM to 12:16 PM, Operational from 12:16 PM to 2:06 PM, Degraded performance from 2:06 PM to 9:47 PM, Operational from 9:47 PM to 8:47 AM

MQTT Server

Degraded performance from 10:31 AM to 8:47 AM

Reports

Degraded performance from 10:31 AM to 12:16 PM, Operational from 12:16 PM to 2:06 PM, Degraded performance from 2:06 PM to 9:47 PM, Operational from 9:47 PM to 8:47 AM

Outgoing Webhooks

Degraded performance from 10:31 AM to 8:47 AM

Updates
  • Resolved
    Resolved

    Dear customers,

    We are informing you that some of our services on the Datacake IoT platform have recently experienced delays. Please be assured that there was no downtime and no data was lost. The protection and integrity of your data remain our top priority.

    The issue affected a subset of devices, causing delays of up to two hours for certain tasks. While some devices continued to operate at full performance, others were impacted by the delays.

    The cause was increased wait times for specific task processing, including Payload Decoder and API requests. Although the platform's core API remained stable, queues became overwhelmed, leading to longer processing times.

    To ensure the platform's stability, we have temporarily disabled some non-essential features and are monitoring the situation closely. We are taking measures to optimize internal processes and will keep you informed of any changes.

    Thank you for your understanding and patience during this time.

    Best regards,
    The Datacake Team

  • Monitoring
    Monitoring

    Queue processing is largely complete, with only outgoing MQTT and webhooks remaining to be finished.

  • Identified
    Identified

    We are still working on a mitigation of the delayed processing of device data.

  • Investigating
    Investigating
    We are currently investigating this incident.
  • Identified
    Identified
    We are continuing to work on a fix for this incident.
  • Investigating
    Investigating

    Our system is experiencing processing delays. No data has been lost, and the system should be caught up shortly.