Datacake - Notice history

Dear customers,

We are informing you that some of our services on the Datacake IoT platform have recently experienced delays. Please be assured that there was no downtime and no data was lost. The protection and integrity of your data remain our top priority.

The issue affected a subset of devices, causing delays of up to two hours for certain tasks. While some devices continued to operate at full performance, others were impacted by the delays.

The cause was increased wait times for specific task processing, including Payload Decoder and API requests. Although the platform's core API remained stable, queues became overwhelmed, leading to longer processing times.

To ensure the platform's stability, we have temporarily disabled some non-essential features and are monitoring the situation closely. We are taking measures to optimize internal processes and will keep you informed of any changes.

Thank you for your understanding and patience during this time.

Best regards,
The Datacake Team

Queue processing is largely complete, with only outgoing MQTT and webhooks remaining to be finished.

We are still working on a mitigation of the delayed processing of device data.

All queues have been processed. We are still investigating the root cause for the unusual buildup and have already taken measures in the form of automated scaling to prevent such situations from happening again in the future.

Most queues have been successfully processed. However, MQTT and outgoing Webhooks are still experiencing delays. Our teams are working to resolve these remaining issues as quickly as possible.

We have identified the issue to be related to certain background workers not functioning as expected. We're actively working on implementing improvements to optimize their performance and resolve the issue.

Our team has successfully implemented optimizations and fine-tuned our infrastructure, resulting in improved performance and reduced latencies. We continue to closely monitor the situation to ensure stability. If you encounter any further issues or have questions, please don't hesitate to reach out to our support team for assistance.

The situation has stabilized. We are still investigating a small percentage of requests with unusual latencies and / or timeouts.