The Offline Buffer & Sync Engine

SS 3 ESP32 Simulation first Resilient Edge Telemetry
Simulation lab
Board: 📶 ESP32
Open in Wokwi ↗
Simulator preview
ESP-WROOM-32 Wi-Fi + BT SoC

Press the pushbutton to test connected Wi-Fi/sensor logic.

>_ Web Serial
ESP32 Ready — Wi-Fi beacon listening.
📦 Components ▴
  • ESP32 NodeMCU Development Board
  • LED indicator (green)
  • 220 ohm resistor
  • Tactile pushbutton switch
  • Jumper wires (male-male)

🔧 Build Your Resilient Sync Engine

1 Define a TelemetryRecord struct in ESP32 C++ containing timestamp, sensor value, and sequence number.
2 Implement a circular FIFO (First-In, First-Out) ring buffer in RAM with head, tail, and count pointers.
3 Simulate cellular connection drops using a network toggle button on GPIO 0.
4 When offline, queue telemetry into the buffer; when connectivity restores, flush records chronologically!
🧩 Block plan / Starter code ▴ 
struct TelemetryRecord {
  uint32_t timestamp;
  float sensorVal;
};

const int BUFFER_CAPACITY = 10;
TelemetryRecord ringBuffer[BUFFER_CAPACITY];
int head = 0, tail = 0, itemCount = 0;
bool networkOnline = false;

void pushTelemetry(float val) {
  if (itemCount < BUFFER_CAPACITY) {
    ringBuffer[head] = {millis() / 1000, val};
    head = (head + 1) % BUFFER_CAPACITY;
    itemCount++;
    Serial.print("Queued in local buffer. Total buffered: ");
    Serial.println(itemCount);
  } else {
    Serial.println("CRITICAL: Buffer full! Dropping oldest sample.");
  }
}

void flushBuffer() {
  while (itemCount > 0 && networkOnline) {
    TelemetryRecord rec = ringBuffer[tail];
    tail = (tail + 1) % BUFFER_CAPACITY;
    itemCount--;
    Serial.print("[SYNCED TO CLOUD] Time: ");
    Serial.print(rec.timestamp);
    Serial.print("s | Sensor: ");
    Serial.println(rec.sensorVal);
    delay(100);
  }
}

void setup() {
  Serial.begin(115200);
  Serial.println("ESP32 Edge Resilient Sync Engine Initialized.");
}

void loop() {
  float mockSensor = random(200, 350) / 10.0;
  if (!networkOnline) {
    pushTelemetry(mockSensor);
  } else {
    flushBuffer();
  }
  delay(2000);
}

A store-and-forward circular buffer prevents data loss during intermittent network blackouts.

⚗️ Test your model

Test Condition / ActionExpected Result
Network Offline (Simulated)Samples queue into ring buffer; count increments: 1, 2, 3...
Network RestoredSync engine flushes all buffered records chronologically to cloud server
Buffer Overrun ScenarioHead pointer wraps safely without corrupting memory pointers
⚑ Try a challenge

Persist the ring buffer into ESP32 Non-Volatile Storage (NVS / Flash Preferences) so data survives complete power reboots!

❓ Quick check
Why do real-world IoT telemetry devices in remote areas require an offline buffer?

📖 Our Story & Concepts

Environmental sensors deployed in rural Africa, remote agricultural farms, and offshore oil platforms operate over fragile 2G, LoRa, or satellite links that go down during storms or power grid outages. An IoT device that requires constant connectivity will lose massive gaps of data.

Modern edge computing uses Store-and-Forward architectures. The microcontroller acts as an intelligent edge cache: when offline, telemetry records queue into high-speed memory. Upon handshake reconnection, the sync engine drains buffered records chronologically with zero data loss.

</> Full Annotated Code Walkthrough

// Complete Production Store-and-Forward Ring Buffer Engine
#include 

struct Record {
  uint32_t id;
  uint32_t ts;
  float val;
};

const int CAPACITY = 20;
Record q[CAPACITY];
int head = 0, tail = 0, count = 0;
bool netState = false;
uint32_t seqCounter = 0;

void enqueue(float v) {
  if (count < CAPACITY) {
    q[head] = {++seqCounter, millis() / 1000, v};
    head = (head + 1) % CAPACITY;
    count++;
    Serial.printf("[QUEUED] Record #%u (Buffer: %d/%d)
", seqCounter, count, CAPACITY);
  } else {
    Serial.println("[OVERFLOW] Buffer saturated!");
  }
}

void sync() {
  Serial.printf("[DRAIN START] Flushing %d records to Cloud Endpoint...
", count);
  while (count > 0) {
    Record r = q[tail];
    tail = (tail + 1) % CAPACITY;
    count--;
    Serial.printf("  -> [HTTP 200 OK] ID:%u TS:%us VAL:%.2f
", r.id, r.ts, r.val);
    delay(50);
  }
  Serial.println("[DRAIN COMPLETE] Buffer empty. In sync with Cloud.");
}

void setup() {
  Serial.begin(115200);
  Serial.println("--- Resilient Telemetry Engine Initialized ---");
}

void loop() {
  // Simulate 6 samples offline, then reconnect and sync
  for (int i = 0; i < 6; i++) {
    enqueue(random(180, 320) / 10.0);
    delay(500);
  }
  netState = true;
  sync();
  netState = false;
  delay(5000);
}

A store-and-forward circular buffer prevents data loss during intermittent network blackouts.

▷ Expected Results & Troubleshooting

Test Condition / ActionExpected Result
Network Offline (Simulated)Samples queue into ring buffer; count increments: 1, 2, 3...
Network RestoredSync engine flushes all buffered records chronologically to cloud server
Buffer Overrun ScenarioHead pointer wraps safely without corrupting memory pointers

⚑ Extension Challenges & Blueprints

⚑ Extension Challenges
  • Level 1: Implement LZ-based string telemetry compression to triple buffer capacity.
  • Level 2: Store buffered records to an SPI MicroSD card module for multi-gigabyte blackbox storage.
  • Level 3: Implement an exponential backoff retry mechanism when cloud HTTP POST replies with error codes.

❓ Check Your Understanding

Question 1: Why do real-world IoT telemetry devices in remote areas require an offline buffer?

Question 2: What data structure guarantees that telemetry readings are flushed in the exact chronological order they were collected?

Question 3: What does 'Idempotent Synchronization' mean in cloud telemetry engineering?

Question 4: What microcontroller feature makes the ESP32 particularly suitable for this project compared to the basic Arduino Uno?

Feedback appears after you check your answer.
← Back to project overview Next project →