Interactive lab
Break the pipeline
A live model of a queue-based, event-driven system: producers, an API, a queue, a capped worker pool, an idempotent store and a dead-letter queue. Turn the dials, cause trouble and watch it recover.
Animated diagram: events flow from three producers through an API into a queue, are picked up by a pool of workers and written to a store. Failed attempts loop back with a backoff, messages that keep failing land in the dead-letter queue, duplicates are absorbed at the store. The metrics and the event feed below describe the same state in text.
Scenarios
Traffic well below capacity. The queue stays near empty and the odd failure is retried quietly.
Controls
Workers processing in parallel. More lanes, more capacity.
Events the producers send per second.
Mean time a worker needs per attempt.
Chance that a single attempt fails and is retried.
Chance that an event can never succeed.
At-least-once: chance that a delivery arrives twice.
Attempts before a message is moved to the DLQ.
Wait before the first retry; doubles per attempt.
Live metrics
- Throughput
- –/s
- Queue depth
- –
- In flight
- –
- Oldest message
- –s
- Retries
- –
- Dead letters
- –
- Duplicates absorbed
- –
Arrivals vs. throughput
- Arrivals
- Throughput
- Capacity
What just happened
- Quiet so far. Try a scenario or send a burst.
What to try
- Start “Traffic spike”, wait for the backlog, then raise the concurrency cap and time how long it takes to drain.
- Set transient failures to 40 % with maxReceiveCount 1: every failure goes straight to the DLQ. Then allow 5 attempts.
- Inject a poison message and follow it: three attempts, longer backoffs, then the DLQ, while everything else keeps flowing.
- Turn duplicates up to 20 % and check the store: “processed” never counts an event twice.
A teaching model, not a benchmark: timings and rates are simplified and do not describe any real production system.