Scheduled email campaigns, end to end

Context

In the previous story I wrote about the queue layer. This one covers the full async loop in a more demanding scenario: not "process immediately", but schedule hundreds of sends across business hours, sometimes over several days, track every lifecycle state and show progress live in the UI.

The example is real: scheduled outreach email campaigns with live delivery tracking on the production platform I work on at Formilo (Amplify Gen 2, AppSync, Lambda, DynamoDB). The email content is prepared earlier in the workflow; this part is about delivering it on time and knowing what happened.

Problem

A campaign has to:

send each email at a specific time in the campaign's timezone,
spread sends across business hours: a number of emails per day (20 by default), on selected weekdays, inside time windows,
track every email individually, so one failure doesn't block the rest,
survive the gaps between two systems that can fail independently: the database and the scheduler.

The obvious alternative is a long-running "campaign runner", or a cron job that scans the table for anything due. I didn't want execution state living in memory or inside a process that has to stay alive for days.

Solution

One email = one durable task

When a user provisions a send queue, the backend creates an EmailSendTask record in DynamoDB for every recipient. It holds a recipient snapshot (email, subject), the scheduled send time and an explicit lifecycle state:

Pending → Running → Finished | Failed | Cancelled

The database is the source of truth. The UI, the scheduler and the workers all work against the same contract.

EventBridge Scheduler as the execution layer

Each task gets its own one-time schedule. At the scheduled moment, EventBridge Scheduler invokes a send Lambda with the task's id. Simplified:

await scheduler.send(
  new CreateScheduleCommand({
    Name: scheduleNameFor(task.id),
    GroupName: EMAIL_SCHEDULE_GROUP, // a dedicated group
    ScheduleExpression: `at(${sendAt})`, // e.g. at(2026-05-28T10:15:00)
    ScheduleExpressionTimezone: campaign.timezone ?? "Europe/Berlin",
    FlexibleTimeWindow: { Mode: "OFF" },
    ActionAfterCompletion: "DELETE",
    ClientToken: task.schedulerRequestToken,
    Target: {
      Arn: SEND_FUNCTION_ARN,
      RoleArn: SCHEDULER_ROLE_ARN, // may invoke only the send Lambda
      Input: JSON.stringify({ taskId: task.id }),
      RetryPolicy: { MaximumRetryAttempts: 2 },
      DeadLetterConfig: { Arn: SCHEDULER_DLQ_ARN }, // with an alarm
    },
  }),
);

A few details matter more than they look:

The schedule name is derived from the task id, so every task has exactly one schedule and it's easy to find.
ClientToken makes creation idempotent. It's a random UUID stored on the task as schedulerRequestToken. When the reconciler recreates a schedule, it reuses the same token.
ActionAfterCompletion: DELETE cleans up every schedule after it ran, so nothing accumulates.
Failed invocations retry twice and then go to a dead-letter queue with an alarm, instead of disappearing.

Provisioning without hitting rate limits

A custom mutation provisions the queue (more on that in the next story). EventBridge Scheduler is sensitive to rate limits, so the provisioning Lambda makes at most 10 Scheduler API calls at a time. For large campaigns it stops 90 seconds before its own timeout, saves a checkpoint and re-invokes itself asynchronously to continue from there.

The send worker protects terminal states

If the task is already Finished or Cancelled, it skips. It doesn't throw: throwing would only make the scheduler retry into the DLQ.
It claims the row with a conditional write to Running before sending.
It writes Finished with sentAt, or Failed with the error.

A reconciler, because distributed systems drift

Creating a DynamoDB record and creating a schedule are two separate operations. Eventually, edge cases happen: the task row exists but schedule creation failed, a Lambda crashed after switching the task to Running, a transient AWS error interrupted execution.

So a reconciler runs every 30 minutes. It recreates missing schedules for Pending tasks and recovers stale Running tasks that never reached a terminal state. This is the part many demos skip, and it's the part that makes the system operable.

Live UI: subscriptions first, polling as fallback

On the frontend, React Query loads the campaign's tasks page by page, and Amplify observeQuery subscribes to EmailSendTask changes filtered by campaignId. The send queue updates live as workers write status changes: Pending → Running → Finished. When subscription delivery lags under load, a lightweight background refetch stays enabled while tasks are still active.

Real-time when it works. Resilient when it doesn't.

Trade-offs

One schedule per email means many API calls. That's why provisioning is rate-limited and resumable instead of a single loop.
Two sources can disagree. The task table and the scheduler can drift apart; the reconciler exists because of that, not as an afterthought.
Retries are bounded. After two retries, a failed invocation ends in the DLQ. Someone has to look at it, and the alarm makes sure someone does.
Subscriptions alone aren't enough. The fallback refetch costs a few requests, but the UI never gets stuck on stale state.

Result

No long-running campaign process: every send is one scheduled Lambda invocation.
Sends spread across business hours by emails per day, weekdays and time windows, in the campaign's timezone.
Every execution is isolated; one failure doesn't block the batch, and schedules delete themselves after running.
Missing schedules and stale Running tasks are repaired every 30 minutes.

The skeleton is the same as in simpler async systems: task record → execution layer → status updates → live UI. Only the execution mechanism changed: not DynamoDB Streams, not SQS consumers, but EventBridge Scheduler with one scheduled invocation per send. Once the task model and its lifecycle are stable, the execution layer becomes replaceable without rewriting product logic or frontend state.