Skip to content
serdjukow.eu
All stories

I forgot try/catch in a Lambda. AWS did not forget.

Unhandled errors are not just code errors, they are infrastructure behavior: how synchronous and asynchronous invocations retry, and what I do about it.

  • Lambda
  • SQS
  • EventBridge

Context

Unhandled errors are not just code errors. They are infrastructure behavior.

In a Lambda function, what happens after an exception depends less on your code than on how the function was invoked. On the production platform I work on at Formilo, almost everything runs asynchronously: EventBridge rules and schedules, SQS queues, DynamoDB Streams. So the question "what happens if this throws?" has a different answer depending on who called the function.

Problem

AWS counts unhandled exceptions, timeouts and rejected promises as function errors, and treats them differently per invocation type:

  • Asynchronous invocation (EventBridge, SNS, S3 and others): Lambda retries the function two more times, with backoff. If all attempts fail, the event goes to a dead-letter queue or an on-failure destination, if one is configured.
  • Synchronous invocation (API Gateway, a direct SDK call): the error is returned to the caller immediately. Lambda doesn't retry; retry behavior is up to the caller.
  • Poll-based sources (SQS, streams): the event source mapping retries. An SQS message comes back after the visibility timeout; a stream batch is retried in order.

The async case is the risky one:

  • partial execution can happen before the failure,
  • retries can cause duplicate side effects,
  • downstream systems may be affected twice,
  • and the opposite mistake, an error that's caught and swallowed, loses the event entirely.

What I got wrong first

The last point is one I found in my own code. On an earlier product in 2025, built on DynamoDB Stream → Lambda mappings:

  • one handler returned batchItemFailures: [], but the mapping never enabled ReportBatchItemFailures, so the return value had no effect,
  • another handler caught its errors and carried on, so Lambda saw a success and nothing was retried.

Forgetting a try/catch lets AWS retry. Adding the wrong one hides the failure. Both are decisions about infrastructure behavior, whether you make them on purpose or not. That's why I became more careful with DynamoDB Streams.

What helps in production

Idempotency

Design handlers to be safe on repeated execution. The scraper first checks the task: if it's already Finished or Failed, it acknowledges the message and skips. The email send worker claims its task with a conditional write to Running before it sends anything.

Handle known errors explicitly

Catch, log and handle the errors you expect. The scraper has a FatalScrapeError for cases a retry can't fix, such as an invalid URL. It marks the task Failed and doesn't retry. An unparseable message also fails without retry.

Let unexpected errors fail

Don't swallow what you don't understand. Everything that isn't a known fatal error goes back to SQS:

const batchItemFailures: { itemIdentifier: string }[] = [];

for (const record of event.Records) {
  try {
    await processTask(record);
  } catch (error) {
    if (error instanceof FatalScrapeError) {
      await markTaskFailed(record, error); // no retry
      continue;
    }
    // Unknown error: let SQS deliver it again.
    batchItemFailures.push({ itemIdentifier: record.messageId });
  }
}

return { batchItemFailures };

This only works with reportBatchItemFailures: true on the event source. As the comment in our infrastructure code says: without it, batchItemFailures is ignored and messages are deleted even on failure. (The handler is simplified.)

Protect terminal states

Don't move a task backwards on retry. The email send worker skips tasks that are already Finished or Cancelled, and it does so without throwing: throwing would only make the scheduler retry into the DLQ.

Give failed events a place to go

Failed events need a destination. The scrape queue moves a message to its DLQ after three receives, with an alarm at the first message; the email schedules allow two retry attempts and have their own DLQ with an alarm.

Monitor and reconcile

Retries are not a replacement for reconciliation. A worker that dies mid-task leaves a row in Running that no retry will touch. Reconcilers run on a schedule: every 5 minutes for scrape tasks, every 30 minutes for email tasks. They recreate what's missing and terminalize what's stuck.

Trade-offs

  • Classifying errors is work. Deciding what's fatal and what's retryable means reading failure modes one by one. Getting it wrong either way costs something: endless retries, or lost work.
  • Retries cost time and money. A retried message comes back after the visibility timeout, and every attempt is another invocation.
  • DLQs need an owner. An alarm is only useful if someone acts on it.

Result

  • Known errors are handled explicitly; unknown ones fail into retries and, eventually, a DLQ.
  • Duplicate executions are harmless because terminal states are protected.
  • Stuck tasks are repaired by reconcilers instead of waiting for a retry that will never come.

Lambda will retry. Your architecture should be ready.

Built with Next.js 16, Tailwind CSS and CanvasHosted on VercelSource on GitHub Built