SleekDeploy | Event-Driven Serverless with EventBridge, SQS, and Idempotency (Serverless Framework V4)

Most Serverless Framework tutorials stop at a synchronous API. The interesting half of a production system is what happens after the request returns: order events fan out to a handful of consumers, one of them is a flaky third-party API, and someone has to make sure a retried message does not charge the customer twice.

This tutorial builds that asynchronous half. You get a custom EventBridge bus, a rule that routes matching events into an SQS queue, a Lambda consumer with partial batch failure reporting, a dead-letter queue, and idempotency backed by DynamoDB via Powertools for AWS Lambda. Everything is declared in serverless.yml with Serverless Framework V4, Node.js 22, and TypeScript.

Why a bus and a queue?

The two services solve different problems and the common mistake is picking one and living with its weakness.

	EventBridge	SQS
Model	Publish/subscribe, many consumers per event	Point-to-point, one consumer group per queue
Routing	Content-based rules and filters	None; the producer picks the queue
Batching to Lambda	One event per invocation	Up to 10,000 records per invocation
Backpressure	Retries on the target, limited buffering	The queue is the buffer
Replay	Archive and replay built in	Only what is still in the queue

EventBridge gives you routing and decoupling: producers publish facts and never learn who consumes them. SQS gives you a buffer, batching, and a per-message retry counter. Putting a queue between the bus and each consumer means a slow or broken consumer backs up its own queue instead of dropping events, and you can retry that one consumer without re-delivering to the others. This bus-to-queue-per-consumer shape is the default we reach for on integration work.

1. The service skeleton

mkdir orders-events && cd orders-events && npm init -y
npm install @aws-lambda-powertools/idempotency @aws-lambda-powertools/logger \
            @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb @aws-sdk/client-eventbridge
npm install --save-dev typescript @types/aws-lambda @types/node

2. `serverless.yml`

service: orders-events
frameworkVersion: '4'

provider:
  name: aws
  runtime: nodejs22.x
  region: us-east-1
  architecture: arm64
  memorySize: 512
  timeout: 30
  logRetentionInDays: 30
  environment:
    BUS_NAME: !Ref OrdersBus
    IDEMPOTENCY_TABLE: !Ref IdempotencyTable
    POWERTOOLS_SERVICE_NAME: orders-events
    POWERTOOLS_LOG_LEVEL: INFO
  tracing:
    lambda: true

build:
  esbuild:
    bundle: true
    minify: true
    target: node22
    exclude:
      - '@aws-sdk/*'

functions:
  # Producer: turns an HTTP call into a domain event.
  placeOrder:
    handler: src/place-order.handler
    iamRoleStatements:
      - Effect: Allow
        Action: events:PutEvents
        Resource: !GetAtt OrdersBus.Arn
    events:
      - httpApi:
          path: /orders
          method: post

  # Consumer: drains the SQS buffer, not the bus directly.
  fulfilOrder:
    handler: src/fulfil-order.handler
    iamRoleStatements:
      - Effect: Allow
        Action:
          - dynamodb:GetItem
          - dynamodb:PutItem
          - dynamodb:UpdateItem
          - dynamodb:DeleteItem
        Resource: !GetAtt IdempotencyTable.Arn
    events:
      - sqs:
          arn: !GetAtt FulfilmentQueue.Arn
          batchSize: 10
          maximumBatchingWindow: 5
          functionResponseType: ReportBatchItemFailures

resources:
  Resources:
    OrdersBus:
      Type: AWS::Events::EventBus
      Properties:
        Name: ${self:service}-${sls:stage}-bus

    FulfilmentQueue:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: ${self:service}-${sls:stage}-fulfilment
        VisibilityTimeout: 180          # >= 6x the function timeout
        MessageRetentionPeriod: 1209600 # 14 days
        RedrivePolicy:
          deadLetterTargetArn: !GetAtt FulfilmentDlq.Arn
          maxReceiveCount: 5

    FulfilmentDlq:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: ${self:service}-${sls:stage}-fulfilment-dlq
        MessageRetentionPeriod: 1209600

    # Rule: only paid orders over a threshold reach the fulfilment consumer.
    FulfilmentRule:
      Type: AWS::Events::Rule
      Properties:
        EventBusName: !Ref OrdersBus
        EventPattern:
          source:
            - orders.api
          detail-type:
            - OrderPlaced
          detail:
            status:
              - paid
            total:
              - numeric: ['>', 0]
        Targets:
          - Id: FulfilmentQueue
            Arn: !GetAtt FulfilmentQueue.Arn

    FulfilmentQueuePolicy:
      Type: AWS::SQS::QueuePolicy
      Properties:
        Queues:
          - !Ref FulfilmentQueue
        PolicyDocument:
          Statement:
            - Effect: Allow
              Principal:
                Service: events.amazonaws.com
              Action: sqs:SendMessage
              Resource: !GetAtt FulfilmentQueue.Arn
              Condition:
                ArnEquals:
                  aws:SourceArn: !GetAtt FulfilmentRule.Arn

    IdempotencyTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:service}-${sls:stage}-idempotency
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: id
            AttributeType: S
        KeySchema:
          - AttributeName: id
            KeyType: HASH
        TimeToLiveSpecification:
          AttributeName: expiration
          Enabled: true

    OrdersArchive:
      Type: AWS::Events::Archive
      Properties:
        SourceArn: !GetAtt OrdersBus.Arn
        RetentionDays: 30

Four details in there are the ones that get missed in review:

VisibilityTimeout must be at least six times the function timeout. AWS enforces this for SQS event sources. Too low and the same message is handed to a second invocation while the first is still working - the single most common cause of "duplicate processing" tickets.
functionResponseType: ReportBatchItemFailures switches the event source to partial batch failures. Without it, one bad record in a batch of ten re-delivers all ten.
The queue policy is scoped to the rule ARN, not to events.amazonaws.com generally. Otherwise any EventBridge rule in any account could push into your queue.
The archive is one resource block and gives you replay of any event on the bus for 30 days. It costs almost nothing and it is the difference between "we lost an hour of orders" and "we replayed an hour of orders".

3. The producer

src/place-order.ts:

import type { APIGatewayProxyHandlerV2 } from 'aws-lambda';
import { randomUUID } from 'node:crypto';
import { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge';
import { Logger } from '@aws-lambda-powertools/logger';

const events = new EventBridgeClient({});
const logger = new Logger();

export const handler: APIGatewayProxyHandlerV2 = async (event) => {
  const body = JSON.parse(event.body ?? '{}');
  const orderId = body.orderId ?? randomUUID();

  const result = await events.send(new PutEventsCommand({
    Entries: [{
      EventBusName: process.env.BUS_NAME,
      Source: 'orders.api',
      DetailType: 'OrderPlaced',
      Detail: JSON.stringify({
        eventId: randomUUID(),   // idempotency key for consumers
        orderId,
        customerId: body.customerId,
        total: body.total,
        status: 'paid',
        occurredAt: new Date().toISOString(),
        schemaVersion: 1,
      }),
    }],
  }));

  // PutEvents returns 200 even when individual entries fail. Always check.
  if (result.FailedEntryCount && result.FailedEntryCount > 0) {
    logger.error('put_events_failed', { entries: result.Entries });
    return { statusCode: 502, body: JSON.stringify({ error: 'could not publish event' }) };
  }

  return { statusCode: 202, body: JSON.stringify({ orderId }) };
};

Two habits worth adopting from this handler. First, FailedEntryCount is not an exception - PutEvents happily returns HTTP 200 with per-entry failures, and code that ignores it silently drops events. Second, every event carries its own eventId and schemaVersion. The eventId is what consumers deduplicate on; the version is what lets you evolve the payload later without breaking them.

4. The consumer: partial batch failures

src/fulfil-order.ts:

import type { SQSEvent, SQSBatchResponse } from 'aws-lambda';
import { Logger } from '@aws-lambda-powertools/logger';
import { processOrder } from './lib/process-order.js';

const logger = new Logger();

export const handler = async (event: SQSEvent): Promise<SQSBatchResponse> => {
  const failures: { itemIdentifier: string }[] = [];

  for (const record of event.Records) {
    try {
      // EventBridge wraps the detail; SQS wraps that again in record.body.
      const envelope = JSON.parse(record.body);
      await processOrder(envelope.detail);
    } catch (err) {
      logger.error('record_failed', { messageId: record.messageId, err });
      failures.push({ itemIdentifier: record.messageId });
    }
  }

  return { batchItemFailures: failures };
};

The contract is strict and easy to get wrong:

Return { batchItemFailures: [] } - never undefined - when everything succeeded. A malformed response is treated as a complete batch failure.
Catch per record. If the handler throws, the whole batch returns to the queue.
Every message id you report goes back to the queue individually and its ApproximateReceiveCount increments toward maxReceiveCount.
If ordering matters, use a FIFO queue and stop processing the group after the first failure; otherwise later messages in the group commit before the retried one.

5. Idempotency that actually holds

At-least-once delivery is a guarantee, not a bug. SQS can deliver the same message twice, a visibility timeout can expire mid-work, and EventBridge can retry a target. Any consumer with a side effect - charging a card, sending an email, calling a partner API - needs a deduplication store.

Powertools for AWS Lambda ships this as a decorator-free wrapper. src/lib/process-order.ts:

import { makeIdempotent } from '@aws-lambda-powertools/idempotency';
import { DynamoDBPersistenceLayer } from '@aws-lambda-powertools/idempotency/dynamodb';
import { IdempotencyConfig } from '@aws-lambda-powertools/idempotency';
import { Logger } from '@aws-lambda-powertools/logger';

const logger = new Logger();

const persistenceStore = new DynamoDBPersistenceLayer({
  tableName: process.env.IDEMPOTENCY_TABLE!,
});

const config = new IdempotencyConfig({
  eventKeyJmesPath: 'eventId',   // dedupe on the producer's eventId
  expiresAfterSeconds: 60 * 60 * 24,
  throwOnNoIdempotencyKey: true, // fail loudly rather than dedupe on nothing
});

const chargeAndFulfil = async (detail: { eventId: string; orderId: string; total: number }) => {
  logger.info('fulfilling', { orderId: detail.orderId });
  // ... call the payment provider, write the order, send the confirmation
  return { orderId: detail.orderId, fulfilled: true };
};

export const processOrder = makeIdempotent(chargeAndFulfil, {
  persistenceStore,
  config,
});

What the persistence layer does under the hood: a conditional PutItem claims the key with status INPROGRESS, the function runs, and the stored item is updated to COMPLETED with the serialized result. A second delivery of the same eventId either returns the stored result immediately or, if the first attempt is still running, raises IdempotencyAlreadyInProgressError so the message is retried later. TTL on the expiration attribute expires records automatically, so the table does not grow forever.

Rules that make this work in production:

Key on a business or event identifier, never on the SQS messageId. A message id changes if the producer republishes; the order or event id does not.
Wrap the smallest function that owns the side effect, not the whole handler - otherwise one poisoned record in a batch marks the batch's key as done.
Set the TTL longer than your maximum retry window, including the 14-day queue retention if you plan to redrive from the DLQ.
Make the wrapped function's return value serializable and small. It is stored in DynamoDB.

6. Deploy and exercise it

npx serverless deploy --stage dev

curl -s -X POST "$(npx serverless info --verbose | grep -o 'https://[^ ]*')/orders" \
  -H 'content-type: application/json' \
  -d '{"orderId":"ord-1001","customerId":"cus-7","total":149.5}'

Things to verify, in order:

# The consumer ran once
npx serverless logs -f fulfilOrder --tail

# Replay the same event - the second run should short-circuit
aws events put-events --entries file://same-event.json

# Nothing stuck in the DLQ
aws sqs get-queue-attributes --queue-url "$DLQ_URL" \
  --attribute-names ApproximateNumberOfMessages

A useful failure drill before go-live: make chargeAndFulfil throw for one specific orderId, publish a batch of ten, and confirm that exactly one message is retried five times and then lands in the DLQ while the other nine are deleted. That single test proves partial batch failures, the redrive policy, and the visibility timeout are all configured correctly.

7. Alarms you should not deploy without

Asynchronous systems fail quietly. Three CloudWatch alarms cover most of it:

ApproximateNumberOfMessagesVisible on the DLQ greater than 0 - something is permanently broken.
ApproximateAgeOfOldestMessage on the main queue above your SLA - the consumer cannot keep up.
FailedInvocations on the EventBridge rule - the target itself is rejecting deliveries, usually an IAM or policy problem.

Add IteratorAge-style dashboards and structured logs and you can answer "where is order 1001?" in one Logs Insights query. Our Serverless Monitoring and Logging Solutions page covers the wider setup.

8. When to drop the Lambda entirely: EventBridge Pipes

If a consumer's only job is "read from a queue, transform slightly, call another AWS service", EventBridge Pipes does it without your code:

    AuditPipe:
      Type: AWS::Pipes::Pipe
      Properties:
        Name: ${self:service}-${sls:stage}-audit
        RoleArn: !GetAtt AuditPipeRole.Arn
        Source: !GetAtt AuditQueue.Arn
        SourceParameters:
          SqsQueueParameters:
            BatchSize: 10
        Target: !GetAtt AuditLogGroup.Arn

Pipes gives you source polling, filtering, optional enrichment, and target delivery as configuration. No handler to test, no cold start, no dependency updates. The trade-off is limited transformation logic - the moment you need real branching, go back to a Lambda or a Step Functions state machine.

Where teams get this wrong

Subscribing Lambda directly to the bus for everything. Fine for fire-and-forget; painful when you need batching, backpressure, or a per-message retry count.
Using the default event bus for domain events. A custom bus per domain keeps rules, permissions, and archives separable.
No schema discipline. Turn on EventBridge Schema Registry discovery, or at minimum version your detail payloads from day one.
Treating the DLQ as an archive. It is a queue with a 14-day retention. Redrive it or alarm on it; do not let it accumulate.
Idempotency "handled by the database". A conditional write on the order row protects the order table and nothing else - not the email, not the partner API call.

Next steps

This pattern - custom bus, queue per consumer, partial batch failures, DLQ, idempotency store - is the backbone of most of the integration work we do. If you are wiring events between systems, see Serverless Integration Services and Microservices with Serverless Framework, or get in touch with your current architecture and we will tell you where the duplicates are hiding.

Event-Driven Serverless with EventBridge, SQS, and Idempotency (Serverless Framework V4)