Auto-scaling is the feature everyone buys serverless for, and it is also the thing that takes down the rest of your stack. A Lambda function will happily scale to a thousand concurrent environments and point all of them at a Postgres instance with 100 connections, a third-party API on a 10-requests-per-second plan, or a downstream team's fixed-size cluster. The bill scales too, and a single misbehaving client can eat a month of budget in an afternoon.
The fix is not to stop scaling. It is to put deliberate, layered limits in front of the parts that cannot scale, and to fail in a way clients can retry. This post walks through each layer we configure on Serverless Framework V4 APIs, from cheapest to most involved, with the configuration for each.
The four layers
| Layer | Protects | Cost to implement |
|---|---|---|
| API Gateway account and stage throttling | Everything behind the API | One config block |
| Usage plans and API keys | Per-client fairness, monetized tiers | Low, requires key distribution |
| Reserved and provisioned concurrency | Downstream databases and third-party APIs | One config line per function |
| Application-level token buckets | Per-tenant, per-operation quotas | A table and ~60 lines of code |
Most teams need the first three. The fourth is for multi-tenant SaaS, where "fair" means something specific per customer.
Layer 1: stage and method throttling
Every REST API stage in API Gateway has a default throttle: a steady-state rate and a burst bucket. Requests over the limit get an immediate 429 Too Many Requests without ever reaching Lambda, so they cost you almost nothing.
Set it explicitly rather than relying on the account default:
provider:
name: aws
runtime: nodejs22.x
apiGateway:
# applies to every method in the stage
throttle:
maxRequestsPerSecond: 200
maxConcurrentRequests: 100
Then tighten the expensive endpoints individually. A report generator that fans out to Athena does not deserve the same allowance as a health check:
functions:
generateReport:
handler: src/reports.handler
events:
- http:
path: /reports
method: post
throttling:
maxRequestsPerSecond: 5
burstLimit: 10
For HTTP API (httpApi) events, throttling is configured on the stage's route settings rather than per http event, so either set stage defaults in provider.httpApi or drop to a resources block for per-route values. If per-method throttling and usage plans matter to you, REST API remains the easier choice.
Choosing numbers. Start from what the weakest downstream dependency can take, divide by the number of calls your handler makes per request, and leave 30% headroom. A burst limit of roughly 2x the steady rate absorbs normal jitter without letting a client sustain a flood.
Layer 2: usage plans and API keys
Stage throttling is global: one noisy client can consume the whole allowance. Usage plans attach a rate, a burst, and a daily/weekly/monthly quota to an API key, so limits are per client.
provider:
apiGateway:
apiKeys:
- free:
- name: acme-free
- pro:
- name: acme-pro
usagePlan:
- free:
quota:
limit: 1000
period: DAY
throttle:
burstLimit: 20
rateLimit: 5
- pro:
quota:
limit: 200000
period: MONTH
throttle:
burstLimit: 200
rateLimit: 50
functions:
api:
handler: src/api.handler
events:
- http:
path: /v1/{proxy+}
method: any
private: true # requires x-api-key
Things worth knowing before you commit to this:
- API keys are an identifier, not authentication. They are fine for quota accounting and partner tiering. For real authorization, keep Cognito JWTs or a Lambda authorizer in front, as covered in our post on securing a serverless API.
- Do not put key values in
serverless.yml. Let API Gateway generate them and distribute them out of band, or import a value from SSM Parameter Store at deploy time. - Quota counters reset on a fixed schedule in UTC, not on a rolling window. A client can burn a daily quota in the first minute.
- If you need quotas keyed on something other than an API key — a tenant ID inside a JWT, for instance — skip to layer 4. A Lambda authorizer can also return a
usageIdentifierKey, which maps an authenticated caller onto a usage plan key.
Layer 3: reserved concurrency as a downstream circuit breaker
This is the highest-value single line in the file. reservedConcurrency caps how many environments a function can run at once. Requests beyond the cap are throttled by Lambda itself: synchronous invokers get a 429, asynchronous ones are retried automatically, and event-source-mapped ones (SQS, Kinesis) back off and try again later.
functions:
# talks to a Postgres instance with a small connection ceiling
ledgerWriter:
handler: src/ledger.handler
reservedConcurrency: 20
# calls a partner API rate-limited to 10 rps
partnerSync:
handler: src/partner.handler
reservedConcurrency: 5
Two important side effects:
- Reserved concurrency is carved out of your account's pool. If the account limit is 1,000 and you reserve 20 here and 5 there, the unreserved pool shrinks accordingly. Reserve deliberately, and watch the
ClaimedAccountConcurrencymetric. reservedConcurrency: 0is a kill switch. Setting it to zero stops a function from running at all while leaving the rest of the stack deployed. That is the fastest way to stop a runaway consumer during an incident, and it is reversible in seconds.
For queue consumers, pair the reservation with maximumConcurrency on the event source so SQS does not keep hammering a throttled function:
functions:
worker:
handler: src/worker.handler
reservedConcurrency: 20
events:
- sqs:
arn: !GetAtt JobQueue.Arn
batchSize: 10
maximumConcurrency: 20
functionResponseType: ReportBatchItemFailures
Layer 4: per-tenant token buckets in DynamoDB
When limits have to follow business rules — "the Starter plan gets 60 report generations an hour, the Enterprise plan gets 5,000, and the limit is per tenant, not per key" — you need the count in your own datastore. A token bucket in DynamoDB with a conditional update is the standard pattern: one round trip, atomic, no race conditions.
resources:
Resources:
RateLimitTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:service}-${sls:stage}-ratelimit
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: pk
AttributeType: S
KeySchema:
- AttributeName: pk
KeyType: HASH
TimeToLiveSpecification:
AttributeName: expiresAt
Enabled: true
The handler-side check, using a fixed window per tenant and operation:
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, UpdateCommand } from '@aws-sdk/lib-dynamodb';
const ddb = DynamoDBDocumentClient.from(new DynamoDBClient({}));
const TABLE = process.env.RATE_LIMIT_TABLE;
/**
* Fixed-window counter. Returns { allowed, remaining, resetAt }.
* One conditional UpdateItem: atomic, no read-then-write race.
*/
export async function consume({ tenantId, operation, limit, windowSeconds }) {
const now = Math.floor(Date.now() / 1000);
const windowStart = now - (now % windowSeconds);
const resetAt = windowStart + windowSeconds;
const pk = `${tenantId}#${operation}#${windowStart}`;
try {
const res = await ddb.send(new UpdateCommand({
TableName: TABLE,
Key: { pk },
UpdateExpression:
'SET expiresAt = if_not_exists(expiresAt, :exp) ADD used :one',
ConditionExpression:
'attribute_not_exists(used) OR used < :limit',
ExpressionAttributeValues: {
':one': 1,
':limit': limit,
// keep the row a little past the window so TTL cleans it up
':exp': resetAt + 60,
},
ReturnValues: 'UPDATED_NEW',
}));
return {
allowed: true,
remaining: Math.max(0, limit - res.Attributes.used),
resetAt,
};
} catch (err) {
if (err.name === 'ConditionalCheckFailedException') {
return { allowed: false, remaining: 0, resetAt };
}
throw err;
}
}
Costs and caveats: one write unit per request (fractions of a cent per million on-demand, but do the arithmetic at your volume), TTL cleans up old windows for free, and a fixed window allows up to 2x the limit across a window boundary. If that matters, switch to a sliding window or a leaky bucket storing tokens and lastRefill and computing the refill on read — same conditional-update shape, slightly more arithmetic. If the check itself becomes your hot path, a cache like ElastiCache Serverless or Momento is the next step, but do not reach for it before you have measured.
If you are billing on this counter, remember it is not an accounting ledger. Emit a usage event to EventBridge or a Firehose stream as well, and reconcile from that.
Returning a 429 clients can actually use
A bare 429 with an empty body teaches clients nothing except to retry immediately, which is the worst possible behaviour. Return the standard headers:
const decision = await consume({
tenantId, operation: 'reports:create', limit: 60, windowSeconds: 3600,
});
if (!decision.allowed) {
return {
statusCode: 429,
headers: {
'Content-Type': 'application/problem+json',
'RateLimit-Limit': '60',
'RateLimit-Remaining': '0',
'RateLimit-Reset': String(decision.resetAt - Math.floor(Date.now() / 1000)),
'Retry-After': String(decision.resetAt - Math.floor(Date.now() / 1000)),
},
body: JSON.stringify({
type: 'https://errors.example.com/rate-limit',
title: 'Rate limit exceeded',
status: 429,
detail: 'Limit of 60 report creations per hour for this tenant.',
resetAt: new Date(decision.resetAt * 1000).toISOString(),
}),
};
}
Retry-After is the one that matters: it is what well-behaved SDKs and HTTP clients read to schedule a backoff. Add RateLimit-* headers on successful responses too, so clients can slow down before they hit the wall.
On the client side of your own outbound calls, do the mirror image: honour Retry-After, use exponential backoff with jitter, and set maxAttempts on the AWS SDK v3 clients rather than retrying in a for loop.
Watch the limits you set
A limit you cannot see is a future incident. The alarms we set on every API:
ThrottledRequestson the API Gateway stage — a sustained non-zero value means the stage or usage-plan limits are wrong, or someone is abusing the API.Throttleson each function with reserved concurrency — the signal that the cap is now the bottleneck.ClaimedAccountConcurrencyagainst the account limit, alarmed at 70%, so a quota increase request goes in before you need it.- Application 429 rate by tenant, from an embedded-metric-format log line, so support can answer "why is my integration failing?" in one query.
# emit a structured metric so per-tenant 429s are queryable and alarmable
# see our observability post for the Powertools setup
Our write-up on observability for serverless covers the Powertools metrics wiring for that last one.
A sensible default stack
For a new multi-tenant API, this is where we start on day one:
- Stage throttle at a round number you are comfortable paying for.
- Per-method throttles on the three most expensive endpoints.
reservedConcurrencyon every function that touches a relational database or a third-party API.- Alarms on
ThrottledRequests,Throttles, and claimed account concurrency. - Application-level per-tenant quotas only once plans and pricing exist.
That is an afternoon of work, and it is the difference between a bad client costing you a support ticket and costing you a weekend.
If you would like this reviewed against your own stack, our Serverless Performance Tuning and Serverless Security Consulting engagements cover exactly these controls. Get in touch and tell us what your API is protecting.