← Learnings

Database Rate Limiting: The Missing Piece After a Circuit Breaker

3 min·system design, resilience, backend

This is an extension of an earlier post, A Circuit Breaker Alone Won't Save Your Database. A circuit breaker only decides one thing:

Should I even try calling Redis?

When the circuit is open, your application runs the fallback. For many systems that fallback is the database.

THE CIRCUIT IS OPEN, SO THE FALLBACK RUNS request circuit breaker OPEN — REDIS IS SKIPPED, AS INTENDED database NOW TAKING THE ENTIRE PRODUCTION LOAD
FIG 1 · THE BREAKER DID ITS JOB, AND THAT IS THE PROBLEM

And that is where the real problem begins.

The database becomes the new bottleneck

Suppose your system normally handles 100K requests per second, Redis serves 99K of them, and the database serves 1K. Redis goes down. Without any protection, all 100K arrive at a database that was never designed to handle the entire production workload.

The circuit breaker saved you from Redis timeouts, but it did not save your database. It removed the wait, not the load.

The fix: cap the fallback

Instead of letting every request reach the database, the application enforces a limit of its own.

FALLBACK PATHGO
if circuitBreaker.IsOpen() {
    if !dbRateLimiter.Allow() {
        return 503
    }
    return db.Get(key)
}
FIG 2 · THE FALLBACK GETS A BUDGET

If your database can safely handle 5,000 QPS, only those 5,000 are allowed through. The rest fail fast, or are served from a local cache or stale data where you have it. The goal isn't to serve every request — it's to keep the database alive.

Where does the rate limiter live?

One question I had while learning this was whether the API gateway could do it. Not really. The gateway only sees GET /users/123. It cannot tell whether Redis is down, whether this endpoint goes to Postgres, whether it calls Elasticsearch, whether it reads Kafka, or whether it touches a database at all.

WHERE THE LIMITER GOES HTTP handler business logic circuit breaker DB rate limiter HERE — THE ONLY LAYER THAT KNOWS A DB CALL IS COMING repository database
FIG 3 · THE LAST LAYER THAT STILL KNOWS WHAT IS ABOUT TO HAPPEN

Only the service knows when it is about to hit the database, which is why the limiter is implemented inside the application, immediately before the call.

A simple implementation

At its core it's a token bucket. Suppose the database can safely handle 500 QPS.

GETUSERGO
func GetUser(id string) User {
    if !redisCircuitBreaker.IsOpen() {
        return redis.Get(id)
    }
    if localCache.Has(id) {
        return localCache.Get(id)
    }
    if !dbRateLimiter.Allow() {
        return Error503()
    }
    return db.GetUser(id)
}
FIG 4 · FOUR BRANCHES, CHEAPEST FIRST

If there are no tokens left, the request is rejected immediately — no waiting, and no queue of requests to overwhelm the database with once it recovers.

But what about multiple application servers?

This was the other question that came to mind, because a per-process bucket is only per-process.

BUDGETING THE LIMIT ACROSS INSTANCES app servers 10 allowed per server 500 QPS EACH ONE HOLDS ITS OWN BUCKET reaching the database 5,000 QPS THE NUMBER YOU ACTUALLY SIZED FOR Each instance gets a slice of the database, not a copy of its whole budget.
FIG 5 · THE BUDGET IS DIVIDED, NOT COPIED

Each instance is allocated a portion of the database's total capacity, not the whole of it. Ten servers at 500 QPS each is 5,000 QPS at the database, which is the number you sized for. Larger systems reach for distributed rate limiters or adaptive concurrency limits, but the rule underneath does not change.

Never allow your fallback path to exceed what your database can safely handle.

A circuit breaker prevents wasted calls to a failing dependency; it does not protect your database. The rate limiter that does sits inside the application, just before the database call, and rejects excess deliberately. During an outage, preserving the system matters more than serving every request — a resilient system isn't one that never fails, it's one that fails without taking everything else down.