Scaling FastAPI Applications Beyond "Just Add More Workers"

Scaling a FastAPI application is not simply increasing Uvicorn or Gunicorn workers. More workers can help, but they can also move the bottleneck somewhere else or make it worse.

I think about the whole request path:

Client
-> Load Balancer
-> FastAPI Instances
-> Database
-> Redis
-> External APIs
-> Message Broker

If the database, provider, cache, or broker cannot handle the extra load, adding API workers only increases pressure.

Application Workers

Workers are processes. Each process has its own memory, event loop, and connection pools. More processes can increase parallelism, but they also increase resource usage.

Async helps when requests spend time waiting on I/O. It does not automatically make CPU-heavy work faster. If an endpoint performs expensive CPU work inside the request path, more async syntax will not fix it. That work may need a worker queue, a separate process pool, or a different design.

Database Connection Pools

Scaling application instances can accidentally scale database connections.

10 containers
x 10 DB connections
= 100 connections

If PostgreSQL is configured for a limited number of connections, a deployment change can overload it without changing a single line of application logic.

Connection pooling needs to be planned across the fleet, not only inside one container. The right number depends on database capacity, query cost, transaction duration, and other services sharing the same database.

Slow External Services

External APIs need timeouts, retry limits, connection pooling, and sometimes circuit breakers. Without timeouts, requests can pile up. Without retry limits, an outage can become a retry storm.

If a downstream service is slow, horizontally scaling the API may increase the number of concurrent calls to that same slow dependency. The user-facing service looks larger, but the bottleneck remains.

Caching

Redis can help read-heavy workloads when repeated database work is avoidable. But caching adds stale-data and invalidation problems. It is useful when the access pattern is understood and the product can tolerate the consistency model.

Background Processing

Long-running operations often do not belong in request/response paths.

FastAPI
-> Kafka or Celery
-> Worker

This can improve user-facing responsiveness and make retry behavior easier to control. It also introduces eventual consistency, queue monitoring, idempotency requirements, and worker operations.

Horizontal Scaling

FastAPI instances should generally be stateless if they need to scale horizontally. Session state, in-memory locks, and local-only caches can make scaling unpredictable.

Shared state should be explicit: database, cache, object storage, queue, or another service.

Backpressure

Backpressure means the system has a way to avoid overwhelming downstream dependencies. A queue can absorb bursts, but it is not infinite. API rate limits, bounded worker concurrency, circuit breakers, and load shedding can all be part of the design.

Practical Checklist

• Is the bottleneck CPU, database, network I/O, provider latency, or serialization?
• What is the total database connection count across all instances?
• Are timeouts configured for every external call?
• Are retries bounded and delayed?
• Can long-running work move to a queue?
• Is the API stateless enough for horizontal scaling?
• What metrics show saturation before users report it?

Scaling is a system property. Worker count is only one lever.