FastAPI – Getting It Into Production

You have an image that runs. Getting it in front of users adds a handful of concerns the previous lessons deliberately left alone: what sits in front of it, where configuration comes from, when migrations run, and how to replace a running version without dropping requests.

The shape of it

          TLS terminated here
              │
  internet ──▶ load balancer ──▶ reverse proxy ──▶ [ uvicorn: 4 workers ]  × N containers
                                   (nginx)              │
                                                        ├──▶ Postgres  (pgbouncer)
                                                        ├──▶ Redis
                                                        └──▶ object storage

Some of those collapse depending on where you deploy — a managed platform is often load balancer and proxy in one, and a small service may skip nginx entirely. The concerns do not disappear; something still has to own them.

Workers, and the number that bites

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

A uvicorn process is one event loop on one core, however async the code. Workers are separate processes and are how you use more than one core.

Start around 2 × cores and tune against real traffic. What you must not do is pick the number in isolation, because each worker carries its own connection pool:

connections = containers × workers × (pool_size + max_overflow)

  3 containers × 4 workers × (5 + 10)  =  180
  Postgres default max_connections     =  100      <-- exhausted

The failure arrives all at once under load, as FATAL: sorry, too many clients already. Three fixes, usually combined: lower the pool, raise the database limit, or put PgBouncer in front so many application connections share few database ones. The last is the standard answer above a few instances — with the caveat that transaction-mode pooling breaks prepared statements and session state, so it needs prepare_threshold=0 on psycopg.

Memory is the other constraint. Each worker is a full Python process, so four workers is roughly four times the resident memory of one. A container limit that fits one worker comfortably will get four of them OOM-killed.

Behind a proxy

nginx (or its managed equivalent) does the things uvicorn should not: TLS, buffering slow clients, request size limits, serving static files, and timeouts.

server {
    listen 443 ssl http2;
    server_name api.example.com;

    # ⚠️ Rejects an oversized upload before it ever reaches Python. The application still
    # enforces its own limit — this is the cheap first line, not the guarantee.
    client_max_body_size 5m;

    location / {
        proxy_pass http://api:8000;

        # ⚠️ Without these the app sees nginx as the client: every request appears to come
        # from 127.0.0.1, over http, addressed to the internal host.
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Long enough for a real request, short enough that a stuck one is not held forever.
        proxy_read_timeout 30s;
    }
}

Setting those headers is only half of it. Uvicorn ignores them unless told to trust them:

uvicorn app.main:app --host 0.0.0.0 --proxy-headers --forwarded-allow-ips='*'

Without --proxy-headers, every log line records the proxy's IP, rate limiting keyed on client address limits everyone together, and any URL the application generates comes out as http:// — which breaks OAuth redirects and produces mixed-content warnings.

--forwarded-allow-ips='*' means "trust the header from anyone", which is safe only when nothing can reach the app except the proxy. If the container port is exposed, a client can forge X-Forwarded-For and become any IP it likes. Name the proxy's address instead.

If the API is served under a path prefix — /api on a shared domain — uvicorn needs --root-path /api, or the OpenAPI document advertises URLs without it and the docs page cannot call anything.

Configuration and secrets

    model_config = SettingsConfigDict(
        env_file=".env",
        env_prefix="STAYHUB_",
        extra="ignore",
    )

The .env file is a development convenience. In production the environment carries the values, and the loading order works in your favour: a real environment variable overrides the file, so the same image runs anywhere with different variables and no rebuild.

One change worth making for production. StayHub's settings have defaults for everything, which makes the demo runnable with no configuration:

    jwt_secret: str = "dev-only-change-me-in-any-real-deployment-0123456789"

A deployed service should drop that default entirely. With no default, a missing value is a ValidationError at startup — the container fails to boot and the rollout stops. With one, it starts happily and signs every token with a value published in your repository. Fail loudly at boot rather than quietly forever.

Secrets come from the platform's secret store — AWS Secrets Manager, Kubernetes secrets, your PaaS's variables — and never from the image. Lesson 16 covered why: layers are additive, so a secret copied in at any point stays readable even if a later layer deletes it.

Migrations are a deploy step

The tempting version runs them in the entrypoint. It works with one replica and races with several — they all start at once and all run alembic upgrade head simultaneously.

Run them once, before the new version starts serving:

alembic upgrade head          # a job, a pre-deploy hook, or a one-off task
# then roll out the new image

Which raises the constraint that governs schema changes: during a rollout, old and new code are both running against one database. A migration that drops a column the old version still selects breaks the half of your fleet that has not been replaced yet.

The way out is to make every change backwards-compatible, in two deploys:

Two more things to check before a migration meets a production table. ALTER TABLE takes locks — adding an indexed column to a large table can block writes for the duration — so build indexes with CREATE INDEX CONCURRENTLY, which Alembic will not generate for you. And never edit an applied migration: it means two databases claiming the same version with different schemas, and nothing will tell you.

Rolling out without dropping requests

Three things have to be true, and they are usually all missing at once.

Readiness must actually mean ready. The orchestrator sends traffic when the probe passes, so a probe that answers before the database pool is warm sends real users into errors. StayHub's health check reports each dependency separately, which is what makes it usable here:

{"status":"ok","database":true,"elasticsearch":true}

Note the judgement encoded in status: no database is degraded, no search is not. Only the first should stop traffic — treating a slow search cluster as unreadiness takes down an API that was working.

Shutdown must be graceful. On SIGTERM, uvicorn stops accepting new connections and waits for in-flight requests to finish. That only works if the signal reaches it: a shell-form CMD puts /bin/sh at PID 1 and it does not forward signals, so the container is killed after the grace period with requests still in flight. Exec form — CMD ["uvicorn", ...] — makes uvicorn PID 1 and fixes it.

lifespan shutdown code runs at that point, which is where connections and clients get closed:

    yield

Everything after the yield is shutdown. Keep it fast — the orchestrator's grace period is finite, typically 30 seconds.

The load balancer needs to stop first. Deregistration is not instantaneous, so a container that exits the moment it receives SIGTERM can still be sent requests for a second or two. The usual fix is a preStop hook that sleeps briefly before shutdown begins — unglamorous and the difference between a clean deploy and a handful of 502s each time.

What runs the container

The image is the same everywhere; what supervises it is not, and the choice changes what you have to build yourself.

The honest advice is to take the highest-level option that fits. A managed platform gives you readiness probes, rolling deploys, TLS and log aggregation on day one; Kubernetes gives you all of that as configuration you write and maintain. That is worth it at a certain size and expensive before it.

The serverless option deserves a specific note, because FastAPI runs there via an adapter:

from mangum import Mangum

handler = Mangum(app)

Three things change. Cold starts add latency to the first request after idle, which for a Python image is often over a second. Connection pooling does not work the way it does in a long-lived process — each concurrent invocation is its own environment, so a proxy like RDS Proxy or PgBouncer stops being optional. And background tasks are unreliable: the runtime may freeze the environment as soon as the response is returned, so anything queued after it may simply not run.

A deploy pipeline that is worth having

Four steps cover most of it, and the ordering is the interesting part:

jobs:
  test:
    steps:
      - run: pytest -q --cov=app
      - run: ruff check app tests

  build:
    needs: test
    steps:
      - run: docker build --platform linux/amd64 -t $REGISTRY/api:$GITHUB_SHA .
      - run: docker push $REGISTRY/api:$GITHUB_SHA

  migrate:
    needs: build
    steps:
      - run: alembic upgrade head

  deploy:
    needs: migrate
    steps:
      - run: deploy $REGISTRY/api:$GITHUB_SHA

Tests before build, so a broken image is never produced. Migrations after build and before deploy, so the schema is ready when the new code arrives — and because they are backwards-compatible, the old code keeps working in the gap. Tagged by commit, so the deploy step and a later rollback name the same artefact.

Two things worth adding once the basics work. A smoke test against the deployed URL, so a green pipeline means the thing actually answers rather than that the deploy command exited zero. And a manual approval before production, which costs nothing and catches the deploy nobody meant to trigger.

Rolling back

The most valuable property of the pipeline above is that rollback is one command:

deploy $REGISTRY/api:$PREVIOUS_SHA

Which works for code and does not work for schema. A migration that dropped a column cannot be undone by deploying an older image — the data is gone. That is the real reason for the two-deploy discipline above: expand, deploy, contract, so there is always a version of the schema both releases can run against.

Alembic's downgrade is worth writing and worth distrusting. It is fine for reversing an added column; it cannot recover data a destructive migration removed. The reliable answer for the destructive kind is a backup taken immediately before, and knowing how long a restore takes — which is a number worth measuring before you need it rather than during.

When it works locally and not in the cluster

The same short list explains most of these, in rough order of frequency.

1. Bound to the wrong interface. Missing --host 0.0.0.0. Everything looks healthy and nothing arrives.
2. A hostname that only exists on your machine. localhost:5433 in a container is that container. Service names and container-side ports, always.
3. A missing environment variable. With a default, it starts and misbehaves; without one, it fails at boot, which is what you want.
4. CORS. The deployed frontend's origin is not the localhost one in your allowlist. Exact match — scheme, host and port.
5. Migrations not run. relation "bookings" does not exist.
6. Connection pool exhausted. Fine with one worker locally, not with twelve.
7. Timeouts. A request that takes 40 seconds locally meets a 30-second proxy timeout and returns 504 — from the proxy, so your logs show nothing at all.
8. Wheels for the wrong platform. Building on an Apple Silicon machine for an x86 cluster produces an image that dies at import. Use --platform linux/amd64, and check with file on a compiled .so.

The first diagnostic in every case is the same: docker logs, or the platform's equivalent. Which only helps if PYTHONUNBUFFERED=1 is set — without it, a crashing process loses whatever was still buffered, and you get an empty log for the one event you needed to see.

Scale in the right order

When it is slow, the useful order is roughly the inverse of how tempting each option is.

Find the actual bottleneck first. Duration percentiles per route, from lesson 15. It is nearly always one endpoint and nearly always the database.

Then indexes and N+1. A missing index or one lazy relationship in a list endpoint costs more than the framework spends all day. Lesson 6 covers both.

Then caching. Anything computed repeatedly and changing rarely — the amenity vocabulary, a search facet. Redis, or HTTP caching headers if a proxy can do it for you.

Then more processes. Workers within a container, then more containers, watching the connection arithmetic each time.

Then read replicas, and only if reads genuinely dominate — replication lag means a write followed immediately by a read can return stale data, which surprises users in ways that are hard to explain.

Reaching for horizontal scale before the profile is a way to pay more money for the same latency. Four containers running the same N+1 query serve four times the traffic at exactly the same speed per request.

Turning off what should not be public

A few defaults are right for development and wrong once the API is reachable.

# the FastAPI(...) call from lesson 1, with three arguments made conditional:
    docs_url="/docs" if settings.expose_docs else None,
    redoc_url=None,
    openapi_url="/openapi.json" if settings.expose_docs else None,

Whether to publish the documentation is a real decision rather than an obvious one. A public API should — it is how people integrate. An internal one probably should not, since the OpenAPI document is a complete map of your surface including parameter names and constraints. Making it a setting means the answer can differ per environment without a code change.

The same applies to /metrics, which leaks traffic volumes and error rates, and to anything that reports versions. Stack traces are already handled — lesson 8's handler returns a generic message — but it is worth confirming debug=True is nowhere near a deployed application, because Starlette's debug page renders the traceback and local variables into the response.

Rate limiting, which nothing does for you

FastAPI has none built in, and the endpoints that need it are predictable: login, registration, password reset, anything expensive, anything that sends a message.

Login is the important one. bcrypt makes offline cracking expensive and does nothing about online guessing, and the deliberately vague "email or password is incorrect" means an attacker learns nothing per attempt except by volume — so volume is what has to be limited.

Two places to do it. At the edge, where nginx or a WAF rejects before the request costs anything:

limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;

location /api/v1/auth/login {
    limit_req zone=login burst=3 nodelay;
    proxy_pass http://api:8000;
}

Or in the application, as a dependency, when the limit needs to know something the proxy does not — per user rather than per IP, or per plan. The counter must live in Redis rather than process memory, because workers share nothing: an in-process limit of five is five per worker, so four workers means twenty.

Return 429 with a Retry-After header, so a well-behaved client backs off instead of hammering.

Backups, and the part people skip

Managed databases take automated backups, which is the easy half. The half that matters is whether a restore works, and how long it takes.

Three numbers worth knowing before you need them: how far back the backups reach, how long a full restore takes, and how much data a restore would lose — the gap between the last backup and the failure. All three are measurable in an afternoon and unknowable during an incident.

The one to actually rehearse is the restore. A backup that has never been restored is a file, not a backup, and the failure modes — a missing extension, an incompatible version, a permission that was never captured — only appear when you try.

The first week in production

Three things worth doing once traffic is real, none of which can be done in advance.

Look at the latency distribution, not the average. The p99 is usually a surprise, and it usually points at one endpoint doing something the others do not — an N+1, a missing index, a third-party call nobody timed.

Read the error log rather than counting it. A rate tells you whether something changed; reading the lines tells you what people are actually hitting. Frequently it is a client sending something you did not anticipate, which is a validation message worth improving rather than a bug.

Check the numbers you guessed. Worker count, pool size, timeouts and container memory were all chosen before there was evidence. Now there is: connection pool usage against its ceiling, memory against the limit, and whether any request is anywhere near the proxy timeout.

The one measurement that repays itself fastest is the pool. It is invisible until it is exhausted, and then everything fails at once — so knowing you are at 30% rather than 90% is the difference between a planned change and an incident.

Before the first deploy

• Secrets from the environment, no defaults, nothing in the image.
• --host 0.0.0.0, --proxy-headers, workers set deliberately.
• Connection arithmetic checked against max_connections.
• Migrations as a separate step, backwards-compatible.
• Readiness probe that means ready; exec-form CMD so SIGTERM arrives.
• JSON logs with a request id, going somewhere you can search.
• TLS, a body size limit and timeouts at the proxy.
• /docs and /metrics either protected or disabled — FastAPI(docs_url=None) if the API is not public.
• A rollback you have actually tried.

Next: interview questions — the whole track compressed into the questions a FastAPI role actually asks.