03Building blocksAPI gateways
03 · Building block
API gateways
Centralize routing, authentication, quotas, and request shaping without turning the edge into an opaque bottleneck.Architecture map
See where the component sits in a real system.
Write
Web/mobile/partners enters through API gateway. Request policy chain owns validation and commits the durable record to Config/policy store.
Propagate
Telemetry stream separates the committed write from background work. Control plane can retry safely while it builds Local config snapshot.
Read
Service router serves from Local config snapshot, then checks authoritative state whenever freshness, policy, or correctness requires it. It also consults Domain service fleet as an explicit dependency.
Say this first: The gateway enforces cross-cutting policy and routing while domain services retain business truth.
Open the full whiteboard ↗Explain every boundary before adding more boxes.
The gateway enforces cross-cutting policy and routing while domain services retain business truth.
Centralize routing, authentication, quotas, and request shaping without turning the edge into an opaque bottleneck.
Gateway p99 · connections · policy lookup · backend fan-out · failover. State average and peak load, stored bytes, bandwidth or open connections, and the growth horizon before choosing a partitioning strategy.
End-to-end walkthrough
Trace the architecture in this order.
- 01
Enter and classify the request
Web/mobile/partners → API gatewayREST, GraphQL, gRPC enters over HTTPS / RPC. API gateway handles identity, admission, routing, and request context; it deliberately does not own domain truth.
- 02
Validate, then cross the commit boundary
API gateway → Request policy chain → Config/policy storeRequest policy chain receives the command, checks invariants and retry identity, then uses commit to update Config/policy store. The user-visible mutation is accepted only after this boundary succeeds.
- 03
Move replayable work off the request path
Request policy chain → Telemetry stream → Control plane → Local config snapshotRequest policy chain emits publish after commit; Control plane uses consume and push config to build Local config snapshot. Consumers must tolerate duplicate delivery and stale retries because this path is asynchronous.
- 04
Serve reads from the right authority
API gateway → Service router → Local config snapshot / Config/policy storeService router uses local policy for the common, read-optimized path and strong read when correctness or repair requires authoritative state. The API must state the freshness promise instead of hiding it.
- 05
Contain the dependency boundary
Request policy chain → Domain service fleetHTTP/gRPC crosses into Domain service fleet. Treat timeouts as ambiguous, use a deadline and idempotent retry or reconciliation, and keep the core state recoverable when the dependency is unavailable.
Ownership ledger
Why each box exists—and what it must defend.
| Component | Owns | Why it exists | Interviewer probe |
|---|---|---|---|
| API gatewayTLS, auth, quota, routing | Identity, admission, routing | Protects the system edge and attaches trusted context before domain work begins. | Timeout budgets, quotas, regional routing |
| Request policy chainValidate + transform | Write invariants and retry identity | Serializes or conditionally applies state changes before acknowledging success. | Concurrent writes, deduplication, hot ownership |
| Config/policy storeRoutes + auth + versions | Authoritative durable state | Provides the one record used to resolve disputes, recover, and rebuild projections. | Partition key, replication, consistency |
| Telemetry streamLogs, traces, audit | Durable asynchronous handoff | Absorbs bursts and lets slow or optional work retry independently of the request. | Ordering key, lag, retention, dead letters |
| Control planePublish config + certificates | Replayable processing | Runs expensive, fan-out, or side-effecting work with leases and bounded retries. | Idempotency, poison work, autoscaling |
| Local config snapshotFast path, no remote lookup | Rebuildable query state | Shapes data for the dominant reads without weakening the write-side invariant. | Freshness, versioning, rebuild time |
| Service routerDiscovery + timeout budget | Read composition and freshness policy | Chooses authoritative or derived state and returns a stable client contract. | Fan-out, cache policy, partial results |
| Domain service fleetBusiness logic + state | External capability, not local truth | Keeps a specialized or third-party concern behind a replaceable contract. | Ambiguous timeout, circuit breaking, fallback |
Physical design
Name the database, shard key, indexes, and guarantees.
- Database + storage
- Stateless gateways use local route/policy snapshots from etcd/Consul/config DB; Redis is optional for shared coarse quotas.
- Partitioning / sharding
- Partition runtime traffic by region/route; shard control state by organization/service only when configuration volume earns it.
- Indexes
- Route trie by host/path/method, policy map by principal/service, certificate by SNI, and versioned config audit.
- Replication + consistency
- Each request uses one config snapshot. Edge owns authentication/coarse admission; services retain domain authorization and truth.
- Cache, queue + recovery
- Cache signing keys/discovery/config locally; use deadline budgets, circuit breakers, last-known-good config, and async telemetry.
- Capacity math
- Estimate QPS, connections, TLS handshakes, route count, policy p99, bandwidth, and zone/regional failover.
- Alternative rejected
- Business workflows in the gateway create a hidden monolith; centralize edge policy, not domain ownership.
Deep-dive candidates
Pick one risk and explain the mechanism, alternative, and cost.
Core responsibilities
Terminate TLS, route, authenticate, enforce coarse quotas, validate shape, transform protocols, and attach trace context.
Tie the mechanism back to Config/policy store, Local config snapshot, and the stated gateway p99 · connections · policy lookup · backend fan-out · failover envelope.Backend for frontend
Give web, mobile, or partner clients tailored aggregation when their request patterns genuinely differ.
Tie the mechanism back to Config/policy store, Local config snapshot, and the stated gateway p99 · connections · policy lookup · backend fan-out · failover envelope.Authentication
Validate signed tokens locally when possible; use introspection or policy services when revocation and dynamic authorization require it.
Tie the mechanism back to Config/policy store, Local config snapshot, and the stated gateway p99 · connections · policy lookup · backend fan-out · failover envelope.Failure pressure test
Show detection, containment, recovery, and evidence.
The topic-specific correctness risk
Heavy aggregation or synchronous auth dependencies can amplify latency and outages.
Track failed promises at Config/policy store and Local config snapshot.Telemetry stream or Control plane falls behind
Bound admission, scale on oldest-work age, retry with jitter, and isolate poison work before lag becomes unbounded.
Oldest event age · retry rate · dead-letter volume · projection freshnessConfig/policy store is slow or unavailable
Apply a deadline, preserve retry identity, fail over only within the stated consistency model, and reconcile any ambiguous result.
Commit p99 · timeout rate · replication lag · recovery time- Functional requirements and non-goals
- Peak traffic, storage, bandwidth, and growth
- Entities, APIs, idempotency, and pagination
- Source of truth and consistency promise
- Partition key, replicas, caches, and hot spots
- Retries, backpressure, failover, and reconciliation
- Latency, saturation, correctness, and recovery metrics
- Security, migration, cost, and multi-region evolution
Read the solid request path first, stop at the source of truth, then follow the dashed event path into workers and rebuildable read models. Every arrow names a contract you should be ready to defend.
- 01
Client — Sends an external contract Define the output contract before moving to the next owner.
- 02
Gateway fleet — Terminates and authenticates Define the output contract before moving to the next owner.
- 03
Policy layer — Applies quotas and routing Define the output contract before moving to the next owner.
- 04
Backend service — Owns domain behavior Define the output contract before moving to the next owner.
- 05
Aggregator — Combines selected responses Define the output contract before moving to the next owner.
- 06
Tracing — Connects the hop Confirm the result and emit the evidence needed to reconcile it.
Lesson spine
What you need to understand.
An API gateway centralizes edge policy while keeping business rules and authoritative state inside domain services.
Core responsibilities
Terminate TLS, route, authenticate, enforce coarse quotas, validate shape, transform protocols, and attach trace context.
Backend for frontend
Give web, mobile, or partner clients tailored aggregation when their request patterns genuinely differ.
Authentication
Validate signed tokens locally when possible; use introspection or policy services when revocation and dynamic authorization require it.
Aggregation
Parallelize independent reads with deadlines and partial-response policy, but do not create a new distributed monolith at the edge.
Resilience
Run stateless gateways across zones, protect downstreams with budgets and circuit breakers, and keep a minimal fallback route.
Avoid coupling
Version configuration, keep domain logic out, and give services end-to-end identity so the gateway is not the only authorization layer.
Before the boxes
Frame the decision.
What must work
Centralize routing, authentication, quotas, and request shaping without turning the edge into an opaque bottleneck.
What changes the design
Gateway p99 · connections · policy lookup · backend fan-out · failover
What owns the truth
Identify the component that commits authoritative state, then separate synchronous confirmation from derived work.
What stays simple
Do not add global coordination, multi-region writes, or a specialized store until a requirement earns the complexity.
Decision table
Make the trade-offs explicit.
| Decision | Defensible position | Cost to acknowledge |
|---|---|---|
| Primary mechanism | One gateway centralizes governance; BFF gateways reduce client coupling but multiply policy surfaces. | The stronger guarantee usually adds coordination, latency, state, or operational work. |
| Sync vs. async | Keep only correctness-critical confirmation synchronous. Move derived views, notifications, analytics, and cleanup behind a durable boundary. | Async work needs idempotency, lag monitoring, replay, and a product definition for partial completion. |
| Simple vs. scaled | Begin with one logical owner and a clear API. Partition or replicate only the resource proven to be the first bottleneck. | Migration requires stable identities, versioned contracts, backfill, and a rollback path. |
Failure review
Design the recovery path.
Topic-specific risk
Heavy aggregation or synchronous auth dependencies can amplify latency and outages.
ResponsePersist enough identity and state to distinguish retry, resume, compensation, and operator repair.
Dependency timeout
A timeout is ambiguous: the remote side may have failed, succeeded, or still be running.
ResponseUse deadlines, bounded backoff with jitter, idempotency keys, and a status or reconciliation path.
Overload or skew
Average capacity can look healthy while a tenant, key, partition, region, or expensive request saturates one owner.
ResponseExpose queue depth and hot-key share, apply backpressure, isolate tenants, and degrade optional work before correctness.
Evidence + level bar
Prove the design can be operated.
Health of the promise
Measure user-visible latency or freshness, correctness drift, saturation, retry volume, and time to recover. Alert on the failed promise—not only CPU.
Complete and clear
Finish the happy path, identify the state owner, choose reasonable building blocks, and explain one scale mechanism.
Trade-offs and failure
Separate read and write paths, define consistency, explain partitioning, and make duplicate or partial failure safe.
Evolution and operations
Discuss multi-region boundaries, migration, tenant isolation, capacity, observability, and how the architecture changes over time.
Interview language
Open the deep dive with a claim.
“For API gateways, the decision I want to make explicit is this: One gateway centralizes governance; BFF gateways reduce client coupling but multiply policy surfaces. I’ll trace the state-changing path first, show where the result becomes durable, then test the design against the highest-risk failure and our target scale.”
08 · Retrieval check
Can you defend it without the page?
- For API gateways, where is the correctness boundary and which failure would you test first?
- Which component owns committed truth, and what event or response proves the commit?
- Where is the first scaling or coordination bottleneck under the stated envelope?
- What happens after an ambiguous timeout or duplicate operation?
- Which complexity would you remove at one hundredth of the scale?