03Building blocksCaching
03 · Building block
Caching
Use cache-aside, write-through, and write-behind strategies while making invalidation and hot keys explicit.Architecture map
See where the component sits in a real system.
Write
Application clients enters through Application service. Write path owns validation and commits the durable record to Source database.
Propagate
Invalidation channel separates the committed write from background work. Refill/refresh workers can retry safely while it builds Distributed cache.
Read
Cache-aside read path serves from Distributed cache, then checks authoritative state whenever freshness, policy, or correctness requires it. It also consults Consistency policy as an explicit dependency.
Say this first: The cache is an acceleration layer; the database remains truth and refill/invalidation behavior is explicit.
Open the full whiteboard ↗Explain every boundary before adding more boxes.
The cache is an acceleration layer; the database remains truth and refill/invalidation behavior is explicit.
Use cache-aside, write-through, and write-behind strategies while making invalidation and hot keys explicit.
Track hit ratio · origin QPS · p95 latency · eviction · hot-key share · stale age. State average and peak load, stored bytes, bandwidth or open connections, and the growth horizon before choosing a partitioning strategy.
End-to-end walkthrough
Trace the architecture in this order.
- 01
Enter and classify the request
Application clients → Application serviceHot reads + writes enters over HTTPS / RPC. Application service handles identity, admission, routing, and request context; it deliberately does not own domain truth.
- 02
Validate, then cross the commit boundary
Application service → Write path → Source databaseWrite path receives the command, checks invariants and retry identity, then uses write truth to update Source database. The user-visible mutation is accepted only after this boundary succeeds.
- 03
Move replayable work off the request path
Write path → Invalidation channel → Refill/refresh workers → Distributed cacheWrite path emits publish after commit; Refill/refresh workers uses consume and invalidate / refresh to build Distributed cache. Consumers must tolerate duplicate delivery and stale retries because this path is asynchronous.
- 04
Serve reads from the right authority
Application service → Cache-aside read path → Distributed cache / Source databaseCache-aside read path uses GET key for the common, read-optimized path and strong read when correctness or repair requires authoritative state. The API must state the freshness promise instead of hiding it.
- 05
Contain the dependency boundary
Write path → Consistency policydependency call crosses into Consistency policy. Treat timeouts as ambiguous, use a deadline and idempotent retry or reconciliation, and keep the core state recoverable when the dependency is unavailable.
Ownership ledger
Why each box exists—and what it must defend.
| Component | Owns | Why it exists | Interviewer probe |
|---|---|---|---|
| Application serviceCache policy owner | Identity, admission, routing | Protects the system edge and attaches trusted context before domain work begins. | Timeout budgets, quotas, regional routing |
| Write pathCommit then invalidate/update | Write invariants and retry identity | Serializes or conditionally applies state changes before acknowledging success. | Concurrent writes, deduplication, hot ownership |
| Source databaseAuthoritative values | Authoritative durable state | Provides the one record used to resolve disputes, recover, and rebuild projections. | Partition key, replication, consistency |
| Invalidation channelVersioned key changes | Durable asynchronous handoff | Absorbs bursts and lets slow or optional work retry independently of the request. | Ordering key, lag, retention, dead letters |
| Refill/refresh workersSingle-flight + prewarm | Replayable processing | Runs expensive, fan-out, or side-effecting work with leases and bounded retries. | Idempotency, poison work, autoscaling |
| Distributed cacheTTL + version + eviction | Rebuildable query state | Shapes data for the dominant reads without weakening the write-side invariant. | Freshness, versioning, rebuild time |
| Cache-aside read pathHit, miss, coalesce | Read composition and freshness policy | Chooses authoritative or derived state and returns a stable client contract. | Fan-out, cache policy, partial results |
| Consistency policyFreshness + negative cache | External capability, not local truth | Keeps a specialized or third-party concern behind a replaceable contract. | Ambiguous timeout, circuit breaking, fallback |
Physical design
Name the database, shard key, indexes, and guarantees.
- Database + storage
- Redis Cluster/Memcached provides shared cache; optional local L1 serves immutable hot values; the origin database remains truth.
- Partitioning / sharding
- Hash full keys across virtual slots; use hash tags only for required colocation and replicate/memoize proven hot keys.
- Indexes
- Direct key lookup, TTL expiry, frequency sketch/LFU admission, and per-tenant memory accounting.
- Replication + consistency
- Cache-aside is eventual. Source versions or versioned keys are required where stale data changes behavior; cache loss must be survivable.
- Cache, queue + recovery
- Use miss coalescing, stale-while-revalidate, negative caching, jittered TTLs, and origin admission control.
- Capacity math
- Estimate object size, working set, hit target, QPS, TTL churn, hot-key share, eviction, and origin capacity at zero hits.
- Alternative rejected
- Write-through simplifies callers but couples writes to cache health; cache-aside is the default unless synchronous population is required.
Deep-dive candidates
Pick one risk and explain the mechanism, alternative, and cost.
Cache-aside
The application checks cache, reads the origin on a miss, and fills the value. It is flexible but needs stampede control.
Tie the mechanism back to Source database, Distributed cache, and the stated track hit ratio · origin qps · p95 latency · eviction · hot-key share · stale age envelope.Read-through and write-through
A cache layer owns loading or synchronously updates cache and origin, simplifying callers while increasing coupling.
Tie the mechanism back to Source database, Distributed cache, and the stated track hit ratio · origin qps · p95 latency · eviction · hot-key share · stale age envelope.Write-behind
Buffer writes through the cache only when temporary inconsistency and loss risk are acceptable and recovery is designed.
Tie the mechanism back to Source database, Distributed cache, and the stated track hit ratio · origin qps · p95 latency · eviction · hot-key share · stale age envelope.Failure pressure test
Show detection, containment, recovery, and evidence.
The topic-specific correctness risk
Hot-key expiry can cause a thundering herd that overwhelms the origin.
Track failed promises at Source database and Distributed cache.Invalidation channel or Refill/refresh workers falls behind
Bound admission, scale on oldest-work age, retry with jitter, and isolate poison work before lag becomes unbounded.
Oldest event age · retry rate · dead-letter volume · projection freshnessSource database is slow or unavailable
Apply a deadline, preserve retry identity, fail over only within the stated consistency model, and reconcile any ambiguous result.
Commit p99 · timeout rate · replication lag · recovery time- Functional requirements and non-goals
- Peak traffic, storage, bandwidth, and growth
- Entities, APIs, idempotency, and pagination
- Source of truth and consistency promise
- Partition key, replicas, caches, and hot spots
- Retries, backpressure, failover, and reconciliation
- Latency, saturation, correctness, and recovery metrics
- Security, migration, cost, and multi-region evolution
Read the solid request path first, stop at the source of truth, then follow the dashed event path into workers and rebuildable read models. Every arrow names a contract you should be ready to defend.
- 01
Caller — Requests a cacheable object Define the output contract before moving to the next owner.
- 02
Cache — Checks key and TTL Define the output contract before moving to the next owner.
- 03
Origin — Loads canonical state Define the output contract before moving to the next owner.
- 04
Fill path — Stores the result Define the output contract before moving to the next owner.
- 05
Invalidator — Expires or updates keys Define the output contract before moving to the next owner.
- 06
Protection — Coalesces misses Confirm the result and emit the evidence needed to reconcile it.
Lesson spine
What you need to understand.
A cache trades freshness and invalidation work for lower latency, lower cost, or origin protection.
Cache-aside
The application checks cache, reads the origin on a miss, and fills the value. It is flexible but needs stampede control.
Read-through and write-through
A cache layer owns loading or synchronously updates cache and origin, simplifying callers while increasing coupling.
Write-behind
Buffer writes through the cache only when temporary inconsistency and loss risk are acceptable and recovery is designed.
Invalidation
Use TTLs for bounded staleness, events for fast convergence, or versioned keys to make old results unable to validate new reads.
Eviction and hot keys
Choose LRU, LFU, or size-aware policies; replicate or memoize hot immutable values and coalesce concurrent misses.
Failure behavior
Define whether cache loss fails open to the origin, serves stale data, or rejects traffic to protect a fragile dependency.
Before the boxes
Frame the decision.
What must work
Use cache-aside, write-through, and write-behind strategies while making invalidation and hot keys explicit.
What changes the design
Track hit ratio · origin QPS · p95 latency · eviction · hot-key share · stale age
What owns the truth
Identify the component that commits authoritative state, then separate synchronous confirmation from derived work.
What stays simple
Do not add global coordination, multi-region writes, or a specialized store until a requirement earns the complexity.
Decision table
Make the trade-offs explicit.
| Decision | Defensible position | Cost to acknowledge |
|---|---|---|
| Primary mechanism | Cache-aside is resilient and simple; write-through improves coherence with more coupling. | The stronger guarantee usually adds coordination, latency, state, or operational work. |
| Sync vs. async | Keep only correctness-critical confirmation synchronous. Move derived views, notifications, analytics, and cleanup behind a durable boundary. | Async work needs idempotency, lag monitoring, replay, and a product definition for partial completion. |
| Simple vs. scaled | Begin with one logical owner and a clear API. Partition or replicate only the resource proven to be the first bottleneck. | Migration requires stable identities, versioned contracts, backfill, and a rollback path. |
Failure review
Design the recovery path.
Topic-specific risk
Hot-key expiry can cause a thundering herd that overwhelms the origin.
ResponsePersist enough identity and state to distinguish retry, resume, compensation, and operator repair.
Dependency timeout
A timeout is ambiguous: the remote side may have failed, succeeded, or still be running.
ResponseUse deadlines, bounded backoff with jitter, idempotency keys, and a status or reconciliation path.
Overload or skew
Average capacity can look healthy while a tenant, key, partition, region, or expensive request saturates one owner.
ResponseExpose queue depth and hot-key share, apply backpressure, isolate tenants, and degrade optional work before correctness.
Evidence + level bar
Prove the design can be operated.
Health of the promise
Measure user-visible latency or freshness, correctness drift, saturation, retry volume, and time to recover. Alert on the failed promise—not only CPU.
Complete and clear
Finish the happy path, identify the state owner, choose reasonable building blocks, and explain one scale mechanism.
Trade-offs and failure
Separate read and write paths, define consistency, explain partitioning, and make duplicate or partial failure safe.
Evolution and operations
Discuss multi-region boundaries, migration, tenant isolation, capacity, observability, and how the architecture changes over time.
Interview language
Open the deep dive with a claim.
“For Caching, the decision I want to make explicit is this: Cache-aside is resilient and simple; write-through improves coherence with more coupling. I’ll trace the state-changing path first, show where the result becomes durable, then test the design against the highest-risk failure and our target scale.”
08 · Retrieval check
Can you defend it without the page?
- For Caching, where is the correctness boundary and which failure would you test first?
- Which component owns committed truth, and what event or response proves the commit?
- Where is the first scaling or coordination bottleneck under the stated envelope?
- What happens after an ambiguous timeout or duplicate operation?
- Which complexity would you remove at one hundredth of the scale?