04QuestionsTop-K videos

04 · Worked prompt

Top-K videos

Design streaming popularity scores across windows while controlling skew, freshness, and ranking cost.

30 minInterview blueprint

INTERVIEW RUBRIC

What a passing answer must show

100 points · 45 minutes

20pts
Scope the problem
0–5 min
Prioritize the core flows, state the scale, and name the non-goals.
15pts
Define contracts
5–10 min
Identify durable entities, APIs, idempotency, and the source of truth.
30pts
Complete the diagram
10–25 min
Trace one write path and one read path. Label the commit boundary and async work.
20pts
Lead one deep dive
25–38 min
Choose the highest-risk trade-off and explain the mechanism, alternative, and cost.
15pts
Prove reliability
38–45 min
Walk a failure, recovery, metric, bottleneck, and evolution path.

DRAW THIS FIRST

One complete box-and-arrow design

Design streaming popularity scores across windows while controlling skew, freshness, and ranking cost.

Top-K videos system architecture. Event-time windows and watermarks make popularity reproducible; candidate heaps bound global merge cost. Request path: Playback clients to Event/query edge to View ingestion to View event log. Asynchronous path: Window partitions to Window aggregators. Read path: Ranking API to Ranking snapshots. External dependency: Video metadata service. — Top-K videos · system architecture

DEFEND THE DIAGRAM

Explain every boundary before adding more boxes.

Event-time windows and watermarks make popularity reproducible; candidate heaps bound global merge cost.

INTERVIEW CONTRACT

Design streaming popularity scores across windows while controlling skew, freshness, and ranking cost.

CAPACITY QUESTIONS TO QUANTIFY

Huge event volume · Zipfian popularity · minute freshness · many windows. State average and peak load, stored bytes, bandwidth or open connections, and the growth horizon before choosing a partitioning strategy.

End-to-end walkthrough

Trace the architecture in this order.

01
Enter and classify the request
Playback clients → Event/query edge
View events + chart reads enters over HTTPS / RPC. Event/query edge handles identity, admission, routing, and request context; it deliberately does not own domain truth.
02
Validate, then cross the commit boundary
Event/query edge → View ingestion → View event log
View ingestion receives the command, checks invariants and retry identity, then uses append event to update View event log. The user-visible mutation is accepted only after this boundary succeeds.
03
Move replayable work off the request path
View ingestion → Window partitions → Window aggregators → Ranking snapshots
View ingestion emits publish after commit; Window aggregators uses consume and merge candidates to build Ranking snapshots. Consumers must tolerate duplicate delivery and stale retries because this path is asynchronous.
04
Serve reads from the right authority
Event/query edge → Ranking API → Ranking snapshots / View event log
Ranking API uses ordered list for the common, read-optimized path and strong read when correctness or repair requires authoritative state. The API must state the freshness promise instead of hiding it.
05
Contain the dependency boundary
Ranking API → Video metadata service
dependency call crosses into Video metadata service. Treat timeouts as ambiguous, use a deadline and idempotent retry or reconciliation, and keep the core state recoverable when the dependency is unavailable.

Ownership ledger

Why each box exists—and what it must defend.

Component	Owns	Why it exists	Interviewer probe
Event/query edgeValidate + sample abuse	Identity, admission, routing	Protects the system edge and attaches trusted context before domain work begins.	Timeout budgets, quotas, regional routing
View ingestionDedupe + event time	Write invariants and retry identity	Serializes or conditionally applies state changes before acknowledging success.	Concurrent writes, deduplication, hot ownership
View event logDurable raw events	Authoritative durable state	Provides the one record used to resolve disputes, recover, and rebuild projections.	Partition key, replication, consistency
Window partitionsVideo + event-time key	Durable asynchronous handoff	Absorbs bursts and lets slow or optional work retry independently of the request.	Ordering key, lag, retention, dead letters
Window aggregatorsCounts + local top-K	Replayable processing	Runs expensive, fan-out, or side-effecting work with leases and bounded retries.	Idempotency, poison work, autoscaling
Ranking snapshotsMerged ordered candidates	Rebuildable query state	Shapes data for the dominant reads without weakening the write-side invariant.	Freshness, versioning, rebuild time
Ranking APIWindow + rank version	Read composition and freshness policy	Chooses authoritative or derived state and returns a stable client contract.	Fan-out, cache policy, partial results
Video metadata serviceAvailability + policy	External capability, not local truth	Keeps a specialized or third-party concern behind a replaceable contract.	Ambiguous timeout, circuit breaking, fallback

Physical design

Name the database, shard key, indexes, and guarantees.

Database + storage: Kafka stores views; Flink/Kafka Streams computes windows; Redis ZSETs serve top-K; ClickHouse/Druid stores history; SQL stores video metadata.
Partitioning / sharding: Partition raw events by video_id, compute shard-local candidates, then merge hierarchical top-K per region/category/window.
Indexes: Unique event_id when needed, analytical (window, region, video), and Redis keys by region + window + category.
Replication + consistency: Counts use event time and watermarks. Served top-K is freshness-bounded/versioned; historical reports reconcile late events.
Cache, queue + recovery: Checkpoint stream state, dedup, cache only small top-K lists, and join metadata by version.
Capacity math: Estimate events/sec, unique videos/window, viral skew, leaderboard count, and target update interval.
Alternative rejected: ORDER BY raw counts cannot update continuously at scale; incremental windows plus hierarchical top-K bound the work.

Deep-dive candidates

Pick one risk and explain the mechanism, alternative, and cost.

Heavy hitters

Keep exact counts per active video when affordable or use Space-Saving for bounded candidate memory

Top-K does not require sorting the whole universe on every read

Multiple windows

Aggregate base time buckets and compose larger windows

Independent pipelines multiply compute and correction logic

Late data

Version snapshots and apply bounded corrections until finalization

Ranking freshness and final analytic truth can have different deadlines

Failure pressure test

Show detection, containment, recovery, and evidence.

Hot partition

Salt very popular IDs for counting and merge partials

max partition share

Bot surge

Quarantine suspicious traffic and publish quality flags

filtered view ratio

Processor recovery

Restore checkpoint and idempotently replace snapshot version

reprocessed event count

Before you finish, explicitly cover

Functional requirements and non-goals
Peak traffic, storage, bandwidth, and growth
Entities, APIs, idempotency, and pagination
Source of truth and consistency promise
Partition key, replicas, caches, and hot spots
Retries, backpressure, failover, and reconciliation
Latency, saturation, correctness, and recovery metrics
Security, migration, cost, and multi-region evolution

SAY THIS WHILE YOU DRAW

A four-part talk track

Scope
“I’ll prioritize rank videos over multiple time windows and refresh results within minutes.”
Scale
“The design changes around huge event volume · zipfian popularity · minute freshness · many windows.”
Decision
“Exact counts support audit; heavy-hitter sketches bound memory for ranking.”
Risk
“The first failure I want to pressure-test is: Bots, late events, and hot partitions can destabilize ranks.”

Reference details

Open these only after you can explain the diagram above without reading.

01Requirements and state lifecycle4 requirements

Rank videos over multiple time windows
Refresh results within minutes
Handle extreme popularity skew and bot filtering
Support exact audit totals separately when needed

Top-K videos · state lifecycle

state lifecycle

Each transition must be durable, observable, and safe to retry.

open window

watermark passed

correcting

finalized

expired

02Data model and APIs4 entities · 3 interfaces

Core entities

ViewEventevent_id, video_id, viewer_hash, occurred_atOwner: Event log

WindowCountvideo_id, window, partition, countOwner: Stream state

CandidateSetwindow, partition, top_items, watermarkOwner: Processor

RankingSnapshotwindow, items, generated_at, versionOwner: Serving store

External interfaces

POST /internal/views

Ingest a validated view event

GET /v1/rankings/videos?window=1h&limit=100

Read a versioned ranking snapshot

GET /v1/videos/{id}/views?window=1d

Read an exact or bounded-error count contract

03Deep dives and trade-offsChoose one

Heavy hitters

Keep exact counts per active video when affordable or use Space-Saving for bounded candidate memory

Top-K does not require sorting the whole universe on every read

Multiple windows

Aggregate base time buckets and compose larger windows

Independent pipelines multiply compute and correction logic

Late data

Version snapshots and apply bounded corrections until finalization

Ranking freshness and final analytic truth can have different deadlines

04Failures, recovery, and evidence3 scenarios

Hot partition

Salt very popular IDs for counting and merge partials

max partition share

Bot surge

Quarantine suspicious traffic and publish quality flags

filtered view ratio

Processor recovery

Restore checkpoint and idempotently replace snapshot version

reprocessed event count

05What makes the answer seniorInterviewer signals

Ask whether rankings may be approximate before choosing the algorithm
Keep metadata hydration outside the counting hot path
Expose watermark age so freshness is measurable
Primary trade-off: Exact counts support audit; heavy-hitter sketches bound memory for ranking.

BEFORE THE NEXT QUESTION

Can you redraw it from memory?

Name the source of truth.
Trace the write and read paths.
Defend one trade-off.
Recover from one failure.

Scope the problem

Define contracts

Complete the diagram

Lead one deep dive

Prove reliability

Write

Propagate

Read

Trace the architecture in this order.

Enter and classify the request

Validate, then cross the commit boundary

Move replayable work off the request path

Serve reads from the right authority

Contain the dependency boundary

Why each box exists—and what it must defend.

Name the database, shard key, indexes, and guarantees.

Pick one risk and explain the mechanism, alternative, and cost.

Heavy hitters

Multiple windows

Late data

Show detection, containment, recovery, and evidence.

Hot partition

Bot surge

Processor recovery

Reference details

Core entities

External interfaces

Heavy hitters

Multiple windows

Late data

Hot partition

Bot surge

Processor recovery

Can you redraw it from memory?