Top 30 System
Design Interview
Questions

Scalability is a system's ability to handle growing load — more users, more data, more requests — without a drop in performance. Vertical scaling ("scale up") means adding more power to a single machine: more CPU, RAM, or faster disks. It's simple (no code changes) but has a hard ceiling — eventually you can't buy a bigger box, and it's a single point of failure. Horizontal scaling ("scale out") means adding more machines and distributing load across them. It scales almost indefinitely and survives individual machine failure, but introduces real complexity: you now need a load balancer, and any state (sessions, data) must be shared or replicated across machines instead of living in one process's memory. Example: a database that outgrows one server gets vertically scaled first (bigger instance), then horizontally scaled via read replicas or sharding once vertical scaling hits its ceiling.

CAP theorem states a distributed system can only guarantee two of three properties at any moment: Consistency (every read sees the latest write), Availability (every request gets a response), and Partition tolerance (the system keeps working when nodes can't talk to each other). In practice, network partitions will happen, so partition tolerance isn't optional — the real choice is between C and A during a partition. A banking ledger picks CP: it would rather reject a request than show a stale balance. DynamoDB and Cassandra pick AP by default: they'd rather serve a slightly stale value than return an error, because for a shopping cart or a social feed, availability matters more than perfect freshness.

A load balancer sits in front of multiple backend servers and routes each incoming request to one of them, so no single server gets overwhelmed and traffic keeps flowing if one server dies. Common algorithms: Round Robin (cycle through servers in order — simple, ignores server load), Least Connections (send to the server with fewest active connections — better for uneven request durations), and Consistent Hashing (map both requests and servers onto a hash ring, so a request always lands near the "same" server — critical for caching layers, because adding/removing one server only reshuffles a small slice of keys instead of all of them).

Strong consistency guarantees every read gets the most recent write immediately, everywhere — needed for a bank balance. Eventual consistency guarantees that if writes stop, all replicas will converge to the same value eventually, but a read right after a write might see stale data — fine for a "like" count or follower count. Causal consistency sits in between: operations that are causally related (a comment reply must appear after the original comment) are seen in the correct order everywhere, even though unrelated operations might not be. Picking the weakest model that's still acceptable for the feature is usually the right move, since stronger consistency always costs latency and availability. This is also the model interviewers expect you to reuse when they ask you to justify a database choice later in the round.

Work top-down with round numbers, out loud. Example for a photo-sharing app with 100M daily active users: if each user posts 1 photo/day, that's 100M writes/day ≈ 1,160 writes/sec average (100M ÷ 86,400s). Peak traffic is usually 2–3x average, so design for ~3,000 writes/sec. If each photo averages 2MB, storage grows by 100M × 2MB = 200TB/day. Reads are typically 10–100x writes for a social app, so plan read capacity accordingly and lean on caching/CDN rather than scaling the database for reads. The exact numbers matter less than showing you can derive scale from assumptions methodically.

1) The browser checks its cache for the DNS record; if missing, it queries a DNS resolver, which walks root → TLD → authoritative nameservers to get the IP. 2) The browser opens a TCP connection to that IP (3-way handshake), then negotiates TLS for HTTPS. 3) It sends an HTTP request, which may first hit a CDN edge node (returns cached static content directly) or a load balancer, which forwards it to an app server. 4) The server processes the request — hitting cache, then the database if needed — and returns an HTTP response. 5) The browser parses the HTML, discovers more resources (CSS/JS/images), and repeats the process for each, often in parallel.

A CDN is a network of geographically distributed edge servers that cache content close to users. When a user in Mumbai requests a static asset (image, video segment, CSS file), it's served from a Mumbai edge node instead of round-tripping to an origin server in Virginia — cutting latency from hundreds of milliseconds to tens. Static content (images, videos) can be cached for long periods; dynamic content is either not cached or cached with short TTLs and invalidated on update. Origin servers only get hit on a "cache miss," which also shields them from most of the traffic.

REST: resource-based URLs over HTTP, simple, cacheable, universally understood — good default for public APIs. GraphQL: clients specify exactly which fields they need in one query, avoiding over-fetching (getting fields you don't need) or under-fetching (needing multiple round trips) — great for mobile apps with varied screen needs and many nested relationships. gRPC: binary protocol (protobuf) over HTTP/2, very low latency, strict typed schemas, supports streaming — the default choice for internal service-to-service calls in a microservices backend where every millisecond and byte matters, but not ideal for public-facing APIs since it's harder for external clients to consume.

A message queue lets producers publish messages that consumers process independently, decoupling the two in time and load. Kafka is a distributed, append-only log built for very high throughput and durability — messages persist and can be replayed, and multiple consumer groups can read the same stream independently. It's the right choice for event streaming, analytics pipelines, and audit logs. RabbitMQ is a traditional message broker with flexible routing (direct, topic, fanout exchanges) and simpler per-message acknowledgment — better suited for task queues where each message should be processed exactly once by one worker, like sending a single email or resizing an image.

Long polling: the client sends a request, the server holds it open until new data is available (or a timeout), responds, and the client immediately re-requests. It works over plain HTTP and is easy to deploy, but each cycle has connection overhead and doesn't scale well to very high message frequency. WebSocket upgrades an HTTP connection into a persistent, full-duplex TCP connection — either side can push data anytime with minimal overhead per message, which is why it's the standard choice for chat apps, live scoreboards, and collaborative editing. The tradeoff is infrastructure: load balancers and servers need to support long-lived stateful connections rather than simple stateless request/response.

Token bucket: a bucket refills with tokens at a fixed rate; each request consumes one token, and requests are rejected when the bucket is empty — allows short bursts up to the bucket size. Leaky bucket: requests queue up and are processed at a fixed constant rate, smoothing out bursts entirely. Sliding window: counts requests in a rolling time window (e.g. last 60 seconds), giving more accurate limiting than a fixed window that resets abruptly. For a distributed system, the limiter's counters need to live in a shared store like Redis (using atomic INCR + TTL) so every API gateway instance sees the same count instead of each enforcing its own local limit. In production, return standard X-RateLimit-Remaining and Retry-After headers so clients can back off gracefully.

SQL databases (Postgres, MySQL) enforce a fixed schema, support strong ACID transactions, and let you run complex joins — ideal when data is relational and correctness/consistency matters, like orders, payments, or inventory. NoSQL databases trade some of that structure for flexibility and horizontal scalability: document stores (MongoDB) suit varied, nested data; wide-column stores (Cassandra) suit massive write-heavy workloads; key-value stores (DynamoDB, Redis) suit simple, ultra-fast lookups. Example: use Postgres for a checkout/order system where correctness is critical, and Cassandra for a write-heavy activity log or sensor data feed where you need to ingest millions of writes per second across many nodes.

You pick a shard key (e.g. user_id), route each row to a shard based on that key (via hashing, ranges, or a lookup table), and each shard runs as an independent database holding only its slice of data. What breaks: a query that needs data from multiple shards (e.g. "top 10 users across the whole platform") now requires scatter-gather across shards and merging results in the app layer instead of one SQL query. Transactions spanning multiple shards lose native ACID guarantees. And resharding — adding a new shard as data grows — means physically moving data between machines, which is why consistent hashing is often used to minimize how much data moves.

In leader-follower replication, all writes go to a single leader, which then propagates changes to one or more followers; followers can serve read traffic, spreading read load, but the leader remains a single point of write failure and a potential bottleneck. In multi-leader replication, multiple nodes (often in different regions) can each accept writes, improving write availability and latency for geographically distributed users — but now concurrent writes to the same data on different leaders can conflict, requiring conflict-resolution logic (e.g. last-write-wins, or app-specific merge rules). Most systems default to leader-follower with read replicas unless multi-region write availability is a hard requirement.

Without an index, finding a row means scanning every row (O(n)). An index is a separate, sorted data structure — usually a B-tree — that maps column values to row locations, so lookups, range scans, and sorted retrieval become O(log n). A B-tree keeps data sorted across a shallow, wide tree of nodes, so the database can binary-search its way to the right leaf in just a few disk reads. The tradeoff: every index speeds up reads on that column but slows down writes (the index must be updated on every insert/update/delete) and takes extra storage — so indexing every column is a common beginner mistake.

Cache-aside (lazy loading): the app checks the cache first; on a miss, it reads from the database and populates the cache for next time. Most common pattern — simple, and the cache only holds data that's actually been requested. Write-through: every write goes to the cache and the database at the same time, keeping them always in sync, at the cost of slightly slower writes. Write-back: writes hit the cache immediately and are flushed to the database asynchronously later — very fast writes, but risks data loss if the cache fails before flushing. Redis and Memcached are the standard tools; TTLs and explicit invalidation on write are how you keep cache-aside data from going stale. Also mention eviction: when the cache fills up, LRU (evict the least recently used entry) is the default policy in almost every real system.

An operation is idempotent if performing it multiple times has the same effect as performing it once. In distributed systems, network failures force clients to retry requests when they don't get a response — but if "charge $50" isn't idempotent, a retried request could charge the customer twice. The standard fix is an idempotency key: the client generates a unique ID per logical operation and sends it with the request; the server stores which keys it's already processed and simply returns the original result for a duplicate key instead of repeating the side effect. This is why payment APIs like Stripe require an idempotency key on every charge request.

Build in redundancy so no single machine, zone, or region failing takes down the whole system — run multiple instances behind a load balancer, replicate data across availability zones. Use timeouts and retries with exponential backoff so a slow dependency doesn't hang callers forever or hammer a struggling service. Add fallbacks: if a recommendation service is down, show generic/cached results instead of an error page. And isolate failures with patterns like bulkheads, so one overwhelmed dependency (say, a third-party API) can't exhaust the thread pool or resources needed by unrelated features. The mindset shift interviewers want: design for "some functionality degrades" rather than "everything works or everything's down."

A circuit breaker wraps calls to a downstream dependency and tracks failures. In the Closed state, requests pass through normally. If failures exceed a threshold, it trips to Open: requests fail immediately without even trying the downstream call, protecting both the caller (fails fast instead of timing out slowly) and the struggling downstream service (stops piling on load it can't handle). After a cooldown period, it moves to Half-Open, letting a small number of test requests through — if they succeed, it closes again; if they fail, it re-opens. Netflix's Hystrix library popularized this pattern for microservice resilience.

Auto-increment IDs work fine on one database, but break once you shard: two shards would both generate "ID 501" independently. Snowflake solves this by packing a 64-bit integer into three parts: a timestamp (milliseconds since a custom epoch, giving rough time-ordering), a machine/worker ID (so different nodes never collide), and a sequence number (to handle multiple IDs generated on the same machine within the same millisecond). No central coordinator is needed — each node generates IDs independently and they're guaranteed unique and roughly sortable by creation time, which is valuable for pagination and indexing.

How to structure the answer: clarify scale first (expected reads vs. writes, whether custom aliases or link expiry are needed — you'll find this is a heavily read-skewed system, often 100:1), then walk from write path to read path. Write path: a client submits a long URL → the server generates a short, unique code (base62-encode an auto-incrementing ID, or a Snowflake-style ID from Q20, to avoid collisions without a global lock) → store the mapping {short_code → long_url} in a key-value store like DynamoDB. Read path: the request hits a cache first (Redis, since reads vastly outnumber writes), falling back to the database on a miss, and issues a 301/302 redirect. Then layer in the details that show depth: an expiry field for optional link expiration, and sharding the mapping table by short_code hash once it outgrows one machine. Naming the read-heavy access pattern explicitly, and reaching for caching before anything fancier, is what a strong answer sounds like here.

How to structure the answer: start by asking what this store is for — a cache in front of a primary database, or the primary store itself — since that changes how much durability and consistency you need. Then design outward from data placement to replication to consistency. Use consistent hashing (Q3) to map keys to nodes so scaling the cluster only moves a small fraction of keys. Replicate each key to N nodes (say 3) for durability, using leader-follower or leaderless (quorum-based) replication — a leaderless design (like DynamoDB) uses read/write quorums (e.g. W+R > N) to balance consistency and availability per request. Pick your consistency model deliberately (Q4): DynamoDB defaults to eventual consistency for speed but offers a strongly consistent read option at higher latency cost. Close by naming what happens under a network partition — vector clocks or a Merkle tree to detect and resolve conflicting writes across replicas.

How to structure the answer: clarify 1:1 vs. group chat, whether delivery/read receipts are needed, and rough concurrent-connection scale — this determines whether you need a real-time transport at all. Then trace one message end to end. Each client holds a persistent WebSocket connection (Q10) to a chat server. Sending a message: client → chat server → write to a message store (partitioned by conversation ID) → if the recipient is online, push directly over their WebSocket connection; if offline, the message waits in a per-user queue and is delivered (or triggers a push notification) on reconnect. Use a sequence number per conversation for message ordering, and keep "sent/delivered/read" state as its own lightweight, frequently-updated record rather than bloating the message row itself. For group chats, fan the message out to all N members' queues; for scale, shard chat servers and route users to the same server consistently via a connection-routing layer.

How to structure the answer: establish the read/write ratio and whether ranking matters or the feed is purely chronological, then reason explicitly about the two ways to assemble a feed. Fan-out on write: when a user posts, immediately push the post into every follower's precomputed feed (stored in a fast store like Redis). Reading the feed is then a single cheap lookup — great for typical users, but breaks down for celebrities with millions of followers, since one post triggers millions of writes. Fan-out on read: the feed is assembled at request time by pulling recent posts from everyone the user follows and merging/ranking them — no write explosion, but read latency grows with follow count. Most real systems use a hybrid: fan-out on write for normal users, fan-out on read (or a separate path) for high-follower accounts, merging both at feed-read time. Naming this celebrity edge case unprompted is what typically distinguishes a strong answer here.

How to structure the answer: recognize early that this is fundamentally a real-time geospatial indexing and matching problem, not a generic CRUD app — say that out loud before diving into components. Drivers continuously report GPS location to a location service, which indexes them using geohashing or a quadtree — dividing the map into cells so "find nearby drivers" becomes a cheap lookup in one or a few cells instead of scanning every driver's coordinates. When a rider requests a trip, the matching service queries nearby available drivers, ranks them by ETA/rating, and offers the trip (with a timeout to move to the next driver if it's declined). Use a message queue to decouple location updates from matching, and a separate pricing service that factors in real-time supply/demand for surge pricing.

How to structure the answer: clarify live vs. on-demand, since live streaming adds real-time transcoding and much tighter latency budgets — most interviews mean video-on-demand unless stated otherwise. Then split the design into upload/processing and playback. On upload, the video goes into object storage, then a transcoding pipeline converts it into multiple resolutions and bitrates (240p to 4K), split into small segments. Playback uses adaptive bitrate streaming (HLS/DASH): the client's player continuously monitors network conditions and switches segment quality up or down in real time to avoid buffering. All video segments and thumbnails are served through a CDN (Q7) so playback comes from an edge node near the viewer, not the origin. Keep metadata (titles, view counts, recommendations) in a separate database from the actual video bytes, since their access and scaling patterns are completely different.

How to structure the answer: clarify whether multi-device sync and offline editing are in scope — that's what makes this harder than plain object storage. Split large files into fixed-size chunks, store each chunk's content hash, and store chunks in a distributed blob store (deduplicating identical chunks across users to save space). Keep metadata (file names, folder structure, chunk list, permissions, version history) in a separate database from the actual chunk data, since metadata is small and query-heavy while chunk storage is large and read/write-heavy. For sync, each client tracks a local version vector and uses a delta-sync protocol to upload/download only changed chunks, with conflict resolution (e.g. "keep both versions") when two devices edit offline and reconnect with diverging changes.

Track the four golden signals: latency (how slow requests are), traffic (requests per second), errors (rate of failed requests), and saturation (how "full" the system is — CPU, memory, queue depth). Build dashboards for causes (per-service latency, DB query times, queue backlogs) but only page a human for symptom-based alerts that directly affect users (elevated error rate, p99 latency breach) — paging on every internal metric blip causes alert fatigue and trains engineers to ignore pages. Use tools like Prometheus/Grafana for metrics and a proper on-call rotation with clear escalation paths for anything that fires after hours.

Use optimistic locking for normal-traffic items: read the current stock with a version number, decrement it in an update that only succeeds if the version hasn't changed since the read — if it fails, retry. For extremely high-contention items (a flash sale on one product), a database row lock becomes a bottleneck itself, so instead reserve stock in a fast atomic counter (e.g. Redis DECR) with a short-lived hold, converting it to a confirmed sale only once payment succeeds, and releasing the hold automatically on timeout if checkout is abandoned. Combine this with idempotency keys (Q17) so a retried "confirm payment" call can never decrement inventory twice for the same order.

A structure that consistently works: (1) Clarify requirements & scale (2-3 min) — ask about scope, users, read/write ratio; (2) High-level design (5-10 min) — sketch the main components and API before any deep dive; (3) Deep dive (20-25 min) — the interviewer will steer you toward 2-3 components they care about most (usually the database design and one tricky component like the feed or the matching algorithm); (4) Discuss bottlenecks & tradeoffs (10 min) — proactively name what breaks at 10x scale and how you'd fix it; (5) Wrap with monitoring/failure handling if time allows. This exact five-step shape is the "how to answer" framework used throughout Category 05 — apply it to any "design X" prompt you haven't seen before. Interviewers consistently say structure and clear tradeoff reasoning matter more than knowing every buzzword.