System Design Building Blocks: Caching, Databases, and Messaging

15 questions on load balancers, caches, CDNs, database scaling, consistent hashing, queues, rate limiting, and search.

1. 1. What does a load balancer do, and what algorithms does it use?

   It spreads incoming requests across servers, removes unhealthy ones using health checks, and can terminate TLS. Common algorithms: round robin, weighted round robin, least connections, and IP or key hashing for sticky routing. A layer 4 balancer routes by IP and port and is very fast. A layer 7 balancer understands HTTP, so it can route by path or header.

2. 2. What caching strategies exist?

   Cache-aside (lazy loading): the application checks the cache, loads from the database on a miss, then fills the cache; it is the most common. Read-through: the cache loads data itself. Write-through: writes go to the cache and the database together, which keeps them consistent but adds write latency. Write-back: writes go to the cache and are saved to the database later, which is fast but risks data loss. Write-around skips the cache for writes.

3. 3. What are cache eviction policies?

   LRU evicts the least recently used item, LFU evicts the least frequently used, FIFO evicts the oldest, and TTL expires items after a time. LRU is the usual default. Pick based on access patterns, and always bound the cache size.

4. 4. What problems come with caching?

   Stale data and invalidation (famously hard), a cache stampede or thundering herd when a popular key expires and many requests hit the database at once, cache penetration from requests for keys that never exist, and hot keys. Mitigate with sensible TTLs, request coalescing or locks on refill, caching negative results, adding jitter to expiry times, and replicating hot keys.

5. 5. What is a CDN, and when do you use one?

   A content delivery network caches content at edge locations near users, so static assets (images, scripts, video) and sometimes dynamic responses load faster and with less load on your servers. It also absorbs traffic spikes and can protect against attacks. Control freshness with cache headers and invalidation, and use versioned file names for assets.

6. 6. When would you choose SQL or NoSQL?

   Choose SQL when you need relationships, joins, transactions, and a stable schema; it scales a long way with indexes, replicas, and partitioning. Choose a NoSQL store when the data model or scale fits: key-value or document stores for simple lookups and flexible schemas, wide-column stores for massive write throughput, graph databases for relationship traversal, and search engines for text search. Many systems combine several.

7. 7. What is replication, and how do leader and follower replicas work?

   Replication keeps copies of data on several nodes. In single-leader replication, writes go to the leader and are copied to followers, which serve reads and take over if the leader fails. Asynchronous replication is fast but followers can lag and a failover can lose recent writes; synchronous replication is safer but slower. Multi-leader and leaderless designs allow writes in several places but need conflict resolution.

8. 8. What is sharding, and how do you choose a shard key?

   Sharding splits data across multiple databases so each holds part of it. A good shard key spreads load and data evenly and keeps most queries on a single shard. Strategies: range-based (simple, but can create hot shards), hash-based (even, but range queries are hard), and directory-based. Cross-shard joins, transactions, and resharding are the costs.

9. 9. What is consistent hashing?

   A way to distribute keys across nodes so that adding or removing a node moves only a small share of the keys, instead of reshuffling nearly everything as with key mod N. Nodes and keys are placed on a hash ring, and each key belongs to the next node clockwise. Virtual nodes (many points per server) even out the load. It is used by caches, distributed databases, and load balancers.

10. 10. Why use a message queue, and what is the difference between Kafka and RabbitMQ?

    Queues decouple producers from consumers, absorb traffic spikes, allow retries, and let work happen asynchronously. RabbitMQ is a traditional broker with flexible routing where messages are removed once consumed, suited to task queues. Kafka is a distributed log: messages are kept for a retention period in ordered partitions, many consumer groups can replay them, and it handles very high throughput, suited to event streaming and data pipelines.

11. 11. What delivery guarantees do messaging systems offer?

    At-most-once (may lose messages, never duplicates), at-least-once (never loses, may deliver duplicates), and exactly-once (hard; usually achieved by at-least-once delivery plus idempotent consumers or transactional features). Design consumers to be idempotent, because duplicates are the normal case.

12. 12. How does rate limiting work, and which algorithms are common?

    Rate limiting caps how many requests a client can make in a period to protect the service and ensure fairness. Token bucket allows bursts up to the bucket size with a steady refill rate. Leaky bucket smooths output to a constant rate. Fixed window counters are simple but allow bursts at window edges, and sliding window log or counter approaches are more accurate. Store counters in a shared fast store such as Redis, and return HTTP 429 with a Retry-After header.

13. 13. How would you generate unique IDs in a distributed system?

    Options: database auto-increment (simple, but a bottleneck and leaks counts), UUIDs (no coordination, but large and random, which is unfriendly to indexes; time-ordered variants help), and Snowflake-style 64-bit IDs that combine a timestamp, a machine id, and a sequence number, giving sortable unique IDs without central coordination. Watch clock skew with timestamp-based schemes.

14. 14. How do search engines like Elasticsearch work at a high level?

    They build an inverted index that maps each term to the documents containing it, which makes full-text search fast. Data is sharded and replicated across nodes, and queries run on all shards and merge ranked results. Search indexes are usually fed from the primary database through a pipeline, so they are eventually consistent copies, not the source of truth.

15. 15. What is a service discovery mechanism and an API gateway?

    Service discovery lets services find each other's current addresses as instances come and go, through a registry (such as Consul) or the platform itself (Kubernetes DNS). An API gateway is a single entry point for clients that handles routing, authentication, rate limiting, TLS, and sometimes request aggregation, so individual services do not each implement those concerns.