Dashboard/System design/05. Back-of-the-Envelope Calculations

SYSTEM DESIGN · LESSON 5 OF 52 · 8 MIN VIDEO · 2 TOPICS

Back-of-the-Envelope Calculations

Module Progress (0 / 2 Lessons Completed)0%
System design Track (0 / 52 Modules) · Overall 0/80 (0%)0%
⌁

Narrated HD Video Lesson (8 min) · Pro

Lesson Overview + Section #01 + RESHADED Checklist + Quiz #1 are unlocked free below (no login required). Unlock all 80 HD video lessons and deep-dive sections — or watch our two free sample video lessons first.

0% READSec 01/02 · Put Back-of-the-Envelope Numbers in Perspective
SYSTEM DESIGN VIDEO LESSON & GUIDE

Back-of-the-Envelope Calculations

Back-of-the-envelope (BoE) calculations turn vague user counts into concrete order-of-magnitude bounds on server fleets, storage petabytes, and gigabits-per-second of network transit in under four minutes. Grounded in hardware latency hierarchies (1 ns CPU cache vs 150 ms cross-ocean RTT) and resource-bottleneck models (~4,000 RPS per commodity 64-core server with headroom), quick arithmetic lets you rule out infeasible designs before drawing a single box. Interviewers do not grade mental arithmetic to three decimal places; they evaluate whether you state clear assumptions, size for peak rather than average traffic, and let your derived ratios shape the architecture.

WHAT YOU WILL MASTER IN THIS LESSON
  • Internalize the six-tier hardware and network latency hierarchy (1 ns to 150 ms) to explain why hot paths stay in RAM and never make synchronous transoceanic hops.
  • Derive single-server request capacity (~4,000 RPS planning baseline) by bounding CPU time vs per-request memory concurrency (Little's Law) and applying 60% target headroom.
  • Execute complete compute, storage (including 3x replication over 5 years), and ingress/egress bandwidth (8 bits/Byte) calculations from Daily Active Users (DAU).
  • Connect numerical estimation outputs—such as a 75:1 egress-to-ingress bandwidth ratio and 22 PB/year media growth—directly to CDN, blob storage, and load-balancer design choices.
SYSTEMATIC INTERVIEW CHECKLIST · RESHADED + 45-MIN CLOCK

8-Step RESHADED Execution Checklist

Follow this exact sequence from the Intro narration on every system design prompt so you never miss requirements, math, contracts, or the 13-minute deep dive.

0 / 8 Checked
INTERVIEWER SCORECARD · WHAT MAKES A STRONG HIRE?

5 Signals Bar Raisers Grade (L4 vs. L5 Senior vs. L6 Staff)

Senior (L5 / E5 / Senior SDE) Bar — Autonomous Pacing, Depth & Proactive Trade-Offs: Owns the 45-minute clock without prompting, derives every component quantitatively, surfaces failure modes (hot shards, stampedes, dual writes) BEFORE being asked, and defends consistency vs. latency trade-offs.
1. Scoping
✓ STRONG HIRE SIGNAL

Clarifies users, read/write skew, p99 latency, consistency bounds, and explicit non-goals in minutes 2–8.

✗ NO-HIRE RED FLAG

Jumps straight into drawing load balancers and microservices at minute 2 without asking a single question.

2. Right-Sized Design
✓ STRONG HIRE SIGNAL

Starts with the simplest working architecture justified by the math (e.g., 1 NVMe SQL primary + replicas + Redis for 40 writes/s & 3 TB).

✗ NO-HIRE RED FLAG

Recites a memorized Kafka + multi-DC Cassandra diagram that contradicts their own traffic calculations.

3. Technical Depth
✓ STRONG HIRE SIGNAL

Drives deep into 1–2 hard bottlenecks (partition keys, hot-key mitigation, transactional outbox, KV-cache batching) at minute 27.

✗ NO-HIRE RED FLAG

Skims across 12 boxes at identical buzzword depth and runs out of time before the deep dive.

4. Explicit Trade-Offs
✓ STRONG HIRE SIGNAL

Uses 3-part reasoning: 'A queue here because SMTP spikes to 3 s; checkout stays <60 ms, and we accept eventual email delay + retry dedup.'

✗ NO-HIRE RED FLAG

Name-drops tools ('We put Kafka and ZooKeeper here') without stating the alternative or operational cost.

5. Communication
✓ STRONG HIRE SIGNAL

Thinks aloud every 30–45s, checks alignment at phase transitions, and treats 'What if X fails?' as a cue for where points are awarded.

✗ NO-HIRE RED FLAG

Goes silent for 5 minutes while drawing or defends a flawed assumption defensively when nudged.

SECTION 01 OF 2

1. Put Back-of-the-Envelope Numbers in Perspective

★ FREE PREVIEW
Key Takeaway: RAM (100 ns) is 1,000x faster than an SSD random read (100 µs) and 1,500,000x faster than a cross-ocean packet trip (150 ms), while a typical 64-core server is bounded by whichever resource—CPU, RAM, or I/O—saturates first (~4,000 RPS with headroom).

The Latency Hierarchy Every Systems Engineer Must Know

Human intuition struggles with nanoseconds and microseconds, so scaling 1 ns (a single L1 CPU cache read) to 1 human second exposes the chasm between memory, storage, and geography. On that scaled clock, reading main RAM (100 ns) takes 100 seconds; a random read from an NVMe/SSD (100 µs) takes ~1.15 days; an intra-datacenter network round trip (500 µs / 0.5 ms) takes ~5.8 days; a mechanical HDD seek (10 ms) takes ~4 months; and a round trip across the Atlantic (150 ms) takes ~4.75 years. This 1,000x to 1,500,000x physical gap explains why high-throughput systems serve hot keys from RAM, convert random disk writes into sequential append logs, and terminate global users at regional CDN edges.

Hardware / Network OperationActual LatencyHumanized Scale (1 ns = 1 s)Architectural Consequence
L1 CPU Cache Reference~1 ns1 secondKeep inner-loop data structures cache-line friendly
Main Memory (RAM) Read~100 ns100 seconds (~1.7 min)In-memory caches (Redis/Memcached) handle 100K+ ops/s effortlessly
NVMe / SSD Random Read~100 µs (0.1 ms)~1.15 days1,000x slower than RAM; bound uncached DB disk lookups per request
Intra-Datacenter Round Trip~500 µs (0.5 ms)~5.8 daysAvoid chatty N+1 RPC loops (200 serial RPCs = 100 ms pure wire delay)
Spinning Disk (HDD) Seek~10 ms~3.8 months100,000x slower than RAM; use HDDs only for sequential logs or cold blob archives
Cross-Ocean RTT (US <-> EU)~150 ms~4.75 yearsNever put synchronous cross-ocean quorums on low-latency user read paths

Deriving Single-Server Throughput: CPU vs Memory Ceilings

How many requests per second can one commodity cloud server (64 CPU cores, 256 GB RAM) sustain? Every workload hits a ceiling on its tightest resource—CPU compute, inflight memory concurrency, or network/disk I/O. Suppose each request consumes 10 ms (0.01 s) of active CPU execution and waits 90 ms on downstream database I/O (100 ms total wall-clock lifetime) while holding 50 MB of heap state. One CPU core completes 1 s / 0.01 s = 100 RPS, so 64 cores cap CPU throughput at 6,400 RPS. Meanwhile, reserving 16 GB of RAM for the OS leaves 240 GB (240,000 MB), fitting 240,000 / 50 = 4,800 concurrent inflight requests; since each request finishes in 0.1 s (10 turnovers/s by Little's Law λ = L / W), memory permits up to 4,800 * 10 = 48,000 RPS. The server stops at min(6,400, 48,000) = 6,400 RPS, and sizing at ~60% utilization for burst safety gives our standard ~4,000 RPS planning figure.

Host Parameter / ConstraintDerivation FormulaCalculated Value
CPU Ceiling (64 cores, 10 ms CPU/req)64 cores * (1,000 ms / 10 ms)6,400 RPS
Usable Application Memory256 GB total - 16 GB OS reserve240 GB (~240,000 MB)
Max Concurrent Inflight Requests (50 MB/req)240,000 MB / 50 MB4,800 concurrent slots
Memory Ceiling (100 ms total request duration)4,800 slots * (1,000 ms / 100 ms)48,000 RPS
Binding Bottleneck Ceilingmin(CPU Ceiling, Memory Ceiling)6,400 RPS (CPU-bound)
Safe Production Planning Capacity (~60% load)6,400 RPS * 0.625~4,000 RPS per stateless app server

Reference Throughput Baselines Across Infrastructure Tiers

While ~4,000 RPS anchors typical stateless application servers doing JSON serialization and business logic, stateful storage engines have sharply different ceilings dictated by their persistence guarantees. Keeping four order-of-magnitude baselines in mind lets you spot storage bottlenecks immediately during estimation.

  • Stateless application server (64 cores, standard JSON/RPC handler): ~4,000 RPS (drops to 50–500 RPS for CPU-heavy image encoding or cryptographic hashing).
  • Single-primary ACID relational DB (PostgreSQL / MySQL on NVMe): ~5,000–10,000 write QPS and ~10,000–30,000 indexed point-read QPS.
  • Distributed NoSQL / wide-column node (Cassandra / DynamoDB partition): ~10,000–50,000 QPS per node (DynamoDB enforces 1,000 WCU / 3,000 RCU per single physical partition).
  • In-memory cache instance (Redis / Memcached): ~100,000–500,000 QPS per node because reads bypass disk I/O and complex lock managers entirely.
Interview Signal: Use powers of 1,000 (10^3 = KB, 10^6 = MB, 10^9 = GB, 10^12 = TB, 10^15 = PB) and 1 day = 86,400 s (frequently rounded to 100,000 s or 10^5 s for rapid mental division, as long as you state the rounding aloud to your interviewer).
SECTION 02 OF 2

2. Examples of Resource Estimation

🔒 PRO SECTION
Key Takeaway: Sizing a 100M DAU photo-sharing platform step by step yields ~35 stateless app servers (116K peak RPS), ~22 PB/year (110 PB / 5 yr) of 3x-replicated blob storage, and 278 Gb/s peak egress (75x ingress)—directly proving why object storage and a CDN are mandatory.

Section 1 + Checkpoint Quiz #1 are unlocked free outside the paywall. Unlock all architecture slides, QPS & sharding trade-off tables, and all 80 narrated HD video lessons with Full Access — or inspect our sample payment system lesson first.

ACTIVE RECALL & TRADE-OFF PROBES

System Design Checkpoint Quizzes (1)

CHECKPOINT QUIZ #1

In "Put Back-of-the-Envelope Numbers in Perspective" (The Latency Hierarchy Every Systems Engineer Must Know), how does a production-grade design handle "Main Memory (RAM) Read" (Architectural Consequence)?