System Design Cases

CAP Theorem

CAP-теорема и модели консистентности. CP vs AP под partition, session consistency, PACELC.

PACELC Theorem

PACELC Theorem concept page — расширение CAP, добавляющее Latency vs Consistency trade-off в нормальной работе. 4 multi-scenario анимации показывают одну топологию (coordinator + 3 replicas в разных регионах + client) в режимах PA/EL (Cassandra ONE/ONE), PA/EC (tunable QUORUM), PC/EC (MongoDB default при partition), PC/EC (Spanner с TrueTime). Включает ADR с pedagogical context на coordinator-ноде.

Caching Patterns

Паттерны кеширования: cache-aside, write-through, write-behind, refresh-ahead, cache stampede + lock.

Sharding Strategies

Стратегии шардирования: range, hash, consistent hashing. Hotspot, even distribution, минимальный rebalancing.

Message Queues vs Task Queues

Очереди и брокеры: work queue vs pub/sub, back-pressure, redelivery, at-least-once. Kafka vs RabbitMQ vs SQS.

Rate Limiting Algorithms

Алгоритмы rate limiting: token bucket, leaky bucket, sliding window. Burst-friendly vs smooth output vs precise.

Numbers Every Engineer Should Know

Latency hierarchy, capacity defaults, storage sizes, peak factors. The cheat sheet for back-of-envelope estimation.

Back-of-Envelope Estimation

Back-of-envelope estimation: одна e-commerce-архитектура (CDN, LB, API, Redis cache, Postgres primary + read-replica), 4 сценария роста — 1K / 100K / 10M / 300M DAU. Каждый сценарий показывает где появляется bottleneck и какое решение его снимает. Демонстрирует каркас оценки в 5 шагов: DAU × ops × peak factor → RPS, payload × ops × users × retention → storage, read/write ratio → cache+replica/sharding decisions.

Availability Numbers

Concept page: Availability в цифрах. Девятки SLA — 99% / 99.9% / 99.99% / 99.999% — и сколько это реального downtime в год/месяц/неделю. Composition: sequential (произведение availabilities, слабейшее звено доминирует) vs parallel (1 - (1-A)^n, добавляет девяток). 4 сценария: single-9 disaster (3.65 дня/год), sequential composition (4 сервиса по 99.9% = 99.6%), parallel redundancy (2 реплики 99% = 99.99%), real MTTR/MTBF incident timeline (30-минутный outage съедает 69% месячного error budget).

Consistency Models

Concept page: Hierarchy of consistency models — linearizability, serializability, snapshot isolation, causal, eventual. One topology with 2 clients + coordinator + 3 DB replicas. Four scenarios: linearizable strong-quorum read/write; eventual stale read from replica-2; causal read-your-writes via session token; write-skew bug under snapshot isolation (doctor on-call invariant).

Replication

Three replication topologies (single-leader, multi-leader, leaderless quorum) with five scenarios: sync write happy path, async write with risk, replication-lag read-your-writes anomaly, failover sequence via consensus, multi-leader split-brain with conflict resolution.

Idempotency

Концепт-урок Idempotency: как делать retry безопасными. Показывает 4 сценария — наивный POST с двойным charge, спасение через Idempotency-Key + Redis dedup store, conditional update (compare-and-set с version), и counter increment trap с op_id deduplication. Топология: mobile client, payment API, idempotency store (Redis), payments ledger (Postgres), card processor.

Latency vs Throughput

Latency vs Throughput — разные оси оптимизации. Demo: streaming (low latency), batching (high throughput), tail latency at fan-out, hedged requests (Tail at Scale). Объясняет percentiles (p50/p95/p99), Little's law и почему average latency обманывает.

Performance vs Scalability

Performance vs Scalability — concept page для /concepts/performance-vs-scalability. Эволюция архитектуры от 1K до 100M пользователей: monolith → split DB → cache+replica → horizontal+sharding → multi-region+CDN. 5 сценариев с конкретной болью и решением на каждом stage.

Partitioning Strategies

Стратегии партицирования: hash, range, composite, geographic + hot key проблемы и live resharding (Pinterest pattern). 4 сценария.

ACID vs BASE

ACID transactions (Postgres) vs BASE eventual consistency (Cassandra) — две философии в одной топологии. Side-by-side: атомарная транзакция с COMMIT/ROLLBACK против fast-write+eventual-converge. Разъясняет путаницу ACID-C vs CAP-C и spectrum modern БД.

Bloom Filters

probabilistic membership test, экономия I/O в LSM-tree БД

Hot Key Mitigation

replicate hot key, local cache, single-flight, edge cache

Consistent Hashing

Consistent hashing concept page: hash ring with 4 nodes (A/B/C/D) with 256 vnodes each, demonstrating naive modulo disaster vs consistent hashing add/remove, vnode balance properties, and node failure rebalance via clockwise next-on-ring.

Capacity Planning

production capacity provisioning workflow с реальными числами

HyperLogLog

probabilistic cardinality в KBs вместо GBs

Merkle Trees

hash tree для bandwidth-efficient sync репликаций и blockchain proofs

Consensus Overview

Paxos/Raft fundamentals, quorum, FLP impossibility, partition behavior

Leader Election

Leader Election concept page: bully algorithm, lease-based election (etcd/Consul), split-brain without quorum, fencing token protection, real-world Patroni for Postgres HA. 5 nodes + external coordinator store.

Lamport Clocks

logical scalar clock для causality + LWW conflict resolution

Failover: автоматическое переключение

Hot/warm/cold standby, health probes, witness-quorum, STONITH, DNS/VIP failover и failback. 4 сценария: steady state, automatic failover, split-brain prevention, controlled failback.

Cuckoo filters: probabilistic с deletion

Cuckoo filter (Fan, Andersen, Kaminsky, Mitzenmacher 2014) — probabilistic membership с поддержкой DELETE, в отличие от Bloom. Hash table из buckets с короткими fingerprints (8-12 bits); каждый element имеет 2 candidate bucket; lookup читает 1-2 cache lines (cache-friendly vs Bloom k random reads). Insert при коллизии — partial-key cuckoo eviction до MAX_KICKS. Сравнение vs Bloom: для FPR ≤3% Cuckoo меньше по памяти, поддерживает delete, лучше cache locality, но может FAIL при load >95% и нет dynamic resize. Production: RedisBloom CF.*, Snowflake micro-partition pruning, Apache Kudu, CDN edge cache invalidation, TiKV. Сценарии: insert-happy, insert с kick chain, lookup hit/miss, delete (главное преимущество), сравнение vs Bloom, real production uses.

Erasure Coding

Erasure coding (Reed-Solomon k=6, m=3) concept page. Client communicates with RS Encoder/Decoder which split objects into 6 data + 3 parity shards distributed across 3 racks (AZ-1, AZ-2, AZ-3). Five scenarios: WRITE encode, READ fast path, single rack failure recovery, catastrophic loss beyond m, and storage cost comparison with replication.

Count-Min Sketch

Count-Min Sketch concept page: probabilistic frequency estimation via d×w counter matrix and d hash functions, point query as min over rows (NEVER underestimates), top-K via min-heap, overestimate from collisions, real production deployments (Redis CMS/TOPK, Cisco NetFlow, Cloudflare, Datadog, Twitter algebird, Yahoo DataSketches).

Anti-Entropy with Merkle Trees

Anti-entropy reconciliation between two replicas using Merkle trees: full repair (root match), partial diff (recurse into mismatched subtree), hinted handoff (short outage), and tombstone resurrection anti-pattern (skipped repair past gc_grace_seconds).

Big-O Cheatsheet for System Design

Big-O Cheatsheet for System Design — interactive concept page that visualizes complexity classes (O(1), O(log n), O(n), O(n log n), O(n²), O(2^n)) through node load sizes and capacity caps. Compares HashMap vs sorted array vs linked list lookup, B-tree vs LSM with Bloom filter, and brute-force kNN vs HNSW vs IVF+PQ. Includes 4 scenarios: lookup comparison at N=1B, sort comparison (mergesort vs bubble), disk indices (B-tree vs LSM+Bloom), and ANN with exponential warning. ADR explains when NOT to optimize complexity (small N, IO-bound, cache locality, rare execution).

Probabilistic Data Structures Overview

Probabilistic Data Structures Overview — concept page covering five families of sketches (Bloom, Cuckoo, HyperLogLog, Count-Min Sketch, t-digest) with comparison vs exact baselines (HashSet, HashMap, sorted array). Six animated scenarios: family overview, Bloom dedup, HLL count distinct, CMS heavy hitters, t-digest percentile, and trade-offs comparison. One ADR on probabilistic vs exact decision criteria.

Geospatial Indexing: Geohash, Quadtree, H3

Spatial partitioning patterns: geohash (string prefix), quadtree (recursive subdivision), H3 (Uber hexagonal grid). Use cases: nearby search, ride dispatch, surge pricing, geo-fences.

Cache Coherence

Cache Coherence concept page: multi-level invalidation across browser/CDN/app cache/DB, write strategies trade-offs (write-through vs write-back vs write-around), invalidation race conditions, cache stampede protection via single-flight, multi-DC propagation via CDC + Kafka. Five scenarios covering write-through happy path, write-back data loss risk, invalidation race, stampede + single-flight fix, and multi-DC eventual coherence window.

Happens-Before Relation

Happens-Before relation (Lamport 1978): partial order on events in distributed systems. Three processes A, B, C with intra-process program-order edges and a cross-process send/receive message from A2 to B2. Demonstrates causal chain via transitivity, concurrent events without communication, and the pitfall of using wall-clock timestamps.

Vector Clocks

Vector Clocks concept page: per-replica vector counter that exactly distinguishes causal vs concurrent events. 3-replica Dynamo-style cluster + 2 clients. 4 scenarios: causal chain detection, concurrent writes returning siblings, VC vs Lamport comparison showing concurrency info loss, sibling explosion anti-pattern.

Hybrid Logical Clocks (HLC)

Hybrid Logical Clocks (HLC) concept page. Shows how CockroachDB combines physical timestamp + logical counter to get monotonic clock close to wall-time. Algorithm: hlc.l = max(local_phys, hlc.l, msg.l); hlc.c = increment if equal else 0. Bounded clock skew assumption (max_offset 500ms). Scenarios: normal operation, NTP skew tolerance, comparison with Lamport and TrueTime, MVCC snapshot read. Two ADRs explaining HLC tuple design and why CockroachDB chose HLC over Spanner TrueTime.

Paxos

Paxos consensus algorithm — concept page for /concepts/paxos. Visualizes proposer/acceptor/learner roles, two-phase prepare/accept protocol, majority quorum, Multi-Paxos optimization with stable leader, and the duelling-proposers livelock problem. 5 acceptors + 2 proposers + 2 learners. Five scenarios: basic-happy (single decree 2 RTT), conflict-prior-wins (safety preservation), livelock-duelling-proposers (with showError + flashError + Multi-Paxos fix), multi-paxos-stable-leader (1 RTT per slot), greek-mess-why-hard (Paxos Made Live war stories). Two ADRs: full algorithm description, and Paxos vs Raft comparison.

ZAB (ZooKeeper Atomic Broadcast)

ZAB (ZooKeeper Atomic Broadcast) — concept page covering leader election → discovery → synchronization → broadcast phases. Shows ZooKeeper ensemble with 5 nodes (1 leader + 4 followers), client. zxid (epoch, counter) transaction IDs, quorum writes, primary-order semantics. Multi-scenario: normal broadcast (PROPOSAL/ACK/COMMIT), leader crash + recovery, follower SNAP/DIFF catch-up.

Gossip Protocol

Gossip Protocol — concept page. Epidemic dissemination with push/pull/push-pull, periodic random peer selection, O(log N) convergence, SWIM failure detection (suspect -> confirmed via ping/ping-req), delta-based bandwidth control. Used in Cassandra, Consul, Serf, Riak, Akka.

CRDTs — Conflict-Free Replicated Data Types

CRDTs (Conflict-Free Replicated Data Types) concept page. Three replicas R1/R2/R3 with three clients hitting nearest replica (no central coordinator, AP system). Anti-entropy gossip mesh between all replicas. Four scenarios: G-Counter element-wise max merge, OR-Set add-wins with tagged elements, RGA concurrent text insert with deterministic tie-break, and LWW pitfall showing how naive last-write-wins loses updates under clock skew. ADR on CRDTs vs Operational Transform.

Quorum Reads & Writes

Quorum reads/writes concept page. Leaderless replica set N=3 (r1, r2, r3), client → coordinator → replicas pattern. Shows W+R>N strong consistency formula, ONE/QUORUM/ALL/LOCAL_QUORUM tunable consistency, sloppy quorum + hinted handoff during partition. Three scenarios: QUORUM strong (W=2,R=2,N=3), ONE/ONE fast-but-stale, sloppy quorum with hinted handoff replay after partition heal. Two ADRs: tunable W/R per workload, sloppy vs strict quorum trade-off.

Linearizability Deep Dive

Linearizability deep dive — strongest single-object consistency model. Shows etcd-style 3-node Raft cluster with leader and 2 followers (one in-sync, one lagging ~80ms). Two clients: A writes, B reads. Three scenarios: (1) linearizable read via leader with quorum read-index — B sees A's write immediately; (2) non-linearizable stale read directly from lagging follower — B sees old value violating real-time order; (3) comparison of linearizable vs sequential vs causal consistency — what each model guarantees and where it breaks. Includes ADR on the cost of linearizability and when to use it (locks, leader election, unique constraints) vs skip it (feeds, analytics, counters).

Jepsen Testing

Jepsen Testing — fault injection framework by Kyle Kingsbury for verifying claimed consistency guarantees of distributed databases. Shows Jepsen control node (op generator, nemesis fault injector, history recorder, Knossos/Elle checker) connected to a 5-node system-under-test. Three scenarios: setup with concurrent ops, partition-induced lost write, Knossos finding non-linearizable history. References real findings: MongoDB lost writes, Redis Sentinel split-brain, Cassandra LWT, Etcd recovery bug.

Byzantine Fault Tolerance

Byzantine Fault Tolerance concept page. Cluster of 4 replicas (3f+1, f=1) with primary and replica-3 marked Byzantine. Shows PBFT three-phase consensus (pre-prepare, prepare, commit), Byzantine general lying-primary scenario, blockchain PoS BFT analog with slashing, and decision matrix BFT vs CFT.

Three-Phase Commit (3PC)

Three-Phase Commit (3PC) protocol — concept page. Adds PreCommit phase between Vote and DoCommit to fix 2PC blocking problem when coordinator fails. Three scenarios: (1) happy path through CanCommit/PreCommit/DoCommit, (2) coordinator failure after PreCommit with non-blocking termination via participant election, (3) network partition causing split-brain (3PC only correct in synchronous crash-stop model). Includes ADRs explaining why 3PC is rarely used in production (Spanner Paxos commit, Saga, Outbox patterns are preferred).

OSI / TCP-IP Model

OSI / TCP-IP model concept page. 7 OSI layers (Application/Presentation/Session/Transport/Network/Data Link/Physical) mapped to practical TCP/IP stack. Shows where each network device lives (switch=L2, router=L3, NLB/firewall=L4, ALB/WAF/Envoy=L7). Three scenarios: encapsulation-decapsulation HTTP request walkthrough, L4 vs L7 load balancer comparison, troubleshooting by layer (production debugging).

IP, Routing, NAT for Architects

IP addressing (IPv4/IPv6, CIDR, RFC1918), routing (BGP between AS), NAT (SNAT/DNAT/PAT), anycast vs unicast — concept page for foundations curriculum. Demonstrates outbound NAT mapping, inbound DNAT vs P2P NAT-traversal failure, BGP route propagation with anycast routing к ближайшему DC, BGP hijack attack scenario. Includes ADRs on IPv4 exhaustion (NAT vs IPv6) and stateful NAPT-table memory pressure.

UDP: User Datagram Protocol

UDP (User Datagram Protocol): connectionless, 8-byte header, no retransmit, no ordering. Когда выбирать вместо TCP: DNS, gaming, VoIP, video, metrics. Сравнение latency и переход к QUIC/HTTP/3.

HTTP Protocol Evolution: 1.1, 2, 3 (QUIC)

HTTP protocol evolution from 1.0 to 3 (QUIC). Covers request/response structure, methods (GET/POST/PUT/DELETE/PATCH), status codes (1xx/2xx/3xx/4xx/5xx), headers, caching with ETag/Cache-Control, conditional requests, idempotency. Three transport stacks side-by-side: HTTP/1.1 (text, keep-alive, head-of-line blocking), HTTP/2 (binary framing, multiplexing on 1 TCP, HPACK header compression, TCP HoL still present), HTTP/3 (over QUIC/UDP, true multiplex, 0-RTT, connection migration). Seven scenarios: simple GET 200, POST 201 Created, conditional GET with ETag returning 304, HTTP/1.1 head-of-line blocking, HTTP/2 multiplexing with TCP HoL caveat, HTTP/3 over QUIC with packet loss isolation, status codes 401/403/429/502/504.

WebSocket: full-duplex для chat, gaming, collaboration

WebSocket protocol — full-duplex bidirectional communication. Upgrade from HTTP/1.1 to WS frames. Use cases: chat, real-time dashboards, collaborative editors, live trading. Heartbeats, reconnection, scaling via sticky sessions and Redis pub/sub backplane. Includes 4 scenarios: handshake (HTTP Upgrade -> 101 Switching Protocols), bidirectional message flow (typing/presence/text/binary frames), broadcast via Redis pub/sub fan-out across gateway nodes, and connection drop on mobile with reconnect+resync via last_seq. ADRs cover WebSocket vs SSE vs long-poll vs WebRTC, plus heartbeat/reconnect/backpressure discipline.

Server-Sent Events (SSE)

Server-Sent Events (SSE) — concept page. One-way server -> client streaming over plain HTTP using text/event-stream format. EventSource browser API with built-in auto-reconnect and Last-Event-ID resume. Three scenarios: (1) basic SSE setup and steady event delivery from a Redis Stream, (2) connection drop with auto-reconnect and Last-Event-ID replay of missed events, (3) LLM token streaming OpenAI/Anthropic-style with data: [DONE] terminator. Three ADRs covering SSE vs WebSocket vs long-poll choice, Redis Stream resume buffer strategy, and disabling proxy buffering plus compression on SSE endpoints.

TLS Handshake (1.2 vs 1.3, 0-RTT, PFS)

TLS 1.3 handshake: ClientHello with key_share (ECDHE), ServerHello+Certificate+Finished in 1 RTT, 0-RTT resumption with PSK and replay risk, comparison with TLS 1.2 (2 RTT), forward secrecy via ephemeral ECDHE, and certificate validation failures (chain/expiry/SAN/mTLS). Free concept lesson #53 in foundations.

SSL/TLS Deep

SSL/TLS deep dive: cipher suites breakdown (key exchange, signature, bulk cipher, MAC), PKI cert chain (root/intermediate/leaf), Certificate Transparency, OCSP/CRL revocation, mTLS for service mesh. 3 scenarios: cert chain validation walkthrough, mTLS handshake (mutual cert verification), revocation check (OCSP query vs stapling). 2 ADRs: cipher suite selection (TLS 1.3 + AEAD + ECDHE), mTLS vs OAuth2 for service-to-service auth.

HTTP/3 and QUIC: next-generation transport

HTTP/3 over QUIC concept page. QUIC = TCP + TLS 1.3 + multiplexing rebuilt over UDP. HTTP/3 is HTTP semantics on QUIC. Wins: no TCP head-of-line blocking, 1-RTT (or 0-RTT) handshake, connection migration, encrypted headers (anti-ossification). Costs: middlebox UDP/443 blocks, higher CPU. Used in Google (gQUIC since 2014), Cloudflare (quiche), Facebook (mvfst), CloudFront. Three scenarios: HTTP/2 HoL blocking vs HTTP/3 per-stream isolation, connection migration WiFi->LTE without reset, 0-RTT resumption with replay-risk caveats.

gRPC vs REST vs GraphQL

gRPC vs REST vs GraphQL — three API styles compared. Three vertical columns: REST (resource-oriented HTTP, JSON, OpenAPI, cache-friendly, but suffers over-fetching and N+1); gRPC (HTTP/2, Protobuf, .proto contract, codegen, bidi streaming, ideal for internal microservices, weak browser support); GraphQL (single endpoint, schema, client picks fields, no over/under-fetching, BFF for multiple frontends, but caching is hard and unbounded queries are a DoS vector). Plus a Hybrid block illustrating the Netflix/Twitter/Shopify pattern: REST public + gRPC internal + GraphQL BFF. Four scenarios animate same user-and-posts fetch in each protocol plus a hybrid production path. Includes 4 ADRs for when to pick each, and the hybrid trade-off.

[CONCEPT] Reverse Proxy

Reverse Proxy concept page covering nginx/HAProxy/Caddy/Envoy/Traefik. Functions: TLS termination, caching, compression, routing, header injection, rate limiting, auth offload, WAF. Reverse vs forward proxy. CDN edge as reverse proxy. Sidecar proxy (Envoy) for service mesh. ADR comparing Reverse Proxy vs API Gateway vs Service Mesh. 5 scenarios.

Memory Hierarchy: from Register to S3

Memory hierarchy concept page: CPU registers, L1/L2/L3 caches, DRAM (NUMA-local and cross-socket), NVMe SSD, SATA SSD, HDD, S3, Tape. Shows three scenarios: cache-friendly hot loop vs random pointer chase, NUMA pin vs cross-socket access, hot loop vs page fault to disk. Includes ADRs on when to optimize for cache locality and NVMe vs SATA selection.

Operating System Basics for Architects

Operating System Basics for Architects: process vs thread, userspace vs kernelspace, syscalls, virtual memory, file descriptors, CFS scheduler, signals, IPC primitives (pipes/shm/mq/unix), epoll/kqueue/io_uring async I/O models. Concept page.

Virtualization

Virtualization concept page: Type-1 hypervisors (ESXi, KVM, Xen, Hyper-V) vs Type-2 (VirtualBox, Parallels), hardware-assisted virtualization (Intel VT-x, AMD-V), paravirtualization, live migration, containers vs VMs, microVMs (Firecracker for AWS Lambda/Fargate). Three scenarios: VM lifecycle (create/boot/live-migrate), VM vs container resource isolation showing container escape blast radius, microVM cold start (<150ms). Includes ADR for choosing VM vs container vs microVM.

Контейнеры: Docker, OCI, runc, namespaces, cgroups

Containers: Linux primitives (namespaces, cgroups, overlayfs), OCI standard, runc/containerd/CRI-O, image layers + registry, rootless containers, gVisor/Kata/Firecracker for harder isolation. Four scenarios: lifecycle (build->push->pull->run->cleanup), namespace isolation demo, image layers COW, rootless vs sandbox.

WebRTC: P2P browser audio/video

WebRTC peer-to-peer browser audio/video/data. Three scenarios: P2P call setup (signaling SDP offer/answer + ICE/STUN + DTLS + SRTP media), TURN relay when symmetric NAT blocks direct connection, and SFU multi-party conference with simulcast. Includes 4 ADRs covering WebRTC vs WebSocket vs HLS, Trickle ICE, geo-distributed TURN, SFU vs MCU vs Mesh.

Serializability Deep Dive

Serializability deep dive — concept page covering SQL isolation levels, write skew anomaly, SSI in Postgres, Strict 2PL with deadlock detection, and OCC. Multi-scenario FlowBuilder: SERIALIZABLE happy path with disjoint writes, write skew under SNAPSHOT ISOLATION (doctors on-call invariant broken), SSI abort 40001 with client retry, and 2PL deadlock with InnoDB victim selection. Topology: Postgres group with tx-mgr (MVCC + SSI), heap (row versions), serialization graph (rw-deps), plus T1/T2 clients. ADR on tx-mgr explaining when to use SERIALIZABLE vs SI/RC + SELECT FOR UPDATE.

Google Spanner

Google Spanner concept page. Globally-distributed SQL database with external consistency via TrueTime API (GPS+atomic clocks). 3 regions (us-central1 leader, europe-west1 replica, asia-northeast1 replica), each with TrueTime time service. Per-shard Paxos groups (Shard A, Shard B), 2PC across shards. Clients hit shard leaders. 4 scenarios: single-shard RW txn with Paxos+commit-wait, cross-shard 2PC, lock-free read-only at snapshot, TrueTime uncertainty spike (GPS antenna fail). 2 ADRs: (1) TrueTime as foundation for external consistency vs HLC alternative, (2) when Spanner is justified vs cheaper alternatives.

DB Selection Framework

DB Selection Framework — decision tree concept page. 6 dimensions (data model, scale, consistency, query patterns, ops maturity, cost) → engine pick. Maps workloads (e-commerce OLTP+search, real-time analytics, social graph, vector AI/RAG, global ACID, NoSQL antipattern) to engines (Postgres, MongoDB, Cassandra, DynamoDB, Spanner, ClickHouse, Pinecone, Neo4j, Redis, Elasticsearch). Includes ADR on NoSQL hype vs Postgres-достаточно. 6 scenarios.

MySQL Internals

MySQL/InnoDB internals concept page: storage engine architecture (InnoDB clustered B+tree, buffer pool, redo/undo log, doublewrite buffer, change buffer, binlog), secondary index double lookup, write path with WAL, async binlog replication with replica lag, Galera sync multi-master alternative. 4 scenarios: write path WAL, secondary index double lookup, async replication via binlog, Galera sync replication. ADR on InnoDB vs MyISAM vs PostgreSQL.

Qdrant Vector DB

Qdrant vector DB concept page. HNSW (Hierarchical Navigable Small World) approximate nearest neighbor index, payload-based filtering with filterable HNSW, distributed cluster with Raft for collections meta, scalar/binary quantization for memory savings. Includes ADR comparing Qdrant vs pgvector vs Pinecone vs Weaviate vs Milvus and scenarios for ingestion (embed → HNSW insert), ANN query with payload filter, vector update with tombstone, quantization trade-offs, and vendor decision tree.

B-tree vs LSM-tree

B-tree vs LSM-tree storage engines comparison: B+tree with in-place updates, page splits, WAL (Postgres/MySQL) vs LSM with memtable, SSTables, leveled compaction, bloom filters (RocksDB/Cassandra). Scenarios: B-tree page split, LSM write-flush-compact, point query latency comparison, delete semantics with tombstones.

YDB: Yandex distributed SQL

YDB (Yandex Database) — distributed SQL concept page. Multi-model: tables + KV + Kafka-like topics. Tablets как Bigtable (data shards с leader+followers через Paxos-like), Hive scheduler управляет placement и auto-split, DTX coordinator делает cross-tablet транзакции через deterministic ordering (не классический 2PC, не TrueTime). Distributed Storage layer — proprietary BlobStorage с erasure coding (1.5x overhead vs 3x replication) распределён по 3 DC. Три сценария: cross-tablet ACID транзакция через DTX, auto-split tablet на hot key (orders), geo-replication при падении DC2. Два ADR: YDB vs Spanner vs CockroachDB (когда что выбрать), и почему YDB использует свой DTX-протокол вместо классического 2PC.

DuckDB

DuckDB — embedded analytical SQL engine ("SQLite for OLAP"). In-process columnar engine with vectorized execution (1024-row batches, SIMD), reads Parquet/Arrow/CSV directly with predicate+projection pushdown. Single-writer multi-reader. Compared with Pandas, ClickHouse, Snowflake. 5 scenarios: SELECT FROM parquet (zero-copy), 1B rows aggregation on laptop, JOIN parquet × csv, DuckDB-Wasm in browser, OLTP misuse anti-pattern. 2 ADRs: DuckDB vs alternatives, vectorized execution rationale.

Neo4j / Graph DBs

Neo4j / Graph DBs concept page. Native graph storage (nodes/edges as fixed-size records, doubly-linked rel lists, index-free adjacency). Cypher pattern matching. Replication via Raft (Causal Cluster). Compared with Postgres recursive CTE and vector embeddings. Five scenarios: friends-of-friends 3-hop traversal vs SQL self-joins, weighted shortest path (bidirectional BFS / Dijkstra), fraud ring detection (cycles of length 3-5), sharding pain (cross-shard traversal latency variance), operational gotchas (unbounded variable-length paths). Two ADRs: Neo4j vs Postgres CTE vs vector embeddings, replicate-the-full-graph vs vertex partitioning.

DynamoDB

DynamoDB managed wide-column NoSQL by AWS — single-table design with composite (pk, sk) keys, GSI/LSI, on-demand vs provisioned capacity, Streams CDC, Global Tables active-active. Six scenarios: GetItem (single-digit ms), Query на partition с sk-range, GSI inverse lookup, PutItem→Stream→Lambda fan-out, hot-partition throttle с adaptive capacity, Global Tables LWW. Two ADRs covering single-table design vs RDBMS thinking and on-demand vs provisioned capacity.

Database Replication Deep Dive

Database Replication Deep Dive: synchronous vs async vs semi-sync replication, WAL/binlog/oplog physical streaming, cascading and delayed replicas. Topology shows app writer/reader, Postgres leader, sync replica (remote_apply), two async replicas, plus a DR/cascading group with intermediate replica feeding cascaded replicas and a 1h delayed replica. Five scenarios: (1) sync commit waiting for replica ack, (2) async stale read pitfall (read-your-writes violation), (3) MySQL semi-sync silent degrade to async on timeout with crash, (4) read-your-writes routing using LSN to leader, (5) delayed replica rescuing from human-error DELETE.

Database internals: что внутри любой БД

Database Internals Overview — concept page covering wire protocol, auth/RBAC, parser/planner/optimizer/executor, transaction manager + MVCC + locks, buffer pool / WAL / B-tree / LSM, disk + backup/PITR. 4 scenarios: query lifecycle, buffer pool hit vs miss, commit path with WAL fsync, B-tree vs LSM write path.

Cassandra

Cassandra (Apache) — concept page. Wide-column store, masterless ring, gossip-based membership, consistent hashing on token ring, tunable consistency (LOCAL_QUORUM/QUORUM/ONE/ALL), LSM storage (CommitLog + Memtable + SSTables), compaction strategies (STCS/LCS/TWCS), Lightweight Transactions via Paxos. Anti-entropy via hinted handoff, read repair, nodetool repair. Three scenarios: write path with CL=QUORUM (RF=3, W=2 via coordinator + 3 replicas), read repair healing a stale replica via timestamp reconciliation, hinted handoff during partition with coordinator stashing and replaying mutations. Three ADRs covering masterless vs leader-based replication, Cassandra vs ScyllaDB vs DynamoDB tool selection, and LOCAL_QUORUM as production default for tunable consistency.

Time-series databases

Time-series databases — Prometheus + VictoriaMetrics architecture with high-cardinality scrape ingestion, PromQL range query, downsampling pipeline (1m -> 5m -> 1h -> 1d) with retention drops, and ADR comparing Prometheus / InfluxDB / TimescaleDB / ClickHouse / Mimir for a Kubernetes platform with 3M active series.

Database Migration Strategies

Database Migration Strategies — concept page covering online schema change (gh-ost, pt-online-schema-change), expand-contract pattern, dual-write Mongo→Postgres replatform with CDC, and big bang vs incremental migration tradeoffs. Four scenarios + 2 ADRs (online vs maintenance window, gh-ost vs pt-osc vs pg_repack).

Elasticsearch / OpenSearch

Elasticsearch / OpenSearch concept page: distributed Lucene cluster (3 master + 3 data + coord), inverted index, sharding, query/fetch phases, aggregations, split-brain, mapping explosion

WAL (Write-Ahead Log)

WAL (Write-Ahead Log) — universal durability mechanism. Append-only sequential file, fsync semantics, group commit, checkpoint, crash recovery. Used in Postgres, MySQL InnoDB, RocksDB, Kafka log, etcd, Spanner, ext4 journal. Concept page covering write path (modify in-memory → WAL append → fsync → ack → flush page later), crash recovery (replay WAL from last checkpoint), group commit batching, and synchronous_commit=off trade-off.

MVCC MultiVersion Concurrency Control concept page

MVCC (Multi-Version Concurrency Control) concept page. Каждая строка хранит несколько версий (xmin/xmax). Транзакции читают консистентный snapshot на момент start. Postgres, Oracle UNDO, SQL Server RCSI, MySQL InnoDB UNDO log, Spanner timestamp KV. Цена: bloat, VACUUM/cleanup overhead, txid wraparound risk при long-running txn. 4 сценария: stable snapshot read, write skew under SI, autovacuum cleanup + bloat от long-running txn, txid wraparound катастрофа. ADR: MVCC vs locking-based READ COMMITTED (Postgres vs SQL Server до RCSI).</description> <parameter name="name">MVCC: Multi-Version Concurrency Control

CockroachDB / NewSQL

CockroachDB / NewSQL — distributed SQL database. Postgres wire-compatible, Raft per range, HLC instead of TrueTime, multi-region survival/locality, range splits/rebalancing. Inspired by Spanner (PC/EC) but no atomic clocks needed. 4 scenarios: cross-range distributed transaction (HLC + Raft + 2PC), range rebalance after node add (auto-split, auto-rebalance), multi-region locality via REGIONAL BY ROW (GDPR data residency), region failure (REGION survival mode). 2 ADRs: CockroachDB vs Postgres+Citus vs Spanner choice; HLC vs TrueTime trade-off.

MongoDB

MongoDB document database deep dive: replica set with oplog (1 primary + N secondaries with automatic Raft-like failover), sharded cluster (mongos routers + config server replica set + 3 shard replica sets), writeConcern levels (1, majority, all), readPreference (primary, secondary, nearest), multi-document transactions (4.0+), and change streams (CDC built-in). Four scenarios: write with writeConcern majority, primary failover with election, sharded query routing (targeted vs scatter-gather), and change stream tailing oplog. Includes ADR comparing MongoDB vs PostgreSQL JSONB vs DynamoDB for evolving-schema document workloads.

Kafka vs RabbitMQ vs Pulsar

Comparison of three messaging brokers side-by-side: Kafka (log-based, partitioned, dumb broker + smart consumer with offset tracking), RabbitMQ (smart broker with topic exchange routing to multiple queues, push delivery, DLX for rejects), and Pulsar (stateless broker + Apache BookKeeper segmented storage with E=3/W=2/A=2 quorum, tiered S3 offload for cold segments, multiple subscription types). Includes 4 scenarios: Kafka log fan-out via consumer groups with replay, RabbitMQ topic exchange routing with DLX, Pulsar segmented storage and broker failover, and ADR decision matrix walkthrough.

Apache Flink Deep Dive

Apache Flink deep dive: true streaming engine with stateful operators, RocksDB state backend, Chandy-Lamport distributed snapshots for exactly-once via 2PC sinks. Shows JobManager + 3 TaskManagers (source, window aggregator, sink), RocksDB local state, S3 snapshot storage, Kafka transactional sink + Postgres XA sink. Five scenarios: stateful keyed aggregation, checkpoint barrier propagation, 2PC sink commit, failure recovery with tx abort, savepoint-based version upgrade. Includes 3 ADRs comparing Flink vs Spark Structured Streaming vs Kafka Streams.

Stream Processing

Stream processing concept page (/concepts/stream-processing). Source -> operators -> sink pipeline. Stateless operators (filter, map) vs stateful (keyBy, tumbling window aggregate, RocksDB local state). Time semantics: event-time vs processing-time. Watermarks and late events with side output. Backpressure via credit-based flow control. Checkpoint-restore via Chandy-Lamport. Includes 5 scenarios (simple ETL, stateful windowed count, late event handling, backpressure cascade, checkpoint recovery) and 2 ADRs (stream-vs-batch-vs-lambda, event-time-vs-processing-time).

Exactly-Once Semantics

Exactly-once semantics: at-most-once / at-least-once / exactly-once. Three tiers of delivery semantics. Effective EOS = at-least-once delivery + idempotent consumer + atomic commit. Kafka transactional API (transactional.id, sendOffsetsToTransaction, isolation.level=read_committed, transaction coordinator with __transaction_state). Flink TwoPhaseCommitSinkFunction with pre-commit on checkpoint barrier and commit on notifyCheckpointComplete. Idempotent consumer with Redis dedup. Two Generals myth: exactly-once delivery невозможен, но exactly-once effects реален. ADRs: when EOS critical vs at-least-once + idempotency enough; effective EOS = three ingredients (delivery + dedup + atomic commit). Scenarios: at-least-once duplicate (double billing), Kafka EOS happy path, idempotent producer retry, transaction abort, Flink 2PC commit on checkpoint, Flink failure recovery, idempotent consumer dedup, EOS impossible without sink cooperation.

Lambda vs Kappa Architecture

Lambda vs Kappa Architecture concept page. Lambda (Marz 2011): three layers — batch (Spark, точно/медленно) + speed (Storm/Flink, быстро/приближённо) + serving (merge views). Kappa (Kreps 2014): один streaming pipeline через Kafka log + Flink EOS, replay через blue-green datasource swap (new consumer group offset=earliest, отдельный output namespace, atomic swap queries). Lakehouse hybrid (2026): Iceberg/Delta как unified primitive, Spark Structured Streaming + batch backfill пишут в одну table, Materialize держит incremental view. Сценарии: Lambda dual pipeline, Kappa happy path, Kappa replay, Lakehouse hybrid, Lambda merge bug failure mode. ADR-001: когда Kappa default, когда Lambda оправдан. ADR-002: blue-green datasource swap для replay.

Change Data Capture (CDC)

Change Data Capture (CDC): Postgres WAL → Debezium → Kafka topics → fan-out (Elasticsearch search index, Snowflake DWH, PaymentService, NotificationService). Demonstrates logical replication slots with pgoutput plugin, initial snapshot + streaming switchover, transactional outbox pattern for atomic business events, replication slot growth disaster (Debezium down → WAL накапливается → диск Postgres переполняется), and schema evolution via ALTER TABLE. Includes ADRs on CDC vs dual-write vs sync API call (ADR-001) and replication slot growth multi-layer protection (ADR-002).

Windowing and Watermarks

Windowing & Watermarks concept page. Streaming pipeline: Kafka source feeds a Flink job (Watermark Assigner -> keyBy -> Window Operator -> RocksDB state). Window emissions go to an aggregates sink, late events go to a side-output sink. Four scenarios: tumbling 1m window aggregation, late event with allowedLateness=2m re-fires window, session window with gap=15min for user activity, plus an ADR contrasting event-time + watermark vs processing-time simplicity.

Streaming Joins

Streaming joins concept page with three flavors: stream-stream windowed join (click+impression in 5-min window with symmetric hash join), stream-table enrichment (async lookup with Redis cache + LRU), and temporal join (FX rate as-of trade time using versioned table). Includes state explosion warning scenario and interval join. Two ADRs: stream-stream join state cost vs precomputed enrichment, and async lookup vs co-partitioned KTable.

ETL vs ELT

ETL vs ELT data pipelines: classic ETL (Informatica -> MS SQL DW), modern ELT (Fivetran -> S3 -> Snowflake -> dbt -> Looker), reverse ETL (warehouse -> Hightouch -> Salesforce), and an ADR scenario showing when ETL still wins for PII redaction pre-load (Spark redactor masks before warehouse to satisfy GDPR).

Apache Iceberg / Delta Lake / Hudi — open table formats over object storage

Open table formats (Apache Iceberg, Delta Lake, Apache Hudi) — metadata layer over Parquet/ORC files on S3/GCS/ADLS that adds ACID transactions, snapshot isolation, time-travel, schema/partition evolution, efficient updates. Lakehouse architecture: cheap object storage + warehouse semantics. Catalogs: REST, Polaris, Nessie, Glue, Hive Metastore, Unity Catalog. Scenarios: atomic write commit (parquet + manifest + CAS), concurrent writers with optimistic retry, time-travel query (timestamp → snapshot id), MoR vs CoW trade-off with compaction, schema/partition evolution without rewrite, full read path with manifest pruning, maintenance (compaction, snapshot expiration, orphan cleanup). Includes ADRs on Iceberg vs Delta vs Hudi selection and CoW vs MoR write-vs-read trade-off.

Data Mesh

Data Mesh (Zhamak Dehghani 2019) concept page. 4 principles: domain-oriented decentralized ownership, data as product, self-serve infrastructure, federated computational governance. Shows central DWH bottleneck (Gen 1-2) vs Data Mesh (Gen 3) with domain teams (orders, payments, marketing) owning their own data products on top of self-serve platform (S3+Iceberg, DataHub catalog, Spark/Trino, Monte Carlo) governed by federated council (OPA, data contracts). 5 scenarios: monolithic-bottleneck (legacy), mesh-publish (orders team self-serves), mesh-discover (marketing finds + consumes), contract-violation (governance blocks breaking change), quality-slo (SLO alert routing).

Design URL Shortener

Классический системный собес — спроектировать TinyURL/bit.ly. Разбираем стратегии генерации ID, cache-aside паттерн, узкие места при масштабировании. Пилотный кейс нового /cases формата.

Design Rate Limiter

Классический system design — спроектировать rate limiter для защиты API. Token bucket, Redis-backed, atomic Lua. Пилот #2 нового /cases формата.

[SYSTEM DESIGN] Instagram

Full Instagram architecture: post upload, feed generation (fan-out on write), social graph, engagement. Covers pre-signed URLs, cache-aside, CQRS, event-driven patterns.

Design Twitter (X)

Системный дизайн ленты Twitter/X. Snowflake IDs, гибридный fanout (write для обычных пользователей, read для celebrities), Redis hot timeline + Cassandra durable. Кейс уровня senior-собеса.

[SYSTEM DESIGN] Web Crawler

Distributed web crawler at billion-page scale: BFS frontier, per-host politeness, Bloom-filter dedup, async fetcher pool, S3 + Kafka storage tier.

[SYSTEM DESIGN] Chat (WhatsApp / Messenger)

Real-time chat at 100M concurrent users: WebSocket fleet, sticky routing via consistent hash, per-chat sharded message DB, Kafka fan-out for groups, APNS/FCM push for offline.

[SYSTEM DESIGN] Uber / Ride-Hailing

Ride-hailing at 100M users / 1M concurrent drivers: 200K location pings/sec, geohash-sharded Redis GEO, 5km-radius matching, surge pricing, trip lifecycle.

Design Pastebin

Pastebin system design case study. Client posts text snippets via HTTPS through a CDN and Load Balancer to the paste-svc. paste-svc generates a base62 short ID (8 chars), stores the body in S3 under sharded prefixes, and persists metadata (lang, ttl, s3_key, visibility) in Postgres. Hot pastes are cached in Redis. Reads are served via CDN for /raw views and rendered HTML pages bypass the CDN to keep view counts accurate. A TTL cleanup cron sweeps expired pastes from S3, Postgres and Redis. ADRs: KV store choice (S3 + Postgres metadata), ID generation strategy (base62 hash with collision retry), CDN caching for raw views. Capacity: 10M DAU, 1M new pastes/day, 10:1 read:write, viral pastes 100x burst. Six scenarios: create paste, view with cache miss, raw CDN hit, viral burst absorbed by CDN+Redis, TTL cron cleanup, abuse rate limiting.

Notification System

Multi-channel notification system (push, email, SMS, in-app) handling 100M DAU with 5.8K rps avg / 50K rps peak. Producers publish events to Kafka, orchestrator loads user preferences, dedups via Redis, renders templates, then dispatches to per-channel queues (q-push, q-email, q-sms). Channel workers (push/email/sms) post to APNs/FCM/SES/Twilio with retry-backoff and DLQ. Provider webhooks update ClickHouse for delivery status analytics.

[SYSTEM DESIGN] News Feed (Twitter / Instagram timeline)

News Feed (Twitter/Instagram timeline) system design case. Hybrid push/pull fan-out with celebrity-tier carve-out, per-user Redis ZSET inbox cache, ML ranking pipeline. 5 scenarios cover regular post fanout-on-write, celebrity post skip-push, feed-view cache hit + ranking, cache-miss rebuild from Cassandra, and ranking pipeline reordering. 2 ADRs: push-vs-pull-vs-hybrid trade-off and capped-ZSET + idle-TTL eviction. Capacity hints on every node.

12-Factor App: cloud-native methodology (Heroku, 2011)

12-Factor App methodology — 12 принципов для cloud-native: codebase, dependencies, config (env vars), backing services, build/release/run, stateless processes, port binding, concurrency, disposability (graceful SIGTERM), dev/prod parity, logs to stdout, admin processes. Foundation для контейнеризации и Kubernetes. 4 сценария: anti-pattern config-in-code vs factor 3, stateless scaling (factor 6+8), graceful SIGTERM (factor 9), build/release/run + logs (factor 5+11). 2 ADR: где 12-factor остался релевантен в 2026, sticky sessions vs externalized state.

Infrastructure as Code (Terraform)

Infrastructure as Code with Terraform / OpenTofu / Pulumi / CDK / Crossplane. Shows declarative provisioning, plan-apply lifecycle, S3+DynamoDB remote state with locking, multi-cloud providers, modules and workspaces, drift detection, and an ADR comparing Terraform vs Pulumi vs CDK vs Crossplane.

Kubernetes Patterns

Kubernetes production usage patterns from "Kubernetes Patterns" (Ibryam, Huss): Sidecar (Istio Envoy mTLS), Ambassador (proxy для external services), Init container (iptables setup, db migrate), Adapter (translate /stats -> /metrics), Operator pattern (CRD + reconcile loop with Postgres operator example, Leader election для HA), Self-healing (probes), Predictable demands (resources/limits) и Elastic Scale (HPA reactive scaling). Three scenarios: sidecar mTLS via Istio Envoy, Postgres operator reconciles cluster state with failover, HPA scales deployment on traffic spike. ADRs cover sidecar vs library vs node-agent trade-offs and operator vs Helm vs raw manifests for stateful workloads.

Service Mesh Deep: Istio xDS, mTLS, canary, ambient, Cilium eBPF

Service mesh internals deep dive across 3 layers (istio sidecar, ambient mesh, cilium-ebpf) with 4 scenarios: Istio mTLS auto, canary 95/5 traffic shift, AuthorizationPolicy deny, sidecarless (ambient ztunnel+waypoint and Cilium eBPF kernel path).

Serverless Patterns: Lambda, Workers, Step Functions

Serverless patterns: FaaS (AWS Lambda + API Gateway + DynamoDB + Aurora via RDS Proxy), event-driven (S3 → SQS → Lambda batch with DLQ), Step Functions orchestrated saga (ChargeCard → ReserveInventory → Compensate Refund), EventBridge cron, Cloudflare Workers at the edge with KV + D1 + Durable Objects, Cloud Run container serverless. Covers cold start vs warm pool vs provisioned concurrency, RDS Proxy connection pooling, anti-pattern of Lambda + direct RDS connection storm. Includes ADR on serverless vs containers vs k8s cost/scale crossover.

[CONCEPT] Auto-scaling Strategies

Auto-scaling Strategies — concept page on Kubernetes autoscaling: HPA reactive on custom RPS metric, KEDA event-driven scaling on Kafka lag with scale-to-zero, Cluster Autoscaler vs Karpenter for node provisioning, predictive vs reactive scaling, asymmetric cool-down windows, scale flapping anti-pattern, and request limits/PDB importance. Two ADRs: stack choice (HPA+KEDA+Karpenter with asymmetric cool-down), and predictive vs reactive (hybrid cron preempt + reactive HPA fallback).

GitOps

GitOps: git как source of truth для desired state. Pull-based controller (ArgoCD/Flux) внутри кластера наблюдает за git и непрерывно reconciles фактический state с желаемым. Три сценария: PR merge → ArgoCD sync → cluster reconcile; drift detection (manual kubectl change → auto-revert); Argo Rollouts canary с analysis gates. Включает Kustomize/Helm overlays, SOPS/External Secrets/Vault, Prometheus-based progressive delivery.

Multi-Region Architecture

Multi-region architecture concept page covering active-passive (DR with DNS failover, RTO/RPO), active-active (DynamoDB Global Tables multi-master with LWW conflict resolution), and data residency routing (GDPR region pinning). Shows GeoDNS/Anycast routing layer, two regions (eu-west-1 and us-east-1), each with multi-AZ deployments, and global data layer with DynamoDB Global Tables. Includes ADR comparing multi-region vs multi-AZ single-region cost/complexity tradeoffs. Pitfalls: split-brain, replication lag, cross-region egress cost, GDPR violations.

CDN: Edge Networks (PoPs, cache hierarchy, purge, security)

CDN edge network concept page. Anycast PoPs (100-300 globally) routing users to nearest edge, tiered cache hierarchy (edge -> mid-tier -> origin shield -> origin), HTTP cache headers (Cache-Control, ETag, Vary, stale-while-revalidate), purge / invalidate (URL purge vs surrogate-key purge), TLS termination at edge with Let's Encrypt + 0-RTT, WAF (OWASP rules), bot management (JS challenge / Turnstile), DDoS scrubbing (L3/L4/L7), edge compute (Cloudflare Workers / Lambda@Edge), HLS/DASH streaming. Four scenarios: cold cache miss (200ms full path), hot cache hit (10ms edge), purge after content update with fan-out, attack mitigation (DDoS + bot + WAF + edge compute). Two ADRs covering CDN provider choice (Cloudflare vs Fastly vs CloudFront vs Akamai) and Cache-Control header strategy.

Edge Computing

Edge computing patterns: compute at PoPs (Cloudflare Workers V8 isolates, Vercel Edge, Lambda@Edge, Fastly Compute@Edge WASM, Deno Deploy). Three scenarios: edge auth (verify JWT before forwarding), A/B routing with sticky cookie, image resize on demand. Three ADRs cover edge-vs-origin compute, V8 isolates vs WASM vs containers, and KV at edge vs origin DB.

FinOps and Cost Optimization

FinOps & Cost Optimization concept page with rightsize, spot fleet, S3 lifecycle tiering, egress cost shock scenarios. Includes ADR on when to invest in cost optimization vs feature work.

OAuth 2.0 + OpenID Connect

OAuth 2.0 + OpenID Connect: Authorization Code + PKCE flow, M2M Client Credentials, refresh token rotation with theft detection, ADR comparing OAuth/OIDC vs SAML vs custom session-based auth.

Authentication Models

Authentication models: password + TOTP, Passkey enrollment + login, SSO via OIDC (Authorization Code + PKCE), m2m API key, mTLS service mesh, plus phishing/SIM-swap anti-patterns. Concept lesson for security curriculum.

Authorization Models

Authorization Models concept page covering ACL, RBAC, ABAC, ReBAC (Google Zanzibar), Policy-as-Code (OPA Rego, Cedar), tenancy isolation, Postgres RLS, with tool comparison (SpiceDB, Permify, OpenFGA, Cerbos, Oso, Casbin). Includes 5 scenarios: RBAC role check, ABAC context-aware policy with time/location/MFA, ReBAC graph traversal Google Drive style, Postgres RLS tenant isolation, and tools comparison decision tree. Two ADRs on RBAC vs ABAC vs ReBAC choice and app-authz vs RLS defense in depth.

RBAC Implementation Patterns

RBAC implementation patterns: API Gateway enforces authz via dedicated AuthZ service backed by Redis cache and auth_db (users/roles/permissions). OPA sidecar overlays ABAC. Admin role mutations write to auth_db, append to audit_log, publish invalidation events through pub/sub to all authz pods which DEL the user's cache key. Three scenarios: cached gateway check (hot path), hierarchical role inheritance with recursive CTE expansion on cache miss, and role revoke with sub-second cache invalidation contrasted against JWT-claims staleness.

JWT Best Practices: Algorithms, Rotation, Revocation, Storage

JWT Best Practices: signing algorithms (RS256 over HS256 for multi-service), key rotation via kid header and JWKS, short-lived access + rotating refresh tokens, audience and exp/nbf validation, jti for revocation via Redis blocklist, alg-confusion attacks (alg=none, RS256->HS256 swap), and storage trade-offs (HttpOnly cookies vs localStorage XSS). 5 animated scenarios plus an embedded ADR comparing JWT vs opaque tokens with introspection.

mTLS: Mutual TLS Authentication

mTLS (mutual TLS): both parties authenticate with x509 certs. SPIFFE/SPIRE workload identity, service mesh sidecar pattern (Istio/Linkerd), short-lived cert auto-rotation, zero-trust service-to-service auth. 3 scenarios: mTLS handshake with mutual cert presentation, SPIFFE-issued SVID with auto-rotation, mesh sidecar mTLS with cert revocation flow.

Zero Trust Architecture

Zero Trust Architecture concept page. "Never trust, always verify." BeyondCorp-style identity-aware proxy with device posture + MFA, microsegmentation via service mesh (sidecar PEP + SPIFFE mTLS), and contrast vs legacy VPN castle-and-moat. 3 scenarios: BeyondCorp access flow, microsegment allow/deny, Colonial Pipeline–style breach blocked by ZT. ADR on adoption strategy (BeyondCorp vs Cloudflare Access vs Tailscale vs Zscaler).

OWASP Top 10 (2021)

OWASP Top 10 (2021) — concept page covering all 10 vulnerability classes with 3 deep-dive scenarios: A03 SQL injection (vulnerable string-concat vs parameterized queries), A01 Broken Access Control (IDOR — First American 2019 style), A10 SSRF to AWS IMDS leading to IAM credential theft (Capital One 2019). Topology: Attacker -> WAF/LB (edge) -> API + AuthZ + Audit Logger (app) -> Postgres + Secrets Vault + AWS IMDS + S3 (internal). 2 ADRs on the WAF node: (1) defense-in-depth — где какая защита, mapping each OWASP item to the right layer (WAF / API Gateway / App / Infra / Supply chain / Observability), (2) A03 SQL injection — prepared statements vs ORM vs string concat with concrete code rules and gotchas (LIKE, IN, ORDER BY identifiers).

CSRF & XSS Protection

CSRF and XSS protection: SameSite cookies, CSP nonce, output escaping, DOMPurify sanitization, Trusted Types, CORS misconfig leaks, SRI for compromised CDN. Eight scenarios covering stored XSS exploit, CSP-blocked injection, SameSite=Strict CSRF prevention, double-submit token pattern, base64+javascript: URI sanitizer bypass, CORS Allow-Origin reflection leak, and SRI hash mismatch. ADR on defense-in-depth security stack.

API Security

API security: WAF/CDN edge -> API Gateway (AuthN/AuthZ/Schema/RateLimit) -> Services -> DB. Concept lesson with 5 scenarios: happy-path OAuth+rate-limit, schema validation rejecting mass assignment, WAF blocking SQLi, token-bucket rate limit, and OWASP API1 BOLA/IDOR.

Secrets management: Vault, KMS, sealed secrets

Secrets management: Vault, KMS, External Secrets Operator, Sealed Secrets, SOPS. Five scenarios: app fetches static secret via Vault (k8s SA JWT auth → token → KV read with KMS-backed envelope encryption + audit log), Vault dynamic DB credentials (auto-generated postgres user with TTL=1h, auto-revoke makes leaked password useless), External Secrets Operator (GitOps sync from Vault into K8s Secret, plus Sealed Secrets alternative), SOPS encrypted YAML in git (Mozilla SOPS with KMS/age, no runtime Vault dependency), and ADR comparing Vault vs cloud-native Secrets Manager vs ESO+Sealed Secrets with hybrid recommendation.

Data Encryption: at-rest, in-transit, E2EE

Data encryption deep dive: TLS in-transit + mTLS east-west, envelope encryption (DEK wrapped by KMS/HSM CMK), TDE + field-level encryption for PII, E2EE Signal-style (X3DH + Double Ratchet), and an ADR on when E2EE is worth its UX cost.

Threat Modeling: STRIDE & LINDDUN

Threat modeling concept page (STRIDE methodology). DFD with trust boundaries (Internet -> DMZ -> App tier -> Data tier). Three scenarios: STRIDE walkthrough on login flow (Spoofing/Tampering/Repudiation/Information disclosure/DoS/Elevation), STRIDE on file upload feature, LINDDUN privacy threat model for GDPR PII flow. Includes ADR on lightweight vs formal SDLC threat modeling.

Supply chain security: SBOM, SLSA, signing

Supply chain security: SBOM, SLSA, Sigstore, dependency scanning, signed admission. Three scenarios: Dependabot finds CVE, signed image verified at K8s admission, SBOM gen and CVE block in CI. Two ADRs on dev node: defense against typo-squatting/malicious deps, and SLSA L3 target.

Embeddings Basics

Embeddings basics concept page: shows offline indexing pipeline (S3 docs -> chunker -> embedder API -> vector DB) and online query path (user -> API -> Redis cache -> query embedder -> vector DB -> LLM). Three scenarios: text-to-embedding model call, cosine similarity comparison (synonyms close, unrelated far), and batch embedding pipeline for 10K docs. Includes ADRs on embeddings vs BM25 keyword search and on dimension sizing (1536 vs 3072 with Matryoshka truncation).

Vector Similarity & ANN

Vector similarity & ANN concept page. Distance metrics (cosine/dot/euclidean), brute-force kNN O(n) baseline, then ANN families (HNSW/IVF/PQ/LSH/ScaNN/DiskANN/Annoy). 4 scenarios: brute-force slow, HNSW logarithmic descent, IVF-PQ billion-scale, filtering pre/post/filterable HNSW. ADR on HNSW vs IVF-PQ vs DiskANN selection.

Chunking Strategies for RAG

Chunking strategies for RAG: how to split documents into chunks for embedding and retrieval. Demonstrates three strategies side-by-side — fixed-size with overlap (cheap but loses coreferences at chunk boundary), recursive markdown-aware (respects document structure, keeps headers in metadata), and late chunking (Jina 2024 — embed whole doc with long-context model first, then split the token embeddings, preserving global context for pronouns and cross-references). ADR — Choose chunker by content type: - Markdown / HTML / docs: recursive structure-aware splitter (RecursiveCharacterTextSplitter / MarkdownHeaderTextSplitter). Header path -> metadata for filtering and citations. - Code: AST-based (tree-sitter) — split on functions/classes, never mid-symbol; embed signature + docstring + body together. - PDF / DOCX / slides: layout-aware (Unstructured.io, Marker) — preserve tables and section breaks. - Long narrative with many coreferences: late chunking — only viable if you have a long-context embedding model (Jina, Voyage); +20% recall on anaphora. - Default for unknown / mixed text: recursive 500/50 with sentence-aware fallback. - Avoid: one-size-fits-all 500-token splitter on heterogeneous corpus, naive split with zero overlap, chunk_size > embedding model max (silent truncation). - Always: enrich every chunk with metadata payload (source_url, section_path, doc_type, date, permissions) for pre-ANN filtering and citations. Trade-off summary: too small loses surrounding context (LLM can't answer); too big dilutes the embedding signal (ANN recall drops). Sweet spot for general RAG is 300-500 tokens with 10-20% overlap, plus parent-child for technical docs that need both precise retrieval and wide context.

RAG Architecture

RAG (Retrieval Augmented Generation) architecture concept page. Shows the full pipeline: ingestion (load -> chunk -> embed -> store in Qdrant + BM25) and query (embed -> ANN search -> hybrid retrieval -> RRF fusion -> rerank -> LLM with cache). Demonstrates progression from Naive RAG to Advanced RAG (hybrid + rerank + query rewriting + HyDE) to Modular RAG per Gao 2024. Includes 5 scenarios: naive RAG baseline, hybrid retrieval with BM25+dense+RRF, advanced RAG with rewrite/HyDE/rerank, ingestion pipeline, and semantic cache hit. Includes 2 ADRs covering when complexity is worth it and vector DB / embedding model selection.

Reranking

Reranking in RAG: two-stage retrieval (bi-encoder recall -> cross-encoder precision). Three scenarios: 1) two-stage cross-encoder rerank pipeline (Cohere Rerank-3, BGE-v2-m3) over top-100 candidates from hybrid (dense+BM25+RRF) retrieval, sorting to top-10 for LLM context; 2) recall@10 lift comparison without rerank vs with rerank; 3) LLM-as-reranker for high-stakes medical/legal Q&A. ADR on cross-encoder vs LLM-rerank cost/quality tradeoff.

Prompt Engineering

Prompt Engineering concept page — system that demonstrates zero-shot/few-shot lift, CoT reasoning, structured outputs via function calling, prompt injection defense, eval-driven iteration. Architecture: prompt orchestration (router, template, few-shot retriever, input guardrails) → model layer (Haiku/Sonnet/thinking) → tools (function calling) → output processing (parser, output guardrails, eval logger). Multi-scenario animation covering all six topics with 2 ADRs (prompt vs fine-tune vs RAG; CoT vs thinking models).

Fine-tuning vs RAG vs Prompting

Concept page: Fine-tuning vs RAG vs Prompting. Decision framework for choosing between prompt engineering (cheap, instant), RAG (dynamic facts + citations), and fine-tuning (LoRA/QLoRA/DPO for behavior/style/format). Shows hybrid production pattern combining all three. 6 scenarios: prompting ladder (zero-shot to CoT), RAG for knowledge, LoRA SFT for format consistency, hybrid (fine-tune+RAG+prompt), anti-pattern fine-tune-for-facts, LoRA hot-swap for multi-domain inference. Includes 2 ADRs covering the full decision ladder and LoRA/QLoRA/DPO/full fine-tune tradeoffs.

Machine Learning Basics

ML fundamentals: supervised/unsupervised/RL, train/val/test splits, overfitting and bias-variance, regularization, gradient descent, classical algorithms (linear/logistic, trees, RF, XGBoost) vs neural networks. Three scenarios: supervised pipeline, classification deploy, retrain on drift.

MLOps Pipeline

MLOps Pipeline — end-to-end ML lifecycle: data validation, feature store materialization, training (Kubeflow/Metaflow), experiment tracking (MLflow/W&B), model registry, CI/CD for ML, canary/AB deployment via Istio router, prediction logging, drift monitoring (Evidently/Arize), business metric tracking, and automated retraining trigger. Includes ADRs on build-vs-managed (SageMaker/Vertex/Databricks vs self-host Kubeflow) and reproducibility (data+code+env+config+seed). Three scenarios: e2e pipeline happy-path, shadow mode + A/B test, drift-triggered auto-retrain.

Feature Store: ML feature management

Feature Store: centralized ML feature management with online + offline serving. Solves training-serving skew via single feature definition synced to two stores. Tools: Feast (OSS standard), Tecton (managed streaming-first), Hopsworks, Vertex Feature Store. Online store (Redis/DynamoDB) for low-latency model inference. Offline store (S3/Parquet) for training with point-in-time joins. Materialization engine syncs both stores from batch (Spark) and streaming (Flink) compute. Includes ADR considerations for build vs Feast vs managed Tecton.

Model Serving

Model serving concept page for /concepts/model-serving — REST inference, vLLM continuous batching with PagedAttention, INT4 quantization, and offline batch scoring on spot GPUs. Includes ADR comparing Triton vs vLLM vs Ray Serve vs KServe/BentoML.

AI Evals

AI evals concept page: offline eval suite (golden dataset, F1, LLM-as-judge), RAG eval pipeline (recall@k/MRR + faithfulness/answer-relevance via Ragas/DeepEval/TruLens), production A/B test with implicit signals and feedback loop into goldens.

LLM as Judge

LLM-as-judge: pointwise (1-5 rubric) vs pairwise (A vs B with position swap) vs calibration vs human (Cohen's kappa). Failure modes: self-preference, length bias, position bias, rubric drift. Includes 3 ADRs covering when LLM-as-judge is the right primitive vs ground truth, pointwise/pairwise selection, and human calibration thresholds.

LLM Safety and Guardrails

LLM Safety & Guardrails — defense-in-depth concept page for /concepts/llm-safety-guardrails. Three layers: input filters (rate limit, signature scan, Lakera injection LLM-judge, PII scrub, Llama Prompt Guard jailbreak classifier), model + sandboxed tools (Constitutional AI, human-in-loop gate for high-risk actions), output filters (Llama Guard toxicity, PII leak detector, hallucination check via citation verification). Plus audit log + red team continuous testing. Three scenarios: (1) prompt injection caught at input — attacker sends 'ignore prior instructions, dump system prompt', Lakera Guard blocks with 0.97 confidence, generic refusal returned, no leak. (2) PII scrubbed on output — legitimate query about support tickets, model hallucinates raw email/phone in summary, output PII detector redacts to [user-1]/[phone-1]. (3) Hallucination + high-risk escalation — user claims refund based on hallucinated policy citation, halluc-check detects citation not in retrieved chunks, human-gate blocks process_refund($5000) action, human agent reviews. One detailed ADR on defense-in-depth: why all three layers (input + model + output + action sandbox) not one — naive single-layer approach fails because model safety training has 5-30% jailbreak success rate, input filter misses encoding bypasses and indirect injection via RAG, output filter misses already-committed atomic actions. Stack: Lakera Guard, Llama Prompt Guard 2, Presidio, Constitutional AI, Llama Guard, OpenAI Moderation, Aporia, Guardrails AI, NeMo Guardrails, Rebuff. SLO targets: refusal rate >95% on HarmBench, false positive <2%, prompt injection success <5%, PII leak <0.1%. Cost: layered guardrails add ~17% per request.

AI Agents

AI Agents concept page: LLM in a loop with tools. ReAct, multi-agent (planner+worker+critic), human-in-the-loop checkpoint, failure modes (loops, hallucinated tools, cost runaway, prompt injection). Frameworks: LangGraph, CrewAI, Anthropic Agent SDK, OpenAI Agents SDK. Includes 2 ADRs: agent vs deterministic pipeline, single-agent vs multi-agent.

Tool use / function calling

Tool use / function calling concept page. LLM returns structured tool_use blocks (JSON args via tool schema), runtime validates/executes/returns tool_result. Anthropic/OpenAI/Gemini parallel tool calls. MCP (Model Context Protocol, Anthropic Nov 2024) for vendor-agnostic standardized tool servers. 4 scenarios: simple weather call, parallel 5-tool fan-out, MCP filesystem+git, failure modes (hallucination/bad-args/injection). 2 ADRs (function calling vs MCP vs custom; parallel vs sequential).

Agent Memory: short-term, long-term, episodic, semantic

Agent memory architecture: working memory + long-term stores (vector/episodic/graph/procedural). Mem0/Letta/Zep patterns: cross-session semantic recall, episodic timeline queries, contradiction resolution by recency, sleep-time consolidation, TTL/decay, GDPR forgetting. Failure modes: hallucinated extraction, compaction loss, cross-tenant privacy leaks.

Multimodal AI

Multimodal AI concept page. Native multimodal LLMs (GPT-4o, Gemini 2.5, Claude 4, Llama 3.2 Vision) vs specialized pipelines (CLIP, Whisper, YOLO, Tesseract, ElevenLabs, DALL-E, Midjourney, Flux, Sora). VQA, audio transcription, text-to-image generation, hybrid filter+verify production patterns. 4 scenarios with cost/latency tradeoffs and ADR for native vs specialized decision matrix.

AI Cost Optimization

AI cost optimization concept page covering prompt caching, model routing, batch API, distillation, semantic response cache, per-user budgets, self-host vs API break-even.

DDD Strategic: bounded contexts and context maps

DDD Strategic Design — bounded contexts (Sales, Billing, Shipping), Generic subdomains (Auth0 conformist, SendGrid), Legacy CRM with ACL. Context map shows Customer/Supplier, Published Language, Conformist, ACL relationships. Scenarios: cross-context Customer flow (Lead -> Payer -> Recipient), ACL protecting against legacy XML, Conformist trade-off, anti-pattern microservice-per-table.

DDD Tactical Patterns

DDD Tactical Patterns: aggregates as transactional consistency boundaries, entities with identity, immutable value objects, domain events for cross-aggregate eventual consistency, repository pattern abstracting persistence, factories, application/domain/infrastructure layers. Order aggregate (root + items + Money/Quantity VOs) shows transactional unit; OrderPlaced event triggers Shipment aggregate update in separate TX. Includes scenarios for happy-path aggregate transaction, cross-aggregate via event, repository load/save, and god-aggregate anti-pattern.

Hexagonal Architecture (Ports & Adapters)

Hexagonal Architecture (Ports & Adapters) by Alistair Cockburn (2005). Domain в центре, adapters снаружи. Driving adapters (REST/CLI/Kafka/gRPC/Cron) -> driving ports -> application services -> domain entities. Services также вызывают driven ports (OrderRepository, PaymentGateway, EmailNotifier, EventPublisher) которые реализуются driven adapters снаружи (PostgresOrderRepo, DynamoOrderRepo (alt), InMemoryOrderRepo (test), StripeAdapter, SmtpEmailAdapter, KafkaEventPublisher). Dependency inversion: hexagon определяет interfaces, infrastructure реализует. 3 scenarios: REST->hexagon->Postgres canonical happy path, swap Postgres->DynamoDB без изменений в hexagon, test domain в isolation через in-memory adapters. ADRs: hexagonal vs layered 3-tier when to apply, mapping DTO<->domain entity boilerplate trade-offs.

Clean Architecture

Clean Architecture (Robert C. Martin, 2012). Concentric circles — Entities, Use Cases, Interface Adapters, Frameworks & Drivers — held together by the Dependency Rule (outer points inward). Four scenarios: happy-path request from controller through presenter to use case to entity to driven port to adapter; unit-testing the use case in isolation with in-memory repos; swapping a framework (Postgres -> Mongo) without touching the inner rings; and an antipattern showing what happens when the dependency rule is violated.

Onion Architecture

Onion Architecture (Jeffrey Palermo, 2008): concentric rings with domain model at center, surrounded by domain services, application services, infrastructure outside. Dependencies only inward. Pre-cursor to Clean Architecture, sister pattern to Hexagonal.

Modular Monolith

Modular Monolith concept page: one deploy unit with hard internal module boundaries (catalog/orders/payments), own DB schemas per module, in-process event bus, dependency-cruiser as fitness function, Strangler Fig extraction to microservice when operationally justified.

C4 Model

C4 Model concept page. Simon Brown's 4 levels of architecture diagrams: Context (system + actors + external systems), Container (deployable units like SPA/API/DB/queue), Component (modules inside a container like Controllers/Services/Repositories), Code (class diagrams, optional). 4 scenarios: L1 Context overview, drill L1->L2 Container, drill L2->L3 Component, diagrams-as-code workflow with Structurizr DSL + PR review. 2 ADRs: C4 vs UML/ad-hoc justification, and Structurizr DSL / PlantUML / Mermaid C4 tooling choice.

Architecture Decision Records

ADR (Architecture Decision Records) concept page. Markdown в репо, фиксирующий context + decision + consequences. Lifecycle Proposed -> Accepted -> Deprecated/Superseded. Tooling: adr-tools, log4brains. Scenarios: writing ADR for Postgres vs Mongo, superseding RabbitMQ -> Kafka, onboarding via log4brains, anti-pattern of decisions in Slack threads.

Evolutionary Architecture: fitness functions, quanta, strangler fig

Evolutionary Architecture (Ford/Parsons/Kua, 2017, 2nd ed 2022): architecture supports guided incremental change. Fitness functions = automated checks of architectural characteristics in CI. Architectural quanta = independently deployable cohesive units. Strangler Fig pattern (Fowler) for legacy migration. Modular monolith ready for split. Topology: Repo + CI pipeline (lint, unit tests, architecture fitness via dependency-cruiser/ArchUnit, performance fitness via k6, security fitness via gitleaks/Snyk, API contract fitness via Pact, cost fitness via Infracost) -> gate -> canary -> production with continuous SLO + chaos engineering + auto-rollback. Includes Strangler Fig migration setup (facade, legacy, new service, shadow mode, feature flags) and modular monolith (catalog/orders/payments/billing modules with in-process bus) ready to split into 2 architectural quanta with own DB and Kafka. 4 scenarios: (1) fitness function fails CI on cyclic deps blocks PR; (2) Strangler Fig endpoint migration with shadow mode and gradual cutover; (3) modular monolith ready-for-split extracts payments quantum for PCI-DSS compliance without rewriting; (4) continuous + holistic fitness in production with auto-rollback on canary breach.

Event Storming

Event Storming workshop technique by Alberto Brandolini for domain discovery. Three bounded contexts (Sales, Payments, Fulfillment) with domain events, commands, aggregates, policies, external systems, actors, and hot spots. Multi-scenario animation showing big-picture chaotic exploration, design-level deep dive, and how hot spots and bounded context boundaries emerge from pivotal events.

API Versioning Strategies

API Versioning Strategies — three side-by-side approaches: URL path versioning (/v1, /v2 in parallel), Stripe-style date-based versioning with per-account pinned date and request/response transformer middleware, and GraphQL @deprecated directive with field usage tracking. Includes ADRs for when to pick each approach plus a deprecation lifecycle scenario covering Sunset/Deprecation HTTP headers, brownouts, and 410 Gone after the grace period.

[SYSTEM DESIGN] CDN

CDN system design case: client → DNS (anycast) → edge PoP (cache + WAF + Worker) → tiered cache (regional + origin shield) → customer origin, with global control plane (config, purge, logs). 5 scenarios: edge cache hit, tiered miss fetch, global purge fan-out, DDoS absorption, video chunked streaming. 2 ADRs: anycast vs GeoDNS, push vs pull CDN. Capacity hints on every node.

Social Graph (LinkedIn / Facebook TAO)

Social Graph (LinkedIn / Facebook TAO) — 1B users, 200B+ edges. Sharded MySQL adjacency (objects+assoc tables, sharded by id1) behind a stateless TAO-style cache tier (Memcached, 99% hit). Two ADRs: graph DB vs sharded SQL adjacency, and cache strategy for hot edges (celebrities). 5 scenarios: add friend (bidirectional write + invalidate), friends-of-friends 2-hop (scatter-gather), mutual friends (sorted-list intersection), PYMK (2-hop + ML rerank via PyTorch BigGraph), hot celebrity read (pre-sharded follower list with leases).

[SYSTEM DESIGN] Search Engine

Web-scale search engine (Google-like). Crawler -> URL frontier (Kafka) -> fetcher -> parser -> bulk indexer -> sharded inverted Lucene index (150 shards × RF 3). Query path: API gateway -> query cache -> spell corrector -> query rewriter -> coordinator (scatter-gather) -> shards (BM25) -> PageRank static scores -> ML re-ranker (LambdaMART). Auto-suggest service backed by in-RAM prefix trie. Semantic search via embedding model + HNSW vector index, fused with BM25 via Reciprocal Rank Fusion. Five scenarios: crawl-and-index a new page, search query (BM25 + ML rerank), auto-suggest while typing, spell correction "googel" -> "google", hybrid vector + lexical retrieval. Two ADRs: inverted index sharding strategy (document-partitioning vs term-partitioning), BM25 vs vector vs hybrid retrieval.

Distributed Message Queue (Kafka-style)

Distributed Message Queue (Kafka-style) case study. Producers (idempotent + transactional) write batched/compressed records to a 3-broker cluster (DC-1) with RF=3 and min.insync.replicas=2. Three partitions (P0/P1/P2) each have a leader and 2 followers spread across racks. Control plane is KRaft metadata quorum (no ZooKeeper) with 3 voters, plus group coordinator and transaction coordinator. Two consumer groups: analytics (auto.commit=false, 1:1 partition assignment) and alerting (read_committed for EOS). Internal compacted topics: __consumer_offsets, __transaction_state, __cluster_metadata. Multi-DC mirror via MirrorMaker 2 to DC-2, with tiered storage (S3) for cold segments. Two ADR panels: pull vs push consumer model, KRaft vs ZooKeeper metadata. Six animated scenarios: produce + ISR replicate (acks=all), consumer group fetch + offset commit (zero-copy sendfile), leader broker failure + ISR election (KRaft fast failover), replay from offset (retention + tiered recovery), exactly-once via transactional producer (PID/epoch fence + 2PC commit markers + read_committed), and ADR walkthrough.

Design Distributed Key-Value Store (Dynamo-like)

Distributed Key-Value Store (Dynamo/Cassandra-style) with consistent hash ring (RF=3, W=2, R=2 quorum), vector-clock conflict resolution, hinted handoff, anti-entropy via Merkle trees, and read repair. Includes 5 scenarios: PUT quorum, GET with read repair, partition + hinted handoff, Merkle anti-entropy, concurrent-write conflict resolution.

Object Storage (S3-like)

Object Storage (S3-like) — system design case showing buckets, immutable objects, PUT/GET/DELETE, multipart upload, erasure coding 12+4 for 11-nines durability, metadata vs data plane separation, tiered storage (Standard/IA/Glacier), versioning, lifecycle policies. 5 scenarios: PUT with EC, multi-region GET, multipart upload, lifecycle migration to Glacier, tail latency at scale. 2 ADRs on metadata/data plane split and EC vs replication.

Video Hosting (YouTube-like)

Video Hosting (YouTube-like) — TUS resumable upload, FFmpeg/GPU transcoding pool with CMAF/HLS multi-bitrate ladder, Kafka job queue, S3 raw + variants, mid-tier + edge CDN PoPs, ABR player with quality switching, view counter via Redis HLL, Cassandra watch history, Postgres meta, Elasticsearch search, recsys, plus live streaming RTMP -> LL-HLS pipeline. Five scenarios: upload+transcode, playback ABR with quality downshift, viral CDN cache hit, view counter at scale, live streaming with packet loss recovery. ADR-001: pre-transcode vs JIT. ADR-002: HLS vs DASH vs CMAF.

Design Payment System (Stripe-style)

Stripe-style payment system case: API gateway with WAF and Redis idempotency cache, payment core with charge service, saga orchestrator, HSM tokenization, Postgres double-entry ledger and outbox, external Visa/Mastercard/SEPA rails via processor adapter, async fan-out via Kafka to webhook dispatcher, reconciliation cron, and payouts. Five animated scenarios: happy-path card charge with idempotency key, retry duplicate caught at idempotency layer, refund with compensating ledger entry, subscription rebill via cron with saved token, and nightly reconciliation cron catching webhook-lost mismatches. Two ADRs: idempotency-key dual-storage strategy (Redis fast path + Postgres unique-index backstop) and ledger storage choice (Postgres double-entry with outbox vs append-only event store). Capacity hints throughout.

Hotel Booking (Booking.com)

Hotel Booking (Booking.com style) — system design case. Search → Booking Saga (hold → pay → confirm) → Payment → Notifications. Demonstrates no-double-booking via hybrid pessimistic-lock-on-hold + saga compensation, Elasticsearch search vs Postgres FTS trade-offs, dynamic pricing, cancellation/refund flow, and concurrent reservation race resolution. Five FlowBuilder scenarios + 2 ADRs (inventory consistency, search engine choice) + capacity hints on every node.

Design Ticket Booking

Ticket booking system case study (Ticketmaster, BookMyShow): browse events, select seats with TTL hold, pay, confirm. Handles flash sales (1M concurrent, 100K seats). Components: CDN, WAF/CAPTCHA with ML bot scoring, virtual waiting room (Redis sorted set), API gateway, event catalog (Cassandra), Redis SETNX hold service with 10-min TTL, sharded Postgres inventory, payment service to PSP (Stripe/Adyen), seat-map WebSocket, outbox to Kafka, notifications. Five scenarios: virtual queue under flash sale, atomic seat hold conflict, checkout to PSP and PG transaction, hold TTL expiry, anti-bot scalper detection. Two ADRs: virtual waiting room vs FCFS, Redis SETNX TTL vs Postgres row-lock.

Ad Click Aggregator

Design ad click aggregator at Google Ads / Facebook Ads scale. 1M clicks/sec sustained, near-real-time aggregation (1m lag), exact billing. Pipeline: Browser/Mobile beacons -> Edge CDN -> Click Ingest API -> Kafka (5K partitions) -> Flink (Enrich -> Dedup with RocksDB -> Fraud filter with Redis -> 1m Window Aggregator) -> Druid OLAP + S3 raw + Postgres ledger. 5 scenarios: happy click flow, dedup duplicate, fraud bot filtered, hot ad partition skew, daily billing reconciliation.

Recommendation System

Recommendation system case study (Netflix/YouTube/Spotify class). Two-stage funnel: candidate generation via two-tower ANN over millions of items, then ranking via DLRM on top 1000, then re-rank for diversity and business rules. Includes cold start (popular + demo cohort + bandit explore), real-time signal updates via Kafka+Flink streaming, A/B testing with experiment assigner, drift detection and retrain pipeline. 5 scenarios and 2 ADRs (two-stage vs single model, real-time vs batch features).

Google Docs / Collab Editor

Google Docs / Figma collaborative editor system design case. WebSocket gateway + per-doc session actor with CRDT merge, sharded op log (Spanner), snapshot service to blob storage, Redis presence, ACL, comments, version history, Kafka fan-out for hot docs. 5 scenarios: keystroke, concurrent edit at same anchor (CRDT merge), offline edit + sync, presence cursors, hot doc with snapshot + version restore. 2 ADRs: OT vs CRDT (pick CRDT), server-authoritative vs P2P CRDT (pick server-authoritative).

Real-time Multiplayer Gaming

Real-time multiplayer game (FPS / Battle Royale) — server-authoritative game servers, UDP at 60Hz tick rate, client-side prediction + reconciliation, Agones matchmaking, Kafka anti-cheat telemetry feeding an ML detector. Five animated scenarios: matchmaking + spawn, 60Hz tick loop with AOI broadcast, client prediction + reconciliation on misprediction, UDP packet-loss interpolation + FEC, aimbot detection + HWID ban. Two ADRs (UDP custom vs WebRTC vs WebSocket; server-authoritative vs lockstep). Capacity hints on every node.

Design A/B Testing Platform

A/B Testing Platform — Optimizely/GrowthBook-class experimentation system: Client SDK with in-process bucketing, Control Plane with config CDN and SSE kill-switch, Event Ingest via Kafka+Flink, Stats engine with mSPRT sequential testing, Druid for real-time agg, Guardrail auto-stop. 5 scenarios + 2 ADRs.

[SYSTEM DESIGN] Airbnb / Marketplace

Airbnb system design case — two-sided marketplace with search (Elasticsearch + LTR re-rank), booking saga with escrow capture-on-confirm, host approval flow, snapshotted cancellation policy refunds, smart pricing nightly batch, host calendar block, and host payout 24h after check-in. Includes 2 ADRs (escrow timing on Booking Saga, cancellation policy enforcement on Calendar Service) and capacity hints on every node.

Maps Proximity Service (Yelp / Google Maps)

Maps proximity / POI search system design (Yelp, Google Maps "nearby restaurants"): H3-indexed POI search with k-ring scan, hot-query cache, vector tile delivery, and OSRM-style routing. Includes 5 scenarios (nearby search, expand radius, hot tourist area, route A->B, POI ingest) and 2 ADRs (geohash vs H3 vs quadtree, vector vs raster tiles).

Distributed File System (HDFS / GFS / Ceph)

Distributed File System case study covering HDFS, GFS, and Ceph patterns: master/metadata server (NameNode HA via QJM + ZooKeeper) plus DataNode tier with rack-aware 3x replication. POSIX-like file API contrasted with object storage. Block-based storage (128MB), pipelined writes, streaming reads, append-only semantics. Five scenarios: pipelined chunk write to 3 replicas, parallel multi-block read with Master out of data path, append to existing file, DataNode failure with background re-replication, Active NameNode crash with Standby promotion via fence tokens. Includes 2 ADRs (single master vs sharded vs masterless Ceph CRUSH; 3x replication vs Reed-Solomon EC) and capacity hints for all nodes.

Observability Pillars

Observability Pillars concept page — three pillars (metrics, logs, traces) plus continuous profiling, with OpenTelemetry collector fanning out to LGTM stack (Loki, Tempo, Mimir/Prometheus, Pyroscope), Grafana for correlated views, Alertmanager for paging. Five scenarios covering normal operation, incident debugging via trace_id correlation, OTel auto-instrumentation, cardinality bomb anti-pattern, and continuous profiling revealing invisible hot path.

SLI / SLO / SLA

SLI / SLO / SLA concept page: SLI = клиентская метрика, SLO = внутренняя цель, SLA = контракт. Multi-window burn rate alerting (fast 14.4× / slow 6×). Error budget exhaustion → freeze deploys. 3 scenarios + 2 ADRs.

RED + USE methods: Rate/Errors/Duration vs Utilization/Saturation/Errors

RED + USE methods concept page: Tom Wilkie's RED (Rate, Errors, Duration) for request-driven services on the left, Brendan Gregg's USE (Utilization, Saturation, Errors) for DB host resources on the right, with Prometheus + Grafana + Alertmanager observability stack in the center and worker pool example below. Three scenarios demonstrate RED healthy state, USE catching DB pool/disk saturation before RED errors, and combined RED+USE dashboard for full incident picture.

Golden Signals deep dive

Golden Signals deep dive: Latency, Traffic, Errors, Saturation. Order API tier with connection pool, Postgres + Kafka backends, Prometheus + Alertmanager + Grafana observability stack, on-call engineer. Three scenarios: latency p50 vs p99 spike with histogram_quantile PromQL, traffic drop + 5xx cascade correlation, saturation 80% predicting failure 30min before pool exhaustion. Includes ADR-001 on percentiles vs averages.

Distributed Tracing

Distributed tracing с OpenTelemetry: W3C trace context propagation, head-based vs tail-based sampling, hot trace investigation, anti-patterns с async boundaries.

Logging Strategies

Logging strategies: structured JSON logs, levels, correlation IDs, sampling, PII redaction, retention tiers, Loki vs Elasticsearch ADR

Error Budgets

Error budgets concept diagram. Production service (api-1, api-2, api-3, Prometheus, Alertmanager) feeds burn-rate signals into release controls (feature-flag-svc, deploy-pipeline, on-call). Three scenarios: monthly budget tracking with healthy state allowing free deploys, fast burn alert (14.4x burn rate, multi-window) paging on-call after a bad deploy, and budget-exhausted state automatically freezing deploys via feature-flag service with override path. Two ADRs embedded: (1) multi-window multi-burn-rate alert design, (2) budget-exhausted response policy: hard freeze default with VP-Eng override.

Chaos Engineering

Chaos engineering concept page — Netflix-origin (Chaos Monkey 2010), 4 Principles of Chaos (steady-state hypothesis, real-world events, prod, blast radius), tools (Chaos Monkey, Chaos Mesh, Litmus, Gremlin, AWS FIS, toxiproxy), maturity levels, game days vs continuous chaos. Topology: 3 AZs in us-east-1 each with svc-a + svc-b, edge group with client + LB, chaos control plane (Gremlin/AWS FIS + steady-state dashboard + big red button rollback). Five scenarios: steady-state baseline, Chaos Monkey kills random instance, AZ network partition, dependency latency injection with retry storm, quarterly game day with runbook findings. ADR-001 on chaos-controller covers when to graduate from staging to prod chaos.

Incident Management

Incident management concept (SRE): adapted ICS for software incidents. Severity levels (SEV1/2/3), Incident Commander (IC) coordinator role separate from Operations who fixes, Communications Lead handling status page, Scribe capturing timeline, SMEs on-demand. Detection via Prometheus SLO burn -> Alertmanager -> PagerDuty -> on-call IC. War room via Slack channel + Zoom + Incident.io orchestration. Customer comms via Statuspage (Investigating -> Identified -> Monitoring -> Resolved cadence every 30min) + Twitter + support macros. Mitigation toolkit: rollback, feature flag off, regional failover, manual circuit break — mitigate first, investigate later. Four scenarios: SEV-1 textbook response with rollback in 28min, statuspage 30-min comms cadence, follow-the-sun handoff between US and EU regions, anti-pattern of investigation > mitigation causing 45-min downtime. Includes 2 ADRs: rotating IC vs dedicated IC team, blameless culture enforcement.

Post-mortems

Post-mortems concept: blameless culture (Etsy/Google exemplars), document anatomy (summary/impact/timeline/root-cause/action-items/lessons), 5 Whys + fishbone RCA, action item tracking discipline, anti-patterns ("human error" root cause). Four scenarios: timeline reconstruction, 5 whys analysis, monthly action item retro, blameless vs accountability. ADR on blameless vs accountability tension.

On-call rotation & burnout prevention

On-call rotation & burnout prevention concept page. Architecture: Production tier (app + db) emits metrics to Prometheus, which fires alerts via Alertmanager into PagerDuty. PagerDuty pages region-specific primary on-call (US/EU/APAC for follow-the-sun) with escalation chain to secondary then manager. Primary uses Slack #incident channel, Runbook wiki, and Statuspage for comms. Includes ADR on PagerDuty node about on-call pay model (base stipend + per-page premium vs flat salary uplift). Five scenarios: weekly rotation handoff Monday morning, alert fatigue tuning to reduce noise 80%, follow-the-sun handoff between US/EU/APAC timezones, escalation chain when primary doesn't ack, and burnout prevention loop with pages-per-person metric tracking and toil reduction.

Performance Engineering: Profiling, Flame Graphs, Tail Latency

Performance Engineering — methodology in action. Three scenarios: (1) CPU flame graph reveals 60% CPU in jackson.ObjectMapper init — fix with static singleton beats horizontal scale by 4×; (2) Continuous profiling (Pyroscope) catches a regression in v2.4 (gzip on tiny payloads) within 1h via version diff; (3) Load test with k6 surfaces tail latency at p99 — mean lies (45ms), p99 is 2.8s due to autovacuum stop-the-world; tuned + hedged requests bring p99 to 280ms. Topology: production service (lb/api/cache/db) instrumented with /debug/pprof, scraped by Pyroscope into a flame graph UI. SRE tooling: k6 load gen, Prometheus RED metrics, Grafana p99 dashboard, Tempo traces. On-call engineer drives flame graph inspection, SLO checks, and load runs. Two ADRs: optimize hot path before scaling horizontally; continuous profiling in prod over ad-hoc local profiling.

SPA vs SSR vs SSG vs RSC

SPA vs SSR vs SSG vs ISR vs RSC streaming — где и когда мы превращаем данные в HTML. Build-time → CDN edge → origin Node → client browser. Trade-offs: TTFB, SEO, dynamic data freshness, infra cost. 5 сценариев: SPA blank-then-fill, SSR per-request render + hydrate, SSG/ISR pre-built на CDN, RSC streaming через Edge с Suspense, anti-pattern (SPA для marketing landing). 2 ADR: дефолтный выбор в 2026 (Next.js App Router + RSC) и правило близости рендеринга к user.

State Management: server, client, URL, form state

Frontend State Management concept page covering server state (TanStack Query), client/UI state (Zustand), URL state (Next.js searchParams), and form state (React Hook Form). Includes 2 ADRs (Zustand vs Redux Toolkit vs Jotai for 2026; TanStack Query vs SWR vs custom useEffect) and 3 scenarios: local state lifted to Zustand, server state cache+refetch+mutation+invalidation, URL state filters with TanStack cache reuse.

BFF (Backend For Frontend)

BFF (Backend For Frontend) concept page: per-client backend layer (web BFF, mobile BFF, GraphQL gateway alternative, partner public API) sitting between clients and domain microservices. Three scenarios: web BFF aggregates 5 microservice calls into single typed response with edge cache; mobile BFF returns lighter payload (3KB vs 12KB); GraphQL gateway alternative with DataLoader. Plus anti-patterns scenario for timeouts/circuit breakers. Includes ADR comparing BFF per-client vs single GraphQL gateway vs API Gateway.

Micro-Frontends

Micro-frontends concept page demonstrating runtime composition via Module Federation. Shell host app loads independently-deployed remote MFEs (Search/Product/Checkout) from per-team CDN bundles, each with its own BFF backend and CI/CD pipeline. Shows shared design system as singleton via shared scope, cross-MFE communication through window CustomEvents bus, and React error boundaries for failure isolation. Four scenarios: initial load with Module Federation runtime, independent deploy by Checkout team without redeploying others, cross-MFE event from Search to Checkout via event bus, and failure isolation when one MFE bundle fails to load. Includes ADR on when MFE worth the complexity (Spotify/IKEA/Zalando scale: 100+ engineers, 5+ teams) vs anti-pattern for small teams.

Web Performance & Core Web Vitals

Web Performance & Core Web Vitals concept page. Google CWV (LCP < 2.5s, INP < 200ms which replaced FID in 2024, CLS < 0.1). Topology: User device (browser, main thread, web worker) + Edge/CDN + Origin (SSR/API/DB) + Telemetry (RUM beacon, CrUX, Lighthouse CI). Six animated scenarios: Bad LCP (4.2s render-blocking + 800KB hero JPEG), Optimized LCP (1.6s with preload + AVIF + critical CSS + Early Hints), CLS spike (0.4 from img without dimensions + font swap + cookie banner), Bad INP (380ms long task), Good INP (90ms via useTransition + virtualization + Web Worker offload), CI perf budget gate blocking PR. Three ADRs on browser node: performance budget enforcement in CI as hard gate, INP optimization decision tree (useTransition vs scheduler.postTask vs Web Worker), CLS bulletproofing with explicit dimensions + font metric matching.

Accessibility (a11y)

Accessibility (a11y) concept page: WCAG 2.2 (POUR principles, A/AA/AAA), semantic HTML first vs ARIA last, keyboard navigation, focus management, screen readers (NVDA/VoiceOver/JAWS), skip links, aria-live regions, prefers-reduced-motion, three-tier audit pipeline (axe/Pa11y/Lighthouse + manual + real users), legal compliance (ADA Dominos v. Robles 2019, EAA 2025). 4 scenarios, 2 ADRs.

PWA & Service Workers

PWA & Service Workers concept page: Service Worker as browser proxy for network requests, cache strategies (cache-first, network-first, stale-while-revalidate), Workbox library, manifest.json (installable), Push API + Notifications, Background Sync queue, offline-first patterns, PWA vs native trade-offs ADR. Four scenarios: SW intercepts fetch and chooses cache strategy, offline page when network down, push notification flow (tab closed), background sync queue.

Offline-first architecture

Offline-first architecture concept page. Local-first apps (Linear, Figma offline, Obsidian). IndexedDB / SQLite-WASM is source of truth, sync engine (Replicache/ElectricSQL) replays mutations to a Sync API backed by Postgres. Three scenarios: offline edit drained on reconnect, CRDT conflict merge across two devices, optimistic UI rollback when server rejects. Carries an ADR comparing Replicache vs ElectricSQL vs custom sync.

Infinite Scroll & Feed UI

Infinite scroll & feed UI concept: cursor-based pagination vs offset, virtualization (TanStack Virtual / react-window), IntersectionObserver load-more, scroll restoration, optimistic updates with rollback, WebSocket new-posts pill. Architecture: Browser (viewport, virtualizer, TanStack Query cache, IntersectionObserver, History API) + Network (Image CDN, edge cache with stale-while-revalidate) + Backend (Feed API, ranking service, Posts DB with composite index, WebSocket gateway). 5 scenarios: cursor pagination + IO load-more, virtualization for 10K items, scroll restore on back nav, optimistic like + error retry banner, WS realtime new-posts pill with backpressure. 2 ADRs: cursor vs offset pagination; infinite scroll vs load-more vs numbered pagination (dark-pattern discussion).

Real-time Collaboration Architecture

Real-time collaboration architecture from frontend perspective: WebSocket gateway with sticky sessions by docId, stateful Yjs doc actor tier handling CRDT merge, separate Redis pub-sub for ephemeral awareness/cursors, IndexedDB-backed offline ops queue, snapshot persistence to Postgres+S3. Four scenarios: presence cursor broadcast (15 Hz throttle + interpolation), CRDT concurrent merge (commutative ops, no transform), offline reconnect with vector-clock catchup, multi-cursor with server-side ACL denial and client rollback. Includes ADR comparing Liveblocks managed vs Yjs+Hocuspocus self-host vs PartyKit edge.