Erasure coding

Erasure coding transforms k data shards into n = k + m total shards. A maximum-distance-separable Reed-Solomon code can reconstruct the original data from any k valid shards, so it tolerates any m known erasures when placement assumptions hold.

Capacity math

For large full stripes, the nominal storage factor is:

storage factor = (k + m) / k

RS(6,3) stores nine shards for six shards of logical data, a 1.5x nominal factor. Real overhead is higher for metadata, checksums, padding, small objects, allocator granularity, and temporary recovery copies.

This is not the same as tolerating arbitrary silent corruption. Checksums or authenticated metadata must first identify bad shards as erasures. Error correction without known locations has different bounds.

Write and commit contract

Pin codec, field/word size, shard ordering, stripe generation, padding, checksums, and object length. Place shards across independent failure domains. Publish the stripe map only after the configured number of durable shard writes succeeds; otherwise readers can observe an uncommitted generation.

Updates commonly rewrite a stripe or use a log/replication layer. Do not assume every erasure-coded store supports arbitrary append, truncate, or in-place small writes.

Read and repair

A healthy read may fetch the k data shards. A degraded read chooses any k valid survivors and decodes. Repair must read enough surviving data, consume network and CPU, and write replacement shards in valid failure domains. The minimum information is k shards, but practical I/O can be higher because of retries, placement, checksums, and implementation-specific repair algorithms.

Correlated rack/zone loss can exceed the code's assumptions even when fewer than m + 1 hosts fail. Capacity planning must model the chosen failure domain.

Operational guardrails

Scrub checksums, throttle repair, reserve recovery headroom, version metadata atomically, and test partial writes. Replication can be better for small hot objects or latency-sensitive updates; erasure coding often wins for large colder data where storage efficiency dominates.

Diagram scenarios

The diagram is explicitly RS(6,3): encode and commit, healthy six-shard read, recovery from three erasures, hard failure at four erasures, and corruption/placement handling.

Related concept

Interactive scenarios