SizeTiered vs Leveled Compaction: A 10x Difference in Disk Headroom
Cassandra needs free disk space beyond your replicated dataset just to run compaction, and how much depends entirely on which strategy you pick. This calculator models SizeTieredCompactionStrategy (STCS) as needing headroom equal to roughly 100% of replicated data size — the worst case for compacting large, similarly-sized SSTables together — versus LeveledCompactionStrategy (LCS) at around 10% of replicated size. On a 1TB dataset with RF=3 (3TB replicated), that's the difference between provisioning about 6TB (STCS) and 3.3TB (LCS) of usable disk for the same data.
Why STCS Needs So Much More Free Space
STCS periodically compacts SSTables of similar size together; in the worst case — several large, similarly-sized SSTables merging into one — Cassandra needs enough free space to hold a second full copy of the data being compacted before it can delete the originals, which is where the ~100% headroom modeled here comes from. LCS avoids this by keeping SSTables in fixed-size levels (160MB by default) and compacting small, overlapping key ranges a few at a time, which is why its headroom requirement is an order of magnitude smaller.
The Tradeoff You're Actually Making
- STCS: lower background compaction I/O, generally better for write-heavy workloads — paid for with far more reserved disk headroom.
- LCS: fewer SSTables to check per read (better read latency) but meaningfully higher sustained compaction I/O in exchange for its smaller disk headroom requirement.
- Replication compounds whichever you pick:
totalDiskRequired = replicatedData + compactionHeadroom, andreplicatedData = rawData × replicationFactor, so raising RF from 3 to 5 doesn't just add replicas — it scales your compaction headroom by the same factor.