Why Cassandra fits high-frequency writes

Imagine thousands of devices sending measurements every second. Each measurement is an independent write. Cassandra can handle this workload by spreading writes across servers and grouping disk writes on each server. 1. What happens to one measurement? Cassandra · Commit log, memtable, SSTable Append to commit log + update memtable → flush a batch to an SSTable Each replica appends the measurement to its commit log and adds it to an in-memory structure called a memtable. If the replica crashes before a flush, it can replay durable commit log entries. The commit log sync settings control when those entries reach durable storage. The memtable holds many writes. A flush writes them in sorted order to a new file called an SSTable. This is the write path of a log-structured merge tree, or LSM tree. For an ordinary write, Cassandra can record a new value without reading and modifying an older SSTable. PostgreSQL · WAL, heap pages, indexes Record WAL + update buffered heap and index pages → flush pages later PostgreSQL also has a recovery log, called the write-ahead log (WAL). It can acknowledge a durable commit before the changed table pages reach disk. Each insert adds a row to a heap page and updates the applicable indexes. With a B-tree index, it adds an entry to the appropriate index page. PostgreSQL buffers page changes, but the primary still does the work of maintaining the table and its indexes as measurements arrive. Cassandra groups table writes into new sorted files. Later, compaction merges files and rewrites data. This shifts some work out of the write request and adds background disk I/O. Sustained throughput depends on compaction keeping pace. Cassandra storage engine · PostgreSQL WAL · PostgreSQL index maintenance 2. What happens when the write rate doubles? In Cassandra, the device ID and hour can form the partition key. Cassandra routes each partition to its replica set. Writes for different devices can use different replica sets. Adding nodes can increase write capacity if the partitions and write traffic are well distributed. One heavily used partition still depends on its own replica set. In a standard PostgreSQL primary and standby setup, all inserts go to one primary. Standbys replay its WAL. If incoming writes exceed the primary’s capacity to process rows, maintain indexes, or flush WAL and data pages, requests can queue. Write latency rises, and clients can eventually time out. To spread that load across multiple PostgreSQL primaries, the deployment needs sharding. Cassandra partitioning and replication · PostgreSQL physical replication When Cassandra has the advantage This workload gives Cassandra two ways to sustain writes. The LSM tree groups many changes into sorted disk writes. Partitioning lets multiple servers do that work in parallel. That combination can make Cassandra a better fit when independent writes exceed what one PostgreSQL primary can sustain. PostgreSQL can handle high insert rates within the capacity of its primary. The crossover depends on the hardware, schema, replication and durability settings, and how evenly the application distributes writes. A fair benchmark measures sustained throughput and write latency while compaction and other background work run. Inside Cassandra’s write path On each replica, the same measurement passes through three structures. Each structure has a different job. New measurement → commit log and memtable → SSTable → compaction Commit log · Recover a write after a crash The replica appends the measurement to its local commit log. Tables on that replica share the log. Cassandra writes to the current segment until it reaches its size limit, then starts another segment. Suppose the replica crashes before it flushes the measurement to an SSTable. On restart, Cassandra can replay the durable log entry into the memtable. After all changes in a segment reach SSTables, Cassandra can recycle that segment. In batch sync mode, the replica waits for the log to sync to disk before it acknowledges the write. In periodic mode, it can acknowledge before the next sync. A crash can therefore lose recent unsynced writes on that replica. Memtable · Collect writes in memory Cassandra keeps recent changes for a table in a memtable. Reads can see data in the memtable before Cassandra flushes it. This allows the replica to collect many measurements before it writes a new SSTable. A flush can occur when memtables reach their memory limit or when Cassandra needs to free commit log space. Cassandra switches to a new memtable so that incoming writes can continue while the previous one flushes. SSTable · Store a sorted batch on disk A flush writes the data in sorted order to a new SSTable. An SSTable is immutable. If a device sends a correction for the same measurement later, Cassandra can write the correction to a new SSTable. A read may need data from the memtable and several SSTables. Bloom filters help Cassandra skip files that cannot contain the requested partition. Compaction merges selected SSTables and can remove obsolete values. For the device workload, each flush turns many small changes into a larger sequential write. Compaction rewrites some of that data later, so the disks need enough capacity for both incoming writes and background merges. Apache Cassandra storage engine documentation