We’ve all been there: a critical API endpoint starts lagging, p99 latency spikes into the seconds, and the immediate reaction from the engineering team is to blame PostgreSQL. "The database can't handle the load," someone says. "We need to rewrite these queries, add more composite indexes, or migrate to a NoSQL cash-burner." So you fire up EXPLAIN ANALYZE, tweak your join conditions, tune work_mem, and pray. But what if the problem has nothing to do with your SQL, your schema, or PostgreSQL itself?
What if Postgres is actually running blazingly fast, but it is constantly stalled waiting on your cloud provider's underlying block storage?
In modern cloud infrastructure—whether you are running AWS Aurora, managed GCP Cloud SQL, or self-hosted EC2 nodes—developers frequently confuse database compute constraints with I/O subsystem limitations. Today, we are going to dive deep into the storage mechanics of PostgreSQL, understand why default cloud storage options cripple database performance, and look at how to properly architect and monitor your disk sub-system for maximum throughput.
Understanding the PostgreSQL Write Engine (WAL & Checkpoints)
To understand why storage bottlenecks kill Postgres performance, you first need to understand how Postgres interacts with the filesystem. PostgreSQL uses a Write-Ahead Logging (WAL) architecture to guarantee durability (the "D" in ACID).
When you execute an UPDATE or INSERT query, Postgres does not immediately write those changes to the actual heap tables on disk. Doing so would turn every single write into a random I/O operation, which is brutally slow. Instead, Postgres follows a double-write process:
- Write-Ahead Log (WAL): The change is written sequentially to the WAL files. Sequential I/O is exponentially faster than random I/O.
- Shared Buffers: The relevant database pages are updated in RAM (now called "dirty pages").
- Background Writer & Checkpointer: Periodically, a background process flushes these dirty pages from RAM back to the main heap storage on disk in a controlled, batched fashion.
Here is a conceptual flow of how data moves from a client query down to physical storage:
+