The Grid Is Not Your Friend, and Weather Keeps the Receipts

Every data center tells the same optimistic story about power: utility feed first, backup generators second, batteries in the seam to cover the handoff. On a whiteboard it is airtight. In the field it fails with dispiriting regularity, and the failures cluster around a few recurring themes that are worth learning before the weather teaches them to you at full price.

Start with the grid itself, because the industry tends to treat it as a solved input rather than what it is: a vast, aging, tightly coupled machine that occasionally has a very bad day. The large regional blackouts of the past several decades — the kind that darken tens of millions of people across multiple states or countries — share a structural signature. A local fault, often something as mundane as a transmission line sagging into a tree, triggers a protective trip. That trip shifts load onto neighboring lines, which were already near their limits, so they trip too. Within minutes a small, containable problem has cascaded across an entire interconnection. The lesson for anyone running critical load is sobering: the grid can fail suddenly, over a wide area, for reasons that have nothing to do with you and that no amount of paying your electricity bill will prevent.

This is why serious facilities do not rely on the grid alone. But redundancy is where the second, subtler set of failures lives, and it hinges on a distinction most people gloss over: correlated versus independent failure. Two utility feeds sound twice as safe. They are only genuinely safer if they are actually independent — different substations, different transmission paths, different regional exposure. If both feeds ultimately trace back to the same regional grid, then a wide-area event does not care that you signed up for two of them. It takes both. The same trap hides in geography. Two backup generators in the same flooded basement are not redundant against a flood; they are one failure wearing two costumes. The engineering term of art is 'common-mode failure,' and it is the quiet assassin of redundant designs.

Extreme weather is the most reliable exposer of these correlations, because weather is a wide-area, correlating force by nature. The severe winter events that have stressed major grids in recent years are a clean case study. Cold does not politely knock out one thing. It freezes wind turbines, ices over instrumentation, spikes heating demand across millions of homes, and — the cruel twist — can freeze the wellheads and pipelines feeding the very gas plants the grid was counting on to carry the peak. Supply falls exactly as demand surges. A grid that looked adequately diversified on a spreadsheet discovers that its diverse sources all had the same enemy. That is correlated failure at civilizational scale, and it is the single most instructive pattern in this whole subject.

Heat brings its own version. A prolonged heat wave raises electricity demand for cooling right as high ambient temperatures reduce the efficiency of both the grid's generation and your own chillers. Data centers are enormous consumers of cooling; when the outside air is hot enough for long enough, the cooling plant that keeps servers alive becomes its own point of failure, entirely separate from the power question. Several high-profile facility outages during heat waves have traced not to lost power at all but to cooling systems overwhelmed by conditions beyond their design envelope. The servers did not lose electricity. They cooked.

Water deserves its own line of worry, because it is both a hazard and a dependency. Flooding is the obvious version, and the recurring embarrassment is backup equipment — generators, fuel pumps, switchgear — installed low, in basements and ground floors, precisely where water goes first. But water is also a resource many large facilities quietly depend on for cooling, which means a drought or a municipal water disruption can constrain operations even when power is perfectly healthy. The dependency you did not map is the one that fails you.

The throughline is not that the physical world is out to get you. It is that physical failures correlate in ways digital designs assume away. Software redundancy is cheap and easy to reason about; you can spin up another instance. Physical redundancy is expensive, and worse, it is easy to fake — to build two of something that share a hidden single point of failure. The discipline this module asks of you is to look past the count of your backups and interrogate their independence. Ask of every redundant pair: what single event could take out both? If you can answer that question honestly, you already know more about your true resilience than the reassuring diagram on the wall. And when the weather finally votes, as it does, you will have priced in its ballot instead of being surprised by it.