← Lessons/State — Break It & Recover
← State — Break It & Recover

Why teams need remote state

Learn

Two applies, one state file

Local state works for one person. Now imagine two engineers (or a person and a CI job) running apply against the same state file at the same moment:

Apply #1: Creating...
Apply #2: Creating...
Error: state serial mismatch (expected 3, found 4)
The state file has been corrupted by a concurrent write.

Each apply read the state, changed the world, and wrote its own version back. The second write overwrote the first, so state no longer matches what really exists: both runs think they're right, and the next plan is fiction.

The fix is two features together:

  • Remote state: one shared copy (an S3 bucket, Terraform Cloud), not a file on each laptop.
  • State locking: an apply takes a lock before it reads state and holds it until it writes. A second apply waits or fails with state locked instead of corrupting anything.

That's where the free track ends: remote state and locking are the first thing the production track builds.

Local state works fine for one person. Click Two applies at once to preview what happens when two engineers apply against the same state file with no locking — the state corrupts. We don't fix this here on purpose. Then answer the questions below.

Why did two applies at once corrupt state?
What lets a team share state safely?
Sign in to check your answers

Interview prep

State is the #1 source of Terraform interview questions and real-world incidents.

What is configuration drift and how do you detect it?

Drift is when real infrastructure changes outside Terraform (a console edit, a hotfix, another tool), so reality no longer matches state. You detect it by running terraform plan (or plan -refresh-only): the refresh step reads the real world and the plan surfaces the difference. You resolve it by either applying to bring reality back to your code, or updating your code to match.

Why should you never edit or delete the state file by hand?

State is the only mapping between your config and real resource IDs. Delete it and Terraform forgets what it owns — the next apply tries to recreate resources that already exist (duplicates/conflicts), and the originals become orphaned. Hand-editing risks the same corruption. To re-establish the mapping you use terraform import, not a text editor.

Two engineers run terraform apply at the same time against the same state. What happens, and how do you prevent it?

Without locking, concurrent writes interleave and corrupt the state file (serial mismatch / lost updates). You prevent it with a remote backend that supports state locking (e.g. S3 + DynamoDB, Terraform Cloud, GCS), so the second apply waits for the lock instead of racing.

Loading...

State Inspector

State

Terraform's memory

empty — nothing here yet

Real Infrastructure

the actual world

empty — nothing here yet

Break it & recover

These act outside Terraform — exactly how state gets into trouble in real life.