Your Provider Says Nightly Backups. What Separates That Claim From a Tested Restore

The received wisdom is three copies, two media, one offsite, and it is repeated in vendor decks and audit questionnaires as though it settles the question. It came out of digital photography, where the risk was a failed drive and the recovery was a patient file copy overnight, and for that job it was excellent advice. It is still the right skeleton. What it never claimed to cover is the part that decides whether an organization survives a bad Tuesday: whether anyone has restored from those copies recently, how long it took, and who watched the clock.

The claim everyone makes, and the one question that separates them

Ask a provider whether they back up and the answer is always yes, because the job scheduler has been green for years and nobody has reason to doubt it. Ask instead for the date of the last full restore, the scope of what was restored, the elapsed time from request to working system, and the name of the person who signed it off, and the answers spread out immediately. A good shop produces a document. An adequate one produces a screenshot of a backup console. The gap between those two responses is the whole subject, and it is visible in about ninety seconds.

Backups fail quietly in ways the console cannot see. A database dumped while writes were in flight restores into a file that opens and is wrong. An agent stopped reporting on one subnet after a firmware update and the dashboard counted it as excluded rather than failed. Encryption keys live in a vault that was itself only ever protected by the system now being rebuilt. None of these show up as red. They show up on the day someone needs the data, which is also the day nobody has the spare hours to discover them.

What a restore test actually measures

A real test starts from a stated recovery objective, which is two numbers: how much data the organization can afford to lose, and how long it can afford to be down. Everything else is engineering in service of those numbers. The test then restores to isolated hardware or an isolated network segment, brings the application up, and has someone who uses the system daily confirm that recent records are present and the thing behaves. Restoring files is not the test. Restoring a working service, with its dependencies, credentials and license checks, is the test.

The number that surprises people is throughput. Writing a backup is a background job with all night to finish, while reading one back is a foreground emergency with a queue of people waiting, and the two rates are rarely comparable. Deduplicated storage in particular gives back data slower than it swallowed it, because rehydrating means reading scattered blocks rather than a contiguous stream. An organization that has never timed a restore of its largest system does not know its recovery time. It knows a hope. Timing one full restore, once, converts that hope into a planning figure the finance side can work with.

Where buying cheap ends up costing more

This is a field where the tool choice shows up years later. A cheap appliance that hits the nightly window and bills attractively per terabyte can restore at a rate that turns a four hour objective into a two day outage, and the invoice that revealed nothing about read performance is not refundable. Per-socket or per-workload licensing that looked generous at forty servers becomes the reason a team quietly stops protecting the test environments, which is where an attacker with patience will happily start. Buy for the restore rate and the licensing curve, not the backup window.

Some of it should be hired rather than owned. A tabletop exercise run by someone from outside, for a day or two, will expose the dependency nobody documented: the name resolution that only works because a decommissioned server still answers, or the runbook that assumes an administrator who left in March. Specialist recovery engineering, forensic imaging, large scale rehearsal capacity in a cloud region: all of these are day-rate purchases, and paying for them occasionally is far cheaper than staffing for them permanently. What has to be owned is the schedule, the evidence file and the ownership of the objectives.

Immutability, separation, and the evidence to ask a provider for

The threat model changed and the tooling changed with it. Ransomware operators look for the backup console first, because deleting recovery options is what makes payment feel inevitable, so modern practice is at least one copy that cannot be altered or deleted within a retention window even by an administrator, held under credentials that are not part of the ordinary directory. The National Institute of Standards and Technology is the federal body responsible for the cybersecurity framework most large organizations map their controls against, and recovery sits in it as a function in its own right, not as a footnote to protection.

From a provider, ask for the restore test calendar and the last three completed reports with timings, the retention and immutability settings in writing rather than in a sales sheet, confirmation that their own key material is recoverable independently, and a contractual right to observe or request a sample restore. Most competent providers already have these and are pleased to be asked, because the question distinguishes them from the competitor quoting the same storage for less.

Pick one system that matters, restore it to isolated hardware this quarter, and write down the hour it took. That single number will reshape every other decision in the program.