Array recovery · RAID 5 sets
The sequence rarely varies on the RAID 5 sets that reach us from Maidenhead. One member drops out, the array runs on with nothing in reserve, a rebuild is started to restore it, and part-way through the read a second disk gives up. There is more room here than it sounds: failed is a controller's judgement rather than a diagnosis, and everything returned to you is lifted from sector-by-sector copies, never from your own disks.
Describe the fault to an engineer
0800 6890668
| The symptom | The likely cause | Start here |
|---|---|---|
| One disk has fallen out and the set is degraded | Nothing is protecting the data now; every extra hour is a gamble | Power it off before the second fails |
| A rebuild started and never reached the end | The end-to-end read found weak surface on a second ageing disk, which is how these usually finish | Do not restart it; leave it be |
| Two members are flagged failed by the card | Nearly always one disk truly gone and another with a scattering of unreadable sectors, not two losses | Post the whole set; imaging settles it |
| Dropped disks were forced back into the set | Stale blocks now sit among the live stripes | Go no further |
| New card fitted and the array disappeared | Only the metadata disagrees; the data underneath is untouched | Routine reconstruction work |
| Recovery software has already been run on the members | Your original disks have been written to | Send them in, and say which tool ran |
Nothing becomes chargeable until you have said yes to it. Each item takes a case number of its own the moment it arrives, an engineer establishes what has actually failed, and you are given a candid view of which files stand a realistic chance of returning. The single quote that follows is fixed, written down, and costs nothing.
Each disk goes across to our own storage in its entirety, the ones the card threw out included. Troublesome regions are charted as the copy runs, the poorest surfaces are held back for a final sweep, and at no point is anything written to the media you sent.
Member order, stripe width and parity rotation come from two places: the metadata the controller left on every disk, and the structure of the data underneath. Nothing is assumed, and no working card has to survive.
The volume is put back together across the clones. Every block is taken from whichever image read it most cleanly, and where no member gave a clean answer parity fills the gap, which is how runs held only by a condemned disk come back. The file system sitting on top is then repaired and test-mounted.
Nothing is billed until you have seen the recovered file list in full and told us to go ahead. What comes back arrives on new media, with the return journey at our cost, and the case stays open at this end until you have opened every file yourself.
Work the numbers through once, then leave them alone: the consumer specification permits a single URE, an unrecoverable read error, per 1014 bits, which is about one bad sector per 12.5TB; enterprise disks are rated 1015. Three survivors in a four-bay set of 16TB disks must give up roughly 48TB in a single sitting, so the arithmetic is not imaginary. In practice a great many drives run well past their specification. Neither observation cancels the other, and the conclusion is the same either way: clone everything up front, and a rebuild loses its power to hurt you.
Nobody noticed disk two drop out in June, so when disk four left the array mid-rebuild in August it looked like a double failure. It was not. Imaging found nineteen bad sectors on that fourth drive and nothing worse, so three intact members plus one near-complete fourth rebuilt the stripe, and the filings met their deadline.
In most cases it is. There is a wide gap between a disk the card has ejected and a disk that has stopped turning, and what usually earns the failure flag is a handful of sectors that will not read back. Working from clones, whatever one member withholds is taken from the parity of the rest. Two genuinely ruined disks are unusual; two evicted ones arrive every week.
Not until every member has been copied. A rebuild asks each surviving disk for a complete end-to-end read, which is precisely the workload that finishes off one already close to the edge, and it writes fresh parity across the very stripes a reconstruction needs to read. Once the clones exist, run it as often as you like.
An unrecoverable read error is a sector whose contents the drive cannot return, its own error correction having already failed on it. Consumer disks carry a published rate of roughly one in every 12.5TB read; enterprise models are quoted at one in 125TB. Those numbers set a floor rather than forecast anything, and modern firmware answers a URE by writing off one stripe instead of abandoning the array.
No. Each member holds the controller's metadata setting out the geometry, and when that record has been corrupted the arrangement of the data itself gives the same answer. Reassembly happens on the bench against images; the disks you posted are only ever read.
Each further power-up asks more of a drive that has already begun to fail. Open the case first and let the free diagnosis tell you what is left.