My first in-prod corrupted hard drive problem
7 hours ago
- #data recovery
- #corrupted hard drive
- #IT troubleshooting
- Backup system flagged a server hosting a critical MS SQL database with short downtime acceptance due to potential data loss.
- Event Viewer showed errors; initially used MS SQL backup as a quick fix but later issues with inaccessible analyses.
- Investigation shifted from EDR agent, VSS service, Windows system files repair, to a recent SQL patch as leads.
- Timing of SQL patch suggested heavy I/O exposed weakened magnetic sectors on a dying disk.
- Resolution involved offline repair attempts using software like EaseUS and HDD Regenerator.
- HDD Regenerator worked by rewriting weakly magnetized sectors or triggering firmware remapping.
- Recovered database after restoring readable sectors, luckily corruption was at signal level not physical damage.
- Lessons: verify backups restore correctly, treat vendor patches as major changes, prepare for vendor limited help.
- Stay persistent and open to unconventional tools; RAID wouldn't prevent silent corruption replicated across mirrors.