A database system is of absolutely no use to you when it's not running. Although Informix is a very robust and hardy system, Murphy's Law decrees that something will always go wrong, and it will go wrong at the most inopportune time.
In this 10-Minute Solution, I concentrate on the most common cause of Informix database crashes: hard drive failures. While there are literally thousands of other things that can go wrong with your database system, disk crashes are among the most traumatic.
When that happens, you really have to earn your high salary. In many companies, when the database is down, the company is down. I've seen systems that lose the company more than $100,000 per hour when they're down. It tends to get a little intense when such systems die. CEOs who have never even visited the computer room seem to take up permanent residence until things are fixed.
Things like this happen with the best hardware and the best software. Maybe it's the system load or maybe it's the phase of the moon, but you can be certain that your system will find a way to get your heart pumping with an adrenalin rush and make you consider the relatively quiet life of goat herding in Tajikistan's Pamir Mountains as your next career.
It's happened to all of us. The system crashes an hour before the [choose your poison: big report is due, CEO is about to give a demo of the system at a major conference, year-end closeout, biggest sales day of the year]. You're left with an expensive pile of hardware and a database that is dead to the world. You've just experienced a disk failure.
Disk failures are the bane of database administrators and system administrators everywhere. If you look at your disk drive specs, you'll find an entry for MTBF, or "mean time between failures." Starting with that, you should realize that failure is part of the disk drive specifications. Failure is a predictable occurrence. To make it worse, the probability of failure increases with the number of drives.
Therefore, build your system(s) with the inevitability of failure in mind.
Having said that, I should tell you that there are several things you can do to minimize the probability of a disk failure and to reduce the pain during recovery. Some of these are no-brainers:
- Perform regular backups
- Keep detailed change logs
- Buy quality hard drives
- Improve your disk layout
- Mirror your system
- Use a RAID 5 system
