I like your database example, Nerseus. Explains my point rather well.
I can't agree with this logic, however. (see below for argument)
I can't agree with this either. Take the AutoRecover feature in Microsoft Word. This is an excellent example of an application working correctly to prevent large amounts of data loss after an unrecoverable error has occured. For example, the exception thrown by Word caused the document buffer to be overwritten. If the exception was "handled" (I use this term loosely) the user would continue working, adding to the overwritten buffer. The buffer would contain invalid data however, and once the user saves the document he/she has lost quite a bit unknowingly. Had the application crashed, AutoRecover would have kicked in the next time Word was reopened, allowing the user to continue where they had left off (or close to it). It boils down to either taking a huge risk, putting a whole document on the line, or a small annoyance, restarting the application. Obviously the latter is a much better option.