Hopefully, if you paid any attention at all to the second article of this series, you'll have a good backup of what you wanted. If that's the case, you're golden. All you need to do is go to your backup, grab the missing file(s), and you're back in business - no muss, no fuss, as the laundry detergent commercials used to say.
If the missing file is something you retrieved or downloaded from somewhere - perhaps from the Internet - you're still golden as you can simply go back and grab it again. Again the famous jingle of those old laundry commercials wafts its golden strains through the air. (And perhaps you will save that file off-line somewhere in case the site goes down or they loose the file.)
However, if you find yourself in the incredibly unlucky position of having just deleted an absolutely irreplaceable file, have no backups, and "failure is not an option". . . .
Though you may feel thoroughly screwed to the wall, it is possible that all is not lost.
The descriptions and comments in this article are aimed primarily at the Windows user, because files lost in a Windows-type filesystem are fairly easy to retrieve. If you're not a Windows user, read on anyway. Though it may be much more difficult for you to recover a lost file, the hints and tips in this article apply to you as much as they do anyone else.
There are a number of file recovery methods. Here's a list starting from the easiest to the most difficult:
The Recycle Bin
A feature that has become popular in most every modern operating system is the "Recycle Bin", "Trash", or whatever your particular flavor of operating system calls it. This is a special, hidden, area within the filesystem where deleted files are stored, just in case you decide you want them back. So long as you didn't do a "permanent delete" or a delete that "bypasses the Recycle Bin", you can restore the lost file by simply going to the Recycle Bin / Trash, and restoring the file.
Auto-Save files
Assuming that you are working within some kind of formal environment - a word processor, a spreadsheet, a programming development environment, a database, etc. - these programs can, and often do, create timed automatic backups while you work.
The advantage of this feature is that the program itself periodically takes a snapshot of what you are doing, and saves it in a special file, located somewhere on your hard drive.
In other cases, some programs keep a "one revision back" backup. When you finally save your work, the original file is renamed with a ".BAK" extension and the modified file is written out as a new file with the original file name.
Automatic file backup software
Many backup utilities, Acronis True Image among them, have a feature where earlier versions of files can be automatically preserved. In the case of Acronis, you can allow the software to maintain a special hidden partition at the end of the drive where various important things - like file versions - can be stored away.
Shadow Copies
Windows Vista, Windows 7, (and maybe Windows 8?), have a feature called "Shadow Copies" where the operating system itself maintains a series of older versions of certain files. Mac users will recognize the same idea in their "Time Machine" feature.
It is important to note that the "shadow copy" feature of Windows is usually NOT enabled by default, and depending on your system's configuration, you may have to modify your computer's partition structure to enable it. (Ouch!)
Scattered "temp" files
Occasionally programs like Word, or Word Perfect, don't clean up behind themselves very well and, (sometimes), they leave scattered "temporary" snapshot files laying around; usually beginning with a tilde. (~) It's a bit dicey, but if you're lucky you can recover enough of a file to recreate what was lost.
Undelete Utilities
If all else fails, there are special software utilities that can scan your drive for deleted files and attempt to recover them.
Undeleting a file
All "undelete" utilities depend on one simple fact: Files that have been really and truly deleted are - in fact - not really and truly deleted. The directory entry for that file is marked as "deleted" and all the logical blocks allocated to the file are reset to the "available for use" status in the file system's volume bitmap. However, the actual data that has been written to the disk is NOT deleted - a fact that has lead to many a criminal's downfall. Not to mention things like divorces, getting fired, or even identity theft.
So, what REALLY happens when Windows deletes a file?
When Windows deletes a file, what happens is something like this:
- First, Windows finds all the pieces of the file. That includes the file's directory entry, along with each and every part of the file itself.
- Then Windows places a special "mark" on the directory entry that says this file has been deleted. Because of this, the "deleted" file disappears from the folder it was in.
- Finally, Windows goes to the system's volume bitmap and - for each and every part of the file - turns the corresponding bits OFF, indicating that these parts are "empty", and are available to be re-used.
If you can get back to the file quickly enough - before someone else moves in! - you can take down the big "FOR RENT" sign, set all the volume bitmap bits for that file back ON again, and Voilà! Your file is back.
The problem is this: Windows is constantly doing things with the hard drive; moving things around, updating important information, figuring out what goes where, and deciding where it wants to put the next piece of whatever it's doing. Because of this, "empty apartments" don't stay empty for long, and some of the individual locations and pieces that belonged to your file can get recycled and re-used by other files. If that happens before you get back to the "apartment" you just left, you're out of luck. It's already taken and it just stinks being you.
If that's true, how do undelete utilities work?
Undelete utilities can use different methods to do their job, and one possible way is like this:
- The undelete utility scans the disk until it finds a directory entry that has been marked as "deleted".
- If the directory entry itself isn't corrupted, it attempts to traverse the entire file, end-to-end, using the file's metadata within the directory entry, as well as the "next" and "previous" links in each of the file's individual parts, to verify the integrity of the prospective file.
- If the program can traverse the entire file end-to-end, and the resulting file data matches all the file metadata in the deleted directory entry, (size, etc.), then classify this file as "excellent". At this point you have a pretty good chance of getting the entire file back.
- If the program can apparently traverse the entire file end-to-end and it appears to be complete, but there are discrepancies between the resulting file and the file's metadata, then classify the file as "good". At this point you have a pretty good chance of getting, (at least), part of the file back. Maybe it's OK, maybe not, but it's at least an even-money bet.
- If the program can only traverse part of the file because the file appears to have been overwritten at some point, (i.e. segment "x" points to segment "y", but segment "y" does not point back to "x"), then classify the file as "poor". At this point you know for certain that the file has been damaged, but depending on what kind of file it is, and how much of the file the undelete utility can find, it might still be usable - at least in part.
- If the program cannot traverse any portion of the file because the very first segment of the file has been overwritten, then classify the file as "unrecoverable". At this point you can pretty much throw in the towel unless you want to use the more advanced, (and chancy), features of the program. Your ability to recover even part of the file is now in the hands of God, and the cleverness of the utility's programmer.
- Then, go to the next directory entry marked as "deleted" and repeat all of the above; continuing until all "deleted" directory entries have been examined and classified.
- Once everything it can find has been found and classified, you get the opportunity to select what you want to try and recover.
Even if the file is classified as "excellent", there is still a very real possibility that the file will be borked when you try to recover it. Why? Because Windows is still working with the disk even while YOU are trying to undelete something. Not only is this possible, it's not all that uncommon either. Like I said before, "undeleting" a file is a dicey proposition at best.
But what if all that effort doesn't work, and my file is still missing?
Many undelete utilities also include a special "advanced", "deep", "full", or "exhaustive" scan mode for finding files that the easier methods miss. This is based, (at least in part), on the fact that files of a particular type have distinctive characteristics unique to that type of file. For example, JPEG picture files begin and end with sequences of bytes that are specific and unique to JPEGs.
The advanced scan mode, (using the JPEG file type we mentioned above as an example), scans the entire drive, one tiny piece at a time, looking for the distinctive characteristics of a JPEG file. If it finds the specific characteristic that indicates what might be the beginning of a JPEG file, it tries to follow the trail all the way to the end of that file. Likewise, if it finds the other unique characteristic first, (indicating the possible end of a JPEG file), it tries to follow that trail back to its beginning. And it's very likely to be a false scent. If it is, you start all over again, continuing your search, one tiny piece at a time. Again, and again, and again.
When you're doing this kind of microscopically detailed search, you've entered the wild, woolly, and wonderful world of "forensic analysis", and it is not unusual for some random sector to contain what appears to be the "magic bytes" for a particular file type. Or, maybe you have found a JPEG file, but it's one that hasn't been deleted! Because of all the random factors that can be encountered, searches of this kind can be very time consuming. As in days, (or even longer!), depending on how much experience you have doing this kind of stuff, and the size of the hard drive you're searching.
If we assume that you're lucky enough to find what you believe are all the pieces of a particular file, the undelete utility assumes that you've found a complete file of that type and classifies the file as potentially recoverable.
If the file is not complete, but at least some of it is present, it classifies the found file as either "good" or "poor" depending on what, and how much, of the file has been found.
And, in some cases, the file is just gone; never to be seen again. And that's that.
Once all that is done, if you decide to recover the file(s) that were found, the utility recovers, (or creates if necessary), the file's directory entry and marks all the file's pieces as "occupied" in the system's volume bitmap.
How to improve your chances for recovering a file or files
If you've gotten this far, and are seriously contemplating going the "undelete" path, there are things you can do that will dramatically improve your chances of getting your data back.
- Preparation:
Prior to needing them, get one or more undelete utilities that do not require installation, (this is important!), and save them somewhere you can get your greasy mitts on them in a hurry. (A thumb-drive, an external hard drive, a CD or whatever are all good choices.) - If you have the skill, time, and want to really cover your butt, you might want to consider creating a Bart PE rescue disk, (or something similar), with the undelete software included. If you don't have the requisite skill, you might want to beg a more technical friend or colleague, pretty, pretty, pretty-please, to make one for you. It's a really great idea.
- Have more than one "gun" and plenty of ammo:
Different undelete programs work in different ways, so just because program "A" doesn't find your file, does not mean that program "B" can't. - When you collect your undelete utility(s), get more than one. Professional forensic analysts often use many different programs to examine a disk to increase their chances of getting what they want.
- Consider investing in a combination of freeware, shareware, and possibly even payware programs.
- If you're smart/lucky enough to have something like a Bart PE disk, put all the utilities you've collected on it so that they are all available when you need them.
- Protection:
If you accidentally delete something important, it is absolutely crucial that the disk be taken off-line quickly! You'll want to do this as rapidly as possible to prevent the sectors used by your deleted file from being overwritten. - If possible, shut-down the system, (if it's the system drive), or un-mount it quickly, (if it's a external or data-drive), to reduce the possibility of it being corrupted.
- Investigation:
If at all possible, you'll want to take the drive and mount it externally on another system, or load an external operating system. (like a Bart PE disk for Windows.) - Mount the drive read only if possible to prevent data corruption.
- Use the external system or CD's software, (if you prepared a Bart PE disk), to search for the files.
- If you are using an advanced recovery method, and you know what kind of file you're looking for, (a JPEG for example), have the recovery utility search for only that type of file, (assuming that this type of file is defined in the utility's dictionary). This will speed up the search process considerably, and will result in a much better chance of getting complete files.
- Execution:
If you find your missing files, try to restore them to another disk drive, (like a thumb-drive), if your undelete utility supports that option. - Prayer:
I'm not including this just to be funny or sarcastic. As you can see, undeleting something is a very iffy proposition at it's best, and a successful undelete depends just as much on luck as it does on skill. - Appreciation:
Everyone likes appreciation, and the poor sod of a software writer likes it even more. Especially since the only communication he's likely to get from his users are complaints! - In all seriousness, if the utility is shareware and it just saved your butt bigtime, dig out the Benjamins and buy the license. It's the least you can do.
- If it's freeware - and there's a "donate" option, cough up the dough. At the very least buy him a six-pack of his favorite brew. Or, absent that feature, send him an e-mail telling him how much you appreciate his skill and thoughtfulness.
Linux/Mac Users:
If you're dealing with a Linux/Unix/Mac type operating system, there's a really good possibility that you may just be plain-ole' hozed if you don't have something like "Time Machine" turned on.
Unix type systems have unique kinds of file systems, file deletion methods, and aggressive i-node recovery paradigms that they often use. (i-nodes contain the file descriptors, and all the pointers and metadata for a particular file or files.) Because of their design, it may be almost completely impossible to recover files deleted on a 'nix system without resorting to extremely advanced techniques that are, (usually), beyond the ability and experience of even the most advanced system administrators.
Conclusion:
With any luck I have, (hopefully), scared you s**tless about the prospect of loosing important files or data. And if I have done so, please, please, pretty-please consider using backups as a recovery strategy instead of "a wing and a prayer". OK?
What say ye?
Jim (JR)
