I have discovered some damaged photos in my photo archive, googled around, and found that the cause of this is probably some variant of "bit-rot". This scares me, so I have begun research on how to combat this. Simple backup-strategies don't seem like they are enough, as the "flipped bits" will easily get copied along with other file changes, thereby overwriting the good backup copy. The old rule about backing up to different media is no longer practical or cost-effective, as optical disks are dead, and tape drives are unpractical and too expensive for most independent photographers like myself. I rely solely on RAID arrays and online backup services like Backblaze and Crashplan. I have to make sure my first backup-set stays intact, otherwise I'm in danger of having flipped bits transferred into my backups.
First, I was considering switching to the ZFS file system, but this seems overly complicated. I am still considering buying an Infortrend EONNAS NAS, but I already have two Drobos with a bunch of disks that I'd like to put to use, and scaling a ZFS NAS seems like a lot of hassle, so I'm hesitant. I keep thinking there must be ways to do this without having to go through the drastic steps of changing storage solutions and/or file systems.
I have found articles like this one, which suggests using checksums to verify and even repair corrupted files:http://clusterbuffer.wordpress.com/file-system-tools/checksum/
The problem with the above is that it seems aimed at sysadmins comfortable using command-line tools. I'm not one of them, and I assume most photographers like me prefer to spend their time behind the camera rather than learning Terminal commands.
I also found this article on the controlled vocabulary site, which even has some useful software suggestions:http://www.controlledvocabulary.com/imagedatabases/file-verification.html
The problem here is that the most promising software; Checksum+, seems like it is no longer maintained, and I can't get it to work under Mac OS X 10.8. Also, it doesn't seem like it has any options for restoring data using the checksum, or deduping, which would also be nice to have:http://www.controlledvocabulary.com/imagedatabases/de-dupe.html
So; it seems to me like checksumming, deduping and restoring files would be a killer feature for the upcoming catalogue version of PhotoMechanic.
Something similar has already been put in place for Adobe Lightroom 5, according to Peter Krogh:http://thedambook.com/dng-verification-in-lightroom-5/
The problem with the above is, of course, that it only works with DNG files and with Lightroom.
Time is money, and I would rather not have to use DNGs and Lightroom because both converting to DNGs and using Lightroom is too slow to my liking, even on a maxed out MacPro using SSD's. I also prefer to adjust my RAW files in CaptureOne and would like to keep my RAW files open to cross-software workflows and not be locked into Adobe's universe. Previous experimentation with DNG's suggests that converted DNGs don't always play well with other software.
Another approach I have also tried is to use ImageVerifier, but this fell into the same category as Checksum+ - buggy under OS X 10.8. I could not make it work.
So, after a lot of tinkering, this is my plea:
Please make this a feature under the new PhotoMechanic Catalogue software!
I imagine that most photographers are very concerned about the integrity of their photo archives, so it seems like a natural feature to offer for a photo cataloguing program - a fact proved by the fact that the Adobe people have opted to include this feature in Lightroom.
It's just the way they have opted to link it to DNGs exclusively which seems wrong. I would much rather keep my RAW files intact, and simply store the checksums as sidecar files and/or in a separate database - and I would much rather use this feature in conjunction with my favorite photo browser/tagging application; that is of course PhotoMechanic :-)
It also seems like a half-baked solution without the option to restore corrupted files - what is the use of checking data integrity after all if you can't use the checksums to restore the affected files?
I understand that your top priority right is to release the much-awaited catalogue software, but hope you will add this to your to-do list as soon as v.1 is released :-)