Verifying CDs or DVDs part II

[Oscarsson]

Kirk,

I'm not sure I made myself understood so I try again.

I want to know that it's safe to format my CFs after having everything backed up to hard drives and DVDs after a shoot.
This is my workflow...

1. I ingest the files with PM, to my harddrive and open the contact sheet.
2. I burn a dvd full of raw-files from within PM.
3. I make a backup copy of that dvd.
4. I need a way to control (verify) that all the files on my second dvd are ok and have not been corrupted in any way. This way I can be sure that all the files on the hard drive and the two DVDs are OK.

Just trusting the "verifying disk" that starts after the burning sequence, doesn't feel all that safe. Because it doesn't verify that the files are OK but rather that the data written to the dvd is the same as the one on the HD.

But what if the data from the HD is corrupt to begin with, then the "verifying disk" is only telling me that I have two corrupt DVDs and corrupt files on my HD.

Except I don't know this, because the software didn't tell me that the files were bad from the start.


So my question is:

Can I load the second DVD into PM and be sure that all the raw files are OK if PM is able to render a contact sheet from them?

Or is this only telling me that the JPEG-preview inside the raw file is OK.

I'm not sure if I'm making sense here or not...

Maybe the only thing that I should ask is:

Are all the raw files OK if PM is able to render a contact sheet from them?
Or is this only telling me that the JPEG-preview inside the raw file is OK?

Is there a way to check this other than opening every single file in a raw converter?

Sorry for this possibly confusing post...

[Kirk Baker]

I want to know that it's safe to format my CFs after having everything backed up to hard drives and DVDs after a shoot.
This is my workflow...

1. I ingest the files with PM, to my harddrive and open the contact sheet.
2. I burn a dvd full of raw-files from within PM.
3. I make a backup copy of that dvd.
4. I need a way to control (verify) that all the files on my second dvd are ok and have not been corrupted in any way. This way I can be sure that all the files on the hard drive and the two DVDs are OK.

Just trusting the "verifying disk" that starts after the burning sequence, doesn't feel all that safe. Because it doesn't verify that the files are OK but rather that the data written to the dvd is the same as the one on the HD.

But what if the data from the HD is corrupt to begin with, then the "verifying disk" is only telling me that I have two corrupt DVDs and corrupt files on my HD.

Except I don't know this, because the software didn't tell me that the files were bad from the start.


So my question is:

Can I load the second DVD into PM and be sure that all the raw files are OK if PM is able to render a contact sheet from them?

Or is this only telling me that the JPEG-preview inside the raw file is OK.

I'm not sure if I'm making sense here or not...

Maybe the only thing that I should ask is:

Are all the raw files OK if PM is able to render a contact sheet from them?
Or is this only telling me that the JPEG-preview inside the raw file is OK?

Is there a way to check this other than opening every single file in a raw converter?

The best way I know of to verify that the data on the DVD is exactly the same as what is on your CF card is to use a MD5 checksum generator on the files at both locations.  I expect that the checksums will differ anyway since PM will add a small bit of information for its use onto the end of your RAW files as they are ingested (we are making this optional in version 4.6) so this would cause the checksums to differ.

Currently the only way to get an exact backup of your files to DVD and verify that they're the same as the originals on the CF card would be to not use Photo Mechanic for this purpose at all.  You would need to copy the files using your OS, and burn the disks with another application, and then follow up by generating checksums for each file at both locations.  If the checksums match then your files should be exactly the same.

-Kirk

[Oscarsson]

Thank you for responding so fast Kirk.

Your answer gives me a couple of more questions.

Can you tell me how time consuming this checksum operation is, approximately anyway. Let's say on a Lexar 4 GB CF UDMA card.

And also according to your experience, how big the risk is, that the files get corrupted during ingesting, and if there are ways to minimize that risk further.

If PM can read the JPEG inside the RAW file is it then plausible that the rest of the RAW file would ever be corrupt.
Of course shit happens, but would'nt it be more plausible that the whole file is bad than just the RAW data and not the JPEG data inside?

Oh, and how about when I shoot just JPEG, can I then be sure my DVDs are OK if PM can generate thumbnails from the JPEG files?

I'm running Mac OS 10.4.11 on a powermac G4 867 dual processor with 1.25 GB RAM.
I use a Sandisk Firewire cardreader and Lacie and WD My Book external Firewire hard drives to accompany the two 50GB internal HDs.
I have CFs from Sandisk and Lexar in sizes from 2GB to 4GB, both UDMA cards and the earlier generations down to Sandisk Extreme II cards.
I never use the start up disk for anything else than the OS and other software.

Regards/

Jan

[Kirk Baker]

Jan,

Can you tell me how time consuming this checksum operation is, approximately anyway. Let's say on a Lexar 4 GB CF UDMA card.

Every byte of every file would have to be read from the card.  Checksumming the data will take very little time itself since reading the data in the first place will be the limiting factor.  It should take between a minute and several minutes depending on how fast you can read data from your card.  Current maximum is about 80 MB/second with the right hardware.

And also according to your experience, how big the risk is, that the files get corrupted during ingesting, and if there are ways to minimize that risk further.

The risk is minimal.  If PM were to corrupt data that was perfectly fine on the card then we would soon be out of business.  The only thing that would cause corruption would either be serious bugs in our software, bugs in the OS, or bad hardware (bad RAM or memory controller).

If PM can read the JPEG inside the RAW file is it then plausible that the rest of the RAW file would ever be corrupt.
Of course shit happens, but would'nt it be more plausible that the whole file is bad than just the RAW data and not the JPEG data inside?

Oh, and how about when I shoot just JPEG, can I then be sure my DVDs are OK if PM can generate thumbnails from the JPEG files?

I'm running Mac OS 10.4.11 on a powermac G4 867 dual processor with 1.25 GB RAM.
I use a Sandisk Firewire cardreader and Lacie and WD My Book external Firewire hard drives to accompany the two 50GB internal HDs.
I have CFs from Sandisk and Lexar in sizes from 2GB to 4GB, both UDMA cards and the earlier generations down to Sandisk Extreme II cards.
I never use the start up disk for anything else than the OS and other software.

Since the data on the card itself can be corrupt (bad sectors, power failure in camera when writing data, etc.) then the best way to prove that the data is solid is to render the RAW file.  But even if it were to fail, there is nothing you can do when the original is bad.  Being able to render the JPEG data contained within the RAW file says very little about the state of the RAW data itself.  It is quite likely that if the embedded JPEG preview can be rendered that the rest of the data is also OK, but there is no guarantee.

I know you're looking for some sort of reassurance, but it isn't a simple problem to solve.  The good thing is that corruption like what you're worrying about is very rare.

-Kirk

[Oscarsson]

Thank you very much for a helpful and thorough answer Kirk!

I think you're really good at support and customer relations at PM.

Keep up the good work.

Cheers/

Jan