Any examples of a workflow that includes converting raw files to dng?

[Skids]

For various reasons I have decided to convert my raw files to the dng format.  I am now trying to decide on a reasonable method of converting raw files on a camera card before they are added to my collection of images and  I wonder if anyone has any helpful suggestions.

[alexmacnaughton]

Personally, I would advise against converting to DNG. Why are you thinking of doing it?

[Skids]

Personally, I would advise against converting to DNG. Why are you thinking of doing it?

Well with the demise of Photo Mechanic Plus on Mac I was left looking for yet another database application to use to manage my image collection.  I started out using iView Media Pro then Microsoft basically broke it, next it was Apple Aperture  which Apple killed off, then Lightroom which adopted a subscription payment system which I decided to reject, then Capture One which I think used the remains of iView MediaPro but was unusable when the collection grew larger than 25,000 images and lastly Photo Mechanic Plus which was great while it lasted.  I have tried to use Neo Finder but it has some serious flaws when writing metadata. So I decided that converting to dng will mean that spotlight and other standard file management tools can be used to find images and once found I can use Photo Mechanic to write metadata.

The down side of converting to a dng based workflow is the time taken to convert the files plus the time to write an updated file to backup devices.  The upside is that the metadata is written into the file where it stays.

I conducted a number of tests comparing the edits of both raw and dng copies and found no difference between the files.

So why do you advise against converting to dng?

[alexmacnaughton]

Hi, thanks for the extra information.

Yes, I appreciate what you are saying.

You seem to know that one of the big issues with DNGs is backing them up. It is not so much of an issue when backed up to a local HD, but it is quite painful with cloud storage.

Personally, I use LR. Yes, subscriptions are a pain.

You can continue using the LR library module after cancelling your subscription.

https://community.adobe.com/questions-675/losing-access-to-lightroom-classic-after-cancelling-subscription-962114

[Skids]

Yes, I appreciate what you are saying.

Thank you, :-)

I looked at several options including writing the meta data directly to the raw files using Exiftool and or Photo Mechanic but finally decided that using dng was the better option for me.

Recently, the hard drive that stores my images library failed and then I discovered that while restoring from my first backup drive that it was also "failing" and had a number of bad sectors.  Fortunately, I have a second backup drive and restored my images from that.  However, I shall be purchasing a quality raid from Lacie to take over back-up duties in the very near future.  The restoration process took over six hours reading from a USB 3 drive and writing to a SSD, I would imagine that restoring from cloud storage would take days.

I mention the drive failures because it highlighted the issue of corrupt files.  These are difficult to detect and it seems that the only option is to use rsync to compare one set of files with a second.  One further advantage of dng is that I believe that it writes a checksum into each file thus allowing file integrity to be checked.

Before making my decision to convert to dng I read a lot of information published on the web and even found a forum where the author of Adobe DNG Converter occasionally posts.  One further factor is that dng has now been adopted as an I.S.O standard : ISO 12234-4:2026.  I also read an awful lot of FUD.

Fear, uncertainty, and doubt (FUD) is a manipulative propaganda tactic used in technology sales, marketing, public relations, politics, polling, and cults. FUD is generally a strategy to influence perception by disseminating negative and dubious or false information and is a manifestation of the appeal to fear.

One negative I have discovered is that the most recent versions of Adobe DNG converter default to also converting jpeg files and there is no way of stopping this feature or bug.  My work around has been to create an application that lists raw files and passes the list to the shell version of the converter in batches of 500.

In summary I am happy to convert my images to dng as I can see that dng's offer me a number of advantages over typical raw/xmp pairs.  I fully accept that others are free to disagree.

So does anyone use a workflow that includes converting to dng ?

[RobertE]

The absolute easiest way to convert raw files to DNG is with Lightroom Classic. It lets you choose to only your convert raw files, therefore exclude JPEGs. You can create the MD5 hash for data verification.

Caveats:
1. You may have to subscribe to convert to DNG in Lightroom Classic.
2. As Alex pointed out, most of the Library module in Lightroom remains usable after your after your subscription expires, however...
3. I don't know if the 'Convert to DNG' remains available after your subscription ends.

Usually program you choose to use to create the MD5 hash is also required to run data integrity health checks. If you use Lightroom currently you need to check using an Adobe program. There is nothing stopping anyone creating a tool to check the embedded DNG MD5 hash. It looks like you've got some good experience creating apps. All this is part of the ISO spec.

Have you found a replacement for Photo Mechanic Plus for cataloguing? NeoFinder has been recommended here already, and I'd agree. If like many of us you're an old iView refugee, NeoFinder offers a very close alternative. In addition NeoFinder also allows you to create a MD5 checksum. Unlike the Adobe implementation, if you change the file in any way including metadata it triggers a warning. Adobe only hash the raw data, so changing metadata doesn't trigger a "false" positive. Finally NeoFinder offers a discount if you have a valid iView (and Photo Mechanic) license - but I had to supply my iView MediaPro license number.

I may be biased when it comes to DNG, so take the suggestions above with a grain of salt.

[Skids]

Have you found a replacement for Photo Mechanic Plus for cataloguing? NeoFinder has been recommended here already, and I'd agree. If like many of us you're an old iView refugee, NeoFinder offers a very close alternative. In addition NeoFinder also allows you to create a MD5 checksum. Unlike the Adobe implementation, if you change the file in any way including metadata it triggers a warning. Adobe only hash the raw data, so changing metadata doesn't trigger a "false" positive. Finally NeoFinder offers a discount if you have a valid iView (and Photo Mechanic) license - but I had to supply my iView MediaPro license number.

I used NeoFinder for a while, then I discovered an issue when writing metadata.  If your image collection includes both a raw and a jpeg with the same name (just different extensions) and you write metadata to them both then there is a risk that NeoFinder will add an Adobe attribute to the XMP data in the XMP sidecar file that points the XMP sidecar to the JPEG file.  I reported this but so far have not received any information that the issue has been resolved.  I ended up having to use BBEdit to search for the attribute and change the .jpg of the referenced file name back to the raw file extension.  This was doable because I only had .rw2 raw files in the folders that NeoFinder had worked on.

<rant>
This issue is a further reason why I decided to convert my raw files to dng format; I hope/pray/assume no software will add this attribute to the XMP data stored in Tag 700, Application Notes, inside a dng file and that if an app did any software reading such an attribute would ignore it.  As it stands I believe Carl Siebert ran some tests and reported the results in this forum, these results differed depending on the application processing the .xmp sidecar, with some Adobe products acting on the attribute while other applications mostly ignored it.  This is the problem with extensibility coupled with a specification that allows apps to create there own vocabularies.  Apps can and do add there own "guff" to XMP meaning that some files read like War and Peace.  Obviously I'm exaggerating, but don't forget that Adobe  added extended XMP to the spec back in 2010 to get around the 64K limit of data blocks inside jpeg files.  They did this so they could store even more processing information. </rant>

Simon

[elianoimperato]

Well with the demise of Photo Mechanic Plus on Mac

Hi,
which kind of issues do you have with Photo Mechanic Plus? Because I'm continuing to use it (currently on macOS Golden Gate 27) and I have no problems.

Best,
eliano