Can Photo Mechanic metadata sides be read/updated by Adobe Camera RAW (ACR)?

[Hayo Baan]

Hi Garry,

I don't think you need to be too concerned about all the possible things that the DNG standard allows and doesn't allow; when converting to DNG, there isn't a whole host of settings that can be changed. I have attached my own settings for your perusal.

Of my settings the only thing I'm not too certain about is the “embed fast load data”, this may be something that is not supported by every other software manufacturer. But then, it doesn't seem to add too much to the file size so I'm leaving it turned on.

You may also notice that I don't have embed original turned on. This is because first of all I don't see the need for this; once I have converted to DNG, I don't see the need of ever going back, and second of all, I already have an original from camera as part of my backup strategy

Cheers,
Hayo

[Gary_G]

Hayo;

When do you do your geotagging, before or after conversion to DNG? Since our workflows are similar, I thought I'd ask.

Do you see any issues with renaming and storing the original RAW file on a backup drive and then immediately running a copy through the RAW->DNG converter before doing anything else? Seems to me that most of the software I use (PM, ACR, GeoSetter and RawTherapee) seems happy to process DNG files.

By the way; I'm seriously considering whether to follow your lead and not embed the untouched RAW files. You're correct about already having a backup.  It should suffice, provided I can be sure that all my added info is present in the current generation of the DNG. My main goal in DNG was to avoid the metadata and image from getting separated. So that is still met without an embedded RAW file.

(A bit off-topic, but important. How do you modify your workflow to handle the inevitable JPEGS from a camera and what format do you use for storing the final processed images for both? To date, I've been treating JPEGs as if they were equivalent to DNGs. All my processed outputs (even from JPEGs) are currently stored as 16-bit TIF files until someone wants them in a specific format. For RAW , the reason is obvious. For JPEGs, I didn't think it wise to automatically re-compress an already lossy  image after adjusting it.)

[Hayo Baan]

When do you do your geotagging, before or after conversion to DNG? Since our workflows are similar, I thought I'd ask.

To prevent issues where the geotagging screws up the raw file (especially the camera specific makernotes are important) and confuses the DNG converter, I would do it after converting to DNG. I had this happen to some of my NEFs (where in this case I suspect it was a specific version of Nikon's own photo editor to cause the issues). That said, if your geosetter is well written, it shouldn't actually matter which way you do it

Do you see any issues with renaming and storing the original RAW file on a backup drive and then immediately running a copy through the RAW->DNG converter before doing anything else? Seems to me that most of the software I use (PM, ACR, GeoSetter and RawTherapee) seems happy to process DNG files.

This would actually be my suggestion. In my case, with new files, the only program to have touched the files would be PM which renames and sets some generic IPTC/XMP metadata during ingest. I actually have PM make the copy for me during ingest, so after ingestion I have the same image at two different locations. One I will convert to DNG, the other I will leave alone.

By the way; I'm seriously considering whether to follow your lead and not embed the untouched RAW files. You're correct about already having a backup.  It should suffice, provided I can be sure that all my added info is present in the current generation of the DNG. My main goal in DNG was to avoid the metadata and image from getting separated. So that is still met without an embedded RAW file.

If you already have the backup “clean” raw file, there is indeed no need for embedding at all.

(A bit off-topic, but important. How do you modify your workflow to handle the inevitable JPEGS from a camera and what format do you use for storing the final processed images for both? To date, I've been treating JPEGs as if they were equivalent to DNGs. All my processed outputs (even from JPEGs) are currently stored as 16-bit TIF files until someone wants them in a specific format. For RAW , the reason is obvious. For JPEGs, I didn't think it wise to automatically re-compress an already lossy  image after adjusting it.)

Convert your JPG files to DNG too. The DNG convertor from Adobe doesn't let you do this automatically (it only works on raw files), but you can do this from within ACR easily too (also for multiple images at the same time). The trick is to open your JPGs in ACR and not in straight photoshop. From within ACR you then click on the Save Image… button and choose DNG.
From then on you can treat your JPG just like any other raw file in your workflow

To open a jpg in ACR there are a couple of ways:

• Set the Camera Raw preference to always load JPGs in ACR
• Select Camera Raw as the format in the File Open dialog of Photoshop
• Select open in Camera Raw from within Bridge

Hope this helps,
Hayo

[Gary_G]

Thanks, Hayo;

I appreciate the obvious time you took to reply. You've been an immense help. I've just three questions, then I'm going to start getting this workflow set up and running.

By the way; I should note that I did "stumble upon" an article that talked about using ACR to convert JPGs to DNG. It noted that one should use the "linear" option. I tried it and things looked quite reasonable, in fact it seemed to give me more leeway in adjustments.
QUESTION: Is the "linear" option (rather than compressed") what you use?

So; the 10,000 foot view of the workflow is to use PM to ingest the un-geocoded RAW and JPGs (sending a copy to the archive drive), use Adobe RAW Converter (or ACR) to turn RAW files into DNGs and use ACR to turn JPGs into DNGs, geocode the DNGs, then process both with ACR and finally export as 16-bit TIF. From there one can put out any format needed for the web, prints etc.
QUESTION: Is this a fair description of what you were thinking?

I believe that we did discuss processing legacy image files before, but the result may change based on the current discussion about DNGs. I have a script that uses ExifTool to set the capture date/time to the date taken date/time, remove any XMP, IPTC and GPS segments, and remove Artist and TimeZone. The net effect is to return an annotated Canon RAW or JPG to it's original State. I've done several tests and it seems to work well. However; I've also done tests on the RAW to DNG Converter and ACR. They don't seem to care about the precise structure of the RAW and/or JPG data input. They seem to copy the actual image into their own structure and mine the input file for the rest of the data to fill out their own tags. In effect, you get the original image and preview encapsulated in a new metadata framework. (I did notice that they may need to convert the GPS coordinates into their own format and this can result in an infinitesimal rounding in the hundredth of a second. Not something to worry about). So; it would seem that stripping the input files and then re-introducing the metadata would be a waste of time and could be more of a risk than just converting the legacy files "as is".
QUESTION: What do you think about just saving a copy, then ingesting them as they are?

[Hayo Baan]

Thanks, Hayo;

I appreciate the obvious time you took to reply. You've been an immense help. I've just three questions, then I'm going to start getting this workflow set up and running.

You're welcome!

By the way; I should note that I did "stumble upon" an article that talked about using ACR to convert JPGs to DNG. It noted that one should use the "linear" option. I tried it and things looked quite reasonable, in fact it seemed to give me more leeway in adjustments.
QUESTION: Is the "linear" option (rather than compressed") what you use?

In the DNG converter there is no option to choose linear, so I don't understand where you see this. There is only an option to use lossy (!) compression, which (of course), I do not use.

So; the 10,000 foot view of the workflow is to use PM to ingest the un-geocoded RAW and JPGs (sending a copy to the archive drive), use Adobe RAW Converter (or ACR) to turn RAW files into DNGs and use ACR to turn JPGs into DNGs, geocode the DNGs, then process both with ACR and finally export as 16-bit TIF. From there one can put out any format needed for the web, prints etc.
QUESTION: Is this a fair description of what you were thinking?

Yes, apart from the fact that I do not bother to create a TIFF for every image; I only need those when I want to edit in Photoshop. For everything else the embedded JPG suffices (just make sure you make them full sized).

I believe that we did discuss processing legacy image files before, but the result may change based on the current discussion about DNGs. I have a script that uses ExifTool to set the capture date/time to the date taken date/time, remove any XMP, IPTC and GPS segments, and remove Artist and TimeZone. The net effect is to return an annotated Canon RAW or JPG to it's original State. I've done several tests and it seems to work well. However; I've also done tests on the RAW to DNG Converter and ACR. They don't seem to care about the precise structure of the RAW and/or JPG data input. They seem to copy the actual image into their own structure and mine the input file for the rest of the data to fill out their own tags. In effect, you get the original image and preview encapsulated in a new metadata framework. (I did notice that they may need to convert the GPS coordinates into their own format and this can result in an infinitesimal rounding in the hundredth of a second. Not something to worry about). So; it would seem that stripping the input files and then re-introducing the metadata would be a waste of time and could be more of a risk than just converting the legacy files "as is".
QUESTION: What do you think about just saving a copy, then ingesting them as they are?

Unless the makernotes or other metadata got screwed by other image editing programs, there is no need to “clean” the image before converting to DNG; the DNG converter will happily take over any metadata that it understands and will put it in its own location (this is in fact what happens in my workflow; during ingest PM already fills in a lot of metadata which is then taken over when converting to DNG).

Note: one advantage of running exifTool over your images prior to converting is that it can check for misbehaving files (run it with -warning turned on to get all warnings), you can then have exifTool fix those (if necessary). But this should rarely be the case if you used good tools to edit your files.

Cheers,
Hayo

[Gary_G]

Thanks, Hayo. I think I'm ready to go. Wish me luck!

I went back to check and can't see a linear vs. compressed option. Can't understand why I thought there was one. My mistake.

You make a good point in the answer to the second question. The embedded large JPG should be good enough for many purposes. I'll probably only need TIFs, when I need to pass on a print-ready electronic file. My own prints can be done without an intermediate conversion. Less work is better.

Thanks for reminding me to leverage what Photo Mechanic can do for my workflow. Based on your response to the last question, I think I'll also do much of my metadata entry during ingestion by Photo Mechanic. Unfortunately, PM still doesn't have any location lookup capability in the geo-coding section, so I'll probably pop out to GeoSetter after ingestion to fill in the associated location fields. The invariant/predictable info, I'll do during ingestion and the more shot-specific info, I'll likely do after the initial culling of garbage shots and prior to ranking. That way, I'll more quickly narrow the focus towards working on the photos that count in the shoot.

[FVlcek]

Hi, there IS an option to "create Linear DNG". You shouldn't select this ever. This is basically a developed RAW file, already stripped of the RAW info, for use in software that can't edit raw DNGs. What it does is create a 24(or 32?)bit DNG where the RAW data from the sensor (R, G and B pixels) are debayered (converted from bayer RGB pattern the chip uses) to linear RGB data (like TIFF). What you gain is some speed and maybe compatibility, but you loose on any future advances in debayer technology (bayer pattern - most cameras except Foveon have alternating red, green and blue pixels on the chip, every pixel gets either red, green or blue light - from this, by complex computations, a final image where every pixel has all three colour values is computed)

[Gary_G]

Thank you, Frantisek. I thought I was going crazy!
I'll make a note to NOT select that option.

[Hayo Baan]

Ah, right, linear is an option when you save as DNG in ACR. It is not available in the DNG converter. Anyway, as you rightly mention: do not enable it ever as you will lose the ability to make use of improvements in the de-bayer algorithm of the raw converter.

Thanks for adding to this thread!