Photo Mechanic + Lightroom - What happens to all the information entered into PM

[mikemyers]

After many, many years of wondering about it, I got to watching lots of YouTube videos about PM, then was pleasantly surprised to find the Black Friday discount.  I placed the order.

I have one issue that I can't find an answer for.  After ingesting photos, and filling out the folder names, file names, and all the other information, my goal is to import the selected photos into my Lightroom folders, where they will remain.  The temporary folders I created for PM in order to do the work, will be deleted. 

So, what happens to all the information I typed into the various forms, keywords, and so on, after import into Lightroom?  Was it all wasted effort?

I certainly don't intend to change the way I deal with photos after they are imported and edited in Lightroom.

I certainly feel PM is a very useful tool for getting my photos sorted out, renamed, organized, and so on, but will all the information I typed into PM be lost, once the photos reside in my Lightroom folder structure?

[mikemyers]

Having posted this question, I suppose I could always ingest the photos into a folder that is INSIDE my Lightroom folder structure.  Then, when everything was done, I could use the ADD tool for Lightroom importing.  Presumably, I would then have all the data that PM collected, and two sets of 'keywords', one from PM and another from LR.  Even the IPTC Stationary Pad might be there, somewhere (not sure where it is kept).

[Kirk Baker]

Mike,

You just need to import the folders containing the images in Lightroom.  All of the metadata you entered in PM will be picked up by Lightroom, including crops.  The only metadata that doesn't get represented in Lightroom is Photo Mechanic's 'Tag' checkbox.

-Kirk

[mikemyers]

So far, very good!  That's what I hoped for, but someone in these forums was saying the PM data got over-written by Lightroom, or vice versa....

So, please confirm.  When I ingest a photo into PM, fill out all the labels and data I want, including the stationery, then import the image into LR where it will remain, someplace within the LR folders, the PM keywords will remain with the photo.  If I then add yet more keywords, using LR, they will be added to the PM keywords.  So if I later search for an image in LR using a keyword, LR will find any keywords that have been entered with either program.  That's what I am hoping for. 

Follow-up - does the "stationery" data stay either inside the image, or attached to the image?  If I shot in RAW + JPG, and later delete all the JPG files from LR (not really needed), does all this data remain somehow?

.......and for that matter, can I tell PM when it's ingesting images, to ingest only DNG (raw) images, unless there is a JPG image without a corresponding DNG image?  (My Fuji X100f shoots both, but if I want to use their "digital" focal length setting, or some other settings, it only shoots in JPG.  I usually shoot DNG+JPG, but if it's images I don't care much about, only JPG.  .....and sometimes a mix.


I guess what I'm getting at, is I'd like to use PM as a tool to get images into LR, along with all the bonus features from PM, but at the end of the day, I don't plan to keep the folders that I used for ingesting the images; they will be deleted.

[Kirk Baker]

Mike,

So far, very good!  That's what I hoped for, but someone in these forums was saying the PM data got over-written by Lightroom, or vice versa....

So, please confirm.  When I ingest a photo into PM, fill out all the labels and data I want, including the stationery, then import the image into LR where it will remain, someplace within the LR folders, the PM keywords will remain with the photo.

That is correct.

If I then add yet more keywords, using LR, they will be added to the PM keywords.

You need to force Lr to write its changes out to the files.  Otherwise the changes are only in its database.  You can do this from the Metadata menu manually, but I believe there is a setting to force it to always update the files somewhere in the preferences in Lr.

So if I later search for an image in LR using a keyword, LR will find any keywords that have been entered with either program.  That's what I am hoping for.

If you make changes with PM to an image that has already been imported into Lr, then you need to make Lr read the updated metadata.  But yes, once that is done then Lr will find images that have been edited both by PM and by Lr.

Follow-up - does the "stationery" data stay either inside the image, or attached to the image?  If I shot in RAW + JPG, and later delete all the JPG files from LR (not really needed), does all this data remain somehow?

Yes.  The metadata for RAW files will in general be written to XMP sidecar files and Lr can read the changes when you direct it to do so.

.......and for that matter, can I tell PM when it's ingesting images, to ingest only DNG (raw) images, unless there is a JPG image without a corresponding DNG image?

No.

I guess what I'm getting at, is I'd like to use PM as a tool to get images into LR, along with all the bonus features from PM, but at the end of the day, I don't plan to keep the folders that I used for ingesting the images; they will be deleted.

That is a very common workflow and PM supports Lr quite well.

-Kirk

[mikemyers]

Thank you for reviewing all that.  I think I understand completely.

I guess I need to ingest *everything* on my cards, and if I remember, shoot in either DNG or JPG, but not both.  I wish I had purchased PM long ago, especially back when I was "working" at these things. 


It's probably asking too much, but does PM also work with exporting images into "Raw Therapee", in addition to being able to export into Lightroom?   I think I'm going to keep those images in a separate location.  If I'm shooting in DNG, and especially if I'm taking infrared images, Lightroom doesn't have the same capability as Raw Therapee.  I need to think this through, but I'm trying to ONLY import (ingest) photos into PM, to take advantage of those tools before doing any editing. 

[Kirk Baker]

Mike,

Thank you for reviewing all that.  I think I understand completely.

I guess I need to ingest *everything* on my cards, and if I remember, shoot in either DNG or JPG, but not both.  I wish I had purchased PM long ago, especially back when I was "working" at these things. 

There is nothing inherently wrong with shooting RAW+JPEG.

It's probably asking too much, but does PM also work with exporting images into "Raw Therapee", in addition to being able to export into Lightroom?   I think I'm going to keep those images in a separate location.  If I'm shooting in DNG, and especially if I'm taking infrared images, Lightroom doesn't have the same capability as Raw Therapee.  I need to think this through, but I'm trying to ONLY import (ingest) photos into PM, to take advantage of those tools before doing any editing.

PM is a browser and does not need to import anything, nor does it control your images such that they need to be exported to another application.  So yes, PM can work with "Raw Therapee" and just about any other image editing program that you can think of.

Ingest is used to copy images from camera cards and is simply a "downloader", but you can browse the cards directly with PM if you want instead (we don't recommend this because unless you've copied your images from the card, then those images are the only copy of those images that you have and data loss is a distinct possibility.

HTH,

-Kirk

[mikemyers]

Following what you wrote, for these infrared photos, I will use PM to ingest the raw images from the card, saving them in a properly named folder (with PM), and rename the DNG images using PM.  From filling out all the information, I will have a "sidecar" file for these images, with the added information.  All of this will be more easily findable thanks to the PM naming tools.

The images will be filed away in a logical order in an area separate from my Lightroom images.  I understand now what  you mean - at that point, I'm "done".  Then, I get to edit them in Raw Therapee, which does not modify the original DNG image, but creates another "sidecar" file for each image I have worked on, containing the editing commands.  Presumably when this is done, I will have the original DNG image, the sidecar info file from PM, the sidecar editing file from RT, and most likely an edited and saved JPG image, the result of my RT editing (possibly several, if I re edit several times until I am satisfied).

It's all starting to fall into place.  Thanks again.
This started out as SO confusing, but the various YouTube videos were a huge help to get me started, and this forum seems to be a goldmine of good information and answers.

[Kirk Baker]

Mike,

Following what you wrote, for these infrared photos, I will use PM to ingest the raw images from the card, saving them in a properly named folder (with PM), and rename the DNG images using PM.  From filling out all the information, I will have a "sidecar" file for these images, with the added information.

DNG files won't have XMP sidecar files.  Metadata is written directly into DNG files by PM and Adobe's products.

The images will be filed away in a logical order in an area separate from my Lightroom images.  I understand now what  you mean - at that point, I'm "done".  Then, I get to edit them in Raw Therapee, which does not modify the original DNG image, but creates another "sidecar" file for each image I have worked on, containing the editing commands.  Presumably when this is done, I will have the original DNG image, the sidecar info file from PM, the sidecar editing file from RT, and most likely an edited and saved JPG image, the result of my RT editing (possibly several, if I re edit several times until I am satisfied).

Sounds correct with the exception of a sidecar file created by PM--it won't do that with DNG files.  Rest assured that your keywords and other metadata changes will be seen by Lightroom when you import those images into it.

-Kirk

[Bob M]

It's probably asking too much, but does PM also work with exporting images into "Raw Therapee", in addition to being able to export into Lightroom?   I think I'm going to keep those images in a separate location.  If I'm shooting in DNG, and especially if I'm taking infrared images, Lightroom doesn't have the same capability as Raw Therapee.  I need to think this through, but I'm trying to ONLY import (ingest) photos into PM, to take advantage of those tools before doing any editing.

I use PM and Raw Therapee together.  The main issue that I run into is that PM doesn't understand .pp3 files.  Consequently these need to be handled manually for copy/move, ftp, delete, etc.  But I only use Raw Therapee to develop the final jpg.  These go into a sub folder.  I then use PM to remove all metadata from those files and then add back the metadata that I want.  Raw Therapee does not understand or use .xmp files (I believe) and thus I wouldn't trust metadata to faithfully survive the journey from PM to Raw Therapee to the resultant jpeg (or tiff) although I haven't explicitly tested this.  And adding GIMP into the mix would complicate things even further.  But of course, if Raw Therapee carries through and changes from .pp3 files to .xmp files, then it's a whole new ball game.  Then I guess the issues changes as to whether Raw Therapee and PM play nice with each others .xmp files.

I very much like both PM and Raw Therapee and I have found ways to work around the incompatibilities.  I just wish PM was Linux friendly.

[Kirk Baker]

Bob,

It's probably asking too much, but does PM also work with exporting images into "Raw Therapee", in addition to being able to export into Lightroom?   I think I'm going to keep those images in a separate location.  If I'm shooting in DNG, and especially if I'm taking infrared images, Lightroom doesn't have the same capability as Raw Therapee.  I need to think this through, but I'm trying to ONLY import (ingest) photos into PM, to take advantage of those tools before doing any editing.

I use PM and Raw Therapee together.  The main issue that I run into is that PM doesn't understand .pp3 files.  Consequently these need to be handled manually for copy/move, ftp, delete, etc.  But I only use Raw Therapee to develop the final jpg.  These go into a sub folder.  I then use PM to remove all metadata from those files and then add back the metadata that I want.  Raw Therapee does not understand or use .xmp files (I believe) and thus I wouldn't trust metadata to faithfully survive the journey from PM to Raw Therapee to the resultant jpeg (or tiff) although I haven't explicitly tested this.  And adding GIMP into the mix would complicate things even further.  But of course, if Raw Therapee carries through and changes from .pp3 files to .xmp files, then it's a whole new ball game.  Then I guess the issues changes as to whether Raw Therapee and PM play nice with each others .xmp files.

I very much like both PM and Raw Therapee and I have found ways to work around the incompatibilities.  I just wish PM was Linux friendly.

Is the .pp3 file just a settings file that should be associated with the RAW file, or is it a custom image format?

-Kirk

[Bob M]

It contains all of the edit instructions that Raw Therapee applies to the raw file to produce the final output.  They're text files, but not xml files.

But then it gets complicated and I don't understand all of the nuances.  Here is an example of developing a single raw file with the jpeg going into a sub directory.  The attached zip file contains the following 

The zip was too large but here is the file structure:

/20181023-0002.orf -- original raw file
/20181023-0002.orf.pp3 -- my edits to the raw file
/20181023-0002.xmp -- my meta data
/jpegs/20181023-0002.jpg -- my resultant jpeg
/jpegs/20181023-0002.jpg.out.pp3 -- the edits that produced this particular jpeg
                                                       (In this simple case it should be identical to /20181023-0002.orf.pp3)

I have attached 20181023-0002.orf.pp3

Personally, I mainly care about the upper level .pp3 file and it would be nice if PM recognized its association with 20181023-0002.orf and 20181023-0002.xmp.  But I am aware that PM doesn't and in general I can work around it.  Basically, I don't start to use Raw Therapee until I have finished using PM to add metadata and archive the original raw file.

[Kirk Baker]

Bob,

It contains all of the edit instructions that Raw Therapee applies to the raw file to produce the final output.  They're text files, but not xml files.

But then it gets complicated and I don't understand all of the nuances.  Here is an example of developing a single raw file with the jpeg going into a sub directory.  The attached zip file contains the following 

The zip was too large but here is the file structure:

/20181023-0002.orf -- original raw file
/20181023-0002.orf.pp3 -- my edits to the raw file
/20181023-0002.xmp -- my meta data
/jpegs/20181023-0002.jpg -- my resultant jpeg
/jpegs/20181023-0002.jpg.out.pp3 -- the edits that produced this particular jpeg
                                                       (In this simple case it should be identical to /20181023-0002.orf.pp3)

I have attached 20181023-0002.orf.pp3

Personally, I mainly care about the upper level .pp3 file and it would be nice if PM recognized its association with 20181023-0002.orf and 20181023-0002.xmp.  But I am aware that PM doesn't and in general I can work around it.  Basically, I don't start to use Raw Therapee until I have finished using PM to add metadata and archive the original raw file.

It's actually quite trivial to add another file type that should be associated with an image in PM.  We never had (to our knowledge) anyone request that PM handle .pp3 files.  I'd be happy to add them for RAW files, but they're not a good match for JPEGs since JPEGs are perfectly capable of having embedded metadata and all reasonable JPEG interpreters will both preserve unknown embedded metadata and not be thrown off by its presence.  If that pp3 data is important to the JPEG then it should be embedded instead of lazily written as a sidecar.

I'll make sure that PM6 treats .pp3 files as an associated file type for RAW (non-DNG) files.

-Kirk

[Bob M]

That would be great if PM6 associated the .pp3 file with the raw file.  Thank you.

But just to be clear, the name of the .pp3 file associated with the raw file {filename} is {filename}.pp3 and not {filenamebase}.pp3

[Kirk Baker]

Bob,

That would be great if PM6 associated the .pp3 file with the raw file.  Thank you.

But just to be clear, the name of the .pp3 file associated with the raw file {filename} is {filename}.pp3 and not {filenamebase}.pp3

Yes, I understand.  Bibble did that too so we can handle that case easily.

-Kirk