Author Topic: New user - initial experiences  (Read 6837 times)

Offline StephenG

  • Newcomer
  • *
  • Posts: 4
    • View Profile
New user - initial experiences
« on: September 29, 2021, 07:33:26 AM »
Hi, I'm a new user of PM+, still in my trial, although I think it is highly likely that I will pull the trigger and purchase the software (I've purchased plenty other software in my time and PM seems to cross the bar of value for money for me at the moment).

I'm an amateur photographer with digital photographs from the last twenty years or so and scans from many years prior to that.  Over the years my digital images have been in (and out of) iPhoto, Aperture, Lightroom, Capture One, and PhotoSupreme.  I had high hopes of PS solving my DAM problems but the database seemed to get in a bit of a mess and the import process had to be done in small pieces and was taking for ever - I've already got more images in PM+  than I ever got managed sensibly in PhotoSupreme. 

Because of my history I find that while most of my images are well organised in folders based on dates (and this was the form that was referenced for Aperture / Lightroom etc) because images have been transferred to online services and then re-downloaded or exported to albums, there are also an additional mass of duplicates in various places - some with corrupted EXIF/IPTC information which makes them difficult to identify - which I was hoping to get sorted with a good DAM.  So far by using a combination of EXIF data where it exists, and other clues I'm slowly able  to find duplicates and start to rationalise but it will be a long haul.  Already, I've found images I didn't know I had lurking in my main photo directories that I thought were well organised, so that is a good sign.  A facility to find duplicates / similar images would be a great boon, but one still needs to work out which is the original, the processed ones and those that should be deleted and then move those that are in the wrong places to the correct directories.

My main tools these days are DXO PhotoLab for processing raw images together with Affinity Photo for doing more sophisticated work.  I have used On1 a lot in the past (and am still in their subscription program) but use it less and less as I don't much care for the raw processing and while it used to be a fast browser, it seems a lot slower these days (even on faster hardware).  When I first used it, it was primarily to use On1 Layers as an inexpensive way to get into layers but these days all of that functionality is in Affinity Photo and PM+ would seem much better for the DAM aspects.  While I used Lightroom for a while (because it was necessary for a Open University photography course I was doing), I ceased to use it after Lightroom 6 and the move to subscriptions.  I don't want to have photographs that depend on having software to see them correctly and therefore generating "finished" JPGs seems to be the safe way to go (even if DXO or other software is still keeping the "recipe" in a sidecar file alongside the raw file). 

As I also run websites for a few small local groups, I also tend to acquire images that are used on those websites that need to be managed and organised, typically in their own folders, and possibly in their own catalogues.  The main challenge is having multiple copies of images because of the need for scaling for website use. 

My ultimate aim would be to get my images well organised both for my own benefit and also so that when the inevitable day arrives when I'm not around that my photo hard drives are organised in a way with clearly defined folders of "finished" JPGs so that my partner or children can take what they want. 

I'd love to hear any tips or tricks from others that have come to PM+ with similar needs or experiences.

Otherwise, all I can say at the moment after being a couple of weeks into my trial (and I have lots of other things to do - so I'm not sitting at PM+ all day) is that is seems to work well, fast and looks like it may help me sort out my mess of photographs. 

Offline Mick O (Camera Bits)

  • Camera Bits Staff
  • Hero Member
  • *****
  • Posts: 556
    • View Profile
    • Camera Bits
Re: New user - initial experiences
« Reply #1 on: September 29, 2021, 10:53:50 AM »
Welcome! I'm glad you're finding some things in Photo Mechanic useful.

One thing that helps me from keeping so many different versions of files is knowing that PM can generate resized jpgs on the fly when transferring files (like FTP).  I will keep the source raw file (or often a PSD with the raw file as a smart object) and then if I need a 600px jpg version, I will just use PM to generate that -- either on the fly while I am FTPing to my webserver, or just save it from PM and then delete it right after. (and if you have certain sizes you always need, you can create a snapshot preset for those sizes in the "Save as" or "Upload" dialogs) Keeping multiple size versions is something i rarely have to do now and it has really made my life easier.  (Side note: Knowing that I can do that from PSDs even if I don't have Photoshop is also good for peace of mind.)

Hope that possibly helps

Mick
Mick O
Camera Bits

Offline StephenG

  • Newcomer
  • *
  • Posts: 4
    • View Profile
Re: New user - initial experiences
« Reply #2 on: September 30, 2021, 01:10:37 AM »
Mick,

That's a good idea.  I already use some applets to resize images for various purposes with presets so they are done consistently.  By using snapshots within PM I can achieve the same result with fewer intermediate steps.  It also occurs to me that I could use a keyword or perhaps a colour tag to mark images used for a website in case I need to know exactly which one I used previously out of a batch of similar looking ones. 

Thanks

Stephen

Offline Jack Baty

  • Newcomer
  • *
  • Posts: 20
    • View Profile
Re: New user - initial experiences
« Reply #3 on: October 01, 2021, 04:50:05 AM »



My ultimate aim would be to get my images well organised both for my own benefit and also so that when the inevitable day arrives when I'm not around that my photo hard drives are organised in a way with clearly defined folders of "finished" JPGs so that my partner or children can take what they want. 

Same! And the following suggestion from Mick seems fine...

One thing that helps me from keeping so many different versions of files is knowing that PM can generate resized jpgs

Except...if I've edited the RAW (e.g. DNG) files using, say, Lightroom, the JPGs rendered from PM won't include those edits unless I first remember to force an update in Lightroom using "Update DNG Preview", right? And worse, in Capture One, there's no way to update the embedded preview, since C1 refuses to touch any DNG files themselves.

How do people manage this? It seems inconvenient and error-prone to always have to manually update the DNG previews before creating JPG copies. I always "burn" a JPG after doing edits in Lightroom so that I have a permanent version that will survive an eventual move to a different RAW processor (or a move by me into some afterlife :)). Or does everyone just decide to cross that bridge when they get there? I often hear photographers say that they delete any JPGs they create for sharing/transfer because "I can always generate new ones". Except you can't if you're not using the same RAW processor. I always feel like I'm missing something here.

Offline Stenis

  • Newcomer
  • *
  • Posts: 46
    • View Profile
Re: New user - initial experiences
« Reply #4 on: December 04, 2021, 04:17:00 PM »


My main tools these days are DXO PhotoLab for processing raw images together with Affinity Photo for doing more sophisticated work.  I have used On1 a lot in the past (and am still in their subscription program) but use it less and less as I don't much care for the raw processing and while it used to be a fast browser, it seems a lot slower these days (even on faster hardware).  When I first used it, it was primarily to use On1 Layers as an inexpensive way to get into layers but these days all of that functionality is in Affinity Photo and PM+ would seem much better for the DAM aspects.  While I used Lightroom for a while (because it was necessary for a Open University photography course I was doing), I ceased to use it after Lightroom 6 and the move to subscriptions.  I don't want to have photographs that depend on having software to see them correctly and therefore generating "finished" JPGs seems to be the safe way to go (even if DXO or other software is still keeping the "recipe" in a sidecar file alongside the raw file). 



Hi Steven I have settled for Photolab a long time ago and the last year when I "found" PM Plus I felt it´s close to a perfect match. Photolab gives better image quality than Lightroom and with PM Plus we also get a far better Image Library than the integrated one Lightroom offers. It´s not perfect but really very good if you are aware of the pit holes.

Lightroom has speed problems a lot due to the fact that it´s integrated with a RAW-converter that needs high res. previews like even Photolab does and PM Plus is optimized for speed which Lightroom or Photolab 5 for that matter is not. Lightroom is a general tool and tools like these are not as effective as PM Plus who is a specialized tool. So I ditched LR 10 years ago when I saw that applying metadata took too much of an effort for my taste and I have never looked back. With PM I have finally found that it´s a doable project to add metadata to the best bulk of my 70 000 images.

You have to rig a few things in PM Plus like the forms and the file types you are going to work with and not the least which editor you will integrate the Edit-function in PM Plus with. If you do that PM will move the RAW, the DOP-file and the XMP-file together without any problems if you need to and open the images you slect in PM in Photolab witk a breeze.

If you select say 20 images in PM Plus "contact sheet"-view, right click and select Edit (with Photolab.exe) these files will get imported to Photolabs filmstrip and the first one gets opened for developing. One can also drag the same files from their folder in Windows with Explorer to Photolabs "filmstrip" for the same result.

In Photolab a link to all these imports will be saved as you can see to the left in the screen dump below under the "External Selections"-banner. So you can always recall them if needed from inside Photolab 5. This Edit-function is really important because Photolab can be very slow if you just open a folder with say 500 preview images that has to be rendered on the fly by Photolab. So this way controlled editing import saves you from the wait states you might run into otherwise. Especially if your computer isn´t top notch with the latest Graphics card. So in this case the PM Plus can really boost your productivity considerably. Both applications shall be opened simultaneously.

Metadata:
When it comes to the metadata flow always do all metadata work in PM Plus. Never maintain metadata in a two way flow. It´s to ask for problems. Photolab 5 has got a brand new PhotoLibrary that´s OK really but it´s sort of in a 1.0 state with a few things missing yet and it´s a  lot more archaic than PM. If you want you can index all your metadata from your topfolders PM Plus is using from within Photolab too so you can see the metadata properly even in Photolab. If you do index in Photolab the application will automatically build a keyword list of all the keywords you have used when tagging the images in PM Plus too. These keywords can then be used to search for images in Photolab.

BUT, don´t you ever use PM Plus "Structured Keywords" in an integrated workflow between Photolab 5 and PM Plus yet because they will get scrambled in a mess when exporting and importing them between the applications. Only flat keyword lists works without problems for now. Photolab also is missing the kind of export/import interface we have in PM Plus. Hopefully we will get one soon. I hope Camera Bits and DXO together will sort out this structured keyword mess!

The good news is that a work flow where PM Plus owns the metadata is working very well today with Photolab 5 and 4. If I add metadata to a RAW-images XMP-file in PM Plus exactly the same data will get exported from Photolab to DNG, TIF and JPEG-files without doing anything extra at all. It´s easy to check that fact by opening an exported JPEG from Photolab in PM Plus. Photolab 5 reads the attached RAW when exporting say a JPEG after processing the RAW in Photolab without any problems.
« Last Edit: December 04, 2021, 04:32:45 PM by Stenis »

Offline fuzzygel

  • Newcomer
  • *
  • Posts: 1
    • View Profile
Re: New user - initial experiences
« Reply #5 on: December 10, 2021, 08:02:18 PM »
I am also new to PM+, may be dumb question to start. Can I use PM or PM+ to cull the photos inside my Apple Photos Library on my Mac (running Monterey)

Offline Mick O (Camera Bits)

  • Camera Bits Staff
  • Hero Member
  • *****
  • Posts: 556
    • View Profile
    • Camera Bits
Re: New user - initial experiences
« Reply #6 on: December 13, 2021, 05:22:54 PM »
I am also new to PM+, may be dumb question to start. Can I use PM or PM+ to cull the photos inside my Apple Photos Library on my Mac (running Monterey)

This probably deserves its own thread.  The short answer is NO.  But...

[Post edited to indicate that better solutions are below]
« Last Edit: December 20, 2021, 09:57:28 AM by Mick O (Camera Bits) »
Mick O
Camera Bits

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25025
    • View Profile
    • Camera Bits, Inc.
Re: New user - initial experiences
« Reply #7 on: December 13, 2021, 05:47:25 PM »
From there, right-click in PM's Navigator and choose "Open Folder and Subfolders..." to see your entire Photos library.

If your library is of any large size (say over 20,000 photos), PM won't handle it well.  Though there is another solution.

You can then use PM to re-organize/move the whole collection and move it somewhere easier to manage (creating dated folders)  (and then add to a PMPlus catalog)

I'd suggest using Ingest (folders) and choose the top of the library as the source folder.  Then use {variables} to create your folder structure at another location.  Then when that is done, choose that location in Scan to Catalog to build your catalog from your photos.

https://docs.camerabits.com/support/solutions/articles/48000207639-introduction-to-photo-mechanic-variables
https://docs.camerabits.com/support/solutions/articles/48000358438#Time-and-Date-Variables

Photo Mechanic Plus won't have any problems with this solution, even if you have a million images (or more).

-Kirk

Offline Mick O (Camera Bits)

  • Camera Bits Staff
  • Hero Member
  • *****
  • Posts: 556
    • View Profile
    • Camera Bits
Re: New user - initial experiences
« Reply #8 on: December 14, 2021, 11:12:39 AM »
Kirk's improved suggestion also has the benefit that it won't change anything in that hidden Photos library folder, and should allow you to keep using Photos if you need to do that. :-)

-Mick
« Last Edit: December 14, 2021, 12:05:41 PM by Mick O (Camera Bits) »
Mick O
Camera Bits

Offline ahoward

  • Camera Bits Staff
  • Hero Member
  • *****
  • Posts: 991
    • View Profile
Re: New user - initial experiences
« Reply #9 on: December 14, 2021, 12:42:27 PM »
Just wanted to add a bit more here because I have tried all of the things mentioned above, and while they do get your images, there are some caveats to them, especially if you are using the iCloud Photo Library.

The structure of this library often contains multiple versions of the same image, and it is also possible that the source image is not contained in the Photos Library at all and is instead referencing some other location on the disk (not to mention that if you have the iCloud Photo Library enabled, the actual source file may not be stored locally on your computer at all.)

The best way to do this that I have found is to use the Export option in Apple Photos to collect the images into one parent folder for Photo Mechanic Plus to scan (or if not using Photo Mechanic Plus, a bunch of folders collected under one parent that you can view the contents of with Photo Mechanic 6.0 If you are using the iCloud Photo Library, this will also take care of downloading the source files from iCloud to save them locally to the Export destination (which, if there is a lot of files that are not local and only in iCloud, could take a long time).

To do this, select your images in the Photos application; click on the 'File' menu; and then choose Export.

You can choose choose 'Export X photos' and you will have the option to export images with edits made in the Photos app as a JPEG, TIFF, or PNG.
-or-
You can choose 'Export unmodified originals' and this will export the files in their original format without your edits applied.

We would also recommend checking the box for "Export IPTC as XMP".

For the 'Subfolder format' option, you may want to choose 'Moment Name', otherwise all exported files will go in the one folder that you choose when you click Export.

Once Photos has finished exporting, you can then open those folders that you Exported as a Contact Sheet or Scan them to into your catalog using "Scan to Catalog" are discussed here: https://docs.camerabits.com/support/solutions/articles/48001157493-scan-to-catalog"

Offline ejhutch

  • Member
  • **
  • Posts: 50
    • View Profile
Re: New user - initial experiences
« Reply #10 on: December 16, 2021, 02:29:36 PM »
I would second ahoward's advice.  I've looked at Apple photos in its various incarnations off and on for a while now, and this is the most useful workflow I've managed to discover also.  In fact, it might be a good FAQ or pinned post.

For those who are interested, I've included a little more info after the quote. It got kind of long, so if you don't have the time, stop reading now.

Just wanted to add a bit more here because I have tried all of the things mentioned above, and while they do get your images, there are some caveats to them, especially if you are using the iCloud Photo Library.

The structure of this library often contains multiple versions of the same image, and it is also possible that the source image is not contained in the Photos Library at all and is instead referencing some other location on the disk (not to mention that if you have the iCloud Photo Library enabled, the actual source file may not be stored locally on your computer at all.)

The best way to do this that I have found is to use the Export option in Apple Photos to collect the images into one parent folder for Photo Mechanic Plus to scan (or if not using Photo Mechanic Plus, a bunch of folders collected under one parent that you can view the contents of with Photo Mechanic 6.0 If you are using the iCloud Photo Library, this will also take care of downloading the source files from iCloud to save them locally to the Export destination (which, if there is a lot of files that are not local and only in iCloud, could take a long time).

To do this, select your images in the Photos application; click on the 'File' menu; and then choose Export.

You can choose choose 'Export X photos' and you will have the option to export images with edits made in the Photos app as a JPEG, TIFF, or PNG.
-or-
You can choose 'Export unmodified originals' and this will export the files in their original format without your edits applied.

We would also recommend checking the box for "Export IPTC as XMP".

For the 'Subfolder format' option, you may want to choose 'Moment Name', otherwise all exported files will go in the one folder that you choose when you click Export.

Once Photos has finished exporting, you can then open those folders that you Exported as a Contact Sheet or Scan them to into your catalog using "Scan to Catalog" are discussed here: https://docs.camerabits.com/support/solutions/articles/48001157493-scan-to-catalog"


Apple has certainly improved it's Photos app in the more recent versions of MacOS.  It uses SQLite as it's database back end to manage pretty much everything, from referencing where the original photos are located on your disks, as well as much of the synchronization of data between your local photos library and your iCloud photos storage if you use it.

PM+ also uses SQLite as its backend, as does Lightroom.  I'm not sure about other DAM solutions.

SQLite is a self-contained relational database system that can be included in and distributed with applications on many different kinds of systems.  Apple seems to use it "everywhere" now.  I've come across SQLite databases in many system folders and it's certainly used in many apps on iOS devices. It's incredibly functional, fast if you design your architecture correctly, and seems to be pretty lightweight for what it can do.

How do I know this? By browsing the Apple Photos Library DB files in various locations (and the PM+ files and any others I come across that I'm interested in) with a SQLite database browser, and already having a pretty decent knowledge of database design and architecture myself. Though I wouldn't recommend doing this unless you "know what you're doing" as you could easily end up corrupting the database if you're not careful.  Especially if you do it – live– while the application is actively using the database.

Apple Photos stores all of its originals (that have been copied into the "System" photos library) in the "originals" folder in subfolders named 0 through F. The original photo file names themselves are in the form of a GUID (Globally Unique Identifier), which is a long string of letters and numbers that seems meaningless, but really isn't.  If you peek at this structure, you notice that each subfolder contains all the filenames beginning with it's character (0-F) (yes, that's hexadecimal if you're interested).

Apple Photos maintains a list of all the GUID original filenames in a database table in the photo library database and cross-references that with the filename you'd be familiar with.  I have a feeling (though I have not verified this) that it's the Photos App on your Mac and also the Photos App on your other iDevices that generates the GUID when the photo is saved to your photos library -- regardless of the photo source. And it appears to me that Apple uses this GUID filename to ensure that photo synchronization does not become jumbled across the cloud, iDevices, and shared albums.

However, when you use the Export or Export Unmodified Original functions, Photos provides the expected IMGxxxxx.jpg (or whatever your filename was) filename for the file that gets exported.

In addition, Photos also maintains derivative and other "versions" of image files located in other folders in the photos library.  They may have the same GUID filename, but be a different resolution, or perhaps have a different "rendering" recipe.  I'm not sure, as I haven't really explored that part, yet.

What I do know, is that if you simply use PM+ to "scan to catalog" or ingest the contents of your Apple Photos Library, you probably won't get what you hope for. I've tried various of doing just that.

However, if you use the Export methods, you will.  And you'll also get all the metadata as well if you chose the "Export IPTC as XMP" option.

I also know that programmatically interacting with the Photos Library used to be a royal pain, as the photos browser functionality provided in earlier incarnations was very limited and very slow.

Now, however, it seems to have gotten much better, with much more updated and functional APIs to browse and more fully interact with the photos library.  A good example program that I've used is one called PhotoSweeper (used for finding duplicate images pretty much anywhere you look).  From what I can tell, it uses much newer APIs to interact with the Photos library, and it's amazingly very much faster than the previous experience used to be.

I'd love to see PM+ do something similar, but I don't know how much of the user base would actually be interested in Apple Photos' integration -- even just much easier importing.  Though it does seem to make importing photos from an iPhone much easier if you import on a computer that can access your iCloud photos.

Offline ahoward

  • Camera Bits Staff
  • Hero Member
  • *****
  • Posts: 991
    • View Profile
Re: New user - initial experiences
« Reply #11 on: February 02, 2022, 09:19:50 AM »
I would second ahoward's advice.  I've looked at Apple photos in its various incarnations off and on for a while now, and this is the most useful workflow I've managed to discover also.  In fact, it might be a good FAQ or pinned post.

I only just saw this post but just wanted to thank you for writing it all out. I will be using some of the information you provided for future documentation.