Photo Mechanic 6 + and large database of imagery

[BSM]

I've been a user of PM for 15 plus years and have just recently upgraded to PM 6+. I'm having some struggles and would like any assistance anyone may be willing to give. I am a photo agency for a sports entity and having the ability to search our NAS here at the office is paramount. We were able to do so very easily and efficiently while running PM 5. However with the advance of Catalina, we were forced to move to PM 6. Unfortunately, with PM6 our much needed search tool was rendered useless. I was pointed to PM 6+ by a Camera Bits Support individual and we discussed that this should be the ticket in order to get our search operation back up and running. I was able to dive into the project about a week ago and here's my report up to now.

So, after 5 days of scanning and building the catalog, yes - I definitely could perform the much needed search and all was well. But, by the 6th day, the metadata gathering process of the catalog continued. I had built the catalog to live on my computer not knowing how large it was going to be. The catalog was 192 gigs. Eventually, my computer ran out of disk space and memory and PM 6+ crashed. So…I did some checking and found that I could move the catalog to a different location by dragging and dropping the folder and it’s contents from one location to another. But, before I did, I conducted an experiment. I wanted the Catalog to reside on our NAS. So, I built a quick test catalog from one event, put it on the NAS and conducted a search and it was perfect. So, that told me that the Catalog would be searchable from the NAS. So, I did as the PM guide said and did a drag and drop from my computer to the NAS. PM 6+ then loaded and came back and said the Catalog needed to be “Reintegrated”. After a few attempts of the “Reintegration” process timing out and failing, I did get it to hold and it began the process yesterday morning. It has slowly kept going and 24 hours later, I am at 15% of Reintegration. Now, the Catalog folder is 192 gigs and consists of 1.5 million files. And the first time it performed the Catalog Scan and building it did take 6 days. So, am I just bound to wait patiently as the Reintegration process clicks away at such painfully slow speed?

Yes, the process worked with the Catalog on my computer. But, the catalog was too large to be housed on my computer. So, hopefully it will work and behave properly on the NAS? Who knows? Because, it’s looking like I’ll need to wait until the Reintegration is complete given the catalog remains offline until the reintegration is complete. I found nothing in the forums regarding Reintegration and how it performs, so hopefully I’m on the right track. I’m just so astounded at how slow the process is and has been. Perhaps most people who utilize PM 6+ maybe don’t have catalogs of 1.5 million?

Anyways, that’s my report. Fingers crossed it works! And hopefully it will be completed before my free trial runs out!

Thanks! And I’m up for any help you can give!

[Kirk Baker]

We don't suggest ever putting a catalog on a NAS.  They're just too slow.  The size of your locally stored catalog need not be that large however.  Since you're likely to always have the NAS available, you could turn off proxy generation and save a lot of space.  The catalog would then be in the ~20GB range which shouldn't fill up your entire local hard drive.

Stopping that reintegration, which may take a number of days to complete based on its progress so far, is what should be done first.  There is no direct way to stop it though and usually it takes minutes, not days to complete.  You'll have to quit PM Plus, then potentially kill off any running pm-task processes that remain.  The Activity Monitor application on your system should help there.  In the upper right-corner of the Activity Monitor window is a search field.  Just type "pm" into it (no quotes though) and the list of processes will be filtered down and you may see one or more pm-task entries.  Select them, one by one, and use the 'force quit' button in the toolbar (left-most, looks like a stop sign with an 'x' in it).

Reopen PM Plus and open the Catalog Management window and find your catalog in the list of Local Catalogs.  Expand it by clicking on the triangle widget to the left of its name.  Uncheck the "Generate preview & thumbnail proxy images for offline or remote use" checkbox.  Then click the Forget button.  Quit Photo Mechanic Plus.

Then you can delete the proxies folder inside your catalog folder.  Then move the catalog folder back to your local disk.  Re-launch Photo Mechanic Plus.  In the Catalog Management window, click the "Locate Existing Catalog" button and select the catalog folder on your local disk.  You'll have to run another reintegrate, but it should run faster than when it was on your NAS.

The size of your catalog will now remain quite smaller than when it was managing proxies.

HTH,

-Kirk

[BSM]

Hi Kirk! Hooray! That worked!  Thank you so very much. With your help, I was able to get it going and the reintegration completed within 5 hours give or take. I've stress tested it with our normal search patterns and all seems to be operating as normal!

A few follow up questions:

1) We add event folders weekly to the photo database. In order to continue to build out the Catalog and sync it, do I need to manually tell the Catalog to sync by going to Catalogs>Sync Catalogs>Select Quick Sync>Start. Or is there a way "smart" way that the Catalog will automatically  "look" at the photo database periodically and perform the sync itself?

2) Given we are unable to have the Catalog stored on the NAS, can I now copy my Catalog folder over to another laptop and it will then manage the Catalog in the same way as my laptop does? Essentially, each computer will have its own catalog folder and be responsible for updating and syncing that catalog within its own system.

Thank you so very much for your assistance and help on this. I have a team of photographers who we require to have PM as their workflow so getting this up and running will now allow them to upgrade themselves and I can assist them. Appreciate it!

[Kirk Baker]

Hi Kirk! Hooray! That worked!  Thank you so very much. With your help, I was able to get it going and the reintegration completed within 5 hours give or take. I've stress tested it with our normal search patterns and all seems to be operating as normal!

A few follow up questions:

1) We add event folders weekly to the photo database. In order to continue to build out the Catalog and sync it, do I need to manually tell the Catalog to sync by going to Catalogs>Sync Catalogs>Select Quick Sync>Start. Or is there a way "smart" way that the Catalog will automatically  "look" at the photo database periodically and perform the sync itself?

You'd need to do the Catalog Sync manually like you outlined in your first sentence.

2) Given we are unable to have the Catalog stored on the NAS, can I now copy my Catalog folder over to another laptop and it will then manage the Catalog in the same way as my laptop does? Essentially, each computer will have its own catalog folder and be responsible for updating and syncing that catalog within its own system.

You could copy it from one computer to another and do a Locate Existing Catalog along with a Reintegrate on the destination computer.

-Kirk

[BSM]

Great! Thank you so much for your assistance. So appreciated!