Author Topic: Photo Mechanic 6 plus...slow metadata collection  (Read 2487 times)

Offline Talljeff

  • Newcomer
  • *
  • Posts: 2
    • View Profile
Photo Mechanic 6 plus...slow metadata collection
« on: November 10, 2020, 06:12:52 PM »
I saw on another forum that photo mechanic 6 plus was out and available for download and trial activation. I currently use photo mechanic 6 / Lightroom, and I had been wanting to try another cataloging software. I downloaded the software, activated the trial, created a new catalog. I chose not to use the default catalog. Don’t ask my why...no specific reason. I sent to catalog scan and choose my Photos folder on my NAS. I did not exclude any file extensions. I chose everything in the Photos folder. There was a total of around 28,000 photos. I started the scan. It has been running for around 2 hours and it is showing that there is still around 8 hours of metadata information to get. Does this seem like a normal amount or time for the initial catalog scan? I am using a MacBook Pro I told photo mechanic 6 plus to put my catalog in my documents folder on the Mac.

Please see the two attachments.

Thank You

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24764
    • View Profile
    • Camera Bits, Inc.
Re: Photo Mechanic 6 plus...slow metadata collection
« Reply #1 on: November 10, 2020, 07:11:45 PM »
Jeff (who happens to be tall),

I saw on another forum that photo mechanic 6 plus was out and available for download and trial activation. I currently use photo mechanic 6 / Lightroom, and I had been wanting to try another cataloging software. I downloaded the software, activated the trial, created a new catalog. I chose not to use the default catalog. Don’t ask my why...no specific reason. I sent to catalog scan and choose my Photos folder on my NAS. I did not exclude any file extensions. I chose everything in the Photos folder. There was a total of around 28,000 photos. I started the scan. It has been running for around 2 hours and it is showing that there is still around 8 hours of metadata information to get. Does this seem like a normal amount or time for the initial catalog scan? I am using a MacBook Pro I told photo mechanic 6 plus to put my catalog in my documents folder on the Mac.

It's not out of the realm of reasonableness given that NASes are generally very slow.  It is good that you put your catalog on your local disk.  Having the catalog on a local fast disk is optimal.  You may want to tell Spotlight to not search your catalog folder (System Preferences->Spotlight/Privacy).

-Kirk

Offline Talljeff

  • Newcomer
  • *
  • Posts: 2
    • View Profile
Re: Photo Mechanic 6 plus...slow metadata collection
« Reply #2 on: November 10, 2020, 07:50:21 PM »
Kirk,

Your assessment of the speed of the NAS is reassuring in the sense that this can be a reasonable amount of time. I just checked...there is progress. The files remaining are less than 20,000 now, so I know there is movement. I’ve been waiting on you guys to release this feature and I have high hopes to put it to use. I’ll see what my trial run looks like and if it’s successful, I will definitely upgrade.

Kind Regards,

Jeff...who is tall

Offline Graham1

  • Full Member
  • ***
  • Posts: 148
    • View Profile
Re: Photo Mechanic 6 plus...slow metadata collection
« Reply #3 on: November 21, 2020, 09:09:09 AM »
I do not think this is unreasonably slow.  I recently decided to clear out all my old test catalogues and start again, with a single new catalogue of all my images, nearly 700,000 of them.  The catalogue is on a SSD drive and all the images are on local hard disks.  I am on Windows 10 on a well-specified modern desktop computer.  Building the catalogue took about 4 days, but given the sheer quantity of data and previews to be created, I thought that this was not unreasonable.  I have another DAM, which took much longer to complete a similar task (and is much slower to use having completed it).

The only issue I had was that the database appeared not to include fully a number of files.  Metadata import appeared to be stuck and images were appearing as offline.  I do not rename my images, the file names remain as out of camera.  That means that I have many files with identical file names, spread over my various annual folders (I keep images in folders by year, then sub-folders by location or subject) spread over several drives.  For files marked as offline, sorting by date did not separate them from their similarly named counterparts - they appeared grouped together by name in the contact sheet, even though differently dated.  I concluded that the problem was the database trying to catalogue simultaneous identical file names in parallel (but only a guess, I am no expert).  This was before the recent release which allows a search for offline files, but I could see that most of the apparently offline files were in the most recent folders.  I therefore opened a contact sheet to load all my most recent folders and sub-folders, to force PM to "see" them and that they were online, not confused by identical file names from older folders.  This worked fine, and all the metadata updated quickly, and all images were recognised as being online.

I have not seen this behaviour reported elsewhere (but I may have missed it).  If anyone else has had similar issues, possibly caused by having a lot of identically named files being indexed at the same time, resulting in apparently offline images or stuck metadata, this worked for me, and I hope this helps.  For comparison, creating the catalogue for only my 2020 images, about 45,000, completed fully in about 2 hours.  Perhaps it might be better to drip feed folders into the catalogue rather than select the whole set of several hundred thousand across several drives at the same time for a single addition on creation of the catalogue. 

Graham
« Last Edit: November 21, 2020, 02:52:36 PM by Graham1 »

Offline Graham1

  • Full Member
  • ***
  • Posts: 148
    • View Profile
Re: Photo Mechanic 6 plus...slow metadata collection
« Reply #4 on: November 23, 2020, 08:00:20 AM »
As a bit of an update on this, I have now created separate catalogues for each of my main year collections, 16 (plus my full catalogue of all my nearly 700k images), and I also rescanned the 2020 and the full catalogue, having added a number of images and reorganised some of them on disk.

The actual scanning of files zips along and is completed in a matter of minutes even for tens of thousands of images per catalogue.  What takes the time is metadata gathering and image preview creation - it is not unreasonable that these take a long time.

I tried to see if I could improve performance (under Windows 10).  I used Task Manager to increase priority to the various processes that PM uses.  The one that made most difference was increasing the priority for the instances of the Ruby Interpreter.  Changing this from the default to real time reduced the estimated time for Metadata Gathering from 9.1 days to 2.5 days!

Hope this information helps.

Graham