Author Topic: "Gathering Sort Information"  (Read 12710 times)

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
"Gathering Sort Information"
« on: April 30, 2020, 02:21:24 AM »
When opening a larger folder I get the message "Gathering Sort Information" counting up the percentage (PM 6 build 4538 on Win 10 1909). During this time the contact sheet is locked (somewhere between 5 to 30 seconds). When opening the same folder in PM5 I can immediately start browing the contact sheet.  Both times I am using Modification Time as sort method.

Is there any way to make this as fast in PM6 as PM5 was?

Oliver

ps. I have to admit that even though I bought PM6 I continued working with PM5 since it is faster. Also I couldn't make friends some changes that were made in PM6 (e.g. I dearly miss the possibility to adjust the modification date). I will have to give PM Plus a test run.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #1 on: April 30, 2020, 09:03:12 AM »
When opening a larger folder I get the message "Gathering Sort Information" counting up the percentage (PM 6 build 4538 on Win 10 1909). During this time the contact sheet is locked (somewhere between 5 to 30 seconds). When opening the same folder in PM5 I can immediately start browing the contact sheet.  Both times I am using Modification Time as sort method.

I'd like to see your Photo Mechanic log.  In Photo Mechanic go to the Help menu and choose "Reveal Support Data..." and moments later a Windows Explorer window will come forward with the zipped log file selected.  Use the 'Attachments and other options' link when you're composing your reply to this message and there you'll be able to upload your zipped PM log file.

Thanks,

-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #2 on: May 01, 2020, 02:31:43 AM »
Thanks, Kirk. Here's the log file.


Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #3 on: May 01, 2020, 08:10:48 AM »
Thanks, Kirk. Here's the log file.

While I'm not seeing anything specific related to sorting, I am seeing a number of failures to render video files due to either a missing or broken installation of GStreamer on your system.  Photo Mechanic 6 needs to use GStreamer to render video files.  I suggest you install it and see if that makes any difference to your sorting problem.

https://docs.camerabits.com/support/solutions/articles/48000340654-use-gstreamer-to-view-video-thumbnails-and-previews-on-windows

-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #4 on: May 03, 2020, 12:48:37 AM »
Thank you Kirk. I will try to (re)install GStreamer but never missed it since I do not use PM for folders with videos.

I attached a screen capture of opening a folder containing 2780 pictures on my internal archive hdd (attached via SATA). Opening that same folder in PM5 I can immediately start browsing the pictures. This is one of the reasons why I continue to work with PM5. One difference I see is that in PM6 the initial pictures shown when opening the folder are not the ones that are on top when PM6 is done sorting.

Oliver

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #5 on: May 05, 2020, 09:12:58 AM »
Oliver,

Does your Color Class filter have all colors selected?  I can't see it in the image you posted.  I'm asking because I can't reproduce the modification time sorting issue (mine happen very briskly--no delay at all).  If you have even one color turned off then the metadata for all of the images must be loaded in order to perform the color class filtering in the end.  I notice the number of images displayed after the sort data is gathered is 2780.  When the sort completes, I see only 546 images.

Having any filters active will significantly reduce the speed of sorting unless the sort cache has up-to-date data available (PM builds its sort cache as it works with images so if PM has browsed that folder before, then browsing it again later should be much faster.)  PM5 does the same thing.  The sort cache isn't infallible however, if images change since the last time their sort data was cached, then it is invalidated.  Also, you have control over how long PM keeps sort cache data around.  If the settings dictate that old data is discarded, then PM will flush it as it ages and viewing a folder that you viewed months ago will again be slow.
 
-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #6 on: May 05, 2020, 10:33:33 AM »
Yes, all Color Class filters are selected. There are no filters active when opening the folder. And yes, there are 2780 files in the folder. When opening the folder in PM5 to check this number I can scroll the pictures immediately. Switching over to PM6 I was able to do the same, probably because everything was cached. After purging the cache I am back to the same "counting-up-percentage" routine. Since PM5 is blazingly fast when opening even not cached folders I never bothered about keeping the sort cache data for a long duration but rather purged cache after a few days.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #7 on: May 05, 2020, 11:01:55 AM »
Oliver,

Yes, all Color Class filters are selected. There are no filters active when opening the folder. And yes, there are 2780 files in the folder. When opening the folder in PM5 to check this number I can scroll the pictures immediately. Switching over to PM6 I was able to do the same, probably because everything was cached. After purging the cache I am back to the same "counting-up-percentage" routine. Since PM5 is blazingly fast when opening even not cached folders I never bothered about keeping the sort cache data for a long duration but rather purged cache after a few days.

If you have no filters on, why does the item count drop to 546 at the end of your clip?

-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #8 on: May 05, 2020, 11:38:54 AM »
Quote
If you have no filters on, why does the item count drop to 546 at the end of your clip?

I have no idea. To check for this behavior I opened some other folders but every time the file number was correct. I made another screen capture but this time with all information.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #9 on: May 05, 2020, 12:52:05 PM »
Oliver,

I see the count change again at the end which makes me think that some filtering is happening.  That said, I found a problem preventing the sort info gathering from taking the fast path when no filtering is requested but the sort is by Modification Time.  I can fix that.  But I still think some sort of filtering is going on and even with my fix, you'll still experience the problem.

If you don't mind, can I have you turn on some logging for me and run your test again and then share the log with me?

PMDebug.txt Windows.

In Photo Mechanic's preferences folder you'll create a text file named PMDebug.txt and enter the following text:

CONTACT_SHEET_SORT_LOGGING ON
SORT_MANAGER_LOGGING ON
SORT_CACHE_LOGGING ON

The location of the preferences folder is:

C:\Users\<your username>\AppData\Roaming\Camera Bits, Inc\Photo Mechanic\

Then quit and start Photo Mechanic again.  Then try your test again.  When the sort finishes, choose the "Reveal Support Data..." command on the Help menu.  A Windows Explorer window will come forward with a zip compressed copy of the PM.log file.  Please reply to this message.  Use the 'Attachments and other options' link when you're composing your reply to this message and there you'll be able to upload your zipped log file.

Thanks,

-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #10 on: May 06, 2020, 04:37:55 AM »
Here you go, Kirk.

I first accidentally opened the same folder as yesterday so the sort information was cached. The I tried an uncached folder.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #11 on: May 06, 2020, 05:24:01 PM »
Oliver,

I first accidentally opened the same folder as yesterday so the sort information was cached. The I tried an uncached folder.

I'm going to have to add more logging code and make you a new build.  Let me know if you're willing to try a new build and continue testing.

Thanks,

-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #12 on: May 07, 2020, 06:31:05 AM »
Absolutely no problem, Kirk. I am more than happy to do more testing and help.

Oliver

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: "Gathering Sort Information"
« Reply #13 on: May 07, 2020, 04:17:59 PM »
Oliver,

Excellent.  Please try this build.  Please only open up one folder that is uncached.  Then post your zipped log in your reply.

https://www.camerabits.com/download/PM6SetupR4665_b920498e.msi

Thanks,

-Kirk

Offline olidie

  • Full Member
  • ***
  • Posts: 199
    • View Profile
Re: "Gathering Sort Information"
« Reply #14 on: May 09, 2020, 10:55:39 AM »
Thank you, Kirk. I will try to give it a test run tomorrow and then upload the log.

Oliver