Spinning wheel when reactivating

[timclarke]

Hi,

Wondering if anyone can help here...  We are having an issue with PM5 on Mac OS X 10.8 when we have very large (3000+) folders of images open.  The most obvious symptom is when we have the folder open, activate a different application (for as little as a few seconds), then reactivate PM.  No matter what we were doing, it produces a 'spinning wheel' for anything up to a full minute.

Apart from that it seems quite slow on other occasions, but nothing quite so repeatable.  This happens most noticeably on our field editor's machine, because he's the one opening the biggest folders, but it does happen on others.  We have one machine still running 10.6, and it's not happening on there.  They are getting restarted regularly, but it's got to the point that to get work done he is reverting to PM 4.6.9 which runs perfectly, but without a few functions from PM5 that we use a lot.

Interestingly Activity Monitor is showing that free memory plummets as soon as PM5 starts and is used to view previews, until it hits less than 50-odd mb (about the size of an uncompressed image) then the spinning starts.  After a few hours of this, I've seen the Page Outs stat as high as 37GB.  PM is regularly showing about 1GB of 'Real Memory' in the Activity Monitor list (I suspect it would go higher if the machine had more RAM than the 8GB it has).  In the Memory Cache option we currently have it set to 256mb having read a few posts on here, but it doesn't seem to make any difference to anything, no matter where we put that setting.

To me this just seems like some kind of memory leak issue between PM5 and OS X 10.8 but I wondered if anyone else had experienced anything similar and if there are any known settings issues that might help?

Thanks in advance...

Tim

[Kirk Baker]

Tim,

Wondering if anyone can help here...  We are having an issue with PM5 on Mac OS X 10.8 when we have very large (3000+) folders of images open.  The most obvious symptom is when we have the folder open, activate a different application (for as little as a few seconds), then reactivate PM.  No matter what we were doing, it produces a 'spinning wheel' for anything up to a full minute.

Apart from that it seems quite slow on other occasions, but nothing quite so repeatable.  This happens most noticeably on our field editor's machine, because he's the one opening the biggest folders, but it does happen on others.  We have one machine still running 10.6, and it's not happening on there.  They are getting restarted regularly, but it's got to the point that to get work done he is reverting to PM 4.6.9 which runs perfectly, but without a few functions from PM5 that we use a lot.

Interestingly Activity Monitor is showing that free memory plummets as soon as PM5 starts and is used to view previews, until it hits less than 50-odd mb (about the size of an uncompressed image) then the spinning starts.  After a few hours of this, I've seen the Page Outs stat as high as 37GB.  PM is regularly showing about 1GB of 'Real Memory' in the Activity Monitor list (I suspect it would go higher if the machine had more RAM than the 8GB it has).  In the Memory Cache option we currently have it set to 256mb having read a few posts on here, but it doesn't seem to make any difference to anything, no matter where we put that setting.

To me this just seems like some kind of memory leak issue between PM5 and OS X 10.8 but I wondered if anyone else had experienced anything similar and if there are any known settings issues that might help?

Since PM is a 32-bit application the most RAM that PM could possibly ever account for is 4GB.  Page Outs occur when your system runs low on RAM and needs to allocate more memory.  Least recently used pages of memory are Paged Out to make room.  I can assure you that there is no outside memory leak and that your system is working fine.

Where are the images located?  Are they on a local fast hard drive?

Each time you leave PM and resume it, PM scans each folder in each Contact Sheet for new, removed, or changed images.  It must do this to maintain integrity.  What sort type are you using?

What kind of images are being browsed?

One thing you could try is going into the Caching section of the Preferences and empty all of the caches and see if that helps.

-Kirk

[timclarke]

Hi Kirk,

Thanks very much for your reply.  All good to know. 

By default Filename search order but I've tried it with a few others but there's no difference. 

The drive is usually a LaCie 'little big disk' (I think) which is a pair of RAIDed HDs connected via Thunderbolt with nothing else connected to the Thunderbolt bus, but we also see this on internal drives (I've not tried on an SSD however).  I think filesharing is enabled and people do sometimes connect to that drive, but I'm certain I have tried all my tests when nobody is connected.

The images are almost always just jpegs direct from Canon EOS 1D X's, occasionally with CR2 raws in the mix, and occasionally just the CR2s.

I forgot to mention the caches, thanks - we have tried clearing those but nothing changes.

Thanks again.

Tim

[Kirk Baker]

Tim,

By default Filename search order but I've tried it with a few others but there's no difference. 

The drive is usually a LaCie 'little big disk' (I think) which is a pair of RAIDed HDs connected via Thunderbolt with nothing else connected to the Thunderbolt bus, but we also see this on internal drives (I've not tried on an SSD however).  I think filesharing is enabled and people do sometimes connect to that drive, but I'm certain I have tried all my tests when nobody is connected.

The images are almost always just jpegs direct from Canon EOS 1D X's, occasionally with CR2 raws in the mix, and occasionally just the CR2s.

I forgot to mention the caches, thanks - we have tried clearing those but nothing changes.

OK.  What build of PM 5 do you have?  Is it recent?  The latest build is 15451.  If you don't have build 15451, please get it.  If the problems persist, then there are some logging flags we can turn on to help find out what is causing the delay.

-Kirk

[timclarke]

Yep, it's 14541. Thanks for your help Kirk, it's really appreciated. Let me know what you need me to do...

Tim

[Kirk Baker]

Tim,

Yep, it's 14541. Thanks for your help Kirk, it's really appreciated. Let me know what you need me to do...

There is a logging facility built into PM5.  You need to create a plain text file named PMDebug.txt and it has to be in PM's Preferences folder.

PM's Preferences folder will be in the following location:

~/Library/Preferences/com.camerabits.PhotoMechanic/

Unfortunately Apple decided a few OS versions ago to hide the Library folder within your home folder.  You can still get there though by using the "Go to Folder..." command on the Go menu in the finder.  Enter the path above in the dialog that appears.

Create your PMDebug.txt file there (make sure you save it as a plain text document and not a special format) and make it have the following contents:

SHOW_DIRSCAN_TIME ON

Save the file and relaunch PM5 and get the problem to happen again.  You can then use the "Reveal Support Data..." command on the Help menu and post the zipped log file here.

Thanks,

-Kirk

[timclarke]

Hi Kirk,

Thanks very much for your help.  It might be a few days before I can get to this one but as soon as I have done it I will post the file.

Tim

[timclarke]

Hi Kirk,

Apologies for the delay getting this - I got caught just when people were at shoots abroad so it has taken until now to get the file for you.  It should be attached to this post.

If it helps, a good example of the issue happening was at very roughly 11:30am on Friday 20th June.

Thanks again.

Tim

[attachment deleted by admin]

[Kirk Baker]

Tim,

Apologies for the delay getting this - I got caught just when people were at shoots abroad so it has taken until now to get the file for you.  It should be attached to this post.

If it helps, a good example of the issue happening was at very roughly 11:30am on Friday 20th June.

Most of the directory scans were so short that they didn't exceed one second in total.  There were several that took up to 3 seconds to complete.  Then at 11:20 (the log stops abruptly thereafter so I don't know if you just got the zip file while PM was running or you force quit PM) there was this entry:

[Sat Jun-21-2014 11:20:10.187][ac766a28][2007252] PMDocCommon.cpp:865 STOPWATCH: TIME 18.889 EXCEEDS LIMIT 1.000 BY 17.889 IN /Volumes/DataRAID/ExpandingGalaxy/Development/Camera_Bits/PM4/Source/PMDocCommon.cpp:865: RESCAN: DIRSCAN
[Sat Jun-21-2014 11:20:10.225][ac766a28][2007252] PMDocCommon.cpp:991 STOPWATCH: TIME 18.928 EXCEEDS LIMIT 1.000 BY 17.928 IN /Volumes/DataRAID/ExpandingGalaxy/Development/Camera_Bits/PM4/Source/PMDocCommon.cpp:991: RESCAN TOTAL

To scan the directory contents of the contact sheet being scanned took 18.889 seconds and all of the rest of the work that the scanning code does (determining if there were changes to the directory, joining RAW+JPEG, etc.) took 0.039 seconds.

Something made getting the directory contents take a long time, much longer than the average.  This could be attributed to many things (drive spun down, other processes hitting the same drive such that the drive heads had to move between two positions on the drive while serving up the needs of each process in turn, portions of PM's code or memory being swapped back in by the virtual memory system).

How long in total was this delay?

-Kirk