PM faster than ever

[pdlarry]

I've had PM since around July last year, and have always had some performance issues.  Suspected the external drive, cache settings, etc etc, but never would I have expected it to be a problem with CPU overheating.  We have a Dell laptop running Pentium 4 HT with 1G RAM, XP Pro.  The fans were always on high speed they sounded like jet engines.  Earlier this week I decided to clean out the fans, first time in over 3 years--couldn't believe how much dust was in there!!  This made the fans much quieter, but it also allowed proper cooling so everything is running very fast again, including PM.  I've not truly experienced/enjoyed the speed of PM until now, and just wanted to share my joy!!

For the curious, try googling "CPU overheat" and you might find an interesting clip on youtube...

Shout out to Kirk and the team for an amazing product!!

[david_hill]

On that note, is there a good write-up anywhere on this site explaining how to maximize PM performance on the Mac?

I have seen PM perform really erratically, and most of the time I don't think it's PM's fault. The first thing I did was add every volume on my computer to the privacy list so that Spotlight would not keep trying to index while I worked. That helped. I have increased the RAM setting in PM prefs, and that seems to help. However, I don't know what is optimum there.

I have most recently come to suspect that both Safari and Mail have the effect of killing PM performance simply by being open in the background (and I'm in 10.4.10 at the moment). Thus, I have become suspicious of just about any Apple-authored application running in the background.

A final issue I know that I have is that my system drive has gone over a year without initialization (and in that interval has been filled/emptied several times) and thus is probably quite fragmented. My disk free space is down to 30 GB, which  you'd think is enough, BUT... I installed a second brand new hard drive with leopard on it just to test (have not migrated yet), and what I saw is that Mac OS X was using 32 gigs of virtual memory during a PM operation!!!!! Seeing that made me long for the days of OS 9 when you could turn off VM or run a ram disk. IMHO tying Mac OS X to always-on VM was one of the worst things ever... In case you're wondering, in that case I was browsing an unusually large folder of images, which not-so-coincidentally was about 30 GB worth. But still, I would not expect Mac OS X to try and write the entire folder to VM... But it did suggest that my current startup drive with only 30 GB left free was inadequate for working. Ridiculous. I can remember the good old days when Mac OS X only needed 10 GB to keep from choking on itself.

Back to my point, I think, if there is not one already, that Camerabits should post a reference page on maximizing performance on the Mac. And maybe that other operating system, too.

[Kirk Baker]

David,

On that note, is there a good write-up anywhere on this site explaining how to maximize PM performance on the Mac?

I have seen PM perform really erratically, and most of the time I don't think it's PM's fault. The first thing I did was add every volume on my computer to the privacy list so that Spotlight would not keep trying to index while I worked. That helped. I have increased the RAM setting in PM prefs, and that seems to help. However, I don't know what is optimum there.

I have most recently come to suspect that both Safari and Mail have the effect of killing PM performance simply by being open in the background (and I'm in 10.4.10 at the moment). Thus, I have become suspicious of just about any Apple-authored application running in the background.

A final issue I know that I have is that my system drive has gone over a year without initialization (and in that interval has been filled/emptied several times) and thus is probably quite fragmented. My disk free space is down to 30 GB, which  you'd think is enough, BUT... I installed a second brand new hard drive with leopard on it just to test (have not migrated yet), and what I saw is that Mac OS X was using 32 gigs of virtual memory during a PM operation!!!!! Seeing that made me long for the days of OS 9 when you could turn off VM or run a ram disk. IMHO tying Mac OS X to always-on VM was one of the worst things ever... In case you're wondering, in that case I was browsing an unusually large folder of images, which not-so-coincidentally was about 30 GB worth. But still, I would not expect Mac OS X to try and write the entire folder to VM... But it did suggest that my current startup drive with only 30 GB left free was inadequate for working. Ridiculous. I can remember the good old days when Mac OS X only needed 10 GB to keep from choking on itself.

Back to my point, I think, if there is not one already, that Camerabits should post a reference page on maximizing performance on the Mac. And maybe that other operating system, too.

The manual covers Caching settings somewhat.  How much RAM do you have?  The number one thing you can do to increase the performance of PM is increase the Memory Cache beyond the default of 64 MB.  I run my memory cache at 512 MB.  If you have two or more GB of RAM, then increasing the Memory Cache is in order.  Note that you do have to balance your RAM availability with your other application's needs or increasing PM's Memory Cache can instead be detrimental.

The other thing you can do is increase the speed of your disks by using RAID arrays.  They are costly but their performance is impressive and some of the RAID formats give you both redundancy and speed.  If RAID is out of the budget then de-fragmenting your drives will help things a bit.

As for Mac OS X having huge VM sizes, don't worry about it.  What Mac OS X was doing was caching various resources and as soon as new RAM allocations are needed, Mac OS X will very quickly drop the oldest pages in the system's cache for your immediate needs.

Mac OS X has gotten faster and faster with each release as the engineers profile the OS and tune it for best performance.  Mac OS X 10.0 was a slowpoke compared to 10.4.11 and 10.5.2...

-Kirk

[david_hill]

Kirk -

I have 5 GB ram in my quad core intel. I generally keep the ram cache set to either 1 GB or 2 GB.

The primary issue I have is what what I affectionately call "chugging." Chugging is one of those periods of inexplicable extended disk activity that Mac OS X sometimes does that drastically slows down your other activities. When this occurs, it is so dramatic that any other performance hits look trivial by comparison. Spotlight is an obvious source of chugging, but clearly not the only one.

This frequently also hits me on the first 5 or 6 images of a Photoshop batch process. In other words, the first 5 or 6 photos, which would normally take about 3 seconds each in my action, end up taking about 30 to 40 seconds each until the chugging stops, then the speed magically accelerates back to normal. This also happens when I open a new folder in PM.

The effect across multiple apps shows that it's clearly a system issue and not PM. It's very hard to troubleshoot this issue, because it's so non-specific. And I say this as a former Apple employee and general Apple expert.

Another contributor to chugging on my system is that my startup drive is partitioned (hence full platter excursions). I had reasons for partitioning at the time, but I won't be making that mistake in Leopard. From now on, each volume gets its own dedicated drive in my box.

Anyways, any tips from other chugging sufferers greatly appreciated.

Oh, and does anyone know a good lightweight search application for Mac OS X that works the way the old "find" command did BEFORE spotlight? I absolutely cannot tolerate having spotlight active on my system, so the net effect is that I can't search for anything at all, unless you count grepping in the Terminal.

[Kirk Baker]

David,

I have 5 GB ram in my quad core intel. I generally keep the ram cache set to either 1 GB or 2 GB.

That's a bit high, especially since you're using Photoshop at the same time (as I see below).  I would set that back down to 512-768 MB of Memory Cache.

The primary issue I have is what what I affectionately call "chugging." Chugging is one of those periods of inexplicable extended disk activity that Mac OS X sometimes does that drastically slows down your other activities. When this occurs, it is so dramatic that any other performance hits look trivial by comparison. Spotlight is an obvious source of chugging, but clearly not the only one.

This frequently also hits me on the first 5 or 6 images of a Photoshop batch process. In other words, the first 5 or 6 photos, which would normally take about 3 seconds each in my action, end up taking about 30 to 40 seconds each until the chugging stops, then the speed magically accelerates back to normal. This also happens when I open a new folder in PM.

The effect across multiple apps shows that it's clearly a system issue and not PM. It's very hard to troubleshoot this issue, because it's so non-specific. And I say this as a former Apple employee and general Apple expert.

Because you have such a large Memory Cache in PM and you're running Photoshop (which by default uses 50% of your RAM) you've got about 3.5 to 4.5 GB of RAM devoted to PM and Photoshop's memory caches.  That is nearly all of your memory!

These long pauses are the Virtual Memory system swapping large portions of each app's memory cache out to disk.  Reducing PM's Memory Cache (512-768 MB) and adjusting Photoshop's cache down to 1 to 1.5 GB should help quite a bit with the swapping.  And if you're running other apps like Safari, Mail, etc. then you'll likely need to decrease those settings even further.

Another contributor to chugging on my system is that my startup drive is partitioned (hence full platter excursions). I had reasons for partitioning at the time, but I won't be making that mistake in Leopard. From now on, each volume gets its own dedicated drive in my box.

I don't see how partitioning your hard drive would cause the drive heads (there are two per platter) to make any larger than normal seeks.  It is true that they don't sweep independently but the drive should be able to read blocks off the disc pretty quickly no matter which track contains the desired data.  Having dedicated drives should help somewhat since both drives can be reading and writing independently.  Having PM's disk cache on a different drive from which you are browsing photos will definitely help reduce disk thrashing.

HTH,

-Kirk

[david_hill]

Because you have such a large Memory Cache in PM and you're running Photoshop (which by default uses 50% of your RAM) you've got about 3.5 to 4.5 GB of RAM devoted to PM and Photoshop's memory caches.  That is nearly all of your memory!

These long pauses are the Virtual Memory system swapping large portions of each app's memory cache out to disk.  Reducing PM's Memory Cache (512-768 MB) and adjusting Photoshop's cache down to 1 to 1.5 GB should help quite a bit with the swapping.  And if you're running other apps like Safari, Mail, etc. then you'll likely need to decrease those settings even further.

That will probably help me when I experiment with it. Based on what you're saying, I have my Photoshop RAM set too high. I've always done this (or should I say "failed to worry about it") based on the hype that Mac OS X memory management was supposed to be smart enough to keep the system from choking.

I don't see how partitioning your hard drive would cause the drive heads (there are two per platter) to make any larger than normal seeks.  It is true that they don't sweep independently but the drive should be able to read blocks off the disc pretty quickly no matter which track contains the desired data.  Having dedicated drives should help somewhat since both drives can be reading and writing independently.  Having PM's disk cache on a different drive from which you are browsing photos will definitely help reduce disk thrashing.

Here's the effect of partitioning I see which I would warn anyone against: I have always used my desktop as a dumping ground for temporary copies of "working files" that will ultimately be copied to an external drive for long-term storage. After filling the disk and emptying it a couple of times, my startup drive would end up being rather fragmented. So I got the bright idea to make a separate partition, about half my total disk, for the desktop folder... Since doing that, what I see is that performance is always lower when working with files stored on this "desktop" than with files stored on the startup partition or on any other internal or external hard drive. Based on the noise I'm hearing from the hard drive during these periods of low "desktop" performance, I can only guess that I'm getting more excursions as it reads data files from one partition and virtual memory from another on the same disk. If I try to check my email while doing this, everything grinds to a near halt... As a "control" to this experiment, I can definitely say that the same performance ills do not occur when working with files stored on a second internal hard drive or on the startup partition itself... So while it was a noble experiment in theory, I definitely advise against doing what I did.

[Kirk Baker]

David,

Because you have such a large Memory Cache in PM and you're running Photoshop (which by default uses 50% of your RAM) you've got about 3.5 to 4.5 GB of RAM devoted to PM and Photoshop's memory caches.  That is nearly all of your memory!

These long pauses are the Virtual Memory system swapping large portions of each app's memory cache out to disk.  Reducing PM's Memory Cache (512-768 MB) and adjusting Photoshop's cache down to 1 to 1.5 GB should help quite a bit with the swapping.  And if you're running other apps like Safari, Mail, etc. then you'll likely need to decrease those settings even further.

That will probably help me when I experiment with it. Based on what you're saying, I have my Photoshop RAM set too high. I've always done this (or should I say "failed to worry about it") based on the hype that Mac OS X memory management was supposed to be smart enough to keep the system from choking.

I don't see how partitioning your hard drive would cause the drive heads (there are two per platter) to make any larger than normal seeks.  It is true that they don't sweep independently but the drive should be able to read blocks off the disc pretty quickly no matter which track contains the desired data.  Having dedicated drives should help somewhat since both drives can be reading and writing independently.  Having PM's disk cache on a different drive from which you are browsing photos will definitely help reduce disk thrashing.

Here's the effect of partitioning I see which I would warn anyone against: I have always used my desktop as a dumping ground for temporary copies of "working files" that will ultimately be copied to an external drive for long-term storage. After filling the disk and emptying it a couple of times, my startup drive would end up being rather fragmented. So I got the bright idea to make a separate partition, about half my total disk, for the desktop folder... Since doing that, what I see is that performance is always lower when working with files stored on this "desktop" than with files stored on the startup partition or on any other internal or external hard drive. Based on the noise I'm hearing from the hard drive during these periods of low "desktop" performance, I can only guess that I'm getting more excursions as it reads data files from one partition and virtual memory from another on the same disk. If I try to check my email while doing this, everything grinds to a near halt... As a "control" to this experiment, I can definitely say that the same performance ills do not occur when working with files stored on a second internal hard drive or on the startup partition itself... So while it was a noble experiment in theory, I definitely advise against doing what I did.

Some of this slowness may be happening from Virtual Memory paging on this particular system.  It would be interesting to test copying files on this system but with no applications running besides the Finder.  If you still hear a lot of disk thrashing, then I'd suspect it is because of fragmentation, but at least VM paging would be ruled out.

-Kirk