Author Topic: PM crashed with 220000 files  (Read 4459 times)

Offline JS

  • Full Member
  • ***
  • Posts: 218
    • View Profile
PM crashed with 220000 files
« on: September 20, 2010, 05:38:45 AM »
Hi,

I opened a full HD in one contactsheet, resulting in some 220000 files! I then wanted to select the Grey(trash)-coded ones. PM crashed two times. Seemed like it was to much work for it. Its not something I normally do, i just wanted to find out, since i have my new Mac pro. To my surprise, the Mac Pro isn't faster than my Mac Mini 2,53 Ghz. Both have 4 Gb of Ram.
Regards
John

Mac Pro Quadcore 2,66, 4 GB Ram, Macos 10.6.4, Pm 4.6.4

Offline mbbphoto

  • Sr. Member
  • ****
  • Posts: 312
    • View Profile
Re: PM crashed with 220000 files
« Reply #1 on: September 20, 2010, 06:16:11 AM »
Hi,
 To my surprise, the Mac Pro isn't faster than my Mac Mini 2,53 Ghz. Both have 4 Gb of Ram.
Regards
John

Mac Pro Quadcore 2,66, 4 GB Ram, Macos 10.6.4, Pm 4.6.4
.

Get more RAM, it will help with speed with large numbers of files.
Were they on an external drive?
If so how connected....if by USB 2 this might explain the speed issue.....
Marc

Marc

Offline JS

  • Full Member
  • ***
  • Posts: 218
    • View Profile
Re: PM crashed with 220000 files
« Reply #2 on: September 20, 2010, 06:27:00 AM »
No, they were on a second hd inside the mac pro. I could also put in more Ram in the mac mini, then there would be no difference again. What makes me wonder is, that the Mac pro is not noticeably faster than the mac Mini.
john

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: PM crashed with 220000 files
« Reply #3 on: September 20, 2010, 08:19:28 AM »
Hi,

I opened a full HD in one contactsheet, resulting in some 220000 files! I then wanted to select the Grey(trash)-coded ones. PM crashed two times. Seemed like it was to much work for it. Its not something I normally do, i just wanted to find out, since i have my new Mac pro. To my surprise, the Mac Pro isn't faster than my Mac Mini 2,53 Ghz. Both have 4 Gb of Ram.
Regards
John

Mac Pro Quadcore 2,66, 4 GB Ram, Macos 10.6.4, Pm 4.6.4

Photo Mechanic is not capable, nor was it designed to handle 220,000 files in a single folder.  It's just not something we can support unless PM becomes a 64-bit application.  The metadata alone in that number of files would exceed the 32-bit address space, let alone thumbnails and previews.

You're going to have to split your work up in some fashion and do a bit at a time (we've tested with 30,000 files at once and things worked fine.)

The reason your Mac Pro was no faster than your Mini was that the task you were performing is limited by the amount of RAM you have.  Virtual memory kicked in pretty early in the process, dramatically slowing things down.  Getting more RAM will help, but PM will still run out of RAM since a 32-bit application at most can access a total of 4GB of RAM and that would only be in the case that none of the address space weren't used up by libraries, PM's application code, and various data structures.  I'd say that PM really has about 2.5-3.0 GB of RAM to work with.

Work with fewer files.  There is really nothing that can be done about it from our end other than to artificially limit the maximum number of files, but the maximum number is not determinate and is dependent on a variety of factors including how much metadata each image has, and the total number of images, the size of your thumbnails, and the size of your PM Contact Sheet window (really the number of thumbnails on screen at once.)

-Kirk

Offline JS

  • Full Member
  • ***
  • Posts: 218
    • View Profile
Re: PM crashed with 220000 files
« Reply #4 on: September 20, 2010, 01:33:54 PM »
Thanks for explanation. This was just a try. I normally work with smaller folders. I just wanted to see, if it could display a whole 2 TB HD with all folders and subfolders in one display. And it could, it just crashed when I selected the "Trash"-ones. I hope I will find some action in future where the mac pro would overtake the mac mini.
Regards
John

Offline mbbphoto

  • Sr. Member
  • ****
  • Posts: 312
    • View Profile
Re: PM crashed with 220000 files
« Reply #5 on: September 20, 2010, 09:41:25 PM »
Thanks for explanation. This was just a try. I normally work with smaller folders. I just wanted to see, if it could display a whole 2 TB HD with all folders and subfolders in one display. And it could, it just crashed when I selected the "Trash"-ones. I hope I will find some action in future where the mac pro would overtake the mac mini.
Regards
John
Try crunching RAW files. I would be surprised if you didn't see a difference...
Marc