PM6+ stalling when importing new images to existing database

[Joel Goodman]

EDIT: issue may be resolved.



Win 7 Pro 16GB RAM, latest version of PM6+

Long description of what's occurring here, hoping it'll help nail the issue.

Last night I added four new folders to an exiting database to scan. PM scanned them no problem. Metadata Updates ran fine. But Catalog Metadata Gathering just stopped at preparing. I killed PM6+ a couple of times and it once showed the total remaining at 5,270 and another time reverted to showing preparing. After another restart and it showing 5270 remaining, I left it overnight.

By this morning it was still stuck at 5270 so I closed PM. In task manager after closing I could see a single process, pm-task.exe (description Ruby interpreter (CUI) 2.2.6p384 [x64-mswin64-100], still running and using close to 80% of the available memory.

With PM6+ running I see four of the Ruby interpreter PM-tasks running, one of which uses rapidly increasing memory, the other three remain relatively low. If I close PM6+, the high consumption task remains running at the same high level as when I closed it. If I try to kill the process entirely in task manager, the memory use drops to close to zero but the process remains active and will not close.

If I try to kill it again I get "The operation could not be completed access is denied." It never stops.

I've tried this repeatedly and get the same pattern each time.

Going to reboot the entire computer shortly but will wait for any feedback here first.

Thanks.

[Joel Goodman]

Update: I think I may have resolved for now. I'll leave the post here in case I need to return to this. Will post an update with details which may help, when confirmed. Thanks.

[Joel Goodman]

An update.

Based on what I'd asked previously here: http://forums.camerabits.com/index.php?topic=12562.msg61742#msg61742 , I've been trying to find a workaround to enable me to access the PM6+ database, which I have stored on a NAS, from more than one computer on my network (not simultaneously), without damaging the database, as Kirk has previously advised was a risk.

I have no expertise in database design.

I thought I'd try to set a limit on the NAS (Synology) which would lock access to the folder in which the db file is stored to a single user at a time, but there is no means to do this within their interface.

I asked in a Synology forum whether any users had ever tried to limit folder access in this way and, in discussion I described the problem in this post (without naming the application).

The problem in this post resolved when I closed PM6+ on a 2nd computer which, although not in the process of writing/reading from the database, had an open search results window displayed. As soon as I closed that off, the computer on which I was trying to update the database was able to start and complete the scan without issue and the increasingly memory-hungry process that wouldn't die or be closed, closed fine.

I didn't specify that it was PM, just that it was built in SQLite and someone suggested "They are probably holding on to the results cursor when displaying the search results instead of consuming the results and closing the cursor".

Does this make any sense?

Obviously I'm still keen to find a way to access the one database from two different computers on the same network (which I know you've specified wasn't part of the plan), but I'm also hoping to have PM6+ work fluently in my workflow, even if I have to build multiple instances of the database on each computer connecting to the files, or maybe replicate the database file across different computers.

At the very least I thought I should provide a possible explanation for the problem I experienced and what I'm trying to do.

Thanks.

[Bill Kelly]

Hi Joel,

I didn't specify that it was PM, just that it was built in SQLite and someone suggested "They are probably holding on to the results cursor when displaying the search results instead of consuming the results and closing the cursor".

It's a bit more complicated than that, as our catalog database module caches various results for speed. So if two separate instances of Photo Mechanic tried to perform operations directly on the same SQLite catalog file their cached data would become out of sync, and lead to returning incorrect results.

(It's on our TODO list to detect when such attempts at concurrent file access occur, and provide feedback to the user that this mode of multi-instance-direct-access is not supported.)


However: PM's Catalog system was in fact designed around a client/server architecture. It allows one computer to act as the server, while other instances of PM on your network talk to that server to access its catalog(s). With this approach, multiple PM clients may safely access a single catalog concurrently.

This functionality exists, but is disabled for now, as it is not user-friendly to configure in its current form. (One would need to start the server via a command line script, and type in network IP addresses to point instances of PM at the server; etc.) Whereas we'd like it to be closer to a "Start LAN Server" button on one end, and "Scan for LAN Servers" on the other.

I anticipate this feature may make its debut in "beta" form at some point, but this will likely come after our official v1.0 release (due to the need to prioritize core features.)


Regards,

Bill