Search query update

[adrianlambert]

I type in a search query and press enter, then realise I've made an error - correct the error, then press enter again. The old query is still working away for some time so I close the search tab. A new untitled 1 tab appears. I press enter again to submit my query but nothing happens. Untitled 1 is the only tab showing.

[Kirk Baker]

Adrian,

I type in a search query and press enter, then realise I've made an error - correct the error, then press enter again. The old query is still working away for some time so I close the search tab. A new untitled 1 tab appears. I press enter again to submit my query but nothing happens. Untitled 1 is the only tab showing.

Could I have you turn on Catalog debug logging and try again, please?  From the Help menu, choose "Edit Debug Settings..." and find the item titled CATALOG_DEBUG_LOGGING and set its checkbox.  Then click the Close button.  Then restart Photo Mechanic Plus and reproduce the problem.  From the Help menu choose "Reveal Support Data..." and moments later a Finder window will come forward with the zipped log file selected.  Use the 'Attachments and other options' link when you're composing your reply to this message and there you'll be able to upload your zipped PM log file.

Thanks,

-Kirk

[adrianlambert]

Debug log attached

[Bill Kelly]

Hi Adrian,

I type in a search query and press enter, then realise I've made an error - correct the error, then press enter again. The old query is still working away for some

Thanks for the debug logs.

It appears the missing closing double-quote on the first of the two paths in the following expression produced a very long-running search:

path "manchester" AND (path ^"/Volumes/ALP Archive 01/ OR path ^"/Volumes/ALP Archive 02")

Unfortunately, we don't currently have a solution, but we'll explore ways to identify when a search is taking a long time in the database engine so that we can provide an option to cancel it.

Thanks for bringing this to our attention.

Regards,

Bill

[adrianlambert]

I'm seeing the same behaviour when I use browse or filter and choose a parent folder with a lot of images (90,000). I clear the browse or filter query, and try to make a new query but nothing happens. It appears to have hung. No notifications as to what might be happening or if there's an error.

[Bill Kelly]

I'm seeing the same behaviour when I use browse or filter and choose a parent folder with a lot of images (90,000).

If you haven't already, please try the "Optimize SQL Query Planning" button in the Maintenance options for your catalog(s) in the Manage Catalogs pane. It should take a few minutes on a larger catalog.

If that doesn't help, we'll need to troubleshoot further.

[adrianlambert]

About an hour into the SQL analyse now... (210,000+ images). Bit more than a few minutes. Sounding normal?

[Bill Kelly]

About an hour into the SQL analyse now... (210,000+ images). Bit more than a few minutes. Sounding normal?

Strange.

The analyze function here on a million-item catalog takes 87 seconds. (Granted, on a very fast SSD.)

On what type of disk does your catalog reside?

[adrianlambert]

It’s an SSD albeit a 2.5” model on the 2012 Mac mini quad i7 W 16gb ram. Catalog is on what used to be the scratch disk. It has lots of space free.

[Bill Kelly]

I just ran the analyze function on a million-item catalog which is stored on an external spinning platter HDD over a USB interface, and even that completed in about 187 seconds.

So, something indeed peculiar is going on with the Catalog system on your computer.

Since you're on a Mac, it's likely the "sqlite3" command is built-in. We'll write up a follow-up post containing some diagnostic commands we'd like you to try running on your catalog from the Terminal app.

[Kirk Baker]

Adrian,

Here is what we need you to do.

First quit Photo Mechanic Plus.  Make sure there are no lingering pm-task processes running.  Open the Activity Monitor and click on the "CPU" tab.  Then in the search field at the right side of the toolbar enter: pm-task

If you see any listed, click on them one-by-one and click the Stop button (looks like a hexagon with an X inside it.)  You can now quit the Activity Monitor.

Now open a Terminal window.  The Terminal application is in the /Applications/Utilities folder.  When the window opens type 'cd ' without the quotes.  That's the letter c and d followed by a space.

Then in the Finder, find your catalog folder and drag and drop it onto the Terminal window.  Next click in the Terminal window and press enter.  Your terminal session will now have its current directory inside your catalog folder.

Next copy and paste the following into your Terminal window, and press enter:

time sqlite3 catalog.pmdb "pragma integrity_check"

When that finishes, copy and paste the following into your Terminal window, and press enter:

time sqlite3 catalog.pmdb "analyze"

When that finishes, copy the output of both commands and paste them in your reply to this post.

Thanks,

-Kirk

[adrianlambert]

ALP-MacMini:~ adrian$ cd /Volumes/PS/PM_Catalogs/ALP
ALP-MacMini:ALP adrian$ time sqlite3 catalog.pmdb "pragma integrity_check"
ok

real   6m39.322s
user   3m8.062s
sys   0m56.727s
ALP-MacMini:ALP adrian$ time sqlite3 catalog.pmdb "analyse"
Error: near "analyse": syntax error

real   0m0.009s
user   0m0.002s
sys   0m0.006s
ALP-MacMini:ALP adrian$

[Graham1]

To add to the comments about SQL analyze timing, mine (on a 660,000 image catalogue after a reboot to make sure there were no stray processes running) took 1284 seconds (over 21 minutes) on a fast Windows 10 system with the catalogue and cache on a fast internal SSD.

I reran the analysis a few hours later with other applications running.  The timing was almost identical (1281 seconds).

Graham

[adrianlambert]

Thanks. You just made me actually read what I posted! I’ll rerun the commands and repost.

[adrianlambert]

ALP-MacMini:~ adrian$ cd /Volumes/PS/PM_Catalogs/ALP
ALP-MacMini:ALP adrian$ time sqlite3 catalog.pmdb "pragma integrity_check"
ok

real   7m16.892s
user 3m25.443s
sys   0m58.508s
ALP-MacMini:ALP adrian$ time sqlite3 catalog.pmdb "analyze"

real   3m27.502s
user 0m55.285s
sys   0m28.521s