Author Topic: Extremely slow deletes/updates & repeated apply change journal batch timeouts  (Read 65 times)

Offline DavidHoffmanuk

  • Sr. Member
  • ****
  • Posts: 325
    • View Profile
I wrote this last month but didn’t post it due to lack of time & having found a workaround (as below) in always waiting for each process to finish before beginning another. I also had to stop using collections, managing  with colour labels & only making large collection, removal & deletion operations overnight. I’m posting it now in the hope that it’ll be of help to others & perhaps bring out some insights that will allow PM+ to work better with large batches.

I’m seeing severe performance issues with Photo Mechanic Plus catalog updates on a Mac Studio, even in a brand‑new catalog, and I’d like to know if anyone else has run into something similar.

Environment

Mac Studio M2 Max, 64 GB RAM

macOS 15.6.1

Catalogs on internal 2 TB SSD

Image files on fast TB4 NVMe drives

Photo Mechanic Plus PM6+ (current build)

What works well

In both my original catalog (~350k images) and a new catalog (~230k images), all of the following are very fast:

Catalog searches

Browsing contact sheets

Batch renaming files

Applying metadata templates

Applying colour labels/ratings to thousands of images at once

These operations complete in seconds and PM+ stays responsive.

Where the problems are

The performance issue appears specifically when the catalog’s membership changes:

Deleting files from the catalog

Removing files from the catalog (without deleting on disk)

Adding/removing large numbers of images in collections

For example, in the new catalog (~230k images):

I added a colour tag to ~8,800 images (instant).

Deleting that colour‑tagged set from a simple contact sheet (no collections involved) took about 17 hours to finish.

In my original catalog (~350k images), deleting a few hundred images or adding/removing large sets from collections regularly causes very long “Updating 1 catalog, n batches remaining” periods (tens of minutes or more). During this process contact sheets are either frozen or show black or blank squares.

During these operations, the Catalog Status window shows many repeating errors like:

Error: RPCServerConn.rpc_dispatch: exception: CILA::PROTO::RPCException/apply_change_journal_batch failed: [:rpc_timeout]
Error: CatalogMetadataUpdateTask.try_spawn_metadata_update: apply_change_journal_batch failed: [:rpc_timeout]
Error: RPCServerConn.rpc_dispatch: exception: CILA::PROTO::RPCException/count_documents failed: [:rpc_timeout]

So:

apply_change_journal_batch keeps timing out.

CatalogMetadataUpdateTask keeps failing to spawn metadata updates.

Even query_num_documents (count_documents) sometimes times out.

This is happening in two separate catalogs on the same machine:

my long‑lived ~350k‑image catalog, and

a new ~230k‑image catalog built from scratch with the old one disabled while scanning.

What I’ve already tried

Ensured catalogs are on the internal SSD; photos on NVMe.

Reduced disk cache and render cache to more modest sizes (helped open/quit speed, didn’t fix deletes).

Deleted the catalog state files (cat_state… plus -wal and -shm) for the original catalog.

Created a new catalog and rescanned all drives into it.

Performed problematic deletes from simple contact sheets (plain search / folder results, not collections).

The core problem remains: large deletes and large collection membership changes are extremely slow and flood Catalog Status with apply_change_journal_batch and count_documents timeouts, even in the new catalog.

Current workaround

I’ve ended up:

Using colour labels to mark images for removal/deletion.

Doing all “lightweight” work (search, metadata, renaming, tagging) during the day.

Running the actual deletes/collection updates in big batches overnight.

This works, but it’s far from ideal.

Questions for the community

Has anyone else on Apple Silicon / recent macOS seen similar apply_change_journal_batch and count_documents [:rpc_timeout] behaviour when deleting large sets or changing collection membership?

If so, did you find any specific triggers (particular metadata, file types, catalog options) or workarounds beyond “keep batches tiny”?

Are there any catalog maintenance steps (beyond deleting the state file and the standard maintenance commands) that improved this for you?

Thanks in advance for any insights.

David