Author Topic: Auto Update Catalog feature  (Read 983 times)

Offline lanzi

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Auto Update Catalog feature
« on: April 23, 2019, 01:38:21 AM »
Hi Team,
first of all thanks and congrats for starting the beta phase on PMP.
After playing with it for the past few hours and starting a few catalogs my 1st feature request would be an auto update of the catalog target folder.
Meaning if i have a catalog "xyz" cataloging folder "xyz" on my NAS it would be great if PMP would automatically start updating the catalog once stuff is added to folder "xyz". I was looking for that feature but could not find it.

Cheers
Lanzi

Offline timesnaps

  • Newcomer
  • *
  • Posts: 33
    • View Profile
Re: Auto Update Catalog feature
« Reply #1 on: April 23, 2019, 04:49:53 AM »
I would very much second that. (Plus it seems rating, labels, keywords are also not auto-updated)

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2396
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Auto Update Catalog feature
« Reply #2 on: April 23, 2019, 05:20:49 AM »
I would very much second that. (Plus it seems rating, labels, keywords are also not auto-updated)

A future version should have something to improve upon this:
In a future build, we anticipate a faster way to re-scan.

Updates to ratings, labels, etc, should, however propagate. Certainly when updated from within PM (doesn't need to be from the catalog), but I'm pretty sure even when done so outside of PM. It can, however, take some time for the catalog counts for the filter/browser to update themselves. Sometimes collapsing/expanding the “category” helps (imo this is a bug; they should auto update).
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline Matthew

  • Member
  • **
  • Posts: 63
    • View Profile
    • matt-horwood.com
Re: Auto Update Catalog feature
« Reply #3 on: April 24, 2019, 01:04:02 PM »
Hi Team,
first of all thanks and congrats for starting the beta phase on PMP.
After playing with it for the past few hours and starting a few catalogs my 1st feature request would be an auto update of the catalog target folder.
Meaning if i have a catalog "xyz" cataloging folder "xyz" on my NAS it would be great if PMP would automatically start updating the catalog once stuff is added to folder "xyz". I was looking for that feature but could not find it.

Cheers
Lanzi

Yes, this would be useful. Even if it was just one 'hot' folder where any images added were automatically catalogued. I could then export images from ACR/Lightroom to this folder and have them automatically added to the catalog.

Offline Einberger

  • Newcomer
  • *
  • Posts: 9
    • View Profile
Re: Auto Update Catalog feature
« Reply #4 on: April 28, 2019, 11:25:38 PM »
Yes please auto-update the catalog target folder. It is really a key-feature for us...

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20121
    • View Profile
    • Camera Bits, Inc.
Re: Auto Update Catalog feature
« Reply #5 on: April 28, 2019, 11:48:29 PM »
What is this "catalog target folder" that you speak of?  It would be one thing to have something watch a single (or a few folders) for new additions, but we can't be constantly scanning all of your drives for new files being added.  It would perform poorly.

-Kirk

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2396
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Auto Update Catalog feature
« Reply #6 on: April 29, 2019, 04:02:50 AM »
Hi Kirk,

What I think is meant is simply the ability to synchronise the catalog with what is now found on the drive. A very good example what this entails can be found in Lightroom's synchronise folders (especially the separation of the options to add/delete/update). I think the best starting point for this feature would be in the context menu on the folder path entries in the browser.

Does this clarify the request?
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20121
    • View Profile
    • Camera Bits, Inc.
Re: Auto Update Catalog feature
« Reply #7 on: April 29, 2019, 08:45:33 AM »
Hayo,

What I think is meant is simply the ability to synchronise the catalog with what is now found on the drive. A very good example what this entails can be found in Lightroom's synchronise folders (especially the separation of the options to add/delete/update). I think the best starting point for this feature would be in the context menu on the folder path entries in the browser.

Does this clarify the request?

Yes, but what you've described is still a manually triggered process and the thread title is "Auto Update Catalog feature", so I'm wondering what Catalog would be required to monitor for changes in real-time.

-Kirk

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2396
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Auto Update Catalog feature
« Reply #8 on: April 29, 2019, 10:56:16 AM »
I can’t image an automatic update would be a good thing to have; as you rightly mentioned, this would slow down things tremendously.

What I think could be done automatically, is check for metadata updates for files that are “visited” (actively shown in a contact sheet). But as far as I can tell you already seem to be doing that :)
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20121
    • View Profile
    • Camera Bits, Inc.
Re: Auto Update Catalog feature
« Reply #9 on: April 29, 2019, 11:27:50 AM »
Hayo,

I can’t image an automatic update would be a good thing to have; as you rightly mentioned, this would slow down things tremendously.

What I think could be done automatically, is check for metadata updates for files that are “visited” (actively shown in a contact sheet). But as far as I can tell you already seem to be doing that :)

Yes, we do that, but we don't automatically add images that have been added to a folder since it was scanned, even when you browse that folder again.

-Kirk

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2396
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Auto Update Catalog feature
« Reply #10 on: April 29, 2019, 12:16:56 PM »
Yes, we do that, but we don't automatically add images that have been added to a folder since it was scanned, even when you browse that folder again.

Excellent, this is what I noticed and is the automatic behaviour I’d expect. Adding (or removing) files should only ever be triggered manually, I think (and I say that even though I personally will actually always keep them synced).
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline carlseibert

  • Member
  • **
  • Posts: 94
    • View Profile
Re: Auto Update Catalog feature
« Reply #11 on: April 29, 2019, 02:06:55 PM »
ON1 RAW has this feature and I like it (with reservations). They re-scan cataloged folders at program start. It puts quite a hit on CPU load. Fans roar like a DC-3 taxiing across my office. They deprioritize the process if you ask the program to do anything resource consumptive while the scan is going on.

Maybe Scan to Catalog could have Snapshots (which I think it should have anyway) and options to run them at certain times, like at startup, once a day at midnight, once every n minutes. Or continuously, if that woudn't sink the ship.


Offline Einberger

  • Newcomer
  • *
  • Posts: 9
    • View Profile
Re: Auto Update Catalog feature
« Reply #12 on: April 30, 2019, 12:42:41 AM »
What is this "catalog target folder" that you speak of?  It would be one thing to have something watch a single (or a few folders) for new additions, but we can't be constantly scanning all of your drives for new files being added.  It would perform poorly.

-Kirk

So in our case, we do have a folder-system with name 'archive'. There are sub-folders for every year since 1950 in that 'archive'-folder. If the performance is going down when watching all of that folders, maybe there could be an option to choose, what folder should be watched permanently? That would help already...

Offline Joel Goodman

  • Member
  • **
  • Posts: 80
    • View Profile
Re: Auto Update Catalog feature
« Reply #13 on: June 18, 2019, 01:14:23 PM »
I came here this evening to see if anyone else has posted about this.

Obviously I can manually rescan folders myself, but it's not as elegant or complete an experience having to do so and seems like a feature that'd complete "catalogue".

How I'd imagine it working is users would specify folder/s as "active/hot" (as above) and also define a schedule, so that we could choose whether the performance hit is something we want to incur regularly, and when. For instance, I'd set it to run every morning at 4am, on a computer I leave on anyway. If I don't have it on then it'd just pick up at the next 4am when I do have it on. Add a button to force the specified task to run straight away, and I'm good. If you wanted to make it really sophisticated, you could have separate tasks for each collection or "active/hot" folder, specifying to never rescan one collection/folder, monthly for another, daily for another etc. Although I'd be content with the basic single task, I can see value in building in the feature of separate tasks as a future upgrade.

I know nothing about how to programme, so forgive me if this is naive regarding how much resource it'd take. I presume it'd record the last time a scan was carried out and then scan for anything that was added or modified since that date, to only update the database with those files modified since? I've found the scan process impressively fast and the database updates that run in the background will only need to crunch the newly modified files, which I can't imagine being an issue for me at 4am, based on how it runs now, so I don't personally think the resource issue is one I'd be worried about, but maybe I'm missing something obvious in this?

Thx

Offline Joel Goodman

  • Member
  • **
  • Posts: 80
    • View Profile
Re: Auto Update Catalog feature
« Reply #14 on: June 18, 2019, 01:17:35 PM »
Of course, if you really wanted to make it a killer, you could facilitate a server side app for Synology/Drobo/insert popular NAS name. Maybe this is too specific but it would make it competitive with enterprise level image catalogues and more efficient in not requiring a remote computer to run the task.