Catalog Sync window :missing files name are lost when function terminates

[Frederic]

When using 'Catalog sync' function on large catalog, when the function terminates the list of missing files (if any) is 'lost' (outside the scrolling portion of the window), and the log window is full of 'Found .. files: adding to ...'
Impossible to know which files were missing.
I suggest several possibilities:

• missing files name could be written to a disk file
• the lines 'Found .. files ..' are not very informative and could be avoided, just a counter with the number of files found and added would be enough.

These are just suggestions for the solution, but the problem is really there (or I missed something) ...
Thanks

[Kirk Baker]

Frederic,

When using 'Catalog sync' function on large catalog, when the function terminates the list of missing files (if any) is 'lost' (outside the scrolling portion of the window), and the log window is full of 'Found .. files: adding to ...'
Impossible to know which files were missing.
I suggest several possibilities:

• missing files name could be written to a disk file
• the lines 'Found .. files ..' are not very informative and could be avoided, just a counter with the number of files found and added would be enough.

These are just suggestions for the solution, but the problem is really there (or I missed something) ...

We agree that it can be improved. Our plan is to create a dynamic collection that you'll be able to view the thumbnails (if you have proxies generated) for you to review and choose to remove from the catalog or correct their paths and add back in to the catalog.

We will optionally create a dynamic collection that will show the added images so you can decide if you really want them in your catalog or not.

-Kirk

[Frederic]

Kirk,
Thank you for your reply.
What you are planning looks great to me, and I am sure I am using the function properly today.
Frederic.

[Graham1]

We agree that it can be improved. Our plan is to create a dynamic collection that you'll be able to view the thumbnails (if you have proxies generated) for you to review and choose to remove from the catalog or correct their paths and add back in to the catalog.

-Kirk

A simpler one click option to remove all missing files would also be welcome.  If, like me, all your catalogued files are local, you know that any missing files should be removed, without any need to review.  Of course, this would not work well for any catalogue that includes removable media, but a simple remove-all mechanism would be useful for those who know they do not need to review the results.

The bigger issue is the sheer amount of time it takes to scan for missing files: literally hours in the case of my catalogue with 650k or so images.  The only realistic way I can scan for missing images is to leave it running overnight, then make sure I do not accidentally lose the selection in the morning, but even then it takes ages for the removal from catalogue to take effect while the metadata is examined again.  Keeping the catalogue regularly updated to sync with a few known deletions outside of the PM6+ environment is really not a practical proposition, even though routine maintenance such as this should take seconds (or maybe minutes), not hours.

Graham

[Kirk Baker]

Graham,

The bigger issue is the sheer amount of time it takes to scan for missing files: literally hours in the case of my catalogue with 650k or so images.  The only realistic way I can scan for missing images is to leave it running overnight, then make sure I do not accidentally lose the selection in the morning, but even then it takes ages for the removal from catalogue to take effect while the metadata is examined again.  Keeping the catalogue regularly updated to sync with a few known deletions outside of the PM6+ environment is really not a practical proposition, even though routine maintenance such as this should take seconds (or maybe minutes), not hours.

Are you talking about the "Select Missing Images" feature?

It is indeed slow on large catalogs such as yours.  It was never intended to perform well with a contact sheet containing hundreds of thousands of images.  It's done via getting all of the images from the catalog resolved as local images and those that fail are "missing" and are selected.

If you can think of way to make it take seconds, I'm all ears.

-Kirk

[Graham1]

Are you talking about the "Select Missing Images" feature?

It is indeed slow on large catalogs such as yours.  It was never intended to perform well with a contact sheet containing hundreds of thousands of images.  It's done via getting all of the images from the catalog resolved as local images and those that fail are "missing" and are selected.

If you can think of way to make it take seconds, I'm all ears.

-Kirk

Yes, I am.  I am not a programmer, and do not know the art of the possible (or impossible). 

If you carry out a Full Sync, that counts the missing files.  Then, if you select Find ¦ Select Missing for the same entire catalogue, it takes ages to go through the catalogue again to extract (in non-programming terms) the same data.  Would it not be possible somehow to retain that data from the Full Sync which identifies the missing files, and couple this with an option for the immediate removal of those missing files, without having to search all through the metadata and make a new selection all over again?   That would not speed up the Full Sync, but it would make a huge difference to the speed of removal of the missing items from the catalogue, assuming the user is happy to remove the files without further review (which I would be): maybe only a matter of seconds 

Maybe it is not technically possible, but otherwise all a Full Sync is doing (in relation to missing files) is telling you how many missing files there are, not providing a solution for removing them from the catalogue, which may be interesting but it is not particularly useful for the management of the catalogue.

I await being told that this cannot work!   

Graham

[Kirk Baker]

Graham,

Are you talking about the "Select Missing Images" feature?

It is indeed slow on large catalogs such as yours.  It was never intended to perform well with a contact sheet containing hundreds of thousands of images.  It's done via getting all of the images from the catalog resolved as local images and those that fail are "missing" and are selected.

If you can think of way to make it take seconds, I'm all ears.

Yes, I am.  I am not a programmer, and do not know the art of the possible (or impossible). 

If you carry out a Full Sync, that counts the missing files.  Then, if you select Find ¦ Select Missing for the same entire catalogue, it takes ages to go through the catalogue again to extract (in non-programming terms) the same data.  Would it not be possible somehow to retain that data from the Full Sync which identifies the missing files, and couple this with an option for the immediate removal of those missing files, without having to search all through the metadata and make a new selection all over again?   That would not speed up the Full Sync, but it would make a huge difference to the speed of removal of the missing items from the catalogue, assuming the user is happy to remove the files without further review (which I would be): maybe only a matter of seconds 

Maybe it is not technically possible, but otherwise all a Full Sync is doing (in relation to missing files) is telling you how many missing files there are, not providing a solution for removing them from the catalogue, which may be interesting but it is not particularly useful for the management of the catalogue.

As I explained to Frederic, a solution is in the works that will create a temporary collection that you can use to review the new images and another temporary collection that will allow you to review the missing files.  Viewing either of these collections will be optional and up to you.  The collections should present themselves briskly and you should be able to select images and decide what to do with them.  If you open the missing collection and select all and then use Remove From Catalog, it should take mere moments to accomplish.

I think this will be a good solution that will take the tedium out of dealing with missing images.  And if you see images that you didn't want in your catalog, you can remove them as well.

You won't need to use the "Select Missing Items" command at all.  Just open the "Missing" collection.  All of the images will be missing images.  Should you decide you want to track them down and add them back to the catalog (assuming that they weren't automatically found in their new location) then you can do that as you see fit.

-Kirk

[Graham1]

Thanks, Kirk.  Sounds good.

Graham