Unable to open backup catalog

[KeithRJ]

Copying to the same drive works fine but I have tried a couple of different USB drives which both fail.

My original Catalog is on an external USB SSD drive and I copy to another USB normal mechanical drive.

What is the format of the USB drives that fail?

-Kirk

Original disk (external USB SSD drive) = exFAT
Backup drives = NTFS

[KeithRJ]

What is the format of the USB drives that fail?

Also, Keith, what happens if you try to create a NEW catalog on one of these drives?

Creating a new Catalogs works fine and I can even copy it to my SSD drive and open it without any problems.

[Kirk Baker]

Keith,

It's starting to sound like an ACL/Permissions problem is arising when copying from exFAT to NTFS.

-Kirk

[KeithRJ]

Keith,

It's starting to sound like an ACL/Permissions problem is arising when copying from exFAT to NTFS.

-Kirk

I also thought that and checked all permissions and found that the Catalog files were all fully accessible to everyone. I will see if I can test other options and let you know if I find anything.

[KeithRJ]

OK, done some testing and found a few issues:

1. I did not copy the "proxies" folder to my backup disk and that causes the failure. If I create an empty folder named "proxies" then the catalog opens fine.

2. When you "Forget" a catalog it remains open and the .shm and .wal files remain.  Only way to close the catalog is to quit PMP. 

Please closes all databases that have been forgotten or are not in use (no check marks for Search or Add/Modify).

3. After opening my backup catalog and re-integrating it, it points to my original indexed folders.  This is not what I want.  I want it to point to my backup folders.

Can you please give an option to choose where the backup folders are or have an option to use relative paths based on catalog location?  i.e. If catalog is in folder E:\2020\PMP and photos are in E:\2020\Photos then the relative paths for all photos would be ..\Photos.

I want to be able to access backup photos and catalogs without worrying that the catalog is pointing at the wrong photo directories.

Just for comparison sake, I used to do this exact action with Lightroom and their catalogs without any issue - the backup catalog used the correct photos directories as long as they were in the same relative directory to the catalog.

I hope this helps and does not give you too many headaches!

[Kirk Baker]

Keith,

OK, done some testing and found a few issues:

1. I did not copy the "proxies" folder to my backup disk and that causes the failure. If I create an empty folder named "proxies" then the catalog opens fine.

We will cause the proxies folder to be created if it doesn't exist.

2. When you "Forget" a catalog it remains open and the .shm and .wal files remain.  Only way to close the catalog is to quit PMP. 

Please closes all databases that have been forgotten or are not in use (no check marks for Search or Add/Modify).

We will make it so forgetting a catalog will force-close the database.  We won't be forcing it closed when the Search:Add/Modify check boxes are unchecked though.

3. After opening my backup catalog and re-integrating it, it points to my original indexed folders.  This is not what I want.  I want it to point to my backup folders.

Can you please give an option to choose where the backup folders are or have an option to use relative paths based on catalog location?  i.e. If catalog is in folder E:\2020\PMP and photos are in E:\2020\Photos then the relative paths for all photos would be ..\Photos.

I want to be able to access backup photos and catalogs without worrying that the catalog is pointing at the wrong photo directories.

The catalog system tracks your images based on the relative location of the "sharepoint" (.pmshare) files it creates on your drives.  It tries to create these sharepoints as close to the root of the drive as possible.  The paths tracked are relative to that sharepoint.  This allows us to track your files even if the path to the drive changes.  User-defined relative-pathing is not something that we're going to offer.

How is your backup made?  Is it a clone of the drive or are you copying portions of the source drive to the backup drive?

-Kirk

[Bill Kelly]

A couple clarifications...

We will make it so forgetting a catalog will force-close the database.  We won't be forcing it closed when the Search:Add/Modify check boxes are unchecked though.

However, we could perhaps auto-close catalogs after a minute-or-so of inactivity.

(Our "Add/Modify" checkboxes sound like a sort of write-protect, but they aren't. So it may be we should devise a better label for them. When an image file on disk is deleted, renamed, or its metadata is modified, all known catalogs referencing that file are updated to reflect those changes, regardless of "Add/Modify" checkbox settings.)

The catalog system tracks your images based on the relative location of the "sharepoint" (.pmshare) files it creates on your drives.  It tries to create these sharepoints as close to the root of the drive as possible.  The paths tracked are relative to that sharepoint.  This allows us to track your files even if the path to the drive changes.  User-defined relative-pathing is not something that we're going to offer.

We do however provide for some customization of this pathing via the "Manage Shared Folders" window.

For example, in this scenario:

◆ If catalog is in folder E:\2020\PMP and photos are in E:\2020\Photos then the relative paths for all photos would be…

Adding the path "E:\2020\Photos" to the list in "Manage Shared Folders", if setup prior to including images in the catalog, will ensure paths stored in the catalog are "relative" to "E:\2020\Photos".

This is accomplished by way of PM Plus placing a ".pmshare" file at that specified location ("E:\2020\Photos").

The ".pmshare" file holds a Universally Unique Identifier (UUID) to which all images in sub-folders will be considered relative.

This means the "E:\2020\Photos" folder can later be moved elsewhere, and so long as its location is updated in "Manage Shared Folders" the catalog still find these files (regardless of where the catalog itself resides.)


Hope this helps,

Bill

[KeithRJ]

Thanks for your explanation Kirk.

How is your backup made?  Is it a clone of the drive or are you copying portions of the source drive to the backup drive?

I have a large backup drive that receives copies of directory trees from my local drive when I decide to make a backup.  I generally work with a folder per year and within this folder is my catalog folder and photos folder among others.  It is these individual year-folders that are backed up.  In summary, each year-folder is a standalone folder tree for that catalog with everything needed to make it portable to another PC or backup drive.

A couple clarifications...

However, we could perhaps auto-close catalogs after a minute-or-so of inactivity.

This would not be necessary, just the "forgotten" catalogs.

(Our "Add/Modify" checkboxes sound like a sort of write-protect, but they aren't. So it may be we should devise a better label for them. When an image file on disk is deleted, renamed, or its metadata is modified, all known catalogs referencing that file are updated to reflect those changes, regardless of "Add/Modify" checkbox settings.)

The catalog system tracks your images based on the relative location of the "sharepoint" (.pmshare) files it creates on your drives.  It tries to create these sharepoints as close to the root of the drive as possible.  The paths tracked are relative to that sharepoint.  This allows us to track your files even if the path to the drive changes.  User-defined relative-pathing is not something that we're going to offer.

We do however provide for some customization of this pathing via the "Manage Shared Folders" window.

For example, in this scenario:

◆ If catalog is in folder E:\2020\PMP and photos are in E:\2020\Photos then the relative paths for all photos would be…

Adding the path "E:\2020\Photos" to the list in "Manage Shared Folders", if setup prior to including images in the catalog, will ensure paths stored in the catalog are "relative" to "E:\2020\Photos".

This is accomplished by way of PM Plus placing a ".pmshare" file at that specified location ("E:\2020\Photos").

The ".pmshare" file holds a Universally Unique Identifier (UUID) to which all images in sub-folders will be considered relative.

This means the "E:\2020\Photos" folder can later be moved elsewhere, and so long as its location is updated in "Manage Shared Folders" the catalog still find these files (regardless of where the catalog itself resides.)

Thanks for the info Bill.

This may still cause problems with my folder structure which is still available on my local drive when I open my backup catalog.  I would also like to be able to have both my original and backup catalog available simultaneously for comparisons etc.  Each catalog would be required to view their own photos folder.

How can I tell the catalog that the photos have moved to another disk?

Any thoughts on this?

I will do some testing to see if I can come up with a way of managing my catalogs.

Thanks for all the help and information.

Regards
Keith

[Bill Kelly]

I would also like to be able to have both my original and backup catalog available simultaneously for comparisons etc.  Each catalog would be required to view their own photos folder.

How can I tell the catalog that the photos have moved to another disk?

It's the 'simultaneously' part that's proving to be tricky.

The Catalog system is trying to be anti-fragile, locating its files even when the volume mount point may have changed. This accommodates scenarios such as the catalog being on a local SSD, with its files on various removable drives whose mount points may change as they are added and removed from the system. ("Oh, the system previously mounted the drive as '/Volumes/HDD 1', and now it's '/Volumes/HDD 2'? Well no problem, we can still identify which drive contains the pictures in this catalog.")

This approach is compatible with a conventional backup and restore scenario, where one wants to recover a catalog from a backup and restore its functionality when the original has been lost or damaged.

But there's currently no way to sort of divorce a backup catalog from its original files, when those original files are still reachable.

The only way I can think of to do that would be to temporarily make the original files unreachable, then run a Reintegrate operation on the backup catalog (in the Maintenance options) to force it to find and retarget its files to the equivalents it can find on the backup drive.


Regards,

Bill

[KeithRJ]

Thanks for the insight Bill.  I will work within this design.

Regards
Keith