Author Topic: moving a Catalog with external Proxy Storage  (Read 2229 times)

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
moving a Catalog with external Proxy Storage
« on: April 22, 2022, 01:09:52 AM »
Hi,

for very large cataloges I use the function to save the proxy files externally. This works wonderfully.

Now I have the question how to move these catalogs with a external proxy storage to an other computer?
First I choose a external location for the proxy storage on the other computer.
Then I copy the proxies data there (it is difficult to find the right files, because the proxies are not stored under the catalog name, instead the folder has a long number).
The Catalog Database is copied, I select "locate  existing catalog" and make a "reintegration of this forgotten catalog".
Everything ist fine, there are no errors and the catalog is displayed but no previews.

Only when I choose that the proxies should be saved internally again, they are copied and then they are also displayed.
But this results in a mismatch:
the proxies are now stored internaly, but the catalog management still shows "external storage".
It is now impossible to change this.
On the old external location a dadabase called "cache-index.sq3-wal" is left.
(in my example I moved a catalog from a Windows 11 System to macOS 11.6.5, both PM+ build 6424)

How is the correct procedure to move a Catalog with external Proxy Storage?

Thanks
Thomas
« Last Edit: April 22, 2022, 04:03:23 AM by bildbaendiger »

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #1 on: April 22, 2022, 08:14:43 AM »
Thomas,

It would have been better to move the proxies back to internal, then move the catalog to the new system and then make them external again.  I don't have a solution for your problem and I will have to research it next week and get back to you.  The reason the external proxies folder has an unfriendly name is because it is an ID that the catalog knows about and was never intended for users to have to interact with directly.

If you still have the ability to undo what you did and then make them internal again (from within PM Plus), then you can move the catalog with the proxies to the other system and then make them external again.

-Kirk

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
Re: moving a Catalog with external Proxy Storage
« Reply #2 on: April 22, 2022, 10:10:32 AM »
Thanks Kirk,

I tried it, unfortunately it doesn't work anymore if the catalog had stored the proxies externally at some point.
The catalog stores the original external location, even if the proxies are stored internally again.
When I try to integrate this catalog (with proxies stored internal) on my other computer, I get a proxy location error message (see screenshot) that cannot be resolved.
With the "Fix this issue ..." button I can choose the Location for external proxies (in the catalog preferences), but none is accepted.

after some searching, the problem seems to be already with the location selection:
On the Windows 11 machine (PM+ build 6424) I have a Catalog with a proxy location external.
I now choose internal and the proxies are copied back again.
Everything is fine. The Proxy Storage shows the right selection: "Store with catalog".
I quit PM and start again. The catalog works normal, but when I now go to the Proxy Storage, it shows the wrong option: "Use external storage"
And this conflict is no longer solvable. This seems to be a bug.

Thomas
« Last Edit: April 22, 2022, 10:38:07 AM by bildbaendiger »

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #3 on: April 22, 2022, 10:48:45 AM »
I may need for you to send me the catalog.pmdb file to fix it.

-Kirk

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
Re: moving a Catalog with external Proxy Storage
« Reply #4 on: April 22, 2022, 12:29:54 PM »
Hi Kirk,
here comes the file

Thomas

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #5 on: April 22, 2022, 03:07:55 PM »
Thomas,

I have attached the updated file which makes the proxies internal again.  It's a zip file so you'll have to un-zip it to get the catalog.pmdb file out of it.

-Kirk

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
Re: moving a Catalog with external Proxy Storage
« Reply #6 on: April 23, 2022, 01:25:10 AM »
Hello Kirk,

oh, I didn't expect you to fix it.
Many thanks for that, it worked!

I had specially used a small catalog for testing. What do I do with the really big catalogs? Some are now on external storage and I would like to avoid this error with these catalogs. There is no hurry, I do not need to move these catalogs at the moment. I can certainly wait for an update.

thanks again,
Thomas

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #7 on: April 23, 2022, 09:45:30 PM »
Thomas,

I had specially used a small catalog for testing. What do I do with the really big catalogs? Some are now on external storage and I would like to avoid this error with these catalogs. There is no hurry, I do not need to move these catalogs at the moment. I can certainly wait for an update.

I'm not sure what to fix.  If you manipulate the external proxies outside of Photo Mechanic Plus, there's not much I can do to deal with that.  Move the proxies internal, move the catalog to the other system (forget it on the old system, locate existing on the new system), then make the proxies external again.

I expect that to work correctly.

-Kirk

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
Re: moving a Catalog with external Proxy Storage
« Reply #8 on: April 24, 2022, 02:01:09 AM »
Hello Kirk,

there is a basic error with the external proxy storage (without any manipulation).
It happens on both mac and windows systems:

- Choose for a Catalog the proxy location external. The Proxies are copied, everything is fine.
- now choose internal and the proxies are copied back again. Everything is fine too.
  The Proxy Storage shows the right selection: "Store with catalog".
- Now quit PM and start again.
- The catalog works normal, but now the Proxy Storage shows the wrong option: "Use external storage"
  This conflict is no longer solvable.

This happens with the normal use of the Proxy Storage option without any manipulation.

Thomas

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #9 on: April 25, 2022, 12:05:51 PM »
Thomas,

there is a basic error with the external proxy storage (without any manipulation).
It happens on both mac and windows systems:

- Choose for a Catalog the proxy location external. The Proxies are copied, everything is fine.
- now choose internal and the proxies are copied back again. Everything is fine too.
  The Proxy Storage shows the right selection: "Store with catalog".
- Now quit PM and start again.
- The catalog works normal, but now the Proxy Storage shows the wrong option: "Use external storage"
  This conflict is no longer solvable.

This happens with the normal use of the Proxy Storage option without any manipulation.

Thanks.  We're able to reproduce the problem and have added the issue to our bug tracking system.  A future build (not the next one) will fix this problem.  If you're interested in testing a solution, I can provide you an interim build to test with.

-Kirk

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
Re: moving a Catalog with external Proxy Storage
« Reply #10 on: April 26, 2022, 02:23:51 AM »
Thanks Kirk,

I like to test.

I also noticed the following:
When such a catalog is deleted permanently (Proxy storage was changed), folders with the label .partially_deleted remain behind (catalog_name.partially_deleted as well as catalog_uuid.partially_deleted).
Sometimes they are empty, sometimes they contain an empty Folder "proxies"

Thomas


Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #11 on: April 26, 2022, 06:36:12 AM »
Thomas,

I also noticed the following:
When such a catalog is deleted permanently (Proxy storage was changed), folders with the label .partially_deleted remain behind (catalog_name.partially_deleted as well as catalog_uuid.partially_deleted).
Sometimes they are empty, sometimes they contain an empty Folder "proxies"

Can you please post a screenshot that illustrates what you're describing?

Thanks,

-Kirk

Offline bildbaendiger

  • Member
  • **
  • Posts: 67
    • View Profile
    • bildbaendiger
Re: moving a Catalog with external Proxy Storage
« Reply #12 on: April 26, 2022, 07:06:43 AM »
Hi Kirk,

here is a screenshot.
For testing I create a catalog named "ProxyTest" and choose for the external storage location an other folder "PM Proxies".
When I permanently delete the catalog "ProxyTest" you get these folders (ProxyTest.partially_deleted and 4d65d9e3-cc38- ... .partially_deleted) left.

Thomas

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24731
    • View Profile
    • Camera Bits, Inc.
Re: moving a Catalog with external Proxy Storage
« Reply #13 on: April 26, 2022, 09:13:08 AM »
Thomas,

here is a screenshot.
For testing I create a catalog named "ProxyTest" and choose for the external storage location an other folder "PM Proxies".
When I permanently delete the catalog "ProxyTest" you get these folders (ProxyTest.partially_deleted and 4d65d9e3-cc38- ... .partially_deleted) left.

OK.  Photo Mechanic Plus was unable to delete those folders at the time of deleting the catalog.  You can delete them manually from the Finder.

-Kirk