Author Topic: Partial Offline Storage Catalog  (Read 2984 times)

Offline mike@breezewaystudio.com

  • Newcomer
  • *
  • Posts: 6
    • View Profile
Partial Offline Storage Catalog
« on: November 12, 2023, 08:16:11 AM »
I'm new to PM+ cataloging, I've done a brief search on this subject to no avail, forgive me if this has been covered.

I've created a catalog of all my files on a NAS. The catalog and it's proxies reside on a local drive. I would like to delete a large portion of the NAS files but still have them in my catalog.

I've created a test catalog and found that if I delete a file in Navigator/Favorites, the proxy is also deleted in the catalog. I've tried several times of restoring the deleted files from the Recycle Bin, rescanning the test catalog, rebooting PM+ and creating a new Organizer "" search. The files are present. I delete as mentioned, create a new "" search, and the files are gone from the catalog.

If I delete the files via Windows Explorer, when I view the catalog the proxies are present with the yellow offline indicator.

Please advise if I'm deleting files in PM+ incorrectly; or if I'm to use, in my case, Bridge to browse and delete files off the NAS.

Many thanks.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Partial Offline Storage Catalog
« Reply #1 on: November 12, 2023, 08:03:28 PM »
I'm new to PM+ cataloging, I've done a brief search on this subject to no avail, forgive me if this has been covered.

I've created a catalog of all my files on a NAS. The catalog and it's proxies reside on a local drive. I would like to delete a large portion of the NAS files but still have them in my catalog.

I've created a test catalog and found that if I delete a file in Navigator/Favorites, the proxy is also deleted in the catalog. I've tried several times of restoring the deleted files from the Recycle Bin, rescanning the test catalog, rebooting PM+ and creating a new Organizer "" search. The files are present. I delete as mentioned, create a new "" search, and the files are gone from the catalog.

If I delete the files via Windows Explorer, when I view the catalog the proxies are present with the yellow offline indicator.

Please advise if I'm deleting files in PM+ incorrectly; or if I'm to use, in my case, Bridge to browse and delete files off the NAS.

If you delete from within PM, the catalog will be updated to remove the deleted files.

Can I ask what it is that you're trying to accomplish by deleting the originals in such a way as the state of your folders is not tracked by the catalog?  If your catalog were ever to fail (and you had no backup) then you could never rebuild your catalog (the files are gone).

Also, you cannot edit the metadata of images that are inaccessible to the cataloging system.  You can search and preview, but that's it.

-Kirk

Offline mike@breezewaystudio.com

  • Newcomer
  • *
  • Posts: 6
    • View Profile
Re: Partial Offline Storage Catalog
« Reply #2 on: November 13, 2023, 11:56:24 AM »
Thanks for the reply Kirk.

I've got nearly 70 TB on a QNAP from 30 years of commercial/editorial work. I have two sets of offline backups of this material. Most of these assignments have no real stock photography potential, so I want to get them off the server, but I do want to be able to query the work.




Offline DavidHoffmanuk

  • Sr. Member
  • ****
  • Posts: 315
    • View Profile
Re: Partial Offline Storage Catalog
« Reply #3 on: November 14, 2023, 02:08:11 AM »
I had a similar need on a smaller scale and converted the files to very small, low quality jpg files around 50KB & that reduced the files sufficiently.

David Hoffman

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Partial Offline Storage Catalog
« Reply #4 on: November 14, 2023, 09:28:57 AM »
Thanks for the reply Kirk.

I've got nearly 70 TB on a QNAP from 30 years of commercial/editorial work. I have two sets of offline backups of this material. Most of these assignments have no real stock photography potential, so I want to get them off the server, but I do want to be able to query the work.

It would be better to take one of the backups online, catalog the images, and then take the backup back offline.  At least then the paths would be valid (though offline) and you wouldn't be deleting anything.

-Kirk

Offline mike@breezewaystudio.com

  • Newcomer
  • *
  • Posts: 6
    • View Profile
Re: Partial Offline Storage Catalog
« Reply #5 on: November 14, 2023, 02:04:24 PM »
I understand, but the server is almost full, something has to go. ;)

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Partial Offline Storage Catalog
« Reply #6 on: November 14, 2023, 02:29:30 PM »
Mike,

I understand, but the server is almost full, something has to go. ;)

OK, but what then do the backups represent?

-Kirk

Offline mike@breezewaystudio.com

  • Newcomer
  • *
  • Posts: 6
    • View Profile
Re: Partial Offline Storage Catalog
« Reply #7 on: November 14, 2023, 08:17:33 PM »
They're backups of everything on my server.

I think it's very simple. My server is full, I've built a catalog of everything on my server. I'm going to use Bridge to delete a good portion of those files off the server to free up room for future work. If I query the catalog and a result has the yellow offline indicator, I note the job# and fetch the proper offline drive for retrieval.






Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Partial Offline Storage Catalog
« Reply #8 on: November 14, 2023, 10:35:56 PM »
Mike,

They're backups of everything on my server.

I think it's very simple. My server is full, I've built a catalog of everything on my server. I'm going to use Bridge to delete a good portion of those files off the server to free up room for future work. If I query the catalog and a result has the yellow offline indicator, I note the job# and fetch the proper offline drive for retrieval.

OK, understood.  And you understand what you're getting into.  Just do the deleting outside of the purview of Photo Mechanic Plus and you should be good to go.  The only failure spot I can see is if your catalog were to fail and you have no backup.  You wouldn't be able to rebuild from the state of the NAS because the files no longer exist at the source.

Backup the catalog regularly and you should be good.

-Kirk

Offline mike@breezewaystudio.com

  • Newcomer
  • *
  • Posts: 6
    • View Profile
Re: Partial Offline Storage Catalog
« Reply #9 on: November 15, 2023, 11:41:55 AM »
Thank you Kirk, much appreciated. And affirmative on backing up the catalog; one copy on the server, another on my older workstation and two offline. For what it's worth, I rotate my drives by age. One backup set is on newer 14TB drives, the other is 3 years old on 4-6TB drives. In another 2-3 years I'll back up everything to a new set of drives and retire the 4-6TB drives. I think I'm on my fifth set of drives since adopting this strategy.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Partial Offline Storage Catalog
« Reply #10 on: November 15, 2023, 11:44:39 AM »
Thank you Kirk, much appreciated. And affirmative on backing up the catalog; one copy on the server, another on my older workstation and two offline. For what it's worth, I rotate my drives by age. One backup set is on newer 14TB drives, the other is 3 years old on 4-6TB drives. In another 2-3 years I'll back up everything to a new set of drives and retire the 4-6TB drives. I think I'm on my fifth set of drives since adopting this strategy.

You're welcome, that's a sound strategy.

-Kirk