Author Topic: Checking catalog status  (Read 3764 times)

Offline adrianlambert

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Checking catalog status
« on: February 28, 2021, 03:26:23 PM »
During the preview generation phase of my initial ingest of 250,000 images the process was having issues at around 4000 to go. I was able to see this using the task stated window. I clicked on "Stop Preview Generation Tasks" then went to the directory that had the images in that the process was stuck on and chose to exclude it from the scan. The Scan to Catalog dialog appeared with the folder listed in the Folders to Exclude pane. I then clicked Start. The metadata scanning process then started from what appeared to be right from the start saying there were several days until it'd be complete. Soon after it started, this process seemed to stop again. So I chose to stop this task from the Catalog Status pane. I'm a little confused now as to where my catalog is upto. Do I need to start from the start again? Is there a way for me to just ask it to generate previews for the 4000 or so images that haven't been previewed. Do I need to do this or will it happen on the fly anyway? I feel that I'm not getting clear feedback from the app as to what's going right and what's going wrong. And where it's up to at this very moment in terms of completeness.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Checking catalog status
« Reply #1 on: March 01, 2021, 11:26:50 AM »
Adrian,

During the preview generation phase of my initial ingest of 250,000 images the process was having issues at around 4000 to go. I was able to see this using the task stated window. I clicked on "Stop Preview Generation Tasks" then went to the directory that had the images in that the process was stuck on and chose to exclude it from the scan. The Scan to Catalog dialog appeared with the folder listed in the Folders to Exclude pane. I then clicked Start. The metadata scanning process then started from what appeared to be right from the start saying there were several days until it'd be complete. Soon after it started, this process seemed to stop again. So I chose to stop this task from the Catalog Status pane. I'm a little confused now as to where my catalog is upto. Do I need to start from the start again? Is there a way for me to just ask it to generate previews for the 4000 or so images that haven't been previewed. Do I need to do this or will it happen on the fly anyway? I feel that I'm not getting clear feedback from the app as to what's going right and what's going wrong. And where it's up to at this very moment in terms of completeness.

Scan to Catalog is basically starting over again.

For an existing Catalog that needs updating, use Catalog Sync with the Full Sync option.  It should generate any missing previews.  Those that cannot be generated earlier will be attempted again but will still likely fail unless the problems with those images have been corrected in the meantime.  (For instance you had posted a DNG file that had no preview.  If you ran that file through Adobe's DNG Converter and had it embed a preview, then I'd expect the preview to be generated during the Full Sync.)

-Kirk

Offline adrianlambert

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Checking catalog status
« Reply #2 on: March 01, 2021, 01:17:09 PM »
Thanks. I actually export lossy DNG's, with no preview image so as to minimise file size, of every capture I make and output them to a staging drive from which they are backed up to Backblaze. Metadata is preserved of course. This is my last ditch off-site back-up.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Checking catalog status
« Reply #3 on: March 01, 2021, 01:35:11 PM »
Adrian,

Thanks. I actually export lossy DNG's, with no preview image so as to minimise file size, of every capture I make and output them to a staging drive from which they are backed up to Backblaze. Metadata is preserved of course. This is my last ditch off-site back-up.

OK, they won't be supported in Photo Mechanic 6/Photo Mechanic Plus.

-Kirk

Offline adrianlambert

  • Full Member
  • ***
  • Posts: 105
    • View Profile
Re: Checking catalog status
« Reply #4 on: March 01, 2021, 01:46:13 PM »
I chose to do this purely to be able to upload them as efficiently as possible whilst maintaining only information that couldn't be remade - Pixels and XMP metadata. The raw captures and tiff masters were consistently falling behind the rate at which I was generating media. Now as lossy DNGs they are always ahead. I guess I can switch the preview generation back on.
« Last Edit: March 01, 2021, 02:58:05 PM by adrianlambert »