Author Topic: Duplicate Photos in Label Filter or Browse  (Read 5276 times)

Offline damclarkd

  • Newcomer
  • *
  • Posts: 10
    • View Profile
Duplicate Photos in Label Filter or Browse
« on: March 08, 2021, 02:59:02 PM »
Using PM+ 5560 Build on Mac OSX. Trying to move catalog over from LR Classic and have encounter a couple of issues. Sorted some, but this one has me stumped.
I use the Purple Label to denote files I have uploaded to a web site as cover photos. The Purple labels were applied previously in LR. Once integrated into the PM+ catalog, the Label Purple count is correct. 121. However the number of photos in the contact sheet when I double click on that in Browse or Filter is 126 as 5 of the photos are showing up twice. If I show In Context or in Finder, the duplicates point to the same photo.
I have tried a re-index as well as a sync, with the same results. Any ideas on how or why these might show as duplicates and what could be done to clean this error?
Cheers and thanks.


Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Duplicate Photos in Label Filter or Browse
« Reply #1 on: March 08, 2021, 03:39:25 PM »
Using PM+ 5560 Build on Mac OSX. Trying to move catalog over from LR Classic and have encounter a couple of issues. Sorted some, but this one has me stumped.
I use the Purple Label to denote files I have uploaded to a web site as cover photos. The Purple labels were applied previously in LR. Once integrated into the PM+ catalog, the Label Purple count is correct. 121. However the number of photos in the contact sheet when I double click on that in Browse or Filter is 126 as 5 of the photos are showing up twice. If I show In Context or in Finder, the duplicates point to the same photo.
I have tried a re-index as well as a sync, with the same results. Any ideas on how or why these might show as duplicates and what could be done to clean this error?

How many catalogs do you have in Photo Mechanic Plus?

-Kirk

Offline damclarkd

  • Newcomer
  • *
  • Posts: 10
    • View Profile
Re: Duplicate Photos in Label Filter or Browse
« Reply #2 on: March 08, 2021, 03:48:57 PM »
Two.  Only searching the one in question.

Offline damclarkd

  • Newcomer
  • *
  • Posts: 10
    • View Profile
Re: Duplicate Photos in Label Filter or Browse
« Reply #3 on: March 08, 2021, 03:51:08 PM »
Double checked the other catalog and those photos do not exist there..

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Duplicate Photos in Label Filter or Browse
« Reply #4 on: March 08, 2021, 06:31:51 PM »
We think it may have to do with unicode normalization differences.  The umlaut-u in your image's title can be represented multiple ways in unicode.

Could you create a test catalog and only add that one image to it?  First use Scan to Catalog (copy that image in the Finder to a new folder and scan that folder).  Then you could Reveal in Context and select it and do a Catalog->Include in Catalog.  See if that makes a duplicate.  If you can get your test catalog to have a duplicate, could you then share it with us?

To solve the immediate problem, you could select the duplicates and remove them (Catalog->Remove from Catalog).  I expect that the others will stay in the catalog.

-Kirk

Offline damclarkd

  • Newcomer
  • *
  • Posts: 10
    • View Profile
Re: Duplicate Photos in Label Filter or Browse
« Reply #5 on: March 09, 2021, 05:44:32 AM »
We think it may have to do with unicode normalization differences.  The umlaut-u in your image's title can be represented multiple ways in unicode.

Could you create a test catalog and only add that one image to it?  First use Scan to Catalog (copy that image in the Finder to a new folder and scan that folder).  Then you could Reveal in Context and select it and do a Catalog->Include in Catalog.  See if that makes a duplicate.  If you can get your test catalog to have a duplicate, could you then share it with us?

To solve the immediate problem, you could select the duplicates and remove them (Catalog->Remove from Catalog).  I expect that the others will stay in the catalog.

-Kirk

Good detective work Kirk! I was able to easily get the photo to duplicate. I have the one photo catalog in Dropbox. I have sent you a PM with the link. As added info, based on your comment about the umlaut-u, I researched the others that were duplicating. All have special characters in the naming.
i.e. Città Metropolitana di Firenze, Málaga, Bréhémont, Évora
Further research showed that the duplicates are throughout the catalog anywhere special characters were used. I have not yet tested to see if the cause is the filename only, or other metadata. I use GPS locations in naming of the files, so the metadata is building the filenames.
Look forward to your findings.
Thanks,
Dennis

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Duplicate Photos in Label Filter or Browse
« Reply #6 on: March 09, 2021, 09:58:43 AM »
Dennis,

We think it may have to do with unicode normalization differences.  The umlaut-u in your image's title can be represented multiple ways in unicode.

Could you create a test catalog and only add that one image to it?  First use Scan to Catalog (copy that image in the Finder to a new folder and scan that folder).  Then you could Reveal in Context and select it and do a Catalog->Include in Catalog.  See if that makes a duplicate.  If you can get your test catalog to have a duplicate, could you then share it with us?

To solve the immediate problem, you could select the duplicates and remove them (Catalog->Remove from Catalog).  I expect that the others will stay in the catalog.

-Kirk

Good detective work Kirk! I was able to easily get the photo to duplicate. I have the one photo catalog in Dropbox. I have sent you a PM with the link. As added info, based on your comment about the umlaut-u, I researched the others that were duplicating. All have special characters in the naming.
i.e. Città Metropolitana di Firenze, Málaga, Bréhémont, Évora
Further research showed that the duplicates are throughout the catalog anywhere special characters were used. I have not yet tested to see if the cause is the filename only, or other metadata. I use GPS locations in naming of the files, so the metadata is building the filenames.
Look forward to your findings.

Thank you for the sample catalog.  Can you tell me what operations you did?  Were they the ones I directed or did you do other operations?

-Kirk

Offline damclarkd

  • Newcomer
  • *
  • Posts: 10
    • View Profile
Re: Duplicate Photos in Label Filter or Browse
« Reply #7 on: March 09, 2021, 10:17:05 AM »
Just your steps in that order.
Dennis

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Duplicate Photos in Label Filter or Browse
« Reply #8 on: March 09, 2021, 11:19:04 AM »
Dennis,

Just your steps in that order.

What is the file system of the disk containing those files?  APFS?  Mac OS Extended?

You can view the file system of each of your disks in the Disk Utility app in the Applications/Utilities folder.

Thanks,

-Kirk

Offline damclarkd

  • Newcomer
  • *
  • Posts: 10
    • View Profile
Re: Duplicate Photos in Label Filter or Browse
« Reply #9 on: March 09, 2021, 11:50:16 AM »
The originals are on a Synology file server. I believe that uses BTRFS.

Dennis

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Duplicate Photos in Label Filter or Browse
« Reply #10 on: March 09, 2021, 12:04:11 PM »
The originals are on a Synology file server. I believe that uses BTRFS.

Thanks, Dennis.

-Kirk