Author Topic: Error in Licensors metadata fields?  (Read 3819 times)

Offline Bob Hendricks

  • Member
  • **
  • Posts: 82
    • View Profile
Error in Licensors metadata fields?
« on: June 30, 2021, 11:44:55 AM »
I think there may be an error in the new Licensor metadata Postal Code data field.

The objective:
I wanted to create a licensor that would replace the metadata Contact info because (a) Contact does not have a person's name and (b) because the Contact info is non-searchable.

Here is what I did: I entered the variable names for the Contact into the appropriate fields for the Licensor. e.g., I entered  {cizip} in the Postal Code for the Licensor. Note that this is a typo--the correct variable name is {cizp}. I did the same thing for all of the Contact variables, but spelled them correctly.

The result: When the metadata template was run, the misspelled variable name was treated a text string and the word "{cizip}" was entered into the Licensor Postal Code 19092 times. All of the other variables transferred the correct Contact information to their respective Licensor fields correctly.

When the mistake was discovered, the 19092 images were selected and the correct Zip Code was entered into the Licensor Postal Code. All other Licensor fields were cleared. The concatenate checkbox for the Licensor fields was selected.  As I understand it from tech support, with the checkbox selected and the field left blank, all of the information in the respective Licensor fields  should not be changed, but a blank be concatenated.  Thus, the only change I expected was that all of the Postal Codes with "{cizip}" should have been changed to 01960, the correct Postal Code.

The result:  Nothing happened--the Postal Code remains "{cizip}"

In an attempt to correct this result, the concatenate box was unchecked and the correct data (and not their variable names) were entered into each Licensor field.Every image in the collection was selected (some 36,000 images) and the template was reapplie to every image.  The Postal Code was not corrected. This run took about 4 hours because all of the images are on an external Thunderbolt drive.

Do I misunderstand something, or is there an error in the Postal Code processing software?

Finally, there is another issue.  When I applied the last template to the Licensor, the filled-in fields were applied to most of the metadata fields, but not all.  Typically all but a few hundred fields were filled in. The fields that had <none> for their data entry seemed to be stable--there was no activity.  Then, there might be a small flurry of activity, and then the changes halted.  But, if one waits long enough (say 5 or 10  hours, the number of these fields would change. The number changed in this manner was consistent over the entire database.

Are there any suggestions as to (a) is there a bug that needs to be eradicated? or (b) how I should enter the data such that this situation goes away?

Bob Hendricks

Bob Hendricks

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Error in Licensors metadata fields?
« Reply #1 on: June 30, 2021, 12:13:08 PM »
Bob,

I think there may be an error in the new Licensor metadata Postal Code data field.

The objective:
I wanted to create a licensor that would replace the metadata Contact info because (a) Contact does not have a person's name and (b) because the Contact info is non-searchable.

Here is what I did: I entered the variable names for the Contact into the appropriate fields for the Licensor. e.g., I entered  {cizip} in the Postal Code for the Licensor. Note that this is a typo--the correct variable name is {cizp}. I did the same thing for all of the Contact variables, but spelled them correctly.

The result: When the metadata template was run, the misspelled variable name was treated a text string and the word "{cizip}" was entered into the Licensor Postal Code 19092 times. All of the other variables transferred the correct Contact information to their respective Licensor fields correctly.

When the mistake was discovered, the 19092 images were selected and the correct Zip Code was entered into the Licensor Postal Code. All other Licensor fields were cleared. The concatenate checkbox for the Licensor fields was selected.  As I understand it from tech support, with the checkbox selected and the field left blank, all of the information in the respective Licensor fields  should not be changed, but a blank be concatenated.  Thus, the only change I expected was that all of the Postal Codes with "{cizip}" should have been changed to 01960, the correct Postal Code.

The result:  Nothing happened--the Postal Code remains "{cizip}"

In an attempt to correct this result, the concatenate box was unchecked and the correct data (and not their variable names) were entered into each Licensor field.Every image in the collection was selected (some 36,000 images) and the template was reapplie to every image.  The Postal Code was not corrected. This run took about 4 hours because all of the images are on an external Thunderbolt drive.

Do I misunderstand something, or is there an error in the Postal Code processing software?

I think you may now have two licensors for each of the images.  One that had the original application of the template, and another with just the postal code set.  Check one of your images and let me know what you find.

Finally, there is another issue.  When I applied the last template to the Licensor, the filled-in fields were applied to most of the metadata fields, but not all.  Typically all but a few hundred fields were filled in. The fields that had <none> for their data entry seemed to be stable--there was no activity.  Then, there might be a small flurry of activity, and then the changes halted.  But, if one waits long enough (say 5 or 10  hours, the number of these fields would change. The number changed in this manner was consistent over the entire database.

Where are you seeing this?  In the Metadata Info for specific images or somewhere else?  Screenshots that illustrate the problem are most welcome.

Are there any suggestions as to (a) is there a bug that needs to be eradicated? or (b) how I should enter the data such that this situation goes away?

I can't say just yet.  But I suggest you make smaller runs when trying things out for the first time.  If there is a problem, you'll know right away rather than much later (hours later.)

-Kirk

Offline Bob Hendricks

  • Member
  • **
  • Posts: 82
    • View Profile
Re: Error in Licensors metadata fields?
« Reply #2 on: July 15, 2021, 12:16:10 PM »
Kirk:

Sorry--I got off on some other stuff.  Here is what I have found:

1. To the first question, you are correct--with a number of images, there are two licensors. 1=fields contain the correct info; 2= only the email address is filled in (correctly); all other fields are empty.  Obviously, I need to get rid of the second licensor. However, there is nothing in the licensor 2 that has any info regarding the postal code--that field is completely blank.

2. I was seeing this in the metadata info fields.  I now have new info on this.
 
In one catalog, I had 1011 images.  The info was filled in correctly to 999 images; it was not applied to the other 12. The Browse window said I had 12 images with no postal code info, but if I looked in the metadata info window, the correct zip code was present for each file. I changes the zip code for these 12 files to a dummy number 00000 and everything worked fine. Aha! a counter problem.

But not so fast.  I had another catalog with 4381 image files, of which 2880 indicated no postal code, 36 indicated the postal code with the spelling error {cizip}. and 1465 with the correct zip code. Rats--not a counter stuck at 3 sig figures.  But, if I apply the metadata template to the 36 with the spelling error and use 00000 for the postal code, it works fine.  So I did the same thing to the <none> field--I converted a small batch to 00001--worked fine; I then converted a batch to 00002--also worked fine.  I was able to get rid of all of the none, and those with the spelling error.

I am now playing with trying to merge them all to the correct 01960 code.  Will let you know what I find.

Sorry to be so slow with this.

Bob
Bob Hendricks

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Error in Licensors metadata fields?
« Reply #3 on: July 15, 2021, 12:44:37 PM »
Bob,

I am now playing with trying to merge them all to the correct 01960 code.  Will let you know what I find.

Sorry to be so slow with this.

No problem at all, Bob.  I look forward to your findings.

-Kirk

Offline Bob Hendricks

  • Member
  • **
  • Posts: 82
    • View Profile
Re: Error in Licensors metadata fields?
« Reply #4 on: July 15, 2021, 01:15:29 PM »
Kirk:

Some very strange results:

First, I went to the small catalog with 999 entries with a postal code 01960 and 9 entries of 00000.  I changed all the 00000 to 01960 and the result was (according to the numbers in the Browse fields) 1008 fields had 01960 and there were no 000000 entries remaining.  this seems to be stable. No changes in the last hour. There is only one licensor in the catalog.

Next, I went to the large catalog with 4381 images.

First, I cleared the the second licensor by selecting all of the images in the catalog, going to the Licensor fields in the metadata template, going to the second licensor, clicking the minus sign to make it go away so that there was only one licensor, OK, and then apply to all images.  On completion, check a number of images to be sure all had only one licensor.  All OK

Before I did this, I had no images with the wongly-spelled variable,  no images with no postal id, 1049 images with  postal cooed 00000. 843 with postal code 00001, 1024 with postal code 00002 and 1465 with postal code 01960 for a total of 4381 images.

I went away for a while; came back in 15 minutes or so.  there was no activity with the catalogs implying that everything was up to date.

But now, I had 3 images without a total code, 1049 with 00000, 843 with 00001, and 2486 with 00002, none with 01960 for a total of 4381 images.  It seems that all of the images with 01960 got converted to 00002 while I was gone.

How long must one wait to be sure all the cataloging is finished?  this may be part of the issue--impatience on my part--but my only indicator is the message indicating how many books are yet to be done.

I have not yet tried to merge any of these different postal codes.  Will go to dinner and then try later this evening.

Bob
Bob Hendricks

Offline Bob Hendricks

  • Member
  • **
  • Posts: 82
    • View Profile
Re: Error in Licensors metadata fields?
« Reply #5 on: July 15, 2021, 01:29:27 PM »
Some new strangeness.  So far I had been paying attention only to the postal code.  I just looked at all the other fields.

1. the metadata info is correct for everything so far as I can see.

2. the Browse numbers of images with metadata are really screwed up. For every field (that I populated), Browse says that all but one image has no entry; only one image has my info.  But the metadata info field says it is all there and correct.

Perhaps a Zoom where I can show you some of this stuff with screen sharing might be in order.

Bob
« Last Edit: July 16, 2021, 07:55:03 AM by Bob Hendricks »
Bob Hendricks

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Error in Licensors metadata fields?
« Reply #6 on: July 15, 2021, 05:39:26 PM »
Bob,

Some new strangeness.  So far I had been paying attention only to the postal code.  I just looked at all the other fields.

1. the metadata info is correct for everything so far as I can see.

2. the Browse numbers of images with metadata are really screwed--every other field (that I populated) say that all by one image has no entry for every field;  only one has my info.  But the  info field says it is all there and correct.

Perhaps a Zoom where I can show you some of this stuff with screen sharing might be in order.

I'd be happy to do that when I return from my vacation at the end of the month.

-Kirk

Offline Jerry H

  • Camera Bits Staff
  • Full Member
  • *****
  • Posts: 233
    • View Profile
    • Camera Bits, Inc.
Re: Error in Licensors metadata fields?
« Reply #7 on: July 16, 2021, 10:20:52 AM »
Hi Bob,

2. the Browse numbers of images with metadata are really screwed up. For every field (that I populated), Browse says that all but one image has no entry; only one image has my info.  But the metadata info field says it is all there and correct.

I was able to recreate the problem with image count inconsistencies for files that have multiple Licensor records. I will file a bug in our bug tracking system.

Regards,
Jerry