Author Topic: HELP - Color Class (and more) missing/wrong on 1000's of pics  (Read 7129 times)

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20820
    • View Profile
    • Camera Bits, Inc.
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #15 on: August 09, 2011, 07:56:46 AM »
Juerg,

Quote from: Kirk Baker
Quote from: Kirk Baker
The main issue is that you're trying to use two programs (Capture NX2 and Bridge/ACR) that don't handle metadata in the same way.  CNX2 only works with embedded XMP and completely ignores XMP sidecar files and moreover needs to have the IPTC Urgency field be re-purposed for the use of color labels.  Adobe's apps have various issues when embedded XMP is present and won't update embedded XMP, thus things can get out of sync and do not have the ability to synchronize the label index (no Indexes for Label exist in Adobe's apps!) with the IPTC Urgency field.

Quote from: archer69
So, what are your saying? That my settings as they are are the best and should stay like that?

What I'm saying is that it is difficult to have your cake and eat it too.  You could take PM completely out of the picture and just use CNX2 and Adobe applications together and have all sorts of problems.

No, one of the reasons I use PM is because neiter the Capture nor Adobe applications work well for me ... PM does (just wished the two other companies would ask you guys how to do things right). Anyway, what do I need to change. I don't want to screw up more. And if I have to go back and reset color class, ratings, orientation in the OLD files already screwed up, so I will (sometime). I don't want to screw up the at the moment correct ones. And from now on forward, I need a setting that works best (albeit, probably not perfect).

What I'm trying to say is that although PM is extremely flexible in regards to how metadata is handled, there is no one configuration that will make all applications cooperate together perfectly, especially CNX2 and Adobe's apps.

Quote from: archer69
Taking the "General" Tab:

What do I need to change there?
- Changing color labels from descriptiv back to actuall collors (ie "FinalEdit_Originals" to "Green" and so on)?
- Unckeck the 'Synchronize Color Class with IPTC/XMP Urgency field' checkbox? (But then you said NX needs that ... I think based on a previous discussion we had here http://forums.camerabits.com/index.php?topic=3601.msg16256#msg16256 is why the options in both the "General" and "IPTC/XMP" tabs were set as they are).

And in the "IPTC/XMP" tab:

What do I need to change there?

Please note that there is no way around using NX or CS5, but, as I did for a while until I got a Canon camera, I can start NOT using Bridge again. I believe that where things really get messed up. So setting should be for maximum copadibility and data safety (synchronization).

You'll probably have to experiment with settings until you get it right.  I suggest temporarily making the changes I described earlier, then do some kind of metadata update, like applying an IPTC Stationery Pad that clears out an unused field.  This will then update all available metadata (XMP sidecar, embedded XMP, and embedded IPTC).  Then go back to your old settings.  You may need to turn on the Synchronize color class/IPTC Urgency checkbox and do this updating operation one last time.  If you had PM 4.6.x you would have a snapshot button in the IPTC/XMP settings that you could use to quickly toggle settings.  Instead you should just screen grab the settings so you can refer to them and make the changes manually.

To get all of the Color Classes realigned, you'll have to experiment with the label names so that they sync up again, do a metadata update then set them back one last time and then restore your preferred label names and do a metadata update again.

Quote from: archer69
Quote from: Kirk Baker
Quote from: archer69
Means that the files I sent are from a folder that contains also files that are correct, but were edited at the same time as the files that are wrong.

Honestly, I have no idea.  Unless you gave me every single photo in this folder and I had the time to analyze each and every one of them, I couldn't give you a definite answer.
Well, that's out of question ... unless I upload files for a couple of days ... and you find time to look at them all individually.

But as I said, all the files that are already screwed up I am gonna correct again.

Your files are not 'screwed up'.  The data is there for the most part (with the exception of PM's end of file image prefs which appear to be missing which is OK since the same data is duplicated in IPTC/XMP), and the real issue is which order you tell PM to load the metadata.

Quote from: archer69
IMPORTANT for me now is how to set preferences in the "General" and "IPTC/XMP" Tabs to not screw up more and still can use CaptureNX, ViewNX, CS5 and (if possible in any way) Bridge (if you say it's not possible with Bridge included, I skip Bridge, but the others I need to work with) without screwing up the files that are currently correct.

The issue really depends on what you do in each of the apps.  If you change IPTC Urgency in any Adobe app, then it won't also change the Color Class.  If you change the Color Label in any Adobe app then it won't updated the IPTC Urgency field.  Also no Adobe app will update the embedded IPTC or embedded XMP.  Conversely, Capture NX2 won't update XMP sidecar files.

So if you stay away from making any metadata changes in both apps and rely on PM for that task, that would be best.  When you do make adjustments in CNX2 it would be worthwhile to do some sort of metadata update (like clearing out an unused IPTC field with the IPTC Stationery Pad) to get all of the metadata synchronized again.

Quote from: archer69
And, do you think that some of the issues would be helped by updating from version 4.5.4 to the current 4.6.7? I wanted to wait for the next major release, but if you say it IS important to update, I will do (of course I now I missed the special update deal you had earlier this year  :'( )

It would make switching between settings easier.  The deal we offered earlier wouldn't have been the right deal for you since you really want our next major version anyway.  It was a license for the current version that did not include an upgrade path to the next major version.  That's why it was less expensive.

-Kirk

Offline archer69

  • Full Member
  • ***
  • Posts: 249
    • View Profile
    • JH-Photography
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #16 on: August 09, 2011, 01:13:34 PM »
Kirk,

Before we go any further with troubleshooting, I upgrade to the latest version 4.6.7. A little earlier than expected (please make sure the next major release happens before the 1 year update lisence expires  ;) ), but at least it makes easier for you to help me (I hope).

But:

1. Do I have to uninstall PM 4.5.4 before installing 4.6.7 ?

2. How do I transfer my presets ?

By the way, I was playing a little bit with different preference settings while I had both NX and Bridge open, but only changed settings in PM (stars and color label). I was not able to get the changes I did in PM to show in NX, but it did in Bridge. But as soon as I tagged/untagged the file in PM, the previousely changes to stars/color labe disapered. In the PM log I found this entry regarding that file:

[Tue Aug-09-2011 21:43:57.467][4956] CBFileIO.cpp:51 ($Revision: 1.2.2.4 $) Could not open "Y:\PICTURES_Personal\PICTURES\2007\2007-07_Seattle WA\070723\070723_LND2325_JH_Seattle WA.NEF" for "r+b" access.  Sleeping 500ms and trying again

Any idea what that means in real words?


Thanks,
Juerg
« Last Edit: August 09, 2011, 01:53:46 PM by archer69 »
Regards,
Juerg H.
www.jh-photography.net

Offline archer69

  • Full Member
  • ***
  • Posts: 249
    • View Profile
    • JH-Photography
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #17 on: August 09, 2011, 02:28:45 PM »
Kirk,

I played a little more around with the preference settings on the old version 4.5.4, and following settings in the "General" and "IPTC/XMP" tabs seems to correct the issues of out of sync color labels and ratings (only the orientation of some files is still wrong).

Changes I made are:

"General" Tab:

Uncheck "Synchronise Color Class with ITPC Urgency Field"

"IPTC/XMP" Tab:

Change the reading for TIFF based RAW fotos from

Read embedded XMP -> Read XMP Sidecar File -> Read embedded IPTC

to

Read embedded XMP -> Read embedded IPTC -> Read XMP Sidecar File


Now, what surprises me is that unchecking the "Synchronise Color Class with ITPC Urgency Field" helped, as you mentione that NX needed that.

It also seems that not only the old files are showing rating/color class correct again, but that also the newly in PM edited files still seem to be OK.

BUT, neither in NX nor in Bridge does it now show the correct color classes or ratings, only in PM. But that's OK for me as I don't need either NX nor Bridge to do any metadata editing, but only PM.

Attached the screen shots of the 2 tabs as they are now after the changes. Any other things that I may need to change to prevent other issues?

Thanks
Juerg


[attachment deleted by admin]
« Last Edit: August 09, 2011, 02:43:39 PM by archer69 »
Regards,
Juerg H.
www.jh-photography.net

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20820
    • View Profile
    • Camera Bits, Inc.
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #18 on: August 09, 2011, 02:30:40 PM »
Juerg,

Before we go any further with troubleshooting, I upgrade to the latest version 4.6.7. A little earlier than expected (please make sure the next major release happens before the 1 year update lisence expires  ;) ), but at least it makes easier for you to help me (I hope).

But:

1. Do I have to uninstall PM 4.5.4 before installing 4.6.7 ?

I would recommend that you do uninstall the old version.

Quote from: archer69
2. How do I transfer my presets ?

You don't need to do anything special.  The new version will use your old Preferences.

Quote from: archer69
By the way, I was playing a little bit with different preference settings while I had both NX and Bridge open, but only changed settings in PM (stars and color label). I was not able to get the changes I did in PM to show in NX, but it did in Bridge. But as soon as I tagged/untagged the file in PM, the previousely changes to stars/color labe disapered. In the PM log I found this entry regarding that file:

[Tue Aug-09-2011 21:43:57.467][4956] CBFileIO.cpp:51 ($Revision: 1.2.2.4 $) Could not open "Y:\PICTURES_Personal\PICTURES\2007\2007-07_Seattle WA\070723\070723_LND2325_JH_Seattle WA.NEF" for "r+b" access.  Sleeping 500ms and trying again

Any idea what that means in real words?

Yes.  Capture NX 1 & 2 always keep the entire directory of files open for exclusive access while you're browsing.  You need to either quit Capture NX or have it browse a different folder.  Then PM will be able to update the file.  It is a very well known issue and has been discussed at length on these forums.

-Kirk

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20820
    • View Profile
    • Camera Bits, Inc.
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #19 on: August 09, 2011, 02:35:32 PM »
Juerg,

Any other things that I may need to change to prevent other issues?

You're just going to have to experiment with the settings until you get some thing that works best for your workflow.

The main issues here (that are somewhat incompatible) are:

Capture NX2 only works with embedded metadata and ignores XMP sidecar files.
Adobe's software will use embedded metadata but will not update it which allows the metadata to get out of sync.  Only the XMP sidecar file will be updated.

So, you're just going to have to experiment with the settings until you get some thing that works best for your workflow.

-Kirk

Offline archer69

  • Full Member
  • ***
  • Posts: 249
    • View Profile
    • JH-Photography
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #20 on: August 09, 2011, 11:03:47 PM »
Kirk,
You're just going to have to experiment with the settings until you get some thing that works best for your workflow.

The main issues here (that are somewhat incompatible) are:

Capture NX2 only works with embedded metadata and ignores XMP sidecar files.
Adobe's software will use embedded metadata but will not update it which allows the metadata to get out of sync.  Only the XMP sidecar file will be updated.

So, you're just going to have to experiment with the settings until you get some thing that works best for your workflow.

I actually thought I had that before ... how wrong I was  :( . But I'm glad I am back to a working way. Thanks a lot for your help. But a few more questions:

For TIFF based RAW files, what do or don't do following setting cause:
- Add embedded IPTC -> Yes or No?
- Add embedded IPTC4XMP -> Yes or No? (I assume this one should be Yes (checked) as it is necessary when working with NX to ensure that metadata is transfered from NEF's to TIFF's ect. ?)
- Allways create and/or update XMP sidecar file with IPTC4XMP or If sidecar file exist, add and/or update IPTC4XMP or If sidecar file exist, only update IPTC4XMP when present -> Which one to select?

Again, I don't do any metadata updates in NX and/or Bridge/CS5. NX and CS5 are soley used for editing purpose, Bridge for starting batch ACR development.

Another question regarding the (perhaps presived) slowness of PM, in fact that I have an rahter old (7 years) WinXP system with first generation DualCore processor and 2MB Ram:
- What setting would be ideal to use for the "Momory Cache Size" ? Is higher better?
- What setting would be ideal for the cache disk (which is an internal SATA HD with ca 60GB size, shared also as a cache disk with NX and CS5). PM sais maximum Disk Cache size possible is 65535 MB, of which I allocated at least 10000 MB to PM. Is that a good setting?

Also, after installing version 4.6.7, I notized an new checkbox in the "Files" Preference Tab (see attached screenshot): Allways update XMP Label when changing Color Class ... Do I now need to check that or not? What's the impact?


Thanks,
Juerg

[attachment deleted by admin]
« Last Edit: August 09, 2011, 11:38:21 PM by archer69 »
Regards,
Juerg H.
www.jh-photography.net

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 20820
    • View Profile
    • Camera Bits, Inc.
Re: HELP - Color Class (and more) missing/wrong on 1000's of pics
« Reply #21 on: August 10, 2011, 06:20:14 AM »
You're just going to have to experiment with the settings until you get some thing that works best for your workflow.

The main issues here (that are somewhat incompatible) are:

Capture NX2 only works with embedded metadata and ignores XMP sidecar files.
Adobe's software will use embedded metadata but will not update it which allows the metadata to get out of sync.  Only the XMP sidecar file will be updated.

So, you're just going to have to experiment with the settings until you get some thing that works best for your workflow.

I actually thought I had that before ... how wrong I was  :( . But I'm glad I am back to a working way. Thanks a lot for your help. But a few more questions:

For TIFF based RAW files, what do or don't do following setting cause:
- Add embedded IPTC -> Yes or No?
- Add embedded IPTC4XMP -> Yes or No? (I assume this one should be Yes (checked) as it is necessary when working with NX to ensure that metadata is transfered from NEF's to TIFF's ect. ?)
- Allways create and/or update XMP sidecar file with IPTC4XMP or If sidecar file exist, add and/or update IPTC4XMP or If sidecar file exist, only update IPTC4XMP when present -> Which one to select?

Again, I don't do any metadata updates in NX and/or Bridge/CS5. NX and CS5 are soley used for editing purpose, Bridge for starting batch ACR development.

When you use those apps and you make RAW adjustments, you are indeed making metadata updates.  You just didn't know this until now.
Since you're using an older version of Photoshop/Bridge (yes, CS5 is now old) instead of the current CS5.5 you're going to run into issues with embedded XMP which will cause Photoshop/Bridge to appear to lose their RAW adjustments.

Even with CS5.5, you'll run into synchronization issues when you make adjustments with NX2 and then later return to PM or Photoshop/Bridge.  You'll run into synchronization issues when you make adjustments with ACR and return to NX2 and PM.

This is because ACR won't update embedded XMP or embedded IPTC (though IPTC would never contain RAW adjustment settings anyway) and because NX2 won't update XMP sidecar files.  However, PM will resynchronize all of that data for you but not necessarily in a way that will work for all possible workflows.  Are you ever sending a NX2 edited NEF file off to ACR for adjustment?  Or are you only using Canon CR2 files with ACR?

Quote from: archer69
Another question regarding the (perhaps presived) slowness of PM, in fact that I have an rahter old (7 years) WinXP system with first generation DualCore processor and 2MB Ram:
- What setting would be ideal to use for the "Momory Cache Size" ? Is higher better?

I wouldn't set yours above 256MB, especially if you're using Photoshop at the same time.   You may also want to set Photoshop to use less of your RAM for its caching as well.

Quote from: archer69
- What setting would be ideal for the cache disk (which is an internal SATA HD with ca 60GB size, shared also as a cache disk with NX and CS5). PM sais maximum Disk Cache size possible is 65535 MB, of which I allocated at least 10000 MB to PM. Is that a good setting?

I would set that to 1024MB.  10,000MB is unnecessarily large.

Stay away from Capture Time (or any Custom Sort) sorting and avoid using the Color Class filter and Star Rating filters until after PM has finished loading all of the images.  And hopefully some day you'll get a newer system with more RAM and multiple CPUs.  It really helps.

Quote from: archer69
Also, after installing version 4.6.7, I notized an new checkbox in the "Files" Preference Tab (see attached screenshot): Allways update XMP Label when changing Color Class ... Do I now need to check that or not? What's the impact?

Yes, I advise setting that checkbox since you're using XMP quite a bit now.  Impact is that the XMP 'Label' field will contain the names of your labels whenever you change the Color Class in PM.  You'll see the effect in Bridge/Lightroom that way.

-Kirk