Corruption of JPEGs when geotagging/updating IPTC

[LGoosens]

Hi there everyone,

since I'm new to the forums, I'd like to introduce myself. My name is Lennart Goosens, and I'm from the Netherlands. I'll probably not be posting a lot; just asking some questions and maybe making a couple of suggestions.

CPU:	Pentium 4 Northwood 2.66 GHz (533 MHz FSB, no Hyper Threading, 512 KB L2 cache)
Memory:	1.5 GB DDR-333 in a dual channel configuration
Operating System:	Windows XP SP3
Photo Mechanic version:	4.6.8
Free disk space on system partition:	11 GB

When I was still on 4.5, I never did any geotagging. However, after installing 4.6.8 a couple of days back, I got the urge to geotag a few collections (which instantly became a new hobby ).
After geotagging maybe 150 photos (not sure), and also applying an IPTC stationary to them, I got a crash after clicking Apply in the Set GPS Coordinates window. I restarted Photo Mechanic, and noticed that the JPEG of the picture I tagged last had become corrupted (thankfully not the CR2! ) I didn't really care about that particular picture anyway, so I just continued geotagging. Another crash forced me to restart PM, and two other JPEGs had become corrupted - again, the CR2s are fine (as becomes apparent when turning of RAW+JPEG mode). Both the JPEGs and the CR2s have succesfully acquired the correct geotag and IPTC stationery I applied.

• I have no clue as to what the cause could be. I'm not blaming PM yet, but I'm also abstaining from practicing my new hobby until this is sorted out.
• It seems to have something to do with the geotagging, but that doesn't necessarily have to be the case.
• EDIT: When trying to open the corrupted images in Photoshop CS 2, I get the following message:
  

  Could not complete your request because a SOFn, DQT, or DHT JPEG marker is missing before a JPEG SOS marker.
• I've provided a screenshot and an error report as attachments, and I'd be happy to provide one of the corrupted photos if necessary.

Thanks in advance!

Regards,

Lennart Goosens.

[attachment deleted by admin]

[devenh]

My suggestion would be to try to simplify your test by doing the following:

1. Create a new directory
2. Copy 6 or so jpeg files into the directory.  Try to use files that have their original metadata (in other words, files that have not had their IPTC info edited by PM -- alternatively you can use PM to delete the metadata, Tools | Delete Metadata)
3. Geotag the files
4. Check for corruption with both PM and another editor like Photoshop

Deven

[Kirk Baker]

Lennart,

My suggestion would be to try to simplify your test by doing the following:

1. Create a new directory
2. Copy 6 or so jpeg files into the directory.  Try to use files that have their original metadata (in other words, files that have not had their IPTC info edited by PM -- alternatively you can use PM to delete the metadata, Tools | Delete Metadata)
3. Geotag the files
4. Check for corruption with both PM and another editor like Photoshop

Deven's suggestion is sound, though I would also suggest geotagging files that have had their IPTC data modified (regardless of which application did the modification.)  If you find a JPEG that can reliably cause a crash or corruption, please post it here.  Use the 'Additional Options...' link when you're composing your reply to this message and there you'll be able to upload your JPEG file.  Once I have the file, I can try to reproduce the problem.  If I can reproduce the problem then I can most certainly fix it.

Thanks,

-Kirk

[LGoosens]

Hi Kirk and Deven,

first of all, thank you both for your swift reply. Sorry by the way for posting the huge multi-display screenshot, without realising that everyone would be forced to download it.

These are the results of my testing so far:

• Photo Mechanic will, sometimes, crash when geotagging an Adobe RGB JPEG from a Canon EOS 30D - but only if the JPEG was made in-camera (and not by Adobe Camera RAW or Canon RAW Image Task). sRGB JPEGs don't seem to be affected.
  
  • I'm not sure, but I think this can happen even on the first photo being tagged; however, the odds of PM crashing seem to increase when working quickly (i. e. clicking Apply in the Set GPS Coordinates window relatively often in a short amount of time).
  • Adding GPS coordinates to the aforementioned in-camera JPEGs also seems to be more time consuming - whereas other files will get their geotags with no delay whatsoever.
• On one instance when PM crashed, according to Task Manager, it consumed nearly every available CPU cycle. I haven't been able to reproduce this (but haven't been observant of this either).
• Usually when PM crashes, it will offer to send an error report, and then shut itself down. At some point in my testing however, this changed, and PM just stopped responding altogether (and had to be shutdown manually). I have no clue what caused this change. Later on, it started behaving as usual again.
• When a crash occured, sometimes, the photo tagged last will have its geotag correctly added, sometimes not, and lastly, sometimes the whole file will be corrupted. I haven't been able to reproduce the corruption in my testing, however. Sorry.
• Editing the IPTC data does not seem to have any influence whatsoever.

I will attach a set of image files and crash reports in my next post; for now however, I have to get going. I am interested to know both your thoughts on the issue, and will tune in regularly. Thanks again!

Regards,

Lennart Goosens.

[Kirk Baker]

Lennart,

I will attach a set of image files and crash reports in my next post; for now however, I have to get going. I am interested to know both your thoughts on the issue, and will tune in regularly. Thanks again!

I await your sample files and crash reports.

-Kirk

[LGoosens]

Hi,

I've attached three reports and their respective "suspect images" as a split ZIP file. Good luck!

At this point, I feel I should mention - although I hardly think it has anything to do with it - that the machine I'm working on, has an issue with spurious interrupts, that I have so far been able to trace down to either the sound card or its driver. The difference between the number of interrupts per second, and the number of Deferred Procedure Calls per second, according to Performance Monitor, is roughly 600 (I believe that this is also roughly the number of spurious interrupts). This, however, is as far as my knowledge about interrupts goes.

Regards and thanks for the effort so far,

Lennart Goosens.

NOTE: Even when I keep the size of separate files well under the board's limits, my session times out because of the long time it takes to upload them. If I hadn't backed up my message body in a text editor, I'd have retyped it at least eight times by now. Also, I had to split the file in six parts.

PART SIX


[attachment deleted by admin]

[LGoosens]

PART FIVE

[attachment deleted by admin]

[LGoosens]

PART FOUR

[attachment deleted by admin]

[LGoosens]

PART THREE

[attachment deleted by admin]

[LGoosens]

PART TWO

[attachment deleted by admin]

[LGoosens]

PART ONE

[attachment deleted by admin]

[LGoosens]

Hi there,

just checking back to see how you guys are doing, and if I can be of any further help to you.

I haven't been doing any more geotagging, and haven't yet looked into finding a workaround. If you have any clues on the cause, I'd of course be eager to know; maybe then I can avoid the problem without having to abstain.

Good luck!
Lennart Goosens.

[dennis]

Hi Lennart,

It took a bit of sleuthing to find out what was actually happening here but I have fixed the problem.  This may only be specific to your Canon 30D not sure.  But the camera is incorrectly writing some of the Exif size information and PM was tripping-up on this.  You might call it garbage in - garbage out, but I've added code to protect against this incorrect data.

I'll contact you privately to verify the fix on your end and we will make this update more generally available after we are satisfied with the fix.

Thanks for your help with this!

Regards,

--dennis

[LGoosens]

Hi Dennis,

I already suspected it might have had something to do with my (specific?) camera body. Of course, the 30D is already considered "ancient" by the standards of a lot of people - I bought it about 3,5 years back - so for them this wouldn't be a huge problem. Well, at least on new pictures. Collections of older photos could easily be affected by this.
Since I intend to use this camera for several more years, I am grateful for your efforts to fix this!

I'll have a look btw and check whether I have the latest firmware installed (to my best knowledge, I have) and maybe update it.

Thanks again and regards,

Lennart Goosens.