Author Topic: Problem ingesting some Lumix raw files  (Read 4283 times)

Offline Goughy

  • Newcomer
  • *
  • Posts: 8
    • View Profile
Problem ingesting some Lumix raw files
« on: August 13, 2020, 12:20:35 PM »
I am having a problem ingesting images shot with my Panasonic Lumix DC-G9 when they are geotagged using Panasonic Image App.

Here are the steps to reproduce the issue:

- Start the Panasonic Image App on my iPhone to apply GPS coordinates to the images as I take them
- Shoot RAW (.rw2) images, recording them on an SD card in the camera
- Use Photo Mechanic 6 to ingest the images to my iMac
- Attempt to import these images using Lightroom Classic

I get the message from LRC: “The file appears to be unsupported or damaged. (1)”

The files are not readable by other processing software either (e.g. Capture One 20, Lightroom), Interestingly, the files are readable by DxO PhotoLab 3, but I suspect this particular program does not validate the EXIF data rigorously.

If I copy the files manually from the SD card to the computer, then the files are readable by all the above programs, and the GPS latitude and longitude are present and correct.

If I do not use the Panasonic Image App to tag the images and then ingest them using PM6, then the files are readable. I can then tag them using some other method which writes the EXIF data to the .rw2 file (eg. Geotag Photos Pro 2, HoudahGeo etc.) and all is well.

An additional test. Panasonic Image App only writes latitude and longitude to the file. If I copy the file manually, then add an altitude value using Houdah Geo, then PM6 ingests the edited file correctly.

It seems that PM6 is incorrectly handling the EXIF lat/lon data written by the Panasonic Image App to the raw file resulting in a corrupted .rw2 after ingest.

This is not an urgent issue as I can easily workaround by not using Panasonic Image App (I’m not that keen on this app anyway, as I have to add altitude manually afterwards) or by copying the files manually rather than ingesting.

I can provide a sample of a .rw2 file direct from the SD card and after ingest if it would help.

Thanks,

Richard
Eschew obfuscation

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Problem ingesting some Lumix raw files
« Reply #1 on: August 13, 2020, 03:11:44 PM »
Richard,

I can provide a sample of a .rw2 file direct from the SD card and after ingest if it would help.

That would be a requirement for us to look into the problem.

Can I ask you:

1) Are you making any changes to the file during (or after) the Ingest with Photo Mechanic?  If so, what kind of changes?
2) Are you using XMP sidecar files or are you having PM embed IPTC/XMP metadata into your files?  If you are embedding, have you tried adopting an XMP sidecar workflow instead and see if that makes any difference?

Thanks,

-Kirk

Offline Goughy

  • Newcomer
  • *
  • Posts: 8
    • View Profile
Re: Problem ingesting some Lumix raw files
« Reply #2 on: August 13, 2020, 05:32:39 PM »
Kirk,

1) No. I turned off renaming the file and "Apply Metadata Template to photos" to avoid muddying the issue. After Ingest I launched Lightroom Classic immediately without any changes to the file.

2) I was instructing PM to embed the IPTC/XMP data into the raw files. When I turned off "Allow RAW files to be modified" and selected "Always create and/or update XMP sidecar file" the problem did not occur and Lightroom Classic was able to read the file successfully.

Let me know if you'd like a sample file and the best way to send it. It's around 24MB.

Thanks,

Richard
Eschew obfuscation

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Problem ingesting some Lumix raw files
« Reply #3 on: August 13, 2020, 05:51:00 PM »
Richard,

1) No. I turned off renaming the file and "Apply Metadata Template to photos" to avoid muddying the issue. After Ingest I launched Lightroom Classic immediately without any changes to the file.

2) I was instructing PM to embed the IPTC/XMP data into the raw files. When I turned off "Allow RAW files to be modified" and selected "Always create and/or update XMP sidecar file" the problem did not occur and Lightroom Classic was able to read the file successfully.

Let me know if you'd like a sample file and the best way to send it. It's around 24MB.

If you have a file sharing service like Dropbox, put it there and give me a link.  Otherwise, we have an FTP server to which you can upload the image.  Your choice.

-Kirk

Offline Goughy

  • Newcomer
  • *
  • Posts: 8
    • View Profile
Re: Problem ingesting some Lumix raw files
« Reply #4 on: August 13, 2020, 10:02:13 PM »
Kirk,

I have sent you a PM with a Dropbox link to two files - one straight from the SD card and the other after ingesting that file using PM6.

Looking back, I realize I might have been here before: http://forums.camerabits.com/index.php?topic=10676 from December 2016.

I'm not sure if adding metadata to .rw2 files is still not supported. However, this is slightly different in that, as far as I can tell, I am not adding any data to the file being ingested.

Thanks again,

Richard
« Last Edit: August 14, 2020, 10:42:28 AM by Goughy »
Eschew obfuscation

Offline dennis

  • President
  • Camera Bits Staff
  • Sr. Member
  • *****
  • Posts: 469
    • View Profile
    • Camera Bits, Inc.
Re: Problem ingesting some Lumix raw files
« Reply #5 on: August 17, 2020, 05:59:29 PM »
Richard,

Please disable the modification of these RAW files in your IPTC/XMP preferences and stick with XMP sidecar files until we understand what the problem is. It may just be other applications cannot deal with relocated TIFF tables in this rw2 format. We used to not allow modification of these files due to problems but for some reason we re-allowed modification (by choice) back in January I think. This may have been premature.

To fix these files for other applications, select the rw2 files you are having problems with then use the Revert TIFF-based RAW to Original command from the Tools menu. This will extract any XMP and put it into a sidecar and return the file to its original state.

If this is still a major problem with other applications, we may again disable the ability to modify these rw2 files and force XMP sidecar files. That way you could still embed XMP at least for other RAW formats (not recommended though).

--dennis

Offline Goughy

  • Newcomer
  • *
  • Posts: 8
    • View Profile
Re: Problem ingesting some Lumix raw files
« Reply #6 on: August 18, 2020, 09:08:43 PM »
Thanks Dennis.

Just to make sure there is no misunderstanding-

This problem only occurs if I tag the .rw2 images with the Panasonic app Image App while I am taking the shots. This app uses bluetooth to transfer GPS data (lat and lon only) from my iPhone to the images in camera. If I then use PM to ingest these tagged images, I get the problem.

If I do not use Image App to tag the images and then ingest the untagged images from my SD card to my iMac using PM6, there is no problem.

If I do not use Image App, but instead copy the images to my iMac, then tag them using another app (e.g. Geotag Photos), and then use PM6 to "ingest" the files in place on the iMac, there is again no problem.

So Image App is doing something to the file that upsets PM6.

IMO, it would be a step backwards to again disable the modification of the .rw2 files by PM6. I'd much rather just avoid Image App.

Thanks again,

Richard
Eschew obfuscation

Offline LarsN

  • Member
  • **
  • Posts: 52
    • View Profile
Re: Problem ingesting some Lumix raw files
« Reply #7 on: August 19, 2020, 03:40:40 AM »
Quote
IMO, it would be a step backwards to again disable the modification of the .rw2 files by PM6. I'd much rather just avoid Image App.

Thank you for the above comment. I agree 100%

Lars