Sony RX10 II Video files

[Rick Baumhauer]

I just picked up a Sony RX10 II, primarily for video, though I'm sure I'll shoot some photos with it.

When importing with PM, only the photos show up in the import - the camera appears to write all of its XAVC S MP4 video files to a separate folder (/private/M4ROOT/CLIPS), with all photos in the standard DCIM folder structure. I can point PM at the CLIPS folder and have it import from there, but it doesn't seem to be picking up the date properly - I have my import set to import into YEAR/MONTH/DAY folders based on shoot date (so file creation date), but all the videos get copied to a folder called "default" in the root of the folder where all of my Year folders reside (that is, instead of copying to /2016/04/30/, PM copies to folder "default" at the same level as 2016).

I've copied the folder structure on the card (with one JPG and one MP4 in the relevant locations) and ZIPPed it up - it's still pretty large (~80MB), so let me know if you want me to send it in.

[Kirk Baker]

Rick,

I just picked up a Sony RX10 II, primarily for video, though I'm sure I'll shoot some photos with it.

When importing with PM, only the photos show up in the import - the camera appears to write all of its XAVC S MP4 video files to a separate folder (/private/M4ROOT/CLIPS), with all photos in the standard DCIM folder structure. I can point PM at the CLIPS folder and have it import from there, but it doesn't seem to be picking up the date properly - I have my import set to import into YEAR/MONTH/DAY folders based on shoot date (so file creation date), but all the videos get copied to a folder called "default" in the root of the folder where all of my Year folders reside (that is, instead of copying to /2016/04/30/, PM copies to folder "default" at the same level as 2016).

I've copied the folder structure on the card (with one JPG and one MP4 in the relevant locations) and ZIPPed it up - it's still pretty large (~80MB), so let me know if you want me to send it in.

Can you put the zip archive somewhere and provide a link to it for me?

-Kirk

[Kirk Baker]

Rick,

Thanks for the file.  What software do you use to view your video?  Does it expect the folders to be laid out like they are on the card (PRIVATE directory and its files and sub-directories)?

-Kirk

[Rick Baumhauer]

I pretty much import everything into Year/Month/Day folders (including video), sync the folders in Lightroom, then import into Final Cut Pro as needed from there (leaving the media files in place), so the card folder structure doesn't matter to me at all. Not sure if there are workflows that would have other requirements - mine is very simple. I have no idea why Sony structures the data as they do - I believe that AVCHD video was typically stored in a way that would make Blu-Ray creation straightforward, but for MP4 video, I can't see any reason for it to be stored the way it is.

Lightroom can see everything if I tell it to look at the whole card (instead of going through PM first), but it also then picks up the irrelevant thumbnails (probably only needed by the camera). However, for some odd reason, Lightroom does pick up the date/time correctly on AVCHD video, but adds four hours to MP4 video (if the file creation time is 2pm, it treats it as 6pm). Going to file a bug report with Adobe on that and see what they say.

[Kirk Baker]

Rick,

I pretty much import everything into Year/Month/Day folders (including video), sync the folders in Lightroom, then import into Final Cut Pro as needed from there (leaving the media files in place), so the card folder structure doesn't matter to me at all. Not sure if there are workflows that would have other requirements - mine is very simple. I have no idea why Sony structures the data as they do - I believe that AVCHD video was typically stored in a way that would make Blu-Ray creation straightforward, but for MP4 video, I can't see any reason for it to be stored the way it is.

Lightroom can see everything if I tell it to look at the whole card (instead of going through PM first), but it also then picks up the irrelevant thumbnails (probably only needed by the camera). However, for some odd reason, Lightroom does pick up the date/time correctly on AVCHD video, but adds four hours to MP4 video (if the file creation time is 2pm, it treats it as 6pm). Going to file a bug report with Adobe on that and see what they say.

PM knows nothing about AVCHD video.  To PM, those files are a bunch of unknown file types and it will try to copy them like it would copy any other unknown file type, but variables are not going to evaluate to much if anything at all.

Out of that entire directory tree (in the PRIVATE directory) what is important?  If that directory structure were to be flattened, what would happen in the apps you use to view the video?  Do you simply need the PRIVATE/M4ROOT/CLIP/*.MP4 and *.XML files and the rest of the files in PRIVATE can be ignored?

-Kirk

[Rick Baumhauer]

Yes, that would be ideal - I don't think I even need the XML files for anything I do, but having them wouldn't be a problem. Everything else in there besides the actual MP4 files can be ignored. Not having the AVCHD files is fine with me, as well - I don't plan to use the format for anything, but included it just to see how it effected the file structure on the card.

[Rick Baumhauer]

One other thing: once the MP4 files are imported, PM still doesn't seem to know how to display them. The AVCHD .MTS files just show up as a generic unknown filetype in contact sheets, but the XAVC S MP4 files display with red filenames and can't be played. The files display and play in Finder, which I thought provided some underlying capability to display files to PM, but that doesn't seem to be the case here. This isn't critical for anything I do, but I wanted to mention it.

[Kirk Baker]

Rick,

Yes, that would be ideal - I don't think I even need the XML files for anything I do, but having them wouldn't be a problem. Everything else in there besides the actual MP4 files can be ignored. Not having the AVCHD files is fine with me, as well - I don't plan to use the format for anything, but included it just to see how it effected the file structure on the card.

OK, but what we need to know, since I cannot find any documentation whatsoever, is what is going to work for everyone.  PRIVATE means to me: "hands off, ignore this, you don't know what to do here", whereas the DCIM folder is well-documented.

If we ignore specific files, you may be happy with it, but other people may not be.

We need to get to the bottom of this and find out officially what is important.  This may be near-impossible, unfortunately.

-Kirk

[Kirk Baker]

One other thing: once the MP4 files are imported, PM still doesn't seem to know how to display them. The AVCHD .MTS files just show up as a generic unknown filetype in contact sheets, but the XAVC S MP4 files display with red filenames and can't be played. The files display and play in Finder, which I thought provided some underlying capability to display files to PM, but that doesn't seem to be the case here. This isn't critical for anything I do, but I wanted to mention it.

PM is primarily designed to work with still images.  It has some video features on the Mac version, but they're fairly minimal.  The Windows version has no video features whatsoever (other than it attempts to show thumbnails for movies if Apple's QuickTime is installed.)

I am not surprised that PM is unable to play these movies and I am not certain when PM will be able to play them.

-Kirk

[Rick Baumhauer]

Rick,

Yes, that would be ideal - I don't think I even need the XML files for anything I do, but having them wouldn't be a problem. Everything else in there besides the actual MP4 files can be ignored. Not having the AVCHD files is fine with me, as well - I don't plan to use the format for anything, but included it just to see how it effected the file structure on the card.

OK, but what we need to know, since I cannot find any documentation whatsoever, is what is going to work for everyone.  PRIVATE means to me: "hands off, ignore this, you don't know what to do here", whereas the DCIM folder is well-documented.

If we ignore specific files, you may be happy with it, but other people may not be.

We need to get to the bottom of this and find out officially what is important.  This may be near-impossible, unfortunately.

-Kirk

I totally understand - I wanted to bring it to your attention and see if there was anything that could be done, but can see why there would be pitfalls in many options at your end. It would certainly be my preference to be able to use PM as the first step in my workflow for all filetypes, but I understand that it's always been primarily a photo tool (it's right there in the name, after all), and anything beyond that gets a bit complicated.

If there is anything relatively innocuous that can be done with PM to facilitate importing the MP4 files into my normal file structure, that would be great, but I understand if it's really beyond what can be comfortably accomplished at your end without opening yourselves up to complications down the road.

[Kirk Baker]

Rick,

Yes, that would be ideal - I don't think I even need the XML files for anything I do, but having them wouldn't be a problem. Everything else in there besides the actual MP4 files can be ignored. Not having the AVCHD files is fine with me, as well - I don't plan to use the format for anything, but included it just to see how it effected the file structure on the card.

OK, but what we need to know, since I cannot find any documentation whatsoever, is what is going to work for everyone.  PRIVATE means to me: "hands off, ignore this, you don't know what to do here", whereas the DCIM folder is well-documented.

If we ignore specific files, you may be happy with it, but other people may not be.

We need to get to the bottom of this and find out officially what is important.  This may be near-impossible, unfortunately.

-Kirk

I totally understand - I wanted to bring it to your attention and see if there was anything that could be done, but can see why there would be pitfalls in many options at your end. It would certainly be my preference to be able to use PM as the first step in my workflow for all filetypes, but I understand that it's always been primarily a photo tool (it's right there in the name, after all), and anything beyond that gets a bit complicated.

If there is anything relatively innocuous that can be done with PM to facilitate importing the MP4 files into my normal file structure, that would be great, but I understand if it's really beyond what can be comfortably accomplished at your end without opening yourselves up to complications down the road.

Can I ask you some more questions?

The MP_ROOT folder contained a 100ANV001 folder but it was empty.  How did it get created and do you know why there is no file contained within it?
In the PRIVATE folder, there is an AVCHD file with no file extension.  I take it you shot that video while having the camera produce AVC files.  What happens when you shoot multiple clips?  Do you then start to get numbered AVCHD files?
If you shoot multiple MP4 clips, do you get numbered files?  Does the folder structure get more complicated?
What variables were you using to name your folders when your video files ended up in the wrong place?

Thanks,

-Kirk

[Rick Baumhauer]

Kirk - did some more testing.

The 100ANV001 folder is for "normal" MP4 videos (those not shot with the XAVC S codec, so h.264?). The files are named "MAH#####.MP4" with a corresponding .THM file.

The AVCHD file without extension actually shows as a package on the Mac - inside that is a BDMV package, containing the following folders: CLIPINF, PLAYLIST, and STREAM, along with index.bdm and MOVIEOBJ.BDM files. AVCHD videos are actually stored in the STREAM folder, named #####.MTS (I believe this corresponds to the file structure of Blu-ray discs, but am not positive).

Multiple video files of any format are numbered sequentially, with the caveat that "normal" MP4 files are numbered consecutively with photos in the DCIM folder (if I shoot a photo that is numbered DSC00032.arw, then shoot an MP4, the video will be named MAH00033.mp4). Unhelpfully, it appears that AVCHD and XAVC S files have their numbering reset on every card reformat, with XAVC S files named C####.MP4 and having a corresponding C####M01.XML file to accompany them (AVCHD resets numbering to 0, XAVC S to 1).

Folders appear to only be created as needed after a card reformat, but I have not seen any folders added beyond what I sent you.

My normal ingest copies to {yr4}/{mn0}/{day0}, naming files as {fnm4}_{h24}{min}{sec}. Neither seems to be producing any results with the XAVC S files, which end up in /default, named as in camera (C####.MP4). I just did a test with one photo, one normal MP4, one AVCHD, and one XAVC S MP4 on the card, and PM imported everything except for the THM file (which it knows to ignore) and the XAVC S file into the proper {yr4}/{mn0}/{day0} folder, including the XML file that accompanies the XAVC S file . The only thing copied to /default is the XAVC S video file. It renamed the photo and normal MP4 properly, but didn't rename anything else.

I can send you a ZIP of the card with those files if that will be helpful.

[Kirk Baker]

Rick,

Kirk - did some more testing.

The 100ANV001 folder is for "normal" MP4 videos (those not shot with the XAVC S codec, so h.264?). The files are named "MAH#####.MP4" with a corresponding .THM file.

The AVCHD file without extension actually shows as a package on the Mac - inside that is a BDMV package, containing the following folders: CLIPINF, PLAYLIST, and STREAM, along with index.bdm and MOVIEOBJ.BDM files. AVCHD videos are actually stored in the STREAM folder, named #####.MTS (I believe this corresponds to the file structure of Blu-ray discs, but am not positive).

Multiple video files of any format are numbered sequentially, with the caveat that "normal" MP4 files are numbered consecutively with photos in the DCIM folder (if I shoot a photo that is numbered DSC00032.arw, then shoot an MP4, the video will be named MAH00033.mp4). Unhelpfully, it appears that AVCHD and XAVC S files have their numbering reset on every card reformat, with XAVC S files named C####.MP4 and having a corresponding C####M01.XML file to accompany them (AVCHD resets numbering to 0, XAVC S to 1).

Folders appear to only be created as needed after a card reformat, but I have not seen any folders added beyond what I sent you.

My normal ingest copies to {yr4}/{mn0}/{day0}, naming files as {fnm4}_{h24}{min}{sec}. Neither seems to be producing any results with the XAVC S files, which end up in /default, named as in camera (C####.MP4). I just did a test with one photo, one normal MP4, one AVCHD, and one XAVC S MP4 on the card, and PM imported everything except for the THM file (which it knows to ignore) and the XAVC S file into the proper {yr4}/{mn0}/{day0} folder, including the XML file that accompanies the XAVC S file . The only thing copied to /default is the XAVC S video file. It renamed the photo and normal MP4 properly, but didn't rename anything else.

I can send you a ZIP of the card with those files if that will be helpful.

Yes, that would be most helpful.

-Kirk