Author Topic: auto-ingest depends on the timezone the card was created in  (Read 47 times)

Offline braver

  • Member
  • **
  • Posts: 51
    • View Profile
I have a reproducible Incremental Ingest problem while travelling between time
zones. An existing Leica card resumes correctly under the Mac's original time
zone, but PMP recopies previously ingested files after the Mac changes time zone
and the card is remounted. The images' paths and sizes have not changed.

Environment
- Photo Mechanic Plus; installed app reports 2026.6, build 9175 (2cfdfbb),
  CFBundleVersion 2026.6.9175.
- macOS 26.2, build 25C56, Apple Silicon.
- Leica M10-R, firmware 30.22.23.34; 256 GB SDXC card, exFAT,
  262,144-byte allocation units.
- Original ingest history created while the Mac used Pacific daylight time
  (UTC-7); subsequent mounts in Shanghai (UTC+8) and Tokyo (UTC+9).

Reproduction
1. Ingest part of a card using PMP on a Mac in Pacific daylight time. Keep the
   existing card contents and its .pmingest.dat GUID.
2. Quit PMP and unmount the card.
3. Change the Mac to Shanghai or Tokyo time, then remount the same card.
4. Use Incremental Ingest with the same source card and existing history.

Actual: PMP starts copying old files again.
Expected: unchanged, previously ingested files should remain recognised across
host timezone changes, and ingestion should resume with genuinely new files.

Measured example
Card file: /DCIM/100LEICA/ALXY8703.DNG
Size in all cases: 42,484,224 bytes
Saved history mtime: 1789413132
Native stat mtime after mounting in Shanghai: 1789359132 (minus 54,000 seconds)
Native stat mtime after mounting in Tokyo:    1789355532 (minus 57,600 seconds)

In the first comparison, 4,230 of 4,232 historical entries had identical sizes
and a uniform minus-15-hour timestamp shift. The two exceptions had been
recopied during an interrupted ingest under the other timezone.

After a later interrupted ingest, all 7,777 historical images on a Tokyo mount
had unchanged sizes and a uniform minus-57,600-second shift. Remounting matters:
changing the Mac's timezone while leaving the card mounted did not reproduce
the shift in our test; remounting under the new zone did.

Raw exFAT inspection
We read the actual directory entries. All 9,037 DNG/JPG files had 0x00 in both
CreateUtcOffset and LastModifiedUtcOffset: the offset-valid bit is clear. For
ALXY8703.DNG the on-card modification date/time is 2026-09-14 12:12:12, with an
unspecified UTC offset.

This is consistent with exFAT's documented fallback to the current local UTC
offset when the offset-valid bit is clear. Epoch seconds themselves are not
timezone-dependent; conversion from this card's offset-less wall-clock fields
to epoch seconds changes between mounts. This report is not claiming that
editing the camera's EXIF capture date causes the problem, or that macOS's
fallback behaviour violates the specification.

Successful diagnostic workaround
We built a read-only virtual exFAT disk that forwards original image bytes and
the same .pmingest.dat GUID, but supplies stable modification epochs matching
the existing history. Its virtual directory entries use explicit valid UTC
offsets. No original images or history files were edited by the proxy.

All 7,777 history entries matched this virtual disk by path/size/mtime. The next
DNG, ALXY1480.DNG (39,848,960 bytes), had an identical full-file SHA-256 through
the virtual disk and the physical card. Selecting the virtual volume under
PMP's Disks successfully skipped the old files and resumed at ALXY1480.

Related finding: disk versus folder history paths
An initial WebDAV proxy exposed the same GUID, bytes, sizes and stable mtimes,
but PMP did not list it under Disks, so it was selected as a folder. PMP then
recopied ALXY7589 and ALXY7590 and appended entries to the same GUID history
file using a different path spelling:

Existing disk entry:
/DCIM/100LEICA/ALXY7590.DNG<TAB>43127808<TAB>1788669602

New folder entry:
DCIM/100LEICA/ALXY7590.DNG<TAB>43127808<TAB>1788669602

Both entries have exactly the same size and mtime. Only the leading slash
differs. Please confirm whether this difference is intentional. Canonicalising
equivalent root-relative paths for the same card GUID would avoid this trap
when a source must be ingested as a folder.

Suggested investigation
Please consider making incremental identity robust to offset-less exFAT
timestamps being reinterpreted under another host timezone. Possible approaches
include retaining the original filesystem wall-clock fields/offset-valid
information, or recording the host offset used when creating history and
carefully recognising a uniform mount-time timezone shift. Simply ignoring
mtime changes could conceal genuine source-file edits, so that would need a
separate policy or stronger identity verification.

I can provide the relevant history lines, raw directory entries and comparison
results. The workaround confirms that stable filesystem epochs plus disk-style
paths allow the existing history to resume correctly.

References
Microsoft exFAT specification, section 7.4.10.2 (OffsetValid):
https://learn.microsoft.com/en-us/windows/win32/fileio/exfat-specification#74102-offsetvalid-field

Offline braver

  • Member
  • **
  • Posts: 51
    • View Profile
Re: auto-ingest depends on the timezone the card was created in
« Reply #1 on: Yesterday at 08:02:21 PM »
Since I needed this resolved while traveling, I found two solutions with AI:

(1) Just change the time zone of the MacBook to the original time zone when the card was created, in my case, Pacific (SF).

(2) We created a project, DiskProxy, with Claude that can mount a card as a virtual disk with any preprocessing we need.  I can set the card's timezone to UTC now, and we can also hide the files already in the card's PM ingestion history.  It uses WebDAV, which I first discovered in 2000 for Subversion! (My contribution was hosting Subversion at an apex domain...). Amazing what we can do with AI these days.  I've generalized diskproxy on purpose and made the PMP use case a plugin.  The above bug report was generated with Claude too.  I found that we can fix lots of bugs and understand so much more with AI!  I have it verify the catalog and repair it if bugs prevent me from updating tree locations, for example.  Camera Bits folks: I urge you to reconsider Plus now that you can have this powerful assistant by your side.  No matter how complex the catalog logic is, even from the outside Claude fully understands the SQLite format, can compare files to physical disks, etc.

https://github.com/firstpair/diskproxy