Author Topic: 2nd card from camera causing duplicates when ingesting  (Read 5026 times)

Offline schlotz

  • Member
  • **
  • Posts: 63
    • View Profile
2nd card from camera causing duplicates when ingesting
« on: April 21, 2016, 06:55:55 AM »
This issue was brought up in this thread below.  My two possibilities are at the bottom.

Quote from: ErwinScheriau on April 18, 2016, 09:36:43 am
Most time (especially when doing sports) I`m shooting on two cards, card #2 is sort of backup.
Now here's the problem: I do ingest pics from card #1, when something important happens during ingest, I continue shooting on card #2. But when I want to ingest the pics from card #2 PM is importing all the pics, even those I already have imported from card #1.
As you can imagine this is a big timeloss when there are 300 or more pics on the card.
Any Ideas?

Quote from Kirk here:
So you've got a situation where card #2 has all of the images of card #1, plus new images that card #1 doesn't have?

PM has no way of telling that the two cards are related.  PM puts an ID file on your cards when an Ingest starts and each card will have a different ID.  This ID is used to look up what files have been copied for that particular ID.  Any files that haven't been copied for that card ID are copied.

It's not feasible to do what you ask.  It's a hard problem to solve since at the time of card #1 being Ingested, card #2 is not known to PM.

PM would have to search all card IDs and compare your card #2's files against them to see if a file has ever been ingested from all known sources.  To be completely certain, PM would have to use more than filename, date, and file size information for the comparison.  It would have to checksum each file first and the database would have to be extended to include a checksum.  This could be time consuming.  All Incremental Ingests would have to be exhaustive searches.

-Kirk

The OP describes a real-time issue many sports photographers run into. Right before half time you are searching for a few to be published.  A play ensues so you pick up the camera and fire, now card two has some photos that card 1 does not have.  At the end of the game you want to ingest from card 2 since it has all of the photos but doing so creates duplicates. 

It possibly could be easy though.  What if the fine people at CameraBits were to add a check box & logic (if used, the ID logic is abandoned & replaced with filename lookup) to allow by-passing any duplicate file it encounters during the ingest process?  JMTC  :)  While the filename check process just mentioned will most likely be slower, not having to cull out 100's of duplicates when under a time crunch at the end of a game might make up the difference plus some. 

ALTERNATE (possibly the BETTER SOLUTION): save the ID file from the first card to the computer's hard drive and thru a check box on the ingest window, allow it to be used for the pending ingest of a different card.  Obviously the default would be set to 'off' AND it would be toggled off automatically after the ingest is done.

Matt

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: 2nd card from camera causing duplicates when ingesting
« Reply #1 on: April 21, 2016, 09:14:33 AM »
ALTERNATE (possibly the BETTER SOLUTION): save the ID file from the first card to the computer's hard drive and thru a check box on the ingest window, allow it to be used for the pending ingest of a different card.  Obviously the default would be set to 'off' AND it would be toggled off automatically after the ingest is done.

The second card could already have an ID number on it.  Assigning it a different ID could result in images that had already been ingested being ingested again.

If we add a solution to this problem, we wouldn't want to create new problems.

-Kirk

Offline schlotz

  • Member
  • **
  • Posts: 63
    • View Profile
Re: 2nd card from camera causing duplicates when ingesting
« Reply #2 on: April 21, 2016, 09:44:18 AM »
Totally agree. With the ingest window open, this purposed option is defaulted to always be off, only by checking it will the 2nd CARD ID incremental ingest file logic be modified to instead use the previously saved ID file from card#1 to check for duplicates while ingesting from card #2.  The check box would be automatically removed at the end of card#2 ingest including writing the ID files for card#2 - ergo back to normal.  Basically a one time shot ability to pick up the added photos that only reside on card#2.  IMO, certainly worth serious consideration.
« Last Edit: April 21, 2016, 03:51:54 PM by schlotz »