Author Topic: Multiple delete of duplicated files possible ?  (Read 9404 times)

Offline reefer

  • Member
  • **
  • Posts: 56
    • View Profile
Multiple delete of duplicated files possible ?
« on: March 25, 2012, 11:18:11 AM »
Hi,

Is it possible to do the above ? During an ingest for some reason I have got duplicates of several files, I could go through them manually and delete but I feel this will be too time consuming due to the amount of files I need to remove.

Is there a way I can delete the multiple dupliate files that have been appended alphabetically, ie I have a file named CA1O2509.jpg, the duplicate is CA1O2509A.jpg

Any ideas ?

Thanks,

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2552
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Multiple delete of duplicated files possible ?
« Reply #1 on: March 25, 2012, 11:56:01 AM »
This can not really be done in PM automatically, but if you're not afraid of using the command-line, it can actually be quite easy (and safely).

Run the following on the Mac command-line in the directory with the images you have doubles for:
for a in *.jpg; do if [[ -e ${a/.jpg/A.jpg} ]]; then rm ${a/.jpg/A.jpg}; fi; done

To test it out change rm in echo and you'll get a list of files it will remove this way.

Hope this helps,
Hayo
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25024
    • View Profile
    • Camera Bits, Inc.
Re: Multiple delete of duplicated files possible ?
« Reply #2 on: March 25, 2012, 12:56:11 PM »
Is it possible to do the above ? During an ingest for some reason I have got duplicates of several files, I could go through them manually and delete but I feel this will be too time consuming due to the amount of files I need to remove.

Is there a way I can delete the multiple dupliate files that have been appended alphabetically, ie I have a file named CA1O2509.jpg, the duplicate is CA1O2509A.jpg

Are you ingesting the same card without formatting the card in your camera before you shoot additional photos?  If so, use the 'Incremental Ingest' checkbox and Ingest will ignore photos it has already ingested from that card.

-Kirk

Offline reefer

  • Member
  • **
  • Posts: 56
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #3 on: March 26, 2012, 02:56:57 AM »
Kirk,

Many thanks, I've now set that up. I assume there is no way I can speedily remove this duplicates via PM now that they are ingested though ?

Offline reefer

  • Member
  • **
  • Posts: 56
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #4 on: March 26, 2012, 02:57:54 AM »
This can not really be done in PM automatically, but if you're not afraid of using the command-line, it can actually be quite easy (and safely).

Run the following on the Mac command-line in the directory with the images you have doubles for:
for a in *.jpg; do if [[ -e ${a/.jpg/A.jpg} ]]; then rm ${a/.jpg/A.jpg}; fi; done

To test it out change rm in echo and you'll get a list of files it will remove this way.

Hope this helps,
Hayo

Thanks Hayo, I'm a little worried about going this route for fear of messing it up and losing images that I want to keep.

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25024
    • View Profile
    • Camera Bits, Inc.
Re: Multiple delete of duplicated files possible ?
« Reply #5 on: March 26, 2012, 06:28:23 AM »
Many thanks, I've now set that up. I assume there is no way I can speedily remove this duplicates via PM now that they are ingested though ?

You could try doing a Find (in filenames) for the letter A

This should select all of the files with an 'A' in them.  Review the selection and then use the Delete command on the Edit menu if everything looks good.

HTH,

-Kirk

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2552
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Multiple delete of duplicated files possible ?
« Reply #6 on: March 26, 2012, 12:05:37 PM »
If your filenames are all the same length, you could also try sorting them using e.g., the following custom sort:

{filenamebase:8}_{filenamebase:0,8}

This will put all the files with an "A" as last (actually, ninth) character in their name, together. This allows you to review the duplicates and remove them easily.

Hope this helps,
Hayo
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline reefer

  • Member
  • **
  • Posts: 56
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #7 on: March 26, 2012, 01:19:41 PM »
Many thanks, I've now set that up. I assume there is no way I can speedily remove this duplicates via PM now that they are ingested though ?

You could try doing a Find (in filenames) for the letter A

This should select all of the files with an 'A' in them.  Review the selection and then use the Delete command on the Edit menu if everything looks good.

HTH,

-Kirk



Thanks Kirk, I tried but couldn't get it to work via this method

Offline reefer

  • Member
  • **
  • Posts: 56
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #8 on: March 26, 2012, 01:20:52 PM »
If your filenames are all the same length, you could also try sorting them using e.g., the following custom sort:

{filenamebase:8}_{filenamebase:0,8}

This will put all the files with an "A" as last (actually, ninth) character in their name, together. This allows you to review the duplicates and remove them easily.

Hope this helps,
Hayo


Good news... that works Hayo!  Many thanks, you just saved me some time  ;D

Offline reefer

  • Member
  • **
  • Posts: 56
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #9 on: March 26, 2012, 01:27:33 PM »
Is it possible to do the above ? During an ingest for some reason I have got duplicates of several files, I could go through them manually and delete but I feel this will be too time consuming due to the amount of files I need to remove.

Is there a way I can delete the multiple dupliate files that have been appended alphabetically, ie I have a file named CA1O2509.jpg, the duplicate is CA1O2509A.jpg

Are you ingesting the same card without formatting the card in your camera before you shoot additional photos?  If so, use the 'Incremental Ingest' checkbox and Ingest will ignore photos it has already ingested from that card.

-Kirk



Kirk, I shoot with 2 camera's using 3 memory cards in rotation as I have to swap cards at very frequent intervals. Just to be clear I should always format in camera after a card ingest even though I currently have the 'erase source disk after ingest' checked ?

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25024
    • View Profile
    • Camera Bits, Inc.
Re: Multiple delete of duplicated files possible ?
« Reply #10 on: March 26, 2012, 01:31:53 PM »
Is it possible to do the above ? During an ingest for some reason I have got duplicates of several files, I could go through them manually and delete but I feel this will be too time consuming due to the amount of files I need to remove.

Is there a way I can delete the multiple dupliate files that have been appended alphabetically, ie I have a file named CA1O2509.jpg, the duplicate is CA1O2509A.jpg

Are you ingesting the same card without formatting the card in your camera before you shoot additional photos?  If so, use the 'Incremental Ingest' checkbox and Ingest will ignore photos it has already ingested from that card.

Kirk, I shoot with 2 camera's using 3 memory cards in rotation as I have to swap cards at very frequent intervals. Just to be clear I should always format in camera after a card ingest even though I currently have the 'erase source disk after ingest' checked ?

Absolutely, especially if you're switching camera bodies.  The erase option just deletes the files, no format is done by PM.

-Kirk

Offline Hayo Baan

  • Uber Member
  • ******
  • Posts: 2552
  • Professional Photographer & Software Developer
    • View Profile
    • Hayo Baan - Photography
Re: Multiple delete of duplicated files possible ?
« Reply #11 on: March 27, 2012, 08:43:46 AM »
Good news... that works Hayo!  Many thanks, you just saved me some time  ;D

Glad to be of service :)
Hayo Baan - Photography
Web: www.hayobaan.nl

Offline BrianS

  • Newcomer
  • *
  • Posts: 1
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #12 on: August 11, 2013, 04:12:57 PM »
Just wanted to chime in and say thanks for this info.  I accidentally selected my secondary destination to be the same as my first.  Of course I had about a thousand duplicates that I needed to get rid of.  You saved me a ton of time!!

Cheers!

Brian


Offline Andrew

  • Member
  • **
  • Posts: 61
    • View Profile
Re: Multiple delete of duplicated files possible ?
« Reply #13 on: August 24, 2013, 11:10:04 AM »
I feel it is not a good idea to erase files automatically on ingest.  The ingest may have failed or files may have been corrupted.  It is safer to check that ingest has been successful before deletion from the card (and format in camera is the best way to do this.

Also, if you are travelling, you may want to keep the original card as a back-up until you return home and can run your normal back-up.  Extra cards are cheap.