Sequence number reset on ingest?

[dan]

I see this has been asked a couple times, but I'm hoping it's been fixed and I'm just missing it.

I just bought PM and would love to replace my complex perl script that imports my images with PM importing. However, I have about 30k photos that I have filed in a particular way and would hate to have to change it. My scheme is 2007_0916/09160001.jpg ... etc, so I figured doing {year4}_{mn0}{day0}/{mn0}{day0}{seqn} would fix me right up. Unfortunately {seqn} never resets. Is there any way to restart the sequence number with every date? I guess even cooler would be a variable options, like {seqn:{mn0}{day0}} so it could reset whenever the combo of {mn0}{day0} changed; then people with different organization schemes could do {seqn:{year4}} or whatnot.

Thanks for any advice,


dan

[Kirk Baker]

Dan,

I see this has been asked a couple times, but I'm hoping it's been fixed and I'm just missing it.

I just bought PM and would love to replace my complex perl script that imports my images with PM importing. However, I have about 30k photos that I have filed in a particular way and would hate to have to change it. My scheme is 2007_0916/09160001.jpg ... etc, so I figured doing {year4}_{mn0}{day0}/{mn0}{day0}{seqn} would fix me right up. Unfortunately {seqn} never resets. Is there any way to restart the sequence number with every date? I guess even cooler would be a variable options, like {seqn:{mn0}{day0}} so it could reset whenever the combo of {mn0}{day0} changed; then people with different organization schemes could do {seqn:{year4}} or whatnot.

Interesting.  Well, I talk to the rest of the team about this need and let you know what we come up with.  A solution like what you're talking about would also be useful in filename renaming, IPTC Stationery Pad, etc.

-Kirk

[Hayo Baan]

Kirk, this indeed would be a nice feature and would also allow (for instance) for automatic sequencing within subfolders (where each subfolder numbers again from 1). I think a lot of people would welcome that addition.

Cheers

[dan]

Hi Kirk,

Just wanted to check and see if anything had come up with respect to this. I'd be super-excited to discard my crufty perl script!


dan

[Kirk Baker]

Dan,

Just wanted to check and see if anything had come up with respect to this. I'd be super-excited to discard my crufty perl script!

We discussed it and it isn't doable because when Ingesting, your files are not sorted by date (well it is possible that they may be sorted by date but it isn't guaranteed) so what can happen is that at worst the sequence number could get reset for every single file that is ingested.

In order for this to work properly, Ingest would have to scan the imported directory or directories, sort all images by their capture date or creation date/modification date and then copy them in that order.  Ingest currently does not do this, and if it were to do this it would take a lot longer because it would have to parse each file to determine its capture time (and some files may not be images at all and wouldn't have that data anyway.)

So if we can come up with a way to do this and make it not have much impact on the speed of Ingest (which is vitally important to many of our users) then we would likely implement such a feature.

Ideas are welcome.

-Kirk

[dan]

So if we can come up with a way to do this and make it not have much impact on the speed of Ingest (which is vitally important to many of our users) then we would likely implement such a feature.

So I can think of a couple possible solutions.

1) Assume files are sortable alphabetically and that the sort order approximates date/time order. This would generally work for DSC_0000-named files, except when the number is reset or rolls over.
2) Scan the filesystem and sort files based on the filesystem's timestamp. I don't have a DSLR handy, but a random Canon Powershot s500 shows a definite timestamp ordering on the memory card:

2007-05-07 14:18:52.000000000 -0400 IMG_9697.JPG*
2007-05-07 14:19:12.000000000 -0400 IMG_9698.JPG*
2007-05-07 14:19:38.000000000 -0400 IMG_9699.JPG*
2007-05-07 14:20:18.000000000 -0400 IMG_9700.JPG*

That would fix the rollover issue, and files wouldn't have to be parsed for capture time. I can't think of a situation where this would lead to problems that scanning all files for capture time would fix, but I don't do anything particularly complicated either.

Let me know what you think.


dan

[Kirk Baker]

Dan,

So if we can come up with a way to do this and make it not have much impact on the speed of Ingest (which is vitally important to many of our users) then we would likely implement such a feature.

So I can think of a couple possible solutions.

1) Assume files are sortable alphabetically and that the sort order approximates date/time order. This would generally work for DSC_0000-named files, except when the number is reset or rolls over.
2) Scan the filesystem and sort files based on the filesystem's timestamp. I don't have a DSLR handy, but a random Canon Powershot s500 shows a definite timestamp ordering on the memory card:

2007-05-07 14:18:52.000000000 -0400 IMG_9697.JPG*
2007-05-07 14:19:12.000000000 -0400 IMG_9698.JPG*
2007-05-07 14:19:38.000000000 -0400 IMG_9699.JPG*
2007-05-07 14:20:18.000000000 -0400 IMG_9700.JPG*

That would fix the rollover issue, and files wouldn't have to be parsed for capture time. I can't think of a situation where this would lead to problems that scanning all files for capture time would fix, but I don't do anything particularly complicated either.

Let me know what you think.

I thought you wanted to ingest folders from your hard drive?  Wouldn't these files be likely to have been renamed?

In any rate, we don't like to add features that 'just work some of the time'.

-Kirk

[dan]

Hi Kirk,

I thought you wanted to ingest folders from your hard drive?  Wouldn't these files be likely to have been renamed?

Nope, I want to ingest from my digital camera memory cards, conforming to the naming scheme I've already got on my drive.

In any rate, we don't like to add features that 'just work some of the time'.

I think the second solution would work all of the time. The worst-case scenario I can think of is of someone essentially "re-ingesting" a random collection of files that were taken at different times, but were edited or otherwise saved in a non-time order. I can't think of a situation in which I'd do that, but I'm not a photojournalist or otherwise "hardcore" user. That would seem to be less of "working some of the time," as you put it, as a problem managing expectations, which could easily be solved by terming the feature "disk chronology index" or something to make clear that it's working off of the time values stored on the drive as opposed to those in the photos.

Thoughts?


dan

[IanGoldstein]

Perhaps the ability to have a "local" vs. "global" sequence variable would work here?

The current sequence variable in PM is a "global" one in the sense that unless it is manually reset it is always automatically incremented. A "local" sequence variable could be added which would be applicable only within the current directory. The current value of this local sequence variable could be stored in a .pmlocal file, much like the way a .pmarrangement file is kept (when needed) in a directory.

As an example:

  The files could be ingesting using {year4}_{month0}{day0}/{month0}{day0}{localsequence}
  When PM evaluates {localsequence} it would check the current directory in use for a .pmlocal file
  If a .pmlocal file exists, the value is read, incremented and re-written to the .pmlocal file
  If a .pmlocal file does not exist, the sequence starts at 1 and the .pmlocal file is created

I imagine this could be used in quite a few places other than just ingest and would be a useful addition to PM.

-Ian