Thoughts on interoperability.

[sgbotsford]

I've started the download of a trial for PM.  Hopefully it will be done by tomorrow morning. (Hard to get fast internet in Rural Alberta...)

PM looks like a fine tool.  But it doesn't do everything.  One of the limits on it's spread is interoperability with other applications.

Some of these, such as Aperture have an SDK for programmers to use to create plugins.

Since PM is an auxilary tool -- or rather a component tool -- in a workflow, then increasing interoperability increases it's usefulness.

Consider the following:

PM publishes an API+SDK that allow other vendors to READ PM data.

PM developers use other vendors APIs to READ their data.

In addition PM data carries a timestamp whenever something is changed.

PM encourages other vendors to do similar time stamping.

Now what happens:

PM writes keywords and a time stamp.

Aperture (which through some back and forth discussion, supports PM) can check the timestamps, and update it's own keyword data with the those files that have changed.

In aperture you see a keyword you missed.  You add it, as well as writing a caption.  Both of these are added to the file/sidecar, along with timestamps.

PM sees that the timestamps have changed, and updates accordingly.

Someone has to start. 

Is this workable?

[Kirk Baker]

I've started the download of a trial for PM.  Hopefully it will be done by tomorrow morning. (Hard to get fast internet in Rural Alberta...)

PM looks like a fine tool.  But it doesn't do everything.  One of the limits on it's spread is interoperability with other applications.

Some of these, such as Aperture have an SDK for programmers to use to create plugins.

Since PM is an auxilary tool -- or rather a component tool -- in a workflow, then increasing interoperability increases it's usefulness.

Consider the following:

PM publishes an API+SDK that allow other vendors to READ PM data.

PM developers use other vendors APIs to READ their data.

In addition PM data carries a timestamp whenever something is changed.

PM encourages other vendors to do similar time stamping.

Now what happens:

PM writes keywords and a time stamp.

Aperture (which through some back and forth discussion, supports PM) can check the timestamps, and update it's own keyword data with the those files that have changed.

In aperture you see a keyword you missed.  You add it, as well as writing a caption.  Both of these are added to the file/sidecar, along with timestamps.

PM sees that the timestamps have changed, and updates accordingly.

There is nothing proprietary about the data PM puts in images.  Most of it is industry standard IPTC and XMP.  ExifTool is capable of understanding our only undocumented data (end of file preferences) and since ExifTool actually documents our data on their webpages it is in a way documented.

Aperture can just check the modification date of the files.

As long as everyone continues to use XMP, I don't think any API needs to be created.

-Kirk

[sgbotsford]

You may be right. 

Is file mod time sufficient?  Consider the following scenario.  PM is running and AppX (Doesn't have to be aperture) is running.  In PM I make a change in keyword.  In AppX I make a change in caption.  Depending on how frequently they check file mod times. there is a race condition where one change gets lost.  This is a general problem when two processes access the same files.  Hence my suggestion to include timestamps in the XMP file as to the actual changes.

In another discussion on this board (I think... I've been surfing a lot of different photo boards this week.. ) there was an issue about date changes in one program not showing up in another.  One program used EXIF date the other used ITPC date. 

If PM knows about other programs flakyness, it can compensate.

Ultimately is your call.  I'm trying to suggest how PM can better fit into workflows that involve other programs' non-standard behaviour. If someone who is a diehard AppX fan runs into an interoperability problem while trialing PM, they are far more likely to not follow up a trial with a purchase.  Thus any interoperability problems you can fix at your end should result in increased sales. 

You can accelerate the move toward standard usage (which I think ultimately favours the smaller developers like you...) by publishing the standards, stating clearly what your practice is where they conflict, and by fostering discussion amoung users and vendors alike.  By doing so you create brand awareness of PM as not only a great photo program, but as a communication tool; something that fosters data communicaiton between different programs.

May not be worth your time.  I can't make that decision. 

[TiggerGTO]

If PM knows about other programs flakyness, it can compensate.
...
May not be worth your time.  I can't make that decision. 

Asking PM to compensate for the "flakyness" of other programs is sort of like asking for the impossible.  The guys at CameraBits already do an amazing job making PM one of the most flexible and reliable programs I have ever used.  If I didn't have PM to act as my ingester / browser / metadata editor, I would have ditched NX2 ages ago.  The PM developers have bent over backwards to make PM as compatible as possible with NX2 which is not a program that plays well with others.  However there are some things that they simply can not compensate for.  For example, if I have a file open in NX2 and try to change the color or star rating in PM, the change is lost.  I'm not exactly sure what is going on, because sometimes I get an error message saying the file is locked, but sometimes the change gets lost with no indication.  Sometimes PM can't do things to any files in a directory (folder) simply because NX2 has its browser open on the same directory.  I don't know what NX2 is doing to the files or directory, but that's what I mean by not playing well with others.  I've simply learned that when I want to use PM to change metadata, I need to make sure that none of the files are open in NX2's editor and the directory is not open in NX2's browser.

In case it matters, the above discussion is for running on Windows.  I've recently switched to a MacBook Pro for my primary photography machine but haven't had enough time with my workflow in this new environment to tell if these kinds of conflicts are better or worse on a different OS.

[sgbotsford]

Windows tends to be bad for locking everthing in sight.  If program A has a file open, then B often can't even read it.

As to interoperability:

There are two possible approaches generally to photo management:

1.  The program has to do everything -- import, tag, edit, export, track.

2.  Users assemble their own work flow with a set of tools.

The problem with approach 1 is that different people have different needs.  Aperture has pretty good, but not great editing.  If you want more, you have to export to photoshop, then import it again as a new image.

Aperture keeps it's tags in alphabetical order, and on export, you lose the hierarchy.

Star ratings in aperture get lost elsewhere.

The advantage of approach 2 is that people get the tools to fit what they need to do.  For any given product, the greater the number of programs it can interoperate with, the better the chance it will fit into some one's workflow.

It's moving target.  And you won't be able to work with everyone.  However the following steps make it easier for everyone:

1.  Have a published standard for how you do things.  This should be detailed enough that a good program can read and parse your datafiles.  PM AFAIK uses a pretty standard XML format for this.

2.  Adopt standards as quickly as possible.  E.g. IPTC extensions, PLUS,

3.  Provide conversion tools to get that data to/from  other formats.  Example:  XML to Comma Separated Variable (CSV) or excel. 

4.  Provide mapping tools to cheat so that you can repurpose fields for other things if necessary.  (Yes, now you are doing non-standard things -- but when the standards don't work, you have to improvise)  This already happens on a wide scale.  Newspaper and magazine use of metadata is quite different.

5.  Provide web page resources for how to get your product to work with other programs.  This will be a collection of recipes and gotchas. 

6.  Provide a list of niche products that would make life easier.  For example, a command line tool that would extract star ratings from Aperture, and update ratings in other programs.  In some cases, no such program exists. 

By providing the interop guidelines, you gain credibility (and customers) in the field.


I too am building a work flow.  And I have a fair number of unsolved issues:

*  Tracking derivative images.  Aperture does a partial job with it's concept of master and versions -- but it has no mechanism for different resolutions of an image.  Nor have I found a way to track where you used a given version.  It also doesn't do well with external editors.

*  Finding similar and duplicate images (Photosweepr)

*  I'd like to be able to use controlled vocabulary for fields other than keywords.  E.g. for location, and for people.

*  I'd like to be able to exchange data with facebook.  Extract people tags from forums, and use them as meta data.  (I'm working on an alumni project -- I don't know all the alumni...)

[Luiz Muzzi]

Hi, everybody
I also use PM with a MacBook Pro and I rarely encounter problems of interoperability. In my workflow I use Photoshop when necessary (for minor adjustmensts I use Canon's DPP).
Of course DPP is a program that has some shortcomings but it is free and I really like the jpeg conversion it does...so I keep using it.
Regards,

-Luiz Muzzi

[sgbotsford]

The chief concern about interoperability is in metadata:

Can you apply a keyword in PM, look at, and modify that keyword list in something else, and have the changes reflected in PM.

Can you apply metadata in Photoshop/Adobe Bridge and have those words show up in PM?

If you edit an image in photoshop, is all your metadata kept, or do you lose some of it?

When you convert image formats do the new images still have all the metadata in them?

[Hayo Baan]

The chief concern about interoperability is in metadata:

Can you apply a keyword in PM, look at, and modify that keyword list in something else, and have the changes reflected in PM.

Yes

Can you apply metadata in Photoshop/Adobe Bridge and have those words show up in PM?

Yes

If you edit an image in photoshop, is all your metadata kept, or do you lose some of it?

Yes

When you convert image formats do the new images still have all the metadata in them?

Yes

[ron_hiner]

There are well documented and well implemented standards on how to embed metadata into image files.  Photomechanic adheres to those perfectly, and near as I know.

However, now that we have tens of thousands of images on our computers, any program that wants to read the metadata has to open each an every one of them.  So, if you want to find your 4 star and better images of Julia Roberts, for example, your search tool has to open up all those files to read the metadata.  That is a time consuming process.

This is precisely why catalog applictions are necessary.   Instead of file systems, they use databases to contain the data -- databases are much faster to search.

Catalog tools exist: there is one built into both Lightroom and Aperture.  There is also Phase One's catalog app.   And Camerabits has one in the works.   Those database can't possibly know what has changed in each individual file, without opening each file and looking again.   

Google (and other search engines) index files all the time... they simply apply massive amounts of computing power to dowload every web page they can find and store it in their index.  They also download every image they can find and store and index the metadata.

So it's not about PM being more accessable to other apps -- cataloging apps just simply have to be able to discover and respond appropriately to changed image file metadata.   

And one thing most of us have on our computers is massive unused processing capacity -- constant re-indexing can be run in the background when the computer is not otherwise in use.   I don't know of a catalog program that can do this.

It's my opinion that any catalog appication needs to be able work with whatever standardized metadata that is in the image files, placed there by PM or any other application.   Sadly, both Aperture and Lightroom operate under the assumption that each respective tool can solve any problem you will ever have, so they don't  make updates to the image file metadata upon changes in their catalogs.  (Maybe this has changed, I last test both products for this a couple releases ago.)

I'm hoping that the CameraBits catalog application does not share that philosophy that both Apple and Adobe have followed.  We'll know soon.