live ingest

[Matthew]

Hello,

I’m running Photo Mechanic Version 5.0, build 18359 (5034d39) on an Intel Mac (10.12)

I’m having a few sporadic issues with the Live Ingest feature in Photo Mechanic and was wondering if anyone could help?

I have my home server configured to run a FTP service (Mac OS Server) which is set up receive pictures from my wifi-enabled cameras. These are sent using a mobile broadband connection when I’m out in the field.

Within Photo Mechanic, the Live Ingest source folder is set to be the root FTP folder so when a picture is seen it’s captioned, renamed and moved on.

The problem I am having is that occasionally the Live Ingest feature stops processing the JPGs and a backlog of files can build up in the Live Ingest source folder. To fix this I need to restart the Live Ingest process which clears the backlog and the files are moved.

I am only sending .JPG files. I have attached a screenshot of my Live Ingest settings page.

Is there any reason this is happening?

Thanks in advance,

Matthew

[Kirk Baker]

I'd like to see your Photo Mechanic log.  In Photo Mechanic go to the Help menu and choose "Reveal Support Data..." and moments later a Finder window will come forward with the zipped log file selected.  Use the 'Attachments and other options' link when you're composing your reply to this message and there you'll be able to upload your zipped PM log file.

Thanks,

-Kirk

[Matthew]

Here you go. The problem happened last around June 2, 3 and 4.

Files were named MH_030617_Champions_League_Festival

[Kirk Baker]

Matthew,

Here you go. The problem happened last around June 2, 3 and 4.

Files were named MH_030617_Champions_League_Festival

Looks to me like your source folder was a "Dropbox" which usually means that files can be created and written there, but cannot be read from there.  I don't know if your "/Volumes/MEDIA/Dropbox/Pictures/LIVE/" is really set up that way, but I can tell that Live Ingest is having trouble accessing files at that location.

-Kirk

[Matthew]

Hi Kirk,

Thanks for looking into this.

Changing the Live Ingest source location to a location outside of the Dropbox has fixed this.

[Matthew]

Hi Kirk,

Sorry to bring this up again but I'm still having this problem - despite the Live Ingest source location now being outside of a Dropbox folder.

Since my last message I've upgraded to High Sierra and as Apple have ditched FTP in Mac OS Server I'm now running Crush FTP.

The FTP location where pictures are received is /MEDIA/FTP and the destination is within the Dropbox folder at /MEDIA/Dropbox/Pictures/Live. This is a SSD drive using the new Apple filesystem.

I can't understand what is causing PM's ingest feature to seize up. I'm not always sending from areas with good signal and so it's possible that on occasion the FTP transfer from the camera will stop and then start again shortly after... could this be the issue?

[Kirk Baker]

Hi Kirk,

Sorry to bring this up again but I'm still having this problem - despite the Live Ingest source location now being outside of a Dropbox folder.

Since my last message I've upgraded to High Sierra and as Apple have ditched FTP in Mac OS Server I'm now running Crush FTP.

The FTP location where pictures are received is /MEDIA/FTP and the destination is within the Dropbox folder at /MEDIA/Dropbox/Pictures/Live. This is a SSD drive using the new Apple filesystem.

I can't understand what is causing PM's ingest feature to seize up. I'm not always sending from areas with good signal and so it's possible that on occasion the FTP transfer from the camera will stop and then start again shortly after... could this be the issue?

If the access control lists of the Dropbox folder are inherited by the Pictures/Live folder then they're essentially dropbox folders too.  Is there a reason you need dropbox semantics?

As for stalled uploads, Live Ingest should patiently wait for the images to finish.  But if it can't read them then it isn't going to work.  Live Ingest must be allowed to read directories, write directories, and read files in order to function correctly.

-Kirk

[Matthew]

Hi Kirk,

Just to be clear, the Dropbox folder is only now being used as a destination - the source folder is outside the Dropbox folder.

Yes, it’s key the output folder is a Dropbox one because the idea of the setup is the images are automatically uploaded after being received into the folder. Most of the time it works absolutely fine but occasionally it jams up and there doesn’t seem to be any indication why.

I should add that occasionally, removing particular images that have gathered in /MEDIA/FTP can ‘unblock’ the system and everything moves across nicely.

I understand Live Ingest needs to be able to write and read directories and the files within them and thought moving the source directory out of the Dropbox folder would overcome this. I don’t think I can recall an issue with permissions when writing to a Dropbox folder but maybe PM is operating at a much lower level.

What’s odd is I can reliably send via FTP from the camera into the Dropbox folder (bypassing PM) without issue. The problem here is that I can’t benefit from the Live Ingest features such as captioning renaming etc and the client can end up seeing half complete pictures that are currently uploading.
 

[Kirk Baker]

It could be useful to see another PM.log, when the above case happens.

You're using {seqn} in his renaming, which could indeed cause Live Ingest to hang waiting for a specific file before continuing, if for some reason that file never became "ready" for transfer.

Could you try not using {seqn} in your naming and see if that improves things?  You'll likely need to use something else like {frame4} instead to avoid naming collisions.

Thanks,

-Kirk

[Matthew]

It could be useful to see another PM.log, when the above case happens.

You're using {seqn} in his renaming, which could indeed cause Live Ingest to hang waiting for a specific file before continuing, if for some reason that file never became "ready" for transfer.

Could you try not using {seqn} in your naming and see if that improves things?  You'll likely need to use something else like {frame4} instead to avoid naming collisions.

Thanks,

-Kirk

Thanks. I'll try {frame4} instead and see how I get on with that. If I have the issue again I'll post the log.