Full sync fail

[DavidHoffmanuk]

I've been finding that full sync stops at 98% through looking for missing files, not always but mostly. I've now found that it seems to happen when one of my external drives is offline so I have to reconnect that drive whenever I need to update the catalog after metadata changes - that's quite often.

Is this expected behaviour and, if not, is there anything I might do to avoid the problem.

David Hoffman

[Kirk Baker]

David,

I've been finding that full sync stops at 98% through looking for missing files, not always but mostly. I've now found that it seems to happen when one of my external drives is offline so I have to reconnect that drive whenever I need to update the catalog after metadata changes - that's quite often.

Is this expected behaviour and, if not, is there anything I might do to avoid the problem.

As the feature is written, it will scan what you tell it to scan.  If that includes folders that are on a drive that is offline then apparently it gets stuck waiting for that drive to come online.

You could exclude the folders on that drive that you don't want to make available.

-Kirk

[DavidHoffmanuk]

Thanks Kirk.

I can edit the folder list before syncing but it's a long and complicated list in a cramped window so that's a slow process and remounting the drive(s) will usually be quicker. As nothing can have changed on those drives it shouldn't really be necessary. It would be better if PM+ would either skip missing drives and report that on completion or pause and ask the user how to proceed. I'll make a feature request.

David

[Kirk Baker]

David,

It would be better if PM+ would either skip missing drives and report that on completion or pause and ask the user how to proceed. I'll make a feature request.

Yes, please do.

-Kirk

[Kirk Baker]

David,

I created a catalog for testing and had it include folders on two drives.  One was unmounted and I ran a Full Sync on the catalog.  It completed without issue.  I tried this on both Windows 10 and macOS 10.14 and both worked equally well and finished with no hangs.

One thing I did notice is that the macOS version listed the files as missing whereas the Windows version listed the files as offline.

So something else is likely going wrong in your case.  Would you mind turning on CATALOG_DEBUG_LOGGING and doing another Full Sync and then get me the zipped log, please?

Thanks,

-Kirk

[DavidHoffmanuk]

Hi Kirk

I quit and restarted PM+ last night. Then I ran a full sync with the same drive offline as before and left it to see where it was at in the morning. This morning the sync was stalled as before and at the same file as before. I stopped the sync and saved the log.

I searched for the file that PM+ had stopped at and found 2 in the catalog. One was fine, the other would not open and gave an 'offline' error message although it had a green dot in the corner. The path showed it as amdoc://[long string of characters][filename with spaces shown as %20]. I couldn't find it in a Finder search or delete it from the catalog but 'remove from catalog' worked. I think it was ghost of some kind.

Having removed that file I again ran a full sync. The window showed that it was beginning a search for missing files but no list of files appeared and the window remained blank after the initial 'Begin Sync for 1 catalog' message.
Activity monitor showed around 100% CPU. I left PM+ to itself, not doing anything with it but, after 45 minutes, nothing had changed and I needed to use it so opened a contact sheet. The sync window changed from white to black and the message 'scanning for missing files: 99%' appeared  at the top.

An hour later nothing had changed so I stopped it, generated a new log, quit & restarted. I mounted the missing drive, tried again. Window opens, no scrolling list as it looked for missing files but % at the top increased at a reasonable speed and continued to an apparently successful conclusion.

The sync window is showing Files Added: 25226, Metadata Updated: 24957. Those are very large numbers, far higher than the number of files that I've knowingly added or updated and the files shown are largely ones I've not looked at for a long time.  The sync collection also shows 25226 new files but opening that collection shows only 1456. This difference between the number shown as being in a collection and the number of files that appear when it's opened is a frequent occurrence, mostly with sync collections.

Unlike in your test, I'm consistently failing to do a full sync with this drive offline.

The logs are attached.

I see that there's a new version of PM+ but for consistency I've not yet installed it.

David

[Kirk Baker]

David,

I searched for the file that PM+ had stopped at and found 2 in the catalog. One was fine, the other would not open and gave an 'offline' error message although it had a green dot in the corner. The path showed it as amdoc://[long string of characters][filename with spaces shown as %20]. I couldn't find it in a Finder search or delete it from the catalog but 'remove from catalog' worked. I think it was ghost of some kind.

What was that filename with the %20 (URL-encoded) spaces?

Also, it looks the the first phase is not completing.  Internally it is doing a "Reintegrate Forgotten Catalog" maintenance operation.  If you keep that drive offline and do the "Reintegrate Forgotten Catalog" directly in the Catalog Management window, does that operation fail to complete as well?

-Kirk

[DavidHoffmanuk]

Hi Kirk

Full sync has stalled at different files in the past. The last time it was at
100612 World Naked Bike Ride 0008.nef
The ghost file with the ascii spaces was
[long string of characters]100612%20World%20Naked%20Bike%20Ride%200008.nef
I've attached a screenshot that I took showing the full path.

The reintegrate operation today with the same external drive offline started normally and progressed quickly but by around 80% it had slowed down considerably and at around 96% the yellow 'reintegrate needed' message appeared for a few minutes and then disappeared again. After about 20 minutes it reached 99%.

After 2 or 3 hours it's still at 99% and the management window shows 298011 documents which seems too large a number. It was 306379 earlier and was decreasing in steps of exactly 100 until now when it seems to have stopped. The 'Reintegrate needed' message has just reappeared.

Currently a search for "" in the catalog finds 294770 documents. Many will be duplicates or derivatives.

I left PM+ running for another 4 or 5 hours. The activity monitor at 98-99% shows that it's busy doing something, though not apparently something useful. I quit PM+. I had turned off logging after extracting the logs I sent earlier. I can turn it back on and re-run the reintegrate overnight - let me know if you'd like me to do that.

Restarting PM+ now shows the 'reintegrate needed' message.

David

[Kirk Baker]

David,

Full sync has stalled at different files in the past. The last time it was at
100612 World Naked Bike Ride 0008.nef
The ghost file with the ascii spaces was
[long string of characters]100612%20World%20Naked%20Bike%20Ride%200008.nef
I've attached a screenshot that I took showing the full path.

The reintegrate operation today with the same external drive offline started normally and progressed quickly but by around 80% it had slowed down considerably and at around 96% the yellow 'reintegrate needed' message appeared for a few minutes and then disappeared again. After about 20 minutes it reached 99%.

After 2 or 3 hours it's still at 99% and the management window shows 298011 documents which seems too large a number. It was 306379 earlier and was decreasing in steps of exactly 100 until now when it seems to have stopped. The 'Reintegrate needed' message has just reappeared.

Currently a search for "" in the catalog finds 294770 documents. Many will be duplicates or derivatives.

I left PM+ running for another 4 or 5 hours. The activity monitor at 98-99% shows that it's busy doing something, though not apparently something useful. I quit PM+. I had turned off logging after extracting the logs I sent earlier. I can turn it back on and re-run the reintegrate overnight - let me know if you'd like me to do that.

Would it be possible for you to send me your catalog.pmdb file?  With it I should be able to get to the bottom of the problem.  If a reintegrate cannot complete then a Catalog Sync in Full Sync mode will also fail to complete.

Thanks,

-Kirk

[DavidHoffmanuk]

It's 6.1 GB. I'll FTP it to the incoming folder on your server.

David

[Kirk Baker]

Thanks.  I got the file.

-Kirk

[DavidHoffmanuk]

Hi Kirk

I reconnected the offline drive. Both the reintegration and a full sync ran through to completion without problems. It's curious that offline drives are not a problem in your tests but consistently cause the process to stall on my Mac Pro 5,1 running 10.14.6.

David

[Kirk Baker]

David,

I reconnected the offline drive. Both the reintegration and a full sync ran through to completion without problems. It's curious that offline drives are not a problem in your tests but consistently cause the process to stall on my Mac Pro 5,1 running 10.14.6.

In my case, using your catalog, all of the drives are offline because I don't have any of your drives.

But also, when I test with my own catalog, having a drive offline does not stall.

It's good to have your catalog for my testing: I think I have found a way to make the Organizer more optimal with your dataset (and other user's datasets).  We never expected the fields that are part of the Browse tree in the Organizer to be particularly large and we set no display limits for them.  On your catalog, there are a number of Headline fields that are very long (like a caption) and they take a fair amount of time to send from the catalog to PM, and more time to sort on the PM side, leading to lack of responsiveness (spinning polychromatic cursor) in PM itself.

No matter how wide you make the Organizer, the entire field cannot be seen so it's likely that limiting it to 20-50 characters would be more than adequate for browse purposes, and it would be less data to send and sort through.

But that's an aside.  I need to figure out why a Reintegrate stalls for you.  I will likely need to add additional logging and make you a new build.  Would you be up for that?

Thanks,

-Kirk

[DavidHoffmanuk]

Hi Kirk

Thanks for putting the time into this. Logging & upgrading is no problem.

The long headline fields in those files are a legacy problem from a broken DB and a lack of skill here. I don't want to just delete all the headlines and I can't see an obvious way to shorten them in PM+. I could export them, truncate and embed them back again with Media Pro but would that upset MP+? Or is there a way in PM+ for me to copy the first 50 characters from the caption into a headline field?

David

[Kirk Baker]

David,

I'm sorry for the late reply, I only just now saw your message.

Thanks for putting the time into this. Logging & upgrading is no problem.

OK, great.

The long headline fields in those files are a legacy problem from a broken DB and a lack of skill here. I don't want to just delete all the headlines and I can't see an obvious way to shorten them in PM+. I could export them, truncate and embed them back again with Media Pro but would that upset MP+? Or is there a way in PM+ for me to copy the first 50 characters from the caption into a headline field?

Yes, it's possible, but I'd rather make it so that the Catalog system can handle long fields and shorten them for display purposes such that the burden isn't placed on the user.

-Kirk