Ultra slow reintegration

[DavidHoffmanuk]

Hi all

I saw a maintenance needed notice and started the process. I have a small test catalog, that only took 2 or 3 minutes and was fine but after almost 6 hours the main catalog is still only at 98%. It was at 91% after about 20 minutes but now, while clearly busy (see attached), it seems to be going nowhere and I have no access to my catalog.

Should I force quit and try again or keep waiting?

[Kirk Baker]

David,

I saw a maintenance needed notice and started the process. I have a small test catalog, that only took 2 or 3 minutes and was fine but after almost 6 hours the main catalog is still only at 98%. It was at 91% after about 20 minutes but now, while clearly busy (see attached), it seems to be going nowhere and I have no access to my catalog.

Did you Upgrade your catalog before you started the Reintegrate?

I'd like to see your Photo Mechanic log.  In Photo Mechanic Plus go to the Help menu and choose "Reveal Support Data..." and moments later a Finder window will come forward with the zipped log file selected.  Use the 'Attachments and other options' link when you're composing your reply to this message and there you'll be able to upload your zipped PM log file.

Thanks,

-Kirk

[DavidHoffmanuk]

The upgrade option came up first so I did that before beginning reintegration on the first small test catalog - but I no longer recall if I did that a second time when I began reintegration of the main catalog. Does "upgrade" mean the catalog or the app?

Log attached.

It's still at 98% and Activity Monitor showing pm-task at just below 100%.

Thanks
David

[DavidHoffmanuk]

It's still at 98% and active - about 10 hours. I don't want to do more damage than I have to but I need to get back to work. I'm thinking that I should try to quit but if that doesn't work I'll force quit. Should I make copies of anything beforehand or is there a better way?

David

[Kirk Baker]

David,

It's still at 98% and active - about 10 hours. I don't want to do more damage than I have to but I need to get back to work. I'm thinking that I should try to quit but if that doesn't work I'll force quit. Should I make copies of anything beforehand or is there a better way?

The catalog database will be busy while it is open so it can't be copied.  I'd suggest quitting and if it won't exit even with a retry, you'll have to force quit the app.  Make sure that any lingering pm-task processes are also terminated (you can use the Activity Monitor for those) before you try restarting.

-Kirk

[DavidHoffmanuk]

It's late now (London) so I'll leave it till the morning. You never know...

I do have a few backups from yesterday and earlier. If the current catalog is damaged can I just replace it with an older catalog.pmdb or do I need to also copy the other files (which are all older).

Thanks
David

[Kirk Baker]

David,

I do have a few backups from yesterday and earlier. If the current catalog is damaged can I just replace it with an older catalog.pmdb or do I need to also copy the other files (which are all older).

Just the catalog.pmdb should suffice.  Please do the "Update Catalog Version" maintenance before doing any other type of maintenance or any adding/removing of items to/from your catalog.

-Kirk

[DavidHoffmanuk]

Hi Kirk

The reintegration was still at 98% this morning. I quit PM6+ - that worked without problem and I replaced my catalog.pmdb with one from 4 days ago when all seemed to be working properly. I opened PM6+ and saw that the test catalog was still fine and showed no maintenance required. The main catalog showed that an update was needed. That appeared to go normally, took just a few minutes. The catalog again showed that reintegration was needed but, as before, this began well and ran quickly to start with but slowed more and more as it progressed and does not appear to be going to finish in a finite time.

The problem didn't seem to be in the catalog.pmdb so I replaced the old version with the current one. At a quick look it seemed to be working OK though still showing that reintegration was needed.

I tried a catalog sync to see if that helped. That went as normal and now PM6+ is not showing that any maintenance is needed. All seems normal!

*** Note added 10 minutes later - the catalog is again now showing that reintegration is needed - but I've established that it cannot be completed.***

***And an hour later - I tried reintegration again anyway. It ran quickly to 100% - but then stalled at 100% showing 'reintegration needed' without completing.***

I've attached the current log. Can you see what the problem is and do you have you any suggestions to avoid it in future?

Thanks
David

[Kirk Baker]

David,

The reintegration was still at 98% this morning. I quit PM6+ - that worked without problem and I replaced my catalog.pmdb with one from 4 days ago when all seemed to be working properly. I opened PM6+ and saw that the test catalog was still fine and showed no maintenance required. The main catalog showed that an update was needed. That appeared to go normally, took just a few minutes. The catalog again showed that reintegration was needed but, as before, this began well and ran quickly to start with but slowed more and more as it progressed and does not appear to be going to finish in a finite time.

The problem didn't seem to be in the catalog.pmdb so I replaced the old version with the current one. At a quick look it seemed to be working OK though still showing that reintegration was needed.

I tried a catalog sync to see if that helped. That went as normal and now PM6+ is not showing that any maintenance is needed. All seems normal!

*** Note added 10 minutes later - the catalog is again now showing that reintegration is needed - but I've established that it cannot be completed.***

***And an hour later - I tried reintegration again anyway. It ran quickly to 100% - but then stalled at 100% showing 'reintegration needed' without completing.***

I've attached the current log. Can you see what the problem is and do you have you any suggestions to avoid it in future?

I have looked at the log.  Thank you for providing it.  Unfortunately there's not much to go on in the log.  Could I have you turn on CATALOG_DEBUG_LOGGING, as it would gather the most useful information for me?  You can do this by going to the Help menu and choosing the "Edit Debug Settings..." command and then setting the checkbox next to CATALOG_DEBUG_LOGGING, clicking the Close button and restarting Photo Mechanic Plus.

Then try the reintegrate again.  And give it some time to process at the 100% mark.  Then generate the log and post it here again, please?

Thanks,

-Kirk

[Kirk Baker]

David,

How large is your catalog.pmdb file?  Would it be possible to share it with me?

Thanks,

-Kirk

[DavidHoffmanuk]

Hi Kirk

The catalog is 5.32 GB. I could FTP it on a confidential/delete after use basis. If not FTP then I've heard of this transfer service that goes to 50 GB:
https://www.swisstransfer.com/

I've turned on the debug logging and started a new reintegration. It's evening here so I'll leave it running overnight and send the report in the morning.

David

[Kirk Baker]

David,

The catalog is 5.32 GB. I could FTP it on a confidential/delete after use basis. If not FTP then I've heard of this transfer service that goes to 50 GB:
https://www.swisstransfer.com/

I've turned on the debug logging and started a new reintegration. It's evening here so I'll leave it running overnight and send the report in the morning.

I think a few minutes of logging of the Reintegrate once it reaches 100% should suffice.

Is this large catalog the only catalog that you have that is experiencing this problem?

I can get you FTP credentials for our private server.  I'll send a personal message to you with the details.

Thanks,

-Kirk