Author Topic: Re-Indexing Frequency? Speed?  (Read 5176 times)

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re-Indexing Frequency? Speed?
« on: April 26, 2020, 01:17:18 PM »
Any idea how often we will need to re-index once there is a stable release? I know the current one is caused by the new release. Also, one catalog (relatively small) is finished. But the other (287k records) is still going after starting it two days ago. Just trying to get an idea of how much maintenance time is coming down the road.

Thanks!
Shane

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #1 on: April 26, 2020, 03:44:49 PM »
Shane,

A re-index should only be required when we make a change to the schema of the database.  Once we have a stable release version of Photo Mechanic Plus, you should not need to re-index.

In the future, if the work can be accomplished by a more optimized "Upgrade Catalog Database" feature, we'll provide it.

-Kirk

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #2 on: April 28, 2020, 07:39:38 AM »
Hey Kirk, just checking back. This process is still running after having been started on Friday. The catalog is located on a local drive that's relatively speedy with plenty of room for cache and the catalog. The computer hasn't been shut down and the program has been running constantly. Since starting the reindexing process Friday morning, it has managed to do 173k out of 270k documents. Does that seem to be the speed at which this progresses? Not complaining here, just want to make sure there's not an issue I need to look into further.

Shane

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #3 on: April 28, 2020, 09:14:46 AM »
Shane,

Hey Kirk, just checking back. This process is still running after having been started on Friday. The catalog is located on a local drive that's relatively speedy with plenty of room for cache and the catalog. The computer hasn't been shut down and the program has been running constantly. Since starting the reindexing process Friday morning, it has managed to do 173k out of 270k documents. Does that seem to be the speed at which this progresses? Not complaining here, just want to make sure there's not an issue I need to look into further.

No, it should be faster than that.  What kind of drive, specifically?

-Kirk

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #4 on: April 28, 2020, 12:14:38 PM »
I am going to move it to an SSD, currently it is on a 4tb G-Drive USB3 drive with 3.65 tb available. Not the fastest drive, but no slouch either. From the info I can see, doesn't appear to be a hardware issue. It's the ShaneData drive.



And here is a current test of the drive.



Shane

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #5 on: April 28, 2020, 12:47:04 PM »
Shane,

I am going to move it to an SSD, currently it is on a 4tb G-Drive USB3 drive with 3.65 tb available. Not the fastest drive, but no slouch either. From the info I can see, doesn't appear to be a hardware issue. It's the ShaneData drive.



And here is a current test of the drive.



The process to look for is going to be a "pm-task" of which there should be four, I think.

-Kirk

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #6 on: April 28, 2020, 02:42:10 PM »


I see two... one of them is certainly getting after it. Whatever it is. ha!

Shane

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #7 on: April 28, 2020, 04:54:15 PM »
Shane,

And the pm-task with PID 1392, how much disk bandwidth is it using?

-Kirk

Offline Bill Kelly

  • Software Developer
  • Full Member
  • ***
  • Posts: 131
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #8 on: May 07, 2020, 11:39:21 AM »
Hi Shane,

Also, one catalog (relatively small) is finished. But the other (287k records) is still going after starting it two days ago.

We discovered a performance bottleneck that could cause the re-index operation to run very slowly. A fix has been developed and is currently being tested. It should appear in the next build.


Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 24730
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #9 on: May 07, 2020, 06:38:56 PM »
Please try out the new Beta 17 build: http://forums.camerabits.com/index.php?topic=12753.msg65531#msg65531

Thanks,

-Kirk

Offline KeithRJ

  • Full Member
  • ***
  • Posts: 228
    • View Profile
    • Keith & Vlasta Ross-Jones
Re: Re-Indexing Frequency? Speed?
« Reply #10 on: May 07, 2020, 07:57:43 PM »
Re-indexing IS SIGNIFICANTLY FASTER in latest beta.

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #11 on: May 07, 2020, 08:38:10 PM »
Outstanding! I'll give it a shot tomorrow.