« previous next »
Pages: [1]   Go Down

Author Topic: Re-Indexing Frequency? Speed?  (Read 7180 times)

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re-Indexing Frequency? Speed?
« on: April 26, 2020, 01:17:18 PM »
Any idea how often we will need to re-index once there is a stable release? I know the current one is caused by the new release. Also, one catalog (relatively small) is finished. But the other (287k records) is still going after starting it two days ago. Just trying to get an idea of how much maintenance time is coming down the road.

Thanks!
Shane
Logged

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #1 on: April 26, 2020, 03:44:49 PM »
Shane,

A re-index should only be required when we make a change to the schema of the database.  Once we have a stable release version of Photo Mechanic Plus, you should not need to re-index.

In the future, if the work can be accomplished by a more optimized "Upgrade Catalog Database" feature, we'll provide it.

-Kirk
Logged

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #2 on: April 28, 2020, 07:39:38 AM »
Hey Kirk, just checking back. This process is still running after having been started on Friday. The catalog is located on a local drive that's relatively speedy with plenty of room for cache and the catalog. The computer hasn't been shut down and the program has been running constantly. Since starting the reindexing process Friday morning, it has managed to do 173k out of 270k documents. Does that seem to be the speed at which this progresses? Not complaining here, just want to make sure there's not an issue I need to look into further.

Shane
Logged

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #3 on: April 28, 2020, 09:14:46 AM »
Shane,

Quote from: shaneb on April 28, 2020, 07:39:38 AM
Hey Kirk, just checking back. This process is still running after having been started on Friday. The catalog is located on a local drive that's relatively speedy with plenty of room for cache and the catalog. The computer hasn't been shut down and the program has been running constantly. Since starting the reindexing process Friday morning, it has managed to do 173k out of 270k documents. Does that seem to be the speed at which this progresses? Not complaining here, just want to make sure there's not an issue I need to look into further.

No, it should be faster than that.  What kind of drive, specifically?

-Kirk
Logged

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #4 on: April 28, 2020, 12:14:38 PM »
I am going to move it to an SSD, currently it is on a 4tb G-Drive USB3 drive with 3.65 tb available. Not the fastest drive, but no slouch either. From the info I can see, doesn't appear to be a hardware issue. It's the ShaneData drive.



And here is a current test of the drive.



Shane
Logged

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #5 on: April 28, 2020, 12:47:04 PM »
Shane,

Quote from: shaneb on April 28, 2020, 12:14:38 PM
I am going to move it to an SSD, currently it is on a 4tb G-Drive USB3 drive with 3.65 tb available. Not the fastest drive, but no slouch either. From the info I can see, doesn't appear to be a hardware issue. It's the ShaneData drive.



And here is a current test of the drive.



The process to look for is going to be a "pm-task" of which there should be four, I think.

-Kirk
Logged

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #6 on: April 28, 2020, 02:42:10 PM »


I see two... one of them is certainly getting after it. Whatever it is. ha!

Shane
Logged

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #7 on: April 28, 2020, 04:54:15 PM »
Shane,

And the pm-task with PID 1392, how much disk bandwidth is it using?

-Kirk
Logged

Offline Bill Kelly

  • Software Developer
  • Full Member
  • ***
  • Posts: 131
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #8 on: May 07, 2020, 11:39:21 AM »
Hi Shane,

Quote from: shaneb on April 26, 2020, 01:17:18 PM
Also, one catalog (relatively small) is finished. But the other (287k records) is still going after starting it two days ago.

We discovered a performance bottleneck that could cause the re-index operation to run very slowly. A fix has been developed and is currently being tested. It should appear in the next build.

Logged

Offline Kirk Baker

  • Senior Software Engineer
  • Camera Bits Staff
  • Superhero Member
  • *****
  • Posts: 25020
    • View Profile
    • Camera Bits, Inc.
Re: Re-Indexing Frequency? Speed?
« Reply #9 on: May 07, 2020, 06:38:56 PM »
Please try out the new Beta 17 build: http://forums.camerabits.com/index.php?topic=12753.msg65531#msg65531

Thanks,

-Kirk
Logged

Offline KeithRJ

  • Full Member
  • ***
  • Posts: 228
    • View Profile
    • Keith & Vlasta Ross-Jones
Re: Re-Indexing Frequency? Speed?
« Reply #10 on: May 07, 2020, 07:57:43 PM »
Re-indexing IS SIGNIFICANTLY FASTER in latest beta.
Logged

Offline shaneb

  • Newcomer
  • *
  • Posts: 16
    • View Profile
Re: Re-Indexing Frequency? Speed?
« Reply #11 on: May 07, 2020, 08:38:10 PM »
Outstanding! I'll give it a shot tomorrow.
Logged
Pages: [1]   Go Up
« previous next »