Ord indexing "loops" or "resets" back to a previous level and never completes

[so7ow]

On the Ordicord we've seen many reports of issues since the halving, where ord indexing loops indefinitely, returning to previous levels and never completing successfully. We've been recommending folks add a --commit-interval 100 (or even lower value sometimes) and that generally seems to fix it. Rather than just continue doing that forever, I wanted to bring it up here, and see if any of the devs are aware of the situation and/or working toward a fix. Or maybe our workaround is enough? Don't know, seems like there should be some way to adapt that commit interval when the machine it's running on is struggling.

Low RAM seems to be a factor, but, to be clear, this issue has not affected me directly so I don't have any specifics to report, just generalities!

[cryptoni9n]

On the Ordicord we've seen many reports of issues since the halving, where ord indexing loops indefinitely, returning to previous levels and never completing successfully. We've been recommending folks add a --commit-interval 100 (or even lower value sometimes) and that generally seems to fix it. Rather than just continue doing that forever, I wanted to bring it up here, and see if any of the devs are aware of the situation and/or working toward a fix. Or maybe our workaround is enough? Don't know, seems like there should be some way to adapt that commit interval when the machine it's running on is struggling.

Low RAM seems to be a factor, but, to be clear, this issue has not affected me directly so I don't have any specifics to report, just generalities!

Thanks for reporting this, so7ow. I've been experiencing it for sometime and can confirm that a large swath of Ordicord users experience it as well. I'm not sure the best way to handle it, but my initial thought is to just drastically lower the default commit interval (which is currently at 5000). My machine has 32GB of RAM and I've had to lower the interval to single digits at times.

Perhaps if this setting was changed to be based on GB of RAM rather than number of blocks?

[huguojunsy]

On the Ordicord we've seen many reports of issues since the halving, where ord indexing loops indefinitely, returning to previous levels and never completing successfully. We've been recommending folks add a --commit-interval 100 (or even lower value sometimes) and that generally seems to fix it. Rather than just continue doing that forever, I wanted to bring it up here, and see if any of the devs are aware of the situation and/or working toward a fix. Or maybe our workaround is enough? Don't know, seems like there should be some way to adapt that commit interval when the machine it's running on is struggling.

Low RAM seems to be a factor, but, to be clear, this issue has not affected me directly so I don't have any specifics to report, just generalities!

I have encountered the same problem as well.

[HollywodMeta]

Yup, I'm having the same issue myself. I've spent a few days trying to resolve it with no success. The reporting of any fix or workaround would certainly be appreciated.

[cryptoni9n]

Yup, I'm having the same issue myself. I've spent a few days trying to resolve it with no success. The reporting of any fix or workaround would certainly be appreciated.

The workaround that seems to resolve the issue for everyone is to set the --commit-interval to something much lower like 100, and see if that works. If not, keep lowering it until it does.

[cryptoni9n]

I was digging around and found that this is a dupe of #1630

I think a fix for this needs to be a priority as it is increasingly difficult for many users to index without using the --commit-interval flag. @raphjaph could we add this to the tracker?

[alexagawddess]

Been experiencing this issue for the past 3 weeks. I’ve attempted the —commit-interval flag with different amounts and it still loops. Only way I can successfully index right now is to let it run and then ctrl c to safely shutdown and it registers. It’s just very time consuming doing this and wish I could just let it run like normal 😅

Windows 64GB Ram plenty of storage.

[so7ow]

Been experiencing this issue for the past 3 weeks. I’ve attempted the —commit-interval flag with different amounts and it still loops. Only way I can successfully index right now is to let it run and then ctrl c to safely shutdown and it registers. It’s just very time consuming doing this and wish I could just let it run like normal 😅

Windows 64GB Ram plenty of storage.

How low a value for commit interval did you try?

[alexagawddess]

Been experiencing this issue for the past 3 weeks. I’ve attempted the —commit-interval flag with different amounts and it still loops. Only way I can successfully index right now is to let it run and then ctrl c to safely shutdown and it registers. It’s just very time consuming doing this and wish I could just let it run like normal 😅
Windows 64GB Ram plenty of storage.

How low a value for commit interval did you try?

Went as low as 25

[so7ow]

Went as low as 25

Go lower. 👍🏻

[HollywodMeta]

Yeah, it finally worked for me at 5.

[cryptoni9n]

@raphjaph I think we could use the expertise of mconst on the fix for this issue. It should be easy enough just to lower the default value for the --commit-interval flag. However, I worry that if it is lowered by too much, it could lengthen the index time for a beefier machine that doesn't have any issues with the current default value. Is this an issue that needs benchmarking to determine the optimal default value, or would setting it very low (single digits) not impact the overall indexing time by that much?

It seems like it could be a zero-sum change. For example, an index with a default commit interval might spend 5 minutes every 5000 blocks updating the db with those blocks, but lowering that value to 100 could take the same time if the server then only spent 6s per update. If that were the case, the lower value wouldn't impact overall indexing time. I'm just not sure that is the way it works.

[alexagawddess]

Yeah, it finally worked for me at 5.

I wanted to follow up and say 5 worked for me. Thank you!

[casey]

Is this specific to Windows, or does it affect other OSes?

[cryptoni9n]

Windows here, not sure about the others. I'll start asking everyone in the Ordicord that experiences this issue from now on.

[casey]

@cryptoni9n Thanks! @partialord is a little worried that a future performance improvement might exacerbate this issue, so we want to make sure it's nailed down. Any additional information is super useful.

[casey]

Other things which would be good to know when someone runs into this bug:

• What indexing options are they using? (sats, addresses, runes, inscriptions, etc)
• How much RAM do they have?
• What's the error message when it loops? On Windows they'll need to do set RUST_LOG=info, and on Unix export RUST_LOG=info
• Which block range is it looping on?