Skip to content

Conversation

kenwenzel
Copy link
Contributor

GitHub issue resolved: #5442

Briefly describe the changes proposed in this PR:

Reduce copy operations in ValueStore and slightly improve multi-threaded insertion perfomace.


PR Author Checklist (see the contributor guidelines for more details):

  • my pull request is self-contained
  • I've added tests for the changes I made
  • I've applied code formatting (you can use mvn process-resources to format from the command line)
  • I've squashed my commits where necessary
  • every commit message starts with the issue number (GH-xxxx) followed by a meaningful description of the change

@kenwenzel kenwenzel force-pushed the GH-5442-lmdb-performance branch from a025112 to 376fdeb Compare September 19, 2025 12:29
@hmottestad
Copy link
Contributor

Have you run benchmarks before and after to show how this affects performance?

@kenwenzel
Copy link
Contributor Author

Yes. I can post something tomorrow.

@kenwenzel
Copy link
Contributor Author

kenwenzel commented Sep 23, 2025

main:

Benchmark                                                    Mode  Cnt      Score      Error  Units
TransactionsPerSecondBenchmark.largerTransactionLevelNone   thrpt    3     46.978 ±   16.289  ops/s
TransactionsPerSecondBenchmark.mediumTransactionsLevelNone  thrpt    3  23231.246 ± 5066.380  ops/s

Benchmark                        Mode  Cnt    Score     Error  Units
QueryBenchmark.complexQuery      avgt    3   12.399 ±   2.573  ms/op
QueryBenchmark.lots_of_optional  avgt    3  603.818 ± 150.202  ms/op

this PR:

Benchmark                                                    Mode  Cnt      Score      Error  Units
TransactionsPerSecondBenchmark.largerTransactionLevelNone   thrpt    3     48.927 ±   26.386  ops/s
TransactionsPerSecondBenchmark.mediumTransactionsLevelNone  thrpt    3  25608.047 ± 4287.504  ops/s

Benchmark                        Mode  Cnt    Score    Error  Units
QueryBenchmark.complexQuery      avgt    3   11.414 ±  3.412  ms/op
QueryBenchmark.lots_of_optional  avgt    3  610.069 ± 61.084  ms/op

Unfortunately, I can't reliably reproduce benchmark results with my setup (Windows 11 + Cortex XDR + Hyper-V on a notebook).
Its not a huge improvement but tendency is that it should be a bit faster.

@kenwenzel kenwenzel force-pushed the GH-5442-lmdb-performance branch from 376fdeb to c04a0ae Compare September 23, 2025 07:29
@kenwenzel
Copy link
Contributor Author

kenwenzel commented Sep 23, 2025

@hmottestad I've also added an optimization for Varint that reduces the number of ByteBuffer.put calls.

Benchmark                    Mode  Cnt   Score   Error  Units
QueryBenchmark.complexQuery  avgt    5  10.824 ± 0.607  ms/op

@kenwenzel kenwenzel force-pushed the GH-5442-lmdb-performance branch from 3f0a6e6 to 757cc5a Compare September 26, 2025 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LMDB: Further optimize performance
2 participants