The codebase for "[Experiments & Analysis] Benchmarking, Analyzing, and Optimizing WA of Partial Compaction in RocksDB"

This repository contains two submodules: key-value workload generator and RocksDB (v8.8.1) for partial compaction analysis.

Compiling RocksDB for Partial Compaction Analysis

First, navigate to rocksdb-for-partial-compaction-analysis directory and run:

make static_lib

If you cannot compile the static library successfully due to lack of package, please check here for more info.

Compiling Necessary Utility Programs

Then, navigate to main directory and run:

make tools

which makes programs for workload generation, progress checking of enumeration, and calculate the size of the generated workload.

Running the Example Benchmark for RocksDB

To run the workload, you need to go to main directory and run:

make simple_example

Then, you should generate a workload by running:

./load_gen -I 100 -E 64 -K 8 --output_path ./workload.txt

where [I] is the number of insert, [E] is the entry size, [K] is the key size, [output_path] is the workload path.

More details can be found by running:

./load_gen --help

Finally, you can run the workload by running:

./simple_runner [db_path] [workspace] [workload_path] [write_buffer_size] [target_file_size_base] [max_bytes_for_level_base] [write_buffer_data_structure] [max_bytes_for_level_multiplier]

where
[db_path] is a directory to store the databse
[workspace] is a directory to store the log, experiment data or other information generated by simple_runner
[workload_path] is the path to the workload
[write_buffer_size] is the size of the write buffer in bytes, e.g., 67108864 (64MB)
[target_file_size_base] is target size of the file in base level (level 1), e.g., 67108864 (64MB)
[max_bytes_for_level_base] is the max bytes for the base level (level 1), e.g., 268435456 (256MB)
[write_buffer_data_structure] is the data structure for the write buffer, e.g., Vector, SkipList, HashSkipList, HashLinkList

RefinedMOR Implementation

Details can be found in:

1. rocksdb-for-partial-compaction-analysis/db/compaction/compaction_picker_level.cc:812
2. rocksdb-for-partial-compaction-analysis/db/version_set.cc:7517
3. rocksdb-for-partial-compaction-analysis/enumerate/all_files_enumerator.cc:59