diff --git a/README.md b/README.md index 801a3abd3..1bb4339fe 100644 --- a/README.md +++ b/README.md @@ -6,16 +6,33 @@ BenchBase (formerly [OLTPBench](https://github.com/oltpbenchmark/oltpbench/)) is **Table of Contents** -- [Quickstart](#quickstart) -- [Description](#description) -- [Usage Guide](#usage-guide) -- [Contributing](#contributing) -- [Known Issues](#known-issues) -- [Credits](#credits) -- [Citing This Repository](#citing-this-repository) +- [Prerequisites](#prerequisites) +- [Quickstart](#quickstart) +- [Description](#description) +- [Usage Guide](#usage-guide) +- [Contributing](#contributing) +- [Known Issues](#known-issues) +- [Credits](#credits) +- [Citing This Repository](#citing-this-repository) --- +## Prerequisites + +### Non-Containerized + +Java 17 needs to be installed on the machine in order to run the benchbase commands. + +Java can be downloaded from the [official download page.](https://www.oracle.com/java/technologies/downloads/#java17) + +### Containerized + +Benchbase supports the usage inside of docker containers. + +Docker can be downloaded from the [official download page.](https://docs.docker.com/get-docker/) + +> See the [docker section](#how-to-use-with-docker) for more details on how to use benchbase with docker. + ## Quickstart To clone and build BenchBase using the `postgres` profile, @@ -60,24 +77,24 @@ features, e.g., per-transaction-type latency and throughput logs. The BenchBase framework has the following benchmarks: -* [AuctionMark](https://github.com/cmu-db/benchbase/wiki/AuctionMark) -* [CH-benCHmark](https://github.com/cmu-db/benchbase/wiki/CH-benCHmark) -* [Epinions.com](https://github.com/cmu-db/benchbase/wiki/epinions) -* hyadapt -- pending configuration files -* [NoOp](https://github.com/cmu-db/benchbase/wiki/NoOp) -* [OT-Metrics](https://github.com/cmu-db/benchbase/wiki/OT-Metrics) -* [Resource Stresser](https://github.com/cmu-db/benchbase/wiki/Resource-Stresser) -* [SEATS](https://github.com/cmu-db/benchbase/wiki/Seats) -* [SIBench](https://github.com/cmu-db/benchbase/wiki/SIBench) -* [SmallBank](https://github.com/cmu-db/benchbase/wiki/SmallBank) -* [TATP](https://github.com/cmu-db/benchbase/wiki/TATP) -* [TPC-C](https://github.com/cmu-db/benchbase/wiki/TPC-C) -* [TPC-H](https://github.com/cmu-db/benchbase/wiki/TPC-H) -* TPC-DS -- pending configuration files -* [Twitter](https://github.com/cmu-db/benchbase/wiki/Twitter) -* [Voter](https://github.com/cmu-db/benchbase/wiki/Voter) -* [Wikipedia](https://github.com/cmu-db/benchbase/wiki/Wikipedia) -* [YCSB](https://github.com/cmu-db/benchbase/wiki/YCSB) +- [AuctionMark](https://github.com/cmu-db/benchbase/wiki/AuctionMark) +- [CH-benCHmark](https://github.com/cmu-db/benchbase/wiki/CH-benCHmark) +- [Epinions.com](https://github.com/cmu-db/benchbase/wiki/epinions) +- hyadapt -- pending configuration files +- [NoOp](https://github.com/cmu-db/benchbase/wiki/NoOp) +- [OT-Metrics](https://github.com/cmu-db/benchbase/wiki/OT-Metrics) +- [Resource Stresser](https://github.com/cmu-db/benchbase/wiki/Resource-Stresser) +- [SEATS](https://github.com/cmu-db/benchbase/wiki/Seats) +- [SIBench](https://github.com/cmu-db/benchbase/wiki/SIBench) +- [SmallBank](https://github.com/cmu-db/benchbase/wiki/SmallBank) +- [TATP](https://github.com/cmu-db/benchbase/wiki/TATP) +- [TPC-C](https://github.com/cmu-db/benchbase/wiki/TPC-C) +- [TPC-H](https://github.com/cmu-db/benchbase/wiki/TPC-H) +- TPC-DS -- pending configuration files +- [Twitter](https://github.com/cmu-db/benchbase/wiki/Twitter) +- [Voter](https://github.com/cmu-db/benchbase/wiki/Voter) +- [Wikipedia](https://github.com/cmu-db/benchbase/wiki/Wikipedia) +- [YCSB](https://github.com/cmu-db/benchbase/wiki/YCSB) This framework is design to allow for easy extension. We provide stub code that a contributor can use to include a new benchmark, leveraging all the system features (logging, controlled speed, controlled mixture, etc.) @@ -87,7 +104,8 @@ benchmark, leveraging all the system features (logging, controlled speed, contro ## Usage Guide ### How to Build -Run the following command to build the distribution for a given database specified as the profile name (`-P`). The following profiles are currently supported: `postgres`, `mysql`, `mariadb`, `sqlite`, `cockroachdb`, `phoenix`, and `spanner`. + +Run the following command to build the distribution for a given database specified as the profile name (`-P`). The following profiles are currently supported: `postgres`, `mysql`, `mariadb`, `sqlite`, `cockroachdb`, `phoenix`, and `spanner`. ```bash ./mvnw clean package -P <profile name> @@ -95,23 +113,27 @@ Run the following command to build the distribution for a given database specifi The following files will be placed in the `./target` folder: -* `benchbase-<profile name>.tgz` -* `benchbase-<profile name>.zip` +- `benchbase-<profile name>.tgz` +- `benchbase-<profile name>.zip` ### How to Run -Once you build and unpack the distribution, you can run `benchbase` just like any other executable jar. The following examples assume you are running from the root of the expanded `.zip` or `.tgz` distribution. If you attempt to run `benchbase` outside of the distribution structure you may encounter a variety of errors including `java.lang.NoClassDefFoundError`. + +Once you build and unpack the distribution, you can run `benchbase` just like any other executable jar. The following examples assume you are running from the root of the expanded `.zip` or `.tgz` distribution. If you attempt to run `benchbase` outside of the distribution structure you may encounter a variety of errors including `java.lang.NoClassDefFoundError`. To bring up help contents: + ```bash java -jar benchbase.jar -h ``` To execute the `tpcc` benchmark: + ```bash java -jar benchbase.jar -b tpcc -c config/postgres/sample_tpcc_config.xml --create=true --load=true --execute=true ``` For composite benchmarks like `chbenchmark`, which require multiple schemas to be created and loaded, you can provide a comma separated list: + ```bash java -jar benchbase.jar -b tpcc,chbenchmark -c config/postgres/sample_chbenchmark_config.xml --create=true --load=true --execute=true ``` @@ -151,6 +173,11 @@ mvn clean compile exec:java -P postgres -Dexec.args="-b tpcc -c config/postgres/ this is equivalent to the steps above but eliminates the need to first package and then extract the distribution. +### How to configure Connections & Workloads + +The configuration files are located in the `/config` folder and can be used to adapt the benchbase config to the user's system of choice. +A more detailed explanation can be found [here](/config/README.md). + ### How to Enable Logging To enable logging, e.g., for the PostgreSQL JDBC driver, add the following JVM property when starting... @@ -168,46 +195,46 @@ To modify the logging level you can update [`logging.properties`](src/main/resou ./mvnw -B release:perform ``` -### How use with Docker +### How to use with Docker -- Build or pull a dev image to help building from source: +- Build or pull a dev image to help building from source: - ```sh - ./docker/benchbase/build-dev-image.sh - ./docker/benchbase/run-dev-image.sh - ``` + ```sh + ./docker/benchbase/build-dev-image.sh + ./docker/benchbase/run-dev-image.sh + ``` - or + or - ```sh - docker run -it --rm --pull \ - -v /path/to/benchbase-source:/benchbase \ - -v $HOME/.m2:/home/containeruser/.m2 \ - benchbase.azure.cr.io/benchbase-dev - ``` + ```sh + docker run -it --rm --pull \ + -v /path/to/benchbase-source:/benchbase \ + -v $HOME/.m2:/home/containeruser/.m2 \ + benchbase.azure.cr.io/benchbase-dev + ``` -- Build the full image: +- Build the full image: - ```sh - # build an image with all profiles - ./docker/benchbase/build-full-image.sh + ```sh + # build an image with all profiles + ./docker/benchbase/build-full-image.sh - # or if you only want to build some of them - BENCHBASE_PROFILES='postgres mysql' ./docker/benchbase/build-full-image.sh - ``` + # or if you only want to build some of them + BENCHBASE_PROFILES='postgres mysql' ./docker/benchbase/build-full-image.sh + ``` -- Run the image for a given profile: +- Run the image for a given profile: - ```sh - BENCHBASE_PROFILE='postgres' ./docker/benchbase/run-full-image.sh --help # or other benchbase args as before - ``` + ```sh + BENCHBASE_PROFILE='postgres' ./docker/benchbase/run-full-image.sh --help # or other benchbase args as before + ``` - or + or - ```sh - docker run -it --rm --env BENCHBASE_PROFILE='postgres' \ - -v results:/benchbase/results benchbase.azurecr.io/benchbase --help # or other benchbase args as before - ``` + ```sh + docker run -it --rm --env BENCHBASE_PROFILE='postgres' \ + -v results:/benchbase/results benchbase.azurecr.io/benchbase --help # or other benchbase args as before + ``` > See the [docker/benchbase/README.md](./docker/benchbase/) for further details. @@ -223,9 +250,9 @@ Please see the existing MySQL and PostgreSQL code for an example. We welcome all contributions! Please open a pull request. Common contributions may include: -- Adding support for a new DBMS. -- Adding more tests of existing benchmarks. -- Fixing any bugs or known issues. +- Adding support for a new DBMS. +- Adding more tests of existing benchmarks. +- Fixing any bugs or known issues. ## Known Issues @@ -239,31 +266,31 @@ The original OLTPBench code was largely written by the authors of the original p A significant portion of the modernization was contributed by [Tim Veil @ Cockroach Labs](https://github.com/timveil-cockroach), including but not limited to: -* Built with and for Java ~~11~~ 17. -* Migration from Ant to Maven. - * Reorganized project to fit Maven structure. - * Removed static `lib` directory and dependencies. - * Updated required dependencies and removed unused or unwanted dependencies. - * Moved all non `.java` files to standard Maven `resources` directory. - * Shipped with [Maven Wrapper](https://maven.apache.org/wrapper). -* Improved packaging and versioning. - * Moved to Calendar Versioning (https://calver.org/). - * Project is now distributed as a `.tgz` or `.zip` with an executable `.jar`. - * All code updated to read `resources` from inside `.jar` instead of directory. -* Moved from direct dependence on Log4J to SLF4J. -* Reorganized and renamed many files for clarity and consistency. -* Applied countless fixes based on "Static Analysis". - * JDK migrations (boxing, un-boxing, etc.). - * Implemented `try-with-resources` for all `java.lang.AutoCloseable` instances. - * Removed calls to `printStackTrace()` or `System.out.println` in favor of proper logging. -* Reformatted code and cleaned up imports. -* Removed all calls to `assert`. -* Removed various forms of dead code and stale configurations. -* Removed calls to `commit()` during `Loader` operations. -* Refactored `Worker` and `Loader` usage of `Connection` objects and cleaned up transaction handling. -* Introduced [Dependabot](https://dependabot.com/) to keep Maven dependencies up to date. -* Simplified output flags by removing most of them, generally leaving the reporting functionality enabled by default. -* Provided an alternate `Catalog` that can be populated directly from the configured Benchmark database. The old catalog was proxied through `HSQLDB` -- this remains an option for DBMSes that may have incomplete catalog support. +- Built with and for Java ~~11~~ 17. +- Migration from Ant to Maven. + - Reorganized project to fit Maven structure. + - Removed static `lib` directory and dependencies. + - Updated required dependencies and removed unused or unwanted dependencies. + - Moved all non `.java` files to standard Maven `resources` directory. + - Shipped with [Maven Wrapper](https://maven.apache.org/wrapper). +- Improved packaging and versioning. + - Moved to Calendar Versioning (https://calver.org/). + - Project is now distributed as a `.tgz` or `.zip` with an executable `.jar`. + - All code updated to read `resources` from inside `.jar` instead of directory. +- Moved from direct dependence on Log4J to SLF4J. +- Reorganized and renamed many files for clarity and consistency. +- Applied countless fixes based on "Static Analysis". + - JDK migrations (boxing, un-boxing, etc.). + - Implemented `try-with-resources` for all `java.lang.AutoCloseable` instances. + - Removed calls to `printStackTrace()` or `System.out.println` in favor of proper logging. +- Reformatted code and cleaned up imports. +- Removed all calls to `assert`. +- Removed various forms of dead code and stale configurations. +- Removed calls to `commit()` during `Loader` operations. +- Refactored `Worker` and `Loader` usage of `Connection` objects and cleaned up transaction handling. +- Introduced [Dependabot](https://dependabot.com/) to keep Maven dependencies up to date. +- Simplified output flags by removing most of them, generally leaving the reporting functionality enabled by default. +- Provided an alternate `Catalog` that can be populated directly from the configured Benchmark database. The old catalog was proxied through `HSQLDB` -- this remains an option for DBMSes that may have incomplete catalog support. ## Citing This Repository diff --git a/config/README.md b/config/README.md new file mode 100644 index 000000000..439bcc79a --- /dev/null +++ b/config/README.md @@ -0,0 +1,69 @@ +# Config + +The config files are duplicated in a seperate folder for each of the supported database systems. + +For example, the postgres/sibench config file has the following pieces: + +## Connection Details + +The connection details allow for the configuration of the connection to the database via JDBC. + +```xml + <!-- Connection details --> + <type>POSTGRES</type> + <driver>org.postgresql.Driver</driver> + <url>jdbc:postgresql://localhost:5432/benchbase?sslmode=disable&ApplicationName=sibench&reWriteBatchedInserts=true</url> + <username>admin</username> + <password>password</password> + <isolation>TRANSACTION_SERIALIZABLE</isolation> + <batchsize>128</batchsize> +``` + +## Workload + +- `scalefactor` allows to upscale the workload by the specified factor +- `time` allows to modify the runtime of the workload. It is given in **seconds** +- `weights` define the weight of each [transaction Type](#procedures). The weights are defined as **percentages** + +```xml + + <scalefactor>1</scalefactor> + + <!-- The workload --> + <terminals>1</terminals> + <works> + <work> + <time>60</time> + <rate>unlimited</rate> + <weights>50,50</weights> + </work> + </works> +``` + +## Procedures + +Each workload specifies certain procedures which can be removed by deleting the `transactionType` from the config. By default, all transaction types of the workload are listed in an unmodified config file. + +```xml + <!-- SIBENCH Procedures declaration --> + <transactiontypes> + <transactiontype> + <name>MinRecord</name> + </transactiontype> + <transactiontype> + <name>UpdateRecord</name> + </transactiontype> + </transactiontypes> +</parameters> + +``` + +## Workload Specific Configs + +### Templated + +The templated workload demands a path to a file that holds the templated queries. + +```xml +<query_templates_file>data/templated/example.xml</query_templates_file> +```