Skip to content

Latest commit

 

History

History
238 lines (167 loc) · 8.05 KB

DEVELOPER.md

File metadata and controls

238 lines (167 loc) · 8.05 KB
Raw

Developer Guide

This document describes how to set up your development environment to build and test Valkey GLIDE Node wrapper.

Development Overview

The GLIDE Node wrapper consists of both TypeScript and Rust code. Rust bindings for Node.js are implemented using napi-rs. The Node and Rust components communicate using the protobuf protocol.

Build from source

Prerequisites

Software Dependencies

Note: Nodejs Supported Version

If your Nodejs version is below the supported version specified in the client's documentation, you can upgrade it using NVM.

  • npm
  • git
  • GCC
  • pkg-config
  • protoc (protobuf compiler)
  • openssl
  • openssl-dev
  • rustup

Dependencies installation for Ubuntu

sudo apt update -y
sudo apt install -y nodejs npm git gcc pkg-config protobuf-compiler openssl libssl-dev
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
# Check the installed node version
node -v

Note: Ensure that you installed a supported Node.js version. For Ubuntu 22.04 or earlier, please refer to the instructions here to upgrade your Node.js version.

Dependencies installation for CentOS

sudo yum update -y
sudo yum install -y nodejs git gcc pkgconfig protobuf-compiler openssl openssl-devel gettext
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"

Dependencies installation for MacOS

brew update
brew install nodejs git gcc pkgconfig protobuf openssl
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"

Building and installation steps

Before starting this step, make sure you've installed all software requirments.

  1. Clone the repository:

    git clone https://github.com/valkey-io/valkey-glide.git
cd valkey-glide

  2. Install all node dependencies:

    cd node
npm i
cd rust-client
npm i
cd ..

  3. Build the Node wrapper (Choose a build option from the following and run it from the node folder):

    1. Build in release mode, stripped from all debug symbols (optimized and minimized binary size):
    npm run build:release
    1. Build in release mode with debug symbols (optimized but large binary size):
    npm run build:benchmark
    1. For testing purposes, you can execute an unoptimized but fast build using:
    npm run build

    Once building completed, you'll find the compiled JavaScript code in the./build-ts folder.

  4. Run tests:

    1. Ensure that you have installed server and valkey-cli on your host. You can download Valkey at the following link: Valkey Download page.
    2. Execute the following command from the node folder: 
      npm run build # make sure we have a debug build compiled first
npm test

  5. Integrating the built GLIDE package into your project: Add the package to your project using the folder path with the command npm install <path to GLIDE>/node.

  • For a fast build, execute npm run build. This will perform a full, unoptimized build, which is suitable for developing tests. Keep in mind that performance is significantly affected in an unoptimized build, so it's required to build with the build:release or build:benchmark option when measuring performance.
  • If your modifications are limited to the TypeScript code, run npm run build-external to build the external package without rebuilding the internal package.
  • If your modifications are limited to the Rust code, execute npm run build-internal to build the internal package and generate TypeScript code.
  • To generate Node's protobuf files, execute npm run build-protobuf. Keep in mind that protobuf files are generated as part of full builds (e.g., build, build:release, etc.).

Note: Once building completed, you'll find the compiled JavaScript code in the node/build-ts folder.

Troubleshooting

  • If the build fails after running npx tsc because glide-rs isn't found, check if your npm version is in the range 9.0.0-9.4.1, and if so, upgrade it. 9.4.2 contains a fix to a change introduced in 9.0.0 that is required in order to build the library.

Test

To run tests, use the following command:

npm test

Simplified test suite skips few time consuming tests and runs faster:

npm test-minimum

To execute a specific test, use the testNamePattern option with test-dbg script. For example:

npm run test-dbg -- --testNamePattern="transaction"

IT suite starts the server for testing - standalone and cluster installation using cluster_manager script. To run the integration tests with existing servers, run the following command:

npm run test-dbg -- --cluster-endpoints=localhost:7000 --standalone-endpoints=localhost:6379

# If those endpoints use TLS, add `--tls=true` (applies to both endpoints)
npm run test-dbg -- --cluster-endpoints=localhost:7000 --standalone-endpoints=localhost:6379 --tls=true

Parameters cluster-endpoints, standalone-endpoints and tls could be used with all test suites.

By default, the server modules tests do not run using npm run test. This test suite also does not start the server. In order to run these tests, use:

npm run test-modules -- --cluster-endpoints=<address>:<port>

Note: these tests don't run with standalone server as of now.

REPL (interactive shell)

It is possible to run an interactive shell synced with the currect client code to test and debug it:

npx ts-node --project tsconfig.json

This shell allows executing typescript and javascript code line by line:

import { GlideClient, GlideClusterClient } from ".";
let client = await GlideClient.createClient({
    addresses: [{ host: "localhost", port: 6379 }],
});
let clusterClient = await GlideClusterClient.createClient({
    addresses: [{ host: "localhost", port: 7000 }],
});
await client.ping();

After applying changes in client code you need to restart the shell.

It has command history and bash-like search (Ctrl+R).

Shell hangs on exit (Ctrl+D) if you don't close the clients. Use Ctrl+C to kill it and/or close clients before exit.

Submodules

After pulling new changes, ensure that you update the submodules by running the following command:

git submodule update

Linters

Development on the Node wrapper may involve changes in either the TypeScript or Rust code. Each language has distinct linter tests that must be passed before committing changes.

Language-specific Linters

TypeScript:

  • ESLint
  • Prettier

Rust:

  • clippy
  • fmt

Running the linters

  1. TypeScript

    # Run from the node folder
npm run lint
# To automatically apply ESLint and/or prettier recommendations
npm run lint:fix

  2. Rust

    # Run from the `node/rust-client` folder
rustup component add clippy rustfmt
cargo clippy --all-features --all-targets -- -D warnings
cargo fmt --manifest-path ./Cargo.toml --all

Recommended extensions for VS Code