RocksDB.jl

RocksDB

RocksDB is an LSM based embedded key-value store inspired by Google's LevelDB. The LSM design makes it perform well on SSD devices. RocksDB also has many features that LevelDB doesn't, like snapshots and checkpoints.

RocksDB.jl

RocksDB.jl is a Julia wrapper around librocksdb currently exposing only a subset of functionality. If you want more functions from RocksDB please submit a pull request or raise an issue.

RocksDB.jl calls into librocksdb using Julia's C calling interface. Since IO from librocksdb does not use libuv's event driven mechanism they could stall the Julia process. As a work around RocksDB.jl uses the threadcall mechanism that schedules the call into librocksdb on a separate thread.

Installing

RocksDB.jl works with all minor versions of Julia v1.0

$ mkdir myProject; cd myProject; julia

librocksdb takes about 20 minutes to build.

pkg> activate .

(myProject) pkg> add https://github.com/aj-monk/RocksDB.jl
   Cloning default registries into /home/ajaym/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `https://github.com/aj-monk/RocksDB.jl`
  Updating git-repo `https://github.com/aj-monk/RocksDB.jl`
 Resolving package versions...
 Installed URIParser ─ v0.4.0
 Installed BinDeps ─── v0.8.10
 Installed Compat ──── v1.0.1
  Updating `Project.toml`
  [1341945c] + RocksDB v0.1.0 #master (https://github.com/aj-monk/RocksDB.jl)
  Updating `Manifest.toml`
  [9e28174c] + BinDeps v0.8.10
  [34da2185] + Compat v1.0.1
  [1341945c] + RocksDB v0.1.0 #master (https://github.com/aj-monk/RocksDB.jl)
  [30578b45] + URIParser v0.4.0
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [1a1011a3] + SharedArrays 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode 
  Building RocksDB → `~/.julia/packages/RocksDB/qn86f/deps/build.log`

Basic Usage

julia> using RocksDB

# Opening a db returns a handle
# Set the second parameter to false if you want the method to
# return an exception if the DB does not already exist.
julia> db = open_db("/tmp/test.db", true)

julia> db_put(db, "key1", "value1")

julia> val = db_get(db, "key1")
"value1"

Key and Value can be any Julia type. RocksDB.jl will serialize them while writing and deserialize while reading.

julia> key = "key1"
"key1"

julia> value = rand(Int64)
1196320916215346617

julia> db_put(db, key, value)

julia> val = db_get(db, key)
1196320916215346617

julia> typeof(val)
Int64

# Compound types
julia> struct Foo
                 bar
                 baz::Int
                 qux::Float64
             end

julia> value = Foo("foo", 45, 4.33)
Foo("foo", 45, 4.33)

julia> db_put(db, "key2", value)

julia> val = db_get(db, "key2")
Foo("foo", 45, 4.33)

julia> typeof(val)
Foo

Call db_delete() to delete a key and it's corresponding value.

julia> db_get(db, "key1")
1196320916215346617

julia> db_delete(db, "key1")

If key does not exist then db_get() returns a Void.

julia> val = db_get(db, "key1")

julia> typeof(val)
Void

julia> if val == nothing
           println("Nothing stored at key1")
       end
Nothing stored at key1

In order to read a range of keys from the db call db_range() which returns an iterator for (key, value) tuples.

julia> for i in 1:100
           db_put(db, string(i), i)
       end

julia> r = db_range(db, "3", "15")
RocksDB.Range(Ptr{Void} @0x00007fd1240016e0, Ptr{Void} @0x00007fd1280008c0, UInt8[0x21, 0x01, 0x33, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00  …  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00], UInt8[0x21, 0x02, 0x31, 0x35, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00  …  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00], false, false)

julia> for (key, value) in r
           println("key: ", key, " value: ", value)
       end
key: 3 value: 3
key: 4 value: 4
key: 5 value: 5
key: 6 value: 6
key: 7 value: 7
key: 8 value: 8
key: 9 value: 9
key: 10 value: 10
key: 11 value: 11
key: 12 value: 12
key: 13 value: 13
key: 14 value: 14
key: 15 value: 15

Transactions

RocksDB supports batch writes where every key/value that is put in a batch is atomically committed.

julia> batch = create_write_batch()
Ptr{Void} @0x00007f923c0318f0

julia> batch_put(batch, "one", 1)

julia> batch_put(batch, "two", 2)

julia> batch_put(batch, "three", 3)

julia> write_batch(db, batch)  # Commits the batch

Checkpoints

Checkpoints are persistent snapshots of the entire db. They are typically used to take backups or to keep track of db state.

Call sb_create_checkpoint(db, path) to create a checkpoint in path. path must be an absolute path to a directory where the checkpoint will be created. The last component of path should not exist when making this call. Checkpoints are persistent. They can be opened for reads and writes, as a regular db would, by passing the path.

julia> db_create_checkpoint(db, "/tmp/chkpt1")

julia> chkpt1 = open_db("/tmp/chkpt1", false)
Ptr{Void} @0x00007fd114012fc0

julia> db_get(chkpt1, "one")
1

julia> db_get(chkpt1, "two")
2