LokiKV : ⚠️ WIP

LokiKV is intended to be a simple to use in memory Key-Value Store that can also persist data on disk. This project is specifically for learning purposes only.

To try out loki-kv you can follow these steps:

git clone https://github.com/destrex271/LokiKV

# setup persistnace directory; defaults to a new data directory in the folder where server is running
export PERSIST_DATA=<fullpath>
export CHECKPOINT_DIR=<fullpath>
export CHECKPOINT_INTERVAL=<time in seconds>

cargo run --bin server-db # in a separate terminal
# runs on localhost:8765 by default

# in a separate terminal to start CLI
cargo run  --bin client -- localhost 8765

Current Features

Data Types

• Blob: <BLOB_BEGINS>data of blob<BLOB_ENDS>
• Integer
• Boolean
• Float
• String
• HyperLogLog: To estimate cardinality

Operations

• Set key values
• Get value for key
• Print all values in collection as a string
• Create multiple types of collections (\c_bcol, \c_hcol, \c_bcust)
• Select Collections
• List all available collections

---

Architecture

• Overall Process

• LokiKV structure

• LokiQL Executor

Supported Operations

LokiQL is a custom query language for interacting with the LokiKV database. This document describes the supported commands and their syntax.

Collections

• Create multiple Collections(similar to tables)
• Collections are of the following types:
  • Hashmap
  • BTreeMap
  • Custom BTree
• List collections
• Select one collection at a time

Literals

Whitespace

• Spaces, newlines, carriage returns, and tabs are ignored where applicable.

Data Types

• Integer (INT): Signed or unsigned integer numbers.
• Float (FLOAT): Signed or unsigned floating point numbers.
• Boolean (BOOL): true or false.
• String (STRING): Enclosed in single quotes ('example').
• Blob (BLOB): Enclosed in <BLOB_BEGINS> and <BLOB_ENDS>.
• HyperLogLog(HLL): Init by ADDHLL command

Identifiers

• ID: Any string without whitespace.

Command Syntax

LokiQL supports three types of commands:

Duo Commands (Require a Key and a Value)

Command	Syntax
SET	SET ID (STRING / INT / BOOL / FLOAT / BLOB)
ADDHLL(adds value to a HLL data type)	ADHLL ID (STRING / INT / BOOL / FLOAT / BLOB)

Examples:

SET mykey 'hello'
SET count 42
SET enabled true
SET temperature 98.6
SET file <BLOB_BEGINS>aGVsbG8=<BLOB_ENDS>

Uni Commands (Require a Key Only)

Command	Syntax
GET	GET <ID>
HLLCOUNT(estimated cardinality)	HLLCOUNT <ID>
INCR	INCR <ID>
DECR	DECR <ID>
PERSIST	PERSIST <collection_name>
LOAD_BCUST	LOAD_BCUST <collection_name>
LOAD_BDEF	LOAD_BDEF <collection_name>
LOAD_HMAP	LOAD_HMAP <collection_name>
/c_hcol	/c_hcol <ID>
/c_bcol	/c_bcol <ID>
/c_bcust	/c_bcust <ID>
/selectcol	/selectcol <ID>

Examples:

GET mykey
INCR count
DECR count
/selectcol users

Solo Commands (Do Not Require Arguments)

Command	Syntax
DISPLAY	DISPLAY
/getcur_colname	/getcur_colname
/listcolnames	/listcolnames

Examples:

DISPLAY
/getcur_colname
/listcolnames

Command File Structure

A LokiQL command file follows this structure:

COMMAND
COMMAND
COMMAND
COMMAND; COMMAND; COMMAND;

Example

SET mykey 'hello'
GET mykey
DISPLAY

Multiple command in a single line must be separated by ;. Single commands don't need to follow a ;

---

---

TODO

• Add support for distributed setup via Paxos Algorithm
• Add WAL support