Merklize

A Rust library and HTTP server for building Merkle trees from data sources and generating cryptographic proofs.

This project is part of Efímero: Disposable Chains on Polkadot. Read detailed docs and related concepts at - https://efimero.blockdeep.dev.

Features

• Build Merkle trees from CSV files or SQL databases (SQLite, PostgreSQL, MySQL)
• Generate and verify cryptographic proofs using Blake2-256 hashing
• HTTP API for querying Merkle roots and creating proofs
• SCALE encoding for keys and values

Usage

CLI Server

# From CSV file, this is a example with the included test data
cargo run --bin cli -- --input test/data/test.csv --key-column address csv

# From SQLite database  
cargo run --bin cli -- --input database.db --key-column id sqlite --columns name --columns balance --table accounts

# From PostgreSQL
cargo run --bin cli -- --input "postgresql://user:pass@localhost/db" --key-column id postgres --columns name --columns balance --table accounts

# From MySQL
cargo run --bin cli -- --input "mysql://user:pass@localhost/db" --key-column id mysql --columns name --columns balance --table accounts

Testing the API

# Get Merkle root
curl http://127.0.0.1:3000/root

# Check account balance (hex-encoded account)
curl "http://127.0.0.1:3000/balance?account=ACCOUNT_HEX"

# Create simple proof
curl -X POST http://127.0.0.1:3000/proof/simple \
  -H "Content-Type: application/json" \
  -d '{"account": [ACCOUNT_BYTES]}'

Substrate Account Processing Example

Substrate addresses are SS58-encoded and need to be converted to raw AccountId32 bytes for API calls.

Example with Alice's account:

• SS58 Address: 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
• AccountId32 bytes: [212, 53, 147, 199, 21, 253, 211, 28, 97, 20, 26, 189, 4, 169, 159, 214, 130, 44, 133, 88, 133, 76, 205, 227, 154, 86, 132, 231, 165, 109, 162, 125]

# Get proof for Alice's account
curl -X POST http://127.0.0.1:3000/proof/simple \
  -H "Content-Type: application/json" \
  -d '{
    "account": [212, 53, 147, 199, 21, 253, 211, 28, 97, 20, 26, 189, 4, 169, 159, 214, 130, 44, 133, 88, 133, 76, 205, 227, 154, 86, 132, 231, 165, 109, 162, 125]
  }'

# Get balance for Alice's account (using hex encoding)
# Note: The balance endpoint returns both the balance and the account bytes needed for proof generation
curl "http://127.0.0.1:3000/balance?account=d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d"
# Returns: {"account":[212,53,147,199,...],"balance":10}

The CSV reader automatically converts SS58 addresses to SCALE-encoded AccountId32 bytes when building the trie.

Library

use merklize::{GenerateTrie, GenericTrie, read_from_csv};

// Build trie from data
let data = read_from_csv("data.csv", "address")?;
let trie = GenericTrie::generate_for(data)?;

// Get root hash
let root = trie.root();

// Create proof for a key
let proof = trie.create_proof(key)?;

// Verify proof
let verification_trie = GenericTrie::from_nodes(*root, &proof);
let value = verification_trie.query(key);

API Endpoints

• GET /root - Returns the Merkle tree root hash
• GET /balance?account=HEX - Returns account balance
• POST /proof/simple - Creates a simple proof for an account
• POST /proof - Creates an authenticated proof (requires Google OAuth)

Environment Variables

• SERVER_ADDRESS - Server bind address (default: 127.0.0.1:3000)
• MK_KEY_COLUMN - Column name to use as key
• MK_INPUT - Input file/database connection string

License

This project is licensed under the Apache 2 license.