Skip to content

Commit a27c4ff

Browse files
committed
wip
1 parent 87a3563 commit a27c4ff

File tree

12 files changed

+591
-246
lines changed

12 files changed

+591
-246
lines changed

README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,11 +147,13 @@ Usually, you just want a `ToServer` and `ToClient` union that looks like this:
147147

148148
### Isn't copying the schema going to result in a lot of duplicate code?
149149

150-
Yes, but after enough pain and suffering from running production APIs, this is what you will end up doing manually anyway, but in a much more painful way. Having schema versions also makes it much easier to reason about how clients are connecting to your system and the state of an application. Incremental migrations don't let you consider other properties or structures. This approach also lets you reshape your structures more effectively.
150+
Yes, but after enough pain and suffering from running production APIs, this is what you will end up doing manually anyway — but in a much more painful way.
151+
152+
Having schema versions also makes it much easier to reason about how clients are connecting to your system and the state of an application.
151153

152154
### Don't migration steps get repetitive?
153155

154-
Most of the time, structures will match exactly, and most languages can provide a 1:1 migration. The most complicated migration steps will be for deeply nested structures that changed, but even that is relatively straightforward.
156+
Migration steps are fairly minimal to write. The most verbose migration steps will be for deeply nested structures that changed, but even that is relatively straightforward.
155157

156158
### What are the downsides?
157159

rust/Cargo.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@ members = ["vbare-gen", "vbare-compiler", "vbare", "examples/basic"]
33
resolver = "2"
44

55
[workspace.package]
6-
version = "0.1.0"
6+
version = "0.0.1"
77
edition = "2021"
8-
authors = ["Your Name"]
9-
license = "MIT OR Apache-2.0"
8+
authors = ["Rivet Gaming, Inc. <[email protected]>"]
9+
license = "MIT"
1010

1111
[workspace.dependencies]
1212
anyhow = "1.0"

rust/README.md

Lines changed: 109 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,62 +1,136 @@
11
# Rust Workspace
22

3-
This workspace contains two Rust libraries:
3+
This workspace contains the code generator and runtime for working with BARE schemas and versioned data, plus a runnable example.
44

5-
## Libraries
5+
## Crates
66

7-
### compiler
8-
A build script compiler for processing schema files. This library provides utilities to:
9-
- Process schema files in a directory
10-
- Generate Rust code from schemas
11-
- Create module declarations for generated code
12-
- Handle build script integration with proper cargo rerun-if-changed directives
7+
- `vbare-gen`: TokenStream code generator that parses `.bare` schemas and emits Rust types deriving `serde` and using `serde_bare` for encoding/decoding.
8+
- `vbare-compiler`: Build-script helper that processes a directory of schemas, writes one Rust file per schema into `OUT_DIR`, and emits a `combined_imports.rs` module to include from your crate.
9+
- `vbare`: Runtime traits for versioned data with helpers to serialize/deserialize across versions and with embedded version headers.
10+
- `examples/basic`: End-to-end example that generates types for three schema versions (v1/v2/v3) and shows migrations between them.
11+
12+
## Quick Start (use in your crate)
13+
14+
1) Add dependencies in your `Cargo.toml`:
15+
16+
```toml
17+
[dependencies]
18+
anyhow = "1"
19+
serde = { version = "1", features = ["derive"] }
20+
serde_bare = "0.5"
21+
vbare = { path = "../vbare" } # adjust path as needed
22+
23+
[build-dependencies]
24+
anyhow = "1"
25+
vbare-compiler = { path = "../vbare-compiler" } # or use vbare-gen directly
26+
```
27+
28+
2) In `build.rs`, process your `.bare` schema files directory and generate the modules:
1329

14-
**Usage in build.rs:**
1530
```rust
16-
use compiler::SchemaCompiler;
1731
use std::path::Path;
1832

1933
fn main() -> Result<(), Box<dyn std::error::Error>> {
20-
let schema_dir = Path::new("schemas");
34+
let schemas = Path::new("schemas");
35+
// Or `process_schemas_with_config(schemas, &vbare_compiler::Config::with_hashable_map())`.
36+
vbare_compiler::process_schemas(schemas)?;
37+
Ok(())
38+
}
39+
```
2140

22-
// Use with a custom processor function
23-
SchemaCompiler::process_schemas_for_build_script(
24-
schema_dir,
25-
|path| {
26-
// Your schema processing logic here
27-
Ok(String::from("generated code"))
28-
}
29-
)?;
41+
Note: If you prefer to call the generator directly (as in `examples/basic`), use `vbare-gen` from your `build.rs`, parse the returned `TokenStream` with `syn`, and format with `prettyplease`. In that case add `syn` and `prettyplease` to `[build-dependencies]`.
3042

31-
Ok(())
43+
3) In your `lib.rs` or `mod.rs`, include the auto-generated module that re-exports all generated files:
44+
45+
```rust
46+
// Bring generated schemas into this crate
47+
pub mod schemas {
48+
#![allow(clippy::all)]
49+
include!(concat!(env!("OUT_DIR"), "/combined_imports.rs"));
50+
}
51+
```
52+
53+
4) Implement versioning (example with owned data):
54+
55+
```rust
56+
use anyhow::{bail, Result};
57+
use vbare::OwnedVersionedData;
58+
59+
#[derive(Clone)]
60+
pub enum MyTypeVersioned {
61+
V1(schemas::v1::MyType),
62+
V2(schemas::v2::MyType),
63+
}
64+
65+
impl OwnedVersionedData for MyTypeVersioned {
66+
type Latest = schemas::v2::MyType;
67+
68+
fn latest(latest: Self::Latest) -> Self { Self::V2(latest) }
69+
fn into_latest(self) -> Result<Self::Latest> {
70+
match self { Self::V2(x) => Ok(x), _ => bail!("not latest") }
71+
}
72+
73+
fn deserialize_version(payload: &[u8], version: u16) -> Result<Self> {
74+
Ok(match version {
75+
1 => Self::V1(serde_bare::from_slice(payload)?),
76+
2 => Self::V2(serde_bare::from_slice(payload)?),
77+
_ => bail!("invalid version: {version}"),
78+
})
79+
}
80+
81+
fn serialize_version(self, _version: u16) -> Result<Vec<u8>> {
82+
Ok(match self {
83+
Self::V1(x) => serde_bare::to_vec(&x)?,
84+
Self::V2(x) => serde_bare::to_vec(&x)?,
85+
})
86+
}
87+
88+
fn deserialize_converters() -> Vec<impl Fn(Self) -> Result<Self>> {
89+
vec![Self::v1_to_v2] // order: v1->v2, v2->v3, ...
90+
}
91+
92+
fn serialize_converters() -> Vec<impl Fn(Self) -> Result<Self>> {
93+
vec![Self::v2_to_v1] // optional: latest->older conversions
94+
}
3295
}
3396
```
3497

35-
### vbare
36-
A versioned data serialization library that provides traits for:
37-
- Versioned data serialization/deserialization
38-
- Automatic version conversion between different schema versions
39-
- Embedded version encoding in payloads
40-
- Support for both borrowed and owned data types
98+
Then use `deserialize`/`serialize` or their `*_with_embedded_version` variants:
4199

42-
**Features:**
43-
- `VersionedData<'a>` trait for borrowed data
44-
- `OwnedVersionedData` trait for owned data
45-
- Automatic version migration via converter functions
46-
- Embedded version support for self-describing payloads
100+
```rust
101+
// Decode bytes encoded as version 1 into latest
102+
let latest = MyTypeVersioned::deserialize(&bytes, 1)?;
47103

48-
## Building
104+
// Encode latest as version 1
105+
let v1_bytes = MyTypeVersioned::latest(latest).serialize(1)?;
106+
107+
// Or embed version in the payload header (little-endian u16 prefix):
108+
let bytes = MyTypeVersioned::latest(latest).serialize_with_embedded_version(2)?;
109+
let latest2 = MyTypeVersioned::deserialize_with_embedded_version(&bytes)?;
110+
```
111+
112+
## Example
113+
114+
See `rust/examples/basic/` for a full example:
115+
- `build.rs` normalizes the test fixture schemas and runs codegen (mirrors what `vbare-compiler` does).
116+
- `src/lib.rs` includes the generated `schemas` module and implements `OwnedVersionedData` for `AppVersioned` with v1→v2→v3 migrations.
117+
- `tests/migrator.rs` exercises up/down conversions and BARE encoding with `serde_bare`.
118+
119+
Run just the example crate’s tests:
49120

50121
```bash
51-
cargo build
122+
cargo test -p basic
52123
```
53124

54-
## Testing
125+
## Workspace
126+
127+
Build and test everything:
55128

56129
```bash
130+
cargo build
57131
cargo test
58132
```
59133

60134
## License
61135

62-
MIT OR Apache-2.0
136+
MIT OR Apache-2.0

rust/vbare-gen/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
TODO: This package is a fork, need to find the package & provide correct attribution

scripts/release.ts

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
#!/usr/bin/env -S tsx
2+
3+
import { promises as fs } from 'node:fs';
4+
import path from 'node:path';
5+
import { spawn } from 'node:child_process';
6+
import { fileURLToPath } from 'node:url';
7+
8+
const __filename = fileURLToPath(import.meta.url);
9+
const __dirname = path.dirname(__filename);
10+
const repoRoot = path.resolve(__dirname, '..');
11+
12+
const IGNORED_DIRECTORIES = new Set([
13+
'.git',
14+
'node_modules',
15+
'target',
16+
'dist',
17+
'build',
18+
'.pnpm-store'
19+
]);
20+
21+
async function main(): Promise<void> {
22+
const version = process.argv[2];
23+
24+
if (!version) {
25+
console.error('Usage: release <version>');
26+
process.exitCode = 1;
27+
return;
28+
}
29+
30+
if (!isValidVersion(version)) {
31+
console.error(`Invalid version: ${version}`);
32+
process.exitCode = 1;
33+
return;
34+
}
35+
36+
const manifests = await collectManifests(repoRoot);
37+
38+
await Promise.all(manifests.packageJson.map(file => updatePackageJson(file, version)));
39+
await Promise.all(manifests.cargoToml.map(file => updateCargoToml(file, version)));
40+
41+
await runCommand('pnpm', ['publish'], path.join(repoRoot, 'typescript'));
42+
await runCommand('cargo', ['publish'], path.join(repoRoot, 'rust'));
43+
}
44+
45+
function isValidVersion(version: string): boolean {
46+
return /^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$/.test(version);
47+
}
48+
49+
async function collectManifests(root: string): Promise<{ packageJson: string[]; cargoToml: string[] }> {
50+
const packageJson: string[] = [];
51+
const cargoToml: string[] = [];
52+
53+
await walk(root, async filePath => {
54+
const base = path.basename(filePath);
55+
56+
if (base === 'package.json') {
57+
packageJson.push(filePath);
58+
} else if (base === 'Cargo.toml') {
59+
cargoToml.push(filePath);
60+
}
61+
});
62+
63+
return { packageJson, cargoToml };
64+
}
65+
66+
async function walk(dir: string, onFile: (filePath: string) => Promise<void>): Promise<void> {
67+
const entries = await fs.readdir(dir, { withFileTypes: true });
68+
69+
for (const entry of entries) {
70+
if (IGNORED_DIRECTORIES.has(entry.name)) {
71+
continue;
72+
}
73+
74+
const fullPath = path.join(dir, entry.name);
75+
76+
if (entry.isDirectory()) {
77+
await walk(fullPath, onFile);
78+
} else if (entry.isFile()) {
79+
await onFile(fullPath);
80+
}
81+
}
82+
}
83+
84+
async function updatePackageJson(filePath: string, version: string): Promise<void> {
85+
const raw = await fs.readFile(filePath, 'utf8');
86+
let parsed: unknown;
87+
88+
try {
89+
parsed = JSON.parse(raw);
90+
} catch (error) {
91+
throw new Error(`Failed to parse JSON in ${relative(filePath)}: ${(error as Error).message}`);
92+
}
93+
94+
if (!parsed || typeof parsed !== 'object') {
95+
throw new Error(`Unexpected JSON shape in ${relative(filePath)} (expected object)`);
96+
}
97+
98+
const pkg = parsed as { version?: unknown; [key: string]: unknown };
99+
100+
if (typeof pkg.version !== 'string') {
101+
console.warn(`Skipping ${relative(filePath)} (no version field)`);
102+
return;
103+
}
104+
105+
pkg.version = version;
106+
107+
const newline = raw.includes('\r\n') ? '\r\n' : '\n';
108+
let serialized = JSON.stringify(pkg, null, 2);
109+
serialized = serialized.replace(/\n/g, newline) + newline;
110+
111+
await fs.writeFile(filePath, serialized, 'utf8');
112+
console.log(`Updated ${relative(filePath)} to ${version}`);
113+
}
114+
115+
async function updateCargoToml(filePath: string, version: string): Promise<void> {
116+
const raw = await fs.readFile(filePath, 'utf8');
117+
const newline = raw.includes('\r\n') ? '\r\n' : '\n';
118+
const lines = raw.split(/\r?\n/);
119+
const hadFinalNewline = raw.endsWith('\n') || raw.endsWith('\r\n');
120+
121+
let inPackageSection = false;
122+
let updated = false;
123+
124+
const updatedLines = lines.map(line => {
125+
const trimmed = line.trim();
126+
127+
if (trimmed.startsWith('[') && trimmed.endsWith(']')) {
128+
inPackageSection = trimmed === '[package]';
129+
return line;
130+
}
131+
132+
if (inPackageSection) {
133+
const match = line.match(/^(\s*version\s*=\s*")([^\"]*)(".*)$/);
134+
if (match) {
135+
updated = true;
136+
return `${match[1]}${version}${match[3]}`;
137+
}
138+
}
139+
140+
return line;
141+
});
142+
143+
if (!updated) {
144+
console.warn(`Skipping ${relative(filePath)} (no [package] version field found)`);
145+
return;
146+
}
147+
148+
let content = updatedLines.join(newline);
149+
if (hadFinalNewline && !content.endsWith(newline)) {
150+
content += newline;
151+
}
152+
153+
await fs.writeFile(filePath, content, 'utf8');
154+
console.log(`Updated ${relative(filePath)} to ${version}`);
155+
}
156+
157+
async function runCommand(command: string, args: string[], cwd: string): Promise<void> {
158+
console.log(`Running ${command} ${args.join(' ')} in ${relative(cwd)}`);
159+
160+
await new Promise<void>((resolve, reject) => {
161+
const child = spawn(command, args, {
162+
cwd,
163+
stdio: 'inherit',
164+
env: process.env
165+
});
166+
167+
child.on('error', reject);
168+
child.on('exit', code => {
169+
if (code === 0) {
170+
resolve();
171+
} else {
172+
reject(new Error(`${command} ${args.join(' ')} exited with code ${code}`));
173+
}
174+
});
175+
});
176+
}
177+
178+
function relative(filePath: string): string {
179+
return path.relative(repoRoot, filePath) || '.';
180+
}
181+
182+
main().catch(error => {
183+
console.error(error);
184+
process.exit(1);
185+
});

0 commit comments

Comments
 (0)