You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> BARE is a simple binary representation for structured application data.
10
+
>
11
+
> - Messages are encoded in binary and compact in size. Messages do not contain
12
+
> schema information — they are not self-describing.
13
+
>
14
+
> - BARE is optimized for small messages. It is not optimized for encoding
15
+
> large amounts of data in a single message, or efficiently reading a message
16
+
> with fields of a fixed size. However, all types are aligned to 8 bits,
17
+
> which does exchange some space for simplicity.
18
+
>
19
+
> - BARE's approach to extensibility is conservative: messages encoded today
20
+
> will be decodable tomorrow, and vice-versa. But extensibility is still
21
+
> possible; implementations can choose to decode user-defined types at a
22
+
> higher level and map them onto arbitrary data types.
23
+
>
24
+
> - The specification is likewise conservative. Simple implementations of
25
+
> message decoders and encoders can be written inside of an afternoon.
26
+
>
27
+
> - An optional DSL is provided to document message schemas and provide a
28
+
> source for code generation. However, if you prefer, you may also define
29
+
> your schema using the type system already available in your programming
30
+
> language.
31
+
>
32
+
> [Source](https://baremessages.org/)
33
+
34
+
Also see the [IETF specification](https://www.ietf.org/archive/id/draft-devault-bare-11.html).
11
35
12
36
## Project goals
13
37
@@ -18,6 +42,7 @@ VBARE is a tiny extension to BARE
18
42
non-goals:
19
43
20
44
- data compactness -> that's what gzip is for
45
+
- provide an rpc layer -> this is trivial to do yourself based on your specific requirements
21
46
22
47
## Use cases
23
48
@@ -31,83 +56,144 @@ non-goals:
31
56
- Every message has a version associated with it
32
57
- either pre-negotiated (via something like an http request query parameter/handshake) or embedded int he message itself
33
58
- Applications provide functions to upgrade between protocol versions
34
-
- There is no migration in the schema itself, just copy and paste the schema to write the new one
59
+
- There is no evolution semantics in the schema itself, just copy and paste the schema to write the new one
35
60
36
-
## Migration philosophy
61
+
## evolutino philosophy
37
62
38
63
- declare discrete versions with predefined version indexes
39
-
- manual migrations simplify the application logic by putting complex defaults in your app code
64
+
- manual evolutions simplify the application logic by putting complex defaults in your app code
40
65
- stop making big breaking v1 -> v2 changes, make much smaller changes with more flexibility
41
66
- reshaping structures is important -- not just changing types and names
42
67
43
-
## Code examples
68
+
## specification
44
69
45
-
##Current users
70
+
### versions
46
71
47
-
- Rivet
48
-
- Network protocol
49
-
- All internal communication
50
-
- All data stored at rest
51
-
- RivetKit
52
-
- Protocol for communicating with clients
72
+
each schema version is a monotomically incrementing <TODO: integer type>
53
73
54
-
##FAQ
74
+
### embedded version
55
75
56
-
### Why not include RPC?
76
+
embedded version works by inserting a <TODO: integer type> integer at the beginning of the buffer. this integer is used to define which version of the schema is being used.
57
77
58
-
- why the fuck does your protocol need to define an rpc schema
59
-
- keep it simple, use a union
78
+
the layout looks like this:
60
79
61
-
### Why is copying the entire schema for every version better than using decorators for gradual migrations
80
+
```
81
+
TODO
82
+
```
62
83
63
-
### Isn't copying the schema going to result in a lot of duplicate code?
84
+
### pre-negotiated version
64
85
65
-
- yes. after enough pain and suffering of running production APIS, this is what you will end up doing manually, but in a much more painful way.
66
-
- having schema versions also makes it much easier to reason about how clients are connecting to your system/the state of an application. incremental migrations dno't let you consider other properties/structures.
67
-
- this also lets you reshape your structures.
86
+
often times, you speicty the protocol version outside of the message iteself. for eaxmple, if making an http request with the version in the path like `POST /v3/users`, we can extract version 3 from the path. in this case, VBARE does not insert a version in to the buffer. for this, vbare simply acts as a simple step function for upgrading/downgrading version data structures.
68
87
69
-
### Why copying instead of decorators for migrations?
88
+
##Implementations
70
89
71
-
- decorators are limited and get very complicated
72
-
- it's unclear what version of the protocol a decorator takes effect -- this is helpful
73
-
- generated sdks become more and more bloated with every change
74
-
- you need a validation build step for your validators
-[File system driver](https://github.com/rivet-dev/rivetkit/tree/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/file-system-driver)
109
+
110
+
## Embedded vs Negotiated Version
111
+
112
+
TODO
113
+
114
+
## Clients vs Servers
115
+
116
+
- Only servers need to ahve the evolutions steps
117
+
- clients just send their version
118
+
119
+
## Downsides
120
+
121
+
- extensive migration code
122
+
- the older the version the more migration steps (though these migration steps should be effectively free)
123
+
- migration steps are not portable across langauges, but only the server needs to the migration step. so usually this is only done once.
81
124
82
125
## Comparison
83
126
84
127
- Protobuf (versioned: yes)
85
128
- unbelievably poorly designed protocol
86
-
- makes it your problem by making everything optional
129
+
- makes migrations your problem at runtime by making everything optional
87
130
- even worse, makes properties have a default value (ie integers) which leads to subtle bugs with serious concequenses
88
-
- tracking indexes in the file is ass
131
+
- tracking field numbers in a file is a pain in the ass
89
132
- Cap'n'proto (versioned: yes)
90
-
- simplicity
91
-
- quality of output languages
133
+
- includes the rpc layer as part of the library, this is out of the scope of what we want in our schema design
134
+
- of the schema languages we evaluated, this provides by far the most flexible schema migrations
135
+
- has poor language support. technically most major languages are supported, but the qulaity of the ipmlementations are lacking. i suspect this is largely due to the complexity of capnproto itself compared to other protocols.
136
+
- generics are cool. but we opt for simplicity with more repetition.
137
+
- the learning curve seems the steepest of any other tool
138
+
- cap'n'web (versioned: no)
139
+
- this is focused on rpc with json. not relevant to what we needed.
92
140
- cbor/messagepack/that mongodb one (versioned: self-describing)
93
-
- requires encoding the entire key
141
+
- does not have a schema, it's completley self-describing
142
+
- requires encoding the entire key, not suitable for our needs
94
143
- Flatbuffers (versioned: yes)
144
+
- intented as a high performance encoding similar to protobuf
95
145
- still uses indexes like protobuf, unless you use structs
- considered borsh instead of bare, but bare seemed significantly simpler and more focused
102
158
- rust options like postcard/etc (versioned: no)
159
+
- also provides self-contained binary encoding
103
160
- not cross platform
104
161
105
-
## Implementations
162
+
other deatils not included in this evaluation:
163
+
- number compression (ie static 64 bits vs using minimal bits)
164
+
- zero-copy ser/de
165
+
- json support & extensions
166
+
- rpc
167
+
168
+
## FAQ
169
+
170
+
### Why is copying the entire schema for every version better than using decorators for gradual migrations?
171
+
172
+
- decorators are limited and get very complicated
173
+
- it's unclear what version of the protocol a decorator takes effect -- this is helpful
174
+
- generated sdks become more and more bloated with every change
175
+
- you need a validation build step for your validators
176
+
- things you can do with manual migrations
177
+
178
+
### Why not include RPC?
179
+
180
+
RPC interfaces are trivial to implement yourself. Libraries that provide RPC interfaces tend to add extra bloat & cognitive load over things like abstracting transports, compatibility with the language's async runtime, and complex codegen to implement handlers.
181
+
182
+
Usually, you just want a `ToServer` and `ToClient` union that looks like this: [ToClient example](https://github.com/rivet-dev/rivetkit/blob/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/client-protocol/v1.bare#L34), [ToServer example](https://github.com/rivet-dev/rivetkit/blob/b81d9536ba7ccad4449639dd83a770eb7c353617/packages/rivetkit/schemas/client-protocol/v1.bare#L56)
183
+
184
+
185
+
### Isn't copying the schema going to result in a lot of duplicate code?
186
+
187
+
- yes. after enough pain and suffering of running production APIS, this is what you will end up doing manually, but in a much more painful way.
188
+
- having schema versions also makes it much easier to reason about how clients are connecting to your system/the state of an application. incremental migrations dno't let you consider other properties/structures.
189
+
- this also lets you reshape your structures.
190
+
191
+
### Don't migration steps get repetitive?
192
+
193
+
- most of the time, structures will match exactly. most languages can provide a 1:1 migration.
194
+
- the most complicated migration steps will be very deeply nested structures that changed, but that's pretty simple
106
195
107
-
| Language | BARE | VBARE |
108
-
| --- | --- | --- |
109
-
| TypeScript | X | X |
110
-
| Rust | X | X |
196
+
## License
111
197
112
-
[Full list of BARE implementations](https://baremessages.org/)
0 commit comments