Initial fully snapshot isolated in-memory DB implementation.

Replaces "InMemoryTransient"; is still transient, but is actually properly transactional and a foundation for building a faster backend datastore.

Cargo.toml

@@ -89,10 +89,11 @@ thiserror = "1.0.47"
 ## For macro-ing
 paste = "1.0.12"
 
-# For the DB layer.
+# For the DB / values layer.
 rocksdb = "0.21.0"
 crossbeam-channel = "0.5.8"
 bincode = "2.0.0-rc.3"
+im = "15.1.0"
 
 # Dev dependencies
 tempfile = "3.8.0"

README.md

@@ -35,7 +35,7 @@ execute existing cores, and the 1.0 feature release is targeting this rather amb
 * Have tested against JaysHouseCore, and most of the functionality is there. Bugs are becoming increasingly rare.
 * Hosts websocket, "telnet" (classic line oriented TCP connection), and console connections. MCP clients work, with
   remote editing, etc. support.
-* Objects are stored in a concurrent transactional object database -- safe, consistent and happy. The architecture 
+* Objects are stored in a concurrent transactional object database -- safe, consistent and happy. The architecture
   allows for cleanly adding different storage backends for new scenarios.
 * Monitoring/metrics support via Prometheus-compatible export.
 * Separate network-host vs daemon process architecture means that upgrades/restarts can happen in-place without
@@ -50,21 +50,21 @@ editing a MOO verb in an emacs buffer with syntax highlighting:
 
 The easiest way to get started is to run the `docker compose` setup. This will bring up a complete server with `telnet`
 and `websocket` interfaces. The server will be setup with an initial `JaysHouseCore` core import, and will be set up with
-metrics monitoring via Grafana and VictoriaMetrics. 
+metrics monitoring via Grafana and VictoriaMetrics.
 
 To run, take a look at the local `docker-compose.yml` file, instructions are there, but it really just amounts to:
 
 ```
     docker compose up
 ```
 
-Then connect (with your favourite MUD/MOO client) to `localhost:8888` and follow the login instauctions. Classic 
-clients like `TinyFugue` will work fine, and there are some newer clients like [BlightMud](https://github.com/Blightmud/Blightmud) 
+Then connect (with your favourite MUD/MOO client) to `localhost:8888` and follow the login instauctions. Classic
+clients like `TinyFugue` will work fine, and there are some newer clients like [BlightMud](https://github.com/Blightmud/Blightmud)
 that should work fine. (A partial -- and probably outdated list of clients -- can be found here: https://lisdude.com/moo/#clients)
 
-Once you're familiar with how the docker setup works, you can get more creative. 
+Once you're familiar with how the docker setup works, you can get more creative.
 
-Note this configuration is set up for a development -- it compiles in debug mode, and is to watch the source directory for changes, 
+Note this configuration is set up for a development -- it compiles in debug mode, and is to watch the source directory for changes,
 recompile, and redeploy as needed.
 
 An actual production deployment can be fairly easily derived from the `docker-compose.yml` file, and the provided `Dockerfile` by
@@ -109,29 +109,29 @@ removing the `cargo watch` pieces and adding `--release` etc.
 
 The following are targeted as eventual goals / additions once 1.0 (fall 2023) is out the door:
 
-   * A richer front-end experience. Support for websockets as a connection method means that the server can provide 
-   a richer narrative stream to browsers (with in-core support assistance.) A client which provides a full proper 
-   UI experience with interactive UI components, graphical elements, and so on are the end-goal here.
-   * Support for multiple programming language for programming MOO verbs/objects. The backend has been written such that
-     this is feasible. Authoring verbs in JavaScript/TypeScript will be the first target, and WebAssembly modules are
-     also a possibility. These verbs would still run within the same shared environment and use the same shared object
-     environment, but would allow for a more modern programming experience.
-   * A more scalable server architecture; the system right now is divided into separate "host" frontends for network  
-     connections, and a common backend `daemon` which manages the database, virtual machine, and task scheduler. This
-     can be further split up to permit a distributed database backend or distributing other components, to meet higher
-     scalability goals if that is needed.
-   * Enhancements to the MOO data model and language, to support a richer / smoother authoring experience. Some ideas 
-     are:
-     * Datalog-style relations / predicates; for managing logical relationships between entities. This could allow
-       bidirectional (or more) relationships like already exist with e.g. `location`/`contents`, but more generalized,
-       and to allow for making complex worlds easier to maintain.
-     * Adding a map/dictionary type. MOO predates the existence of dictionary types as a standard type in most languages.  
-       MOO's type system only has lists and uses "associative lists" for maps, which are a bit awkward. Immutable/CoW
-       maps with an explicit syntax would be a nice addition. Other MOO offshoots (Stunt, etc.) do already provided this.
-     * Adding a `binary` type. MOO's type system is very string-oriented, and there's not an elegant way to represent
-       arbitrary binary data. (There's `encode_binary` and `decode_binary` builtins, but these are not the way I'd do it
-       today.)
-     * and so on
+* A richer front-end experience. Support for websockets as a connection method means that the server can provide
+  a richer narrative stream to browsers (with in-core support assistance.) A client which provides a full proper
+  UI experience with interactive UI components, graphical elements, and so on are the end-goal here.
+* Support for multiple programming language for programming MOO verbs/objects. The backend has been written such that
+  this is feasible. Authoring verbs in JavaScript/TypeScript will be the first target, and WebAssembly modules are
+  also a possibility. These verbs would still run within the same shared environment and use the same shared object
+  environment, but would allow for a more modern programming experience.
+* A more scalable server architecture; the system right now is divided into separate "host" frontends for network  
+  connections, and a common backend `daemon` which manages the database, virtual machine, and task scheduler. This
+  can be further split up to permit a distributed database backend or distributing other components, to meet higher
+  scalability goals if that is needed.
+* Enhancements to the MOO data model and language, to support a richer / smoother authoring experience. Some ideas
+  are:
+    * Datalog-style relations / predicates; for managing logical relationships between entities. This could allow
+      bidirectional (or more) relationships like already exist with e.g. `location`/`contents`, but more generalized,
+      and to allow for making complex worlds easier to maintain.
+    * Adding a map/dictionary type. MOO predates the existence of dictionary types as a standard type in most languages.  
+      MOO's type system only has lists and uses "associative lists" for maps, which are a bit awkward. Immutable/CoW
+      maps with an explicit syntax would be a nice addition. Other MOO offshoots (Stunt, etc.) do already provided this.
+    * Adding a `binary` type. MOO's type system is very string-oriented, and there's not an elegant way to represent
+      arbitrary binary data. (There's `encode_binary` and `decode_binary` builtins, but these are not the way I'd do it
+      today.)
+    * and so on
 
 ## License.
 

crates/console-host/src/main.rs

@@ -284,7 +284,7 @@ async fn console_loop(
             //   cleanliness. Need to figure out something for this.
             let input_request_id = input_request_id.lock().await.take();
             let prompt = if let Some(input_request_id) = input_request_id {
-                format!("{} > ", input_request_id.to_string())
+                format!("{} > ", input_request_id)
             } else {
                 "> ".to_string()
             };

crates/db/Cargo.toml

@@ -33,4 +33,6 @@ metrics-macros.workspace = true
 # For the DB layer.
 rocksdb.workspace = true
 crossbeam-channel.workspace = true
-bincode.workspace = true
+bincode.workspace = true
+im.workspace = true
+

crates/db/src/channel_db_tx_client.rs

@@ -106,37 +106,41 @@ impl DbTransaction for DbTxChannelClient {
         get_reply(receive).await?;
         Ok(())
     }
-    async fn get_parent(&self, obj: Objid) -> Result<Objid, WorldStateError> {
+    async fn get_object_parent(&self, obj: Objid) -> Result<Objid, WorldStateError> {
         let (send, receive) = oneshot::channel();
         self.send(DbMessage::GetParentOf(obj, send))?;
         let oid = get_reply(receive).await?;
         Ok(oid)
     }
-    async fn set_parent(&self, obj: Objid, parent: Objid) -> Result<(), WorldStateError> {
+    async fn set_object_parent(&self, obj: Objid, parent: Objid) -> Result<(), WorldStateError> {
         let (send, receive) = oneshot::channel();
         self.send(DbMessage::SetParent(obj, parent, send))?;
         get_reply(receive).await?;
         Ok(())
     }
-    async fn get_children(&self, obj: Objid) -> Result<ObjSet, WorldStateError> {
+    async fn get_object_children(&self, obj: Objid) -> Result<ObjSet, WorldStateError> {
         let (send, receive) = oneshot::channel();
         self.send(DbMessage::GetChildrenOf(obj, send))?;
         let children = get_reply(receive).await?;
         Ok(children)
     }
-    async fn get_location_of(&self, obj: Objid) -> Result<Objid, WorldStateError> {
+    async fn get_object_location(&self, obj: Objid) -> Result<Objid, WorldStateError> {
         let (send, receive) = oneshot::channel();
         self.send(DbMessage::GetLocationOf(obj, send))?;
         let oid = get_reply(receive).await?;
         Ok(oid)
     }
-    async fn set_location_of(&self, obj: Objid, location: Objid) -> Result<(), WorldStateError> {
+    async fn set_object_location(
+        &self,
+        obj: Objid,
+        location: Objid,
+    ) -> Result<(), WorldStateError> {
         let (send, receive) = oneshot::channel();
         self.send(DbMessage::SetLocationOf(obj, location, send))?;
         get_reply(receive).await?;
         Ok(())
     }
-    async fn get_contents_of(&self, obj: Objid) -> Result<ObjSet, WorldStateError> {
+    async fn get_object_contents(&self, obj: Objid) -> Result<ObjSet, WorldStateError> {
         let (send, receive) = oneshot::channel();
         self.send(DbMessage::GetContentsOf(obj, send))?;
         let contents = get_reply(receive).await?;
@@ -226,13 +230,13 @@ impl DbTransaction for DbTxChannelClient {
         }
         Ok(())
     }
-    async fn add_verb(
+    async fn add_object_verb(
         &self,
         location: Objid,
         owner: Objid,
         names: Vec<String>,
-        binary_type: BinaryType,
         binary: Vec<u8>,
+        binary_type: BinaryType,
         flags: BitEnum<VerbFlag>,
         args: VerbArgsSpec,
     ) -> Result<(), WorldStateError> {
@@ -347,7 +351,7 @@ impl DbTransaction for DbTxChannelClient {
         let (prop, value) = get_reply(receive).await?;
         Ok((prop, value))
     }
-    async fn valid(&self, obj: Objid) -> Result<bool, WorldStateError> {
+    async fn object_valid(&self, obj: Objid) -> Result<bool, WorldStateError> {
         if obj.0 < 0 {
             return Ok(false);
         }

crates/db/src/db_loader_client.rs

@@ -24,10 +24,10 @@ impl LoaderInterface for DbTxWorldState {
         Ok(self.tx.create_object(objid, attrs.clone()).await?)
     }
     async fn set_object_parent(&self, obj: Objid, parent: Objid) -> Result<(), WorldStateError> {
-        Ok(self.tx.set_parent(obj, parent).await?)
+        Ok(self.tx.set_object_parent(obj, parent).await?)
     }
     async fn set_object_location(&self, o: Objid, location: Objid) -> Result<(), WorldStateError> {
-        Ok(self.tx.set_location_of(o, location).await?)
+        Ok(self.tx.set_object_location(o, location).await?)
     }
     async fn set_object_owner(&self, obj: Objid, owner: Objid) -> Result<(), WorldStateError> {
         Ok(self.tx.set_object_owner(obj, owner).await?)
@@ -42,12 +42,12 @@ impl LoaderInterface for DbTxWorldState {
         binary: Vec<u8>,
     ) -> Result<(), WorldStateError> {
         self.tx
-            .add_verb(
+            .add_object_verb(
                 obj,
                 owner,
                 names.iter().map(|s| s.to_string()).collect(),
-                BinaryType::LambdaMoo18X,
                 binary,
+                BinaryType::LambdaMoo18X,
                 flags,
                 args,
             )

crates/db/src/db_tx.rs

@@ -33,12 +33,13 @@ pub trait DbTransaction {
     ) -> Result<Objid, WorldStateError>;
     async fn recycle_object(&self, obj: Objid) -> Result<(), WorldStateError>;
     async fn set_object_name(&self, obj: Objid, name: String) -> Result<(), WorldStateError>;
-    async fn get_parent(&self, obj: Objid) -> Result<Objid, WorldStateError>;
-    async fn set_parent(&self, obj: Objid, parent: Objid) -> Result<(), WorldStateError>;
-    async fn get_children(&self, obj: Objid) -> Result<ObjSet, WorldStateError>;
-    async fn get_location_of(&self, obj: Objid) -> Result<Objid, WorldStateError>;
-    async fn set_location_of(&self, obj: Objid, location: Objid) -> Result<(), WorldStateError>;
-    async fn get_contents_of(&self, obj: Objid) -> Result<ObjSet, WorldStateError>;
+    async fn get_object_parent(&self, obj: Objid) -> Result<Objid, WorldStateError>;
+    async fn set_object_parent(&self, obj: Objid, parent: Objid) -> Result<(), WorldStateError>;
+    async fn get_object_children(&self, obj: Objid) -> Result<ObjSet, WorldStateError>;
+    async fn get_object_location(&self, obj: Objid) -> Result<Objid, WorldStateError>;
+    async fn set_object_location(&self, obj: Objid, location: Objid)
+        -> Result<(), WorldStateError>;
+    async fn get_object_contents(&self, obj: Objid) -> Result<ObjSet, WorldStateError>;
     async fn get_max_object(&self) -> Result<Objid, WorldStateError>;
     async fn get_verbs(&self, obj: Objid) -> Result<VerbDefs, WorldStateError>;
     // TODO: this could return SliceRef or an Arc<Vec<u8>>, to potentially avoid copying. Though
@@ -60,13 +61,13 @@ pub trait DbTransaction {
         uuid: Uuid,
         verb_attrs: VerbAttrs,
     ) -> Result<(), WorldStateError>;
-    async fn add_verb(
+    async fn add_object_verb(
         &self,
         location: Objid,
         owner: Objid,
         names: Vec<String>,
-        binary_type: BinaryType,
         binary: Vec<u8>,
+        binary_type: BinaryType,
         flags: BitEnum<VerbFlag>,
         args: VerbArgsSpec,
     ) -> Result<(), WorldStateError>;
@@ -99,7 +100,7 @@ pub trait DbTransaction {
         obj: Objid,
         name: String,
     ) -> Result<(PropDef, Var), WorldStateError>;
-    async fn valid(&self, obj: Objid) -> Result<bool, WorldStateError>;
+    async fn object_valid(&self, obj: Objid) -> Result<bool, WorldStateError>;
     async fn commit(&self) -> Result<CommitResult, WorldStateError>;
     async fn rollback(&self) -> Result<(), WorldStateError>;
 }