Update README.md and CHANGELOG.md

CHANGELOG.md

@@ -8,6 +8,10 @@ What changed in mapiproxy, per version
 - No longer default to --messages. It's not clear what the default should
   be so for the time being it's better to not have a default at all.
 
+- Add experimental `--pcap=FILE` option to read captured network traffic files
+  written by for example, [tcpdump](https://www.tcpdump.org/).
+  So far tqhis has seen very limited testing.
+
 
 ## mapiproxy 0.5.2 - 2024-02-21
 

README.md

@@ -1,36 +1,134 @@
 MAPI PROXY
 ==========
 
-Proxy to inspect [MonetDB] network traffic.
-
-[MonetDB]: https://www.monetdb.org/
+Mapiproxy is a tool for inspecting [MonetDB](https://www.monetdb.org/)
+network traffic by acting as a proxy between clients and servers.
 
 The MonetDB network protocol is called MAPI, hence the name.
 
-The proxy can listen on and connect to TCP sockets, Unix Domain sockets or both.
-The MAPI protocol is slightly different between the two, the proxy will
-automatically adjust it when proxying between the two.
+The proxy can work with TCP sockets, Unix Domain sockets or both at the same
+time. The MAPI protocol is slightly different between TCP and Unix sockets, the
+proxy will automatically adjust it when proxying between the two.
+
+Additionally, Mapiproxy offers experimental support for reading PCAP files:
+captured network traffic files, often generated by tools like
+[tcpdump](https://www.tcpdump.org/) and [Wireshark](https://www.wireshark.org/).
+However, this feature is still in its early stages and may not be fully
+functional in practice.
+
+## Usage
 
-Usage:
+The following is a summary of Mapiproxy's usage:
 
 ```plain
 Usage: mapiproxy [OPTIONS] LISTEN_ADDR FORWARD_ADDR
        mapiproxy [OPTIONS] --pcap PCAP_FILE
 
 LISTEN_ADDR and FORWARD_ADDR:
-    port, for example, 50000
-    host:port, for example, localhost:50000 or 127.0.0.1:50000
+    PORT, for example, 50000
+    HOST:PORT, for example, localhost:50000 or 127.0.0.1:50000
     /path/to/unixsock, for example, /tmp/.s.monetdb.50000
 
 Options:
-    -m --messages       Dump whole messages
-    -b --blocks         Dump individual blocks
-    -r --raw            Dump bytes as they come in
-    -B --binary         Force dumping as binary
-    --color=WHEN        Colorize output, one of 'always', 'auto' or 'never'
-    --help              This help
-    --version           Version info
+    -m, --messages       Dump whole messages
+    -b, --blocks         Dump individual blocks
+    -r, --raw            Dump bytes as they come in
+    -B, --binary         Force dumping as binary
+    --color=WHEN         Colorize output (Options: 'always', 'auto', 'never')
+    --help               Display this help message
+    --version            Show version information
 
 Experimental options:
-    --pcap=FILE         Read network capture data from FILE, use - for stdin
+    --pcap=FILE          Read network capture data from FILE (use '-' for stdin)
 ```
+
+## Installation
+
+Source code is available on [GitHub](https://github.com/MonetDBSolutions/mapiproxy).
+
+Pre-built binaries for x86-64 Linux, MacOS, and Windows are available on the
+[Releases page](https://github.com/MonetDBSolutions/mapiproxy/releases/) of the
+GitHub repository.
+
+Mapiproxy is written in Rust. If you want to build from source and don't have
+the Rust compiler installed, you can obtain it from
+[rustup.rs](https://rustup.rs/).
+
+To build from source, simply execute `cargo install mapiproxy` or clone the
+repository from GitHub and build using `cargo build --release`.
+
+
+Output Modes
+------------
+
+Mapiproxy offers three output modes, corresponding to the layers of the MAPI
+protocol. In a nutshell, MAPI communication consists of a series of messages.
+Each message is sent as of one or more blocks. Each block is preceded by a
+two-byte block header.
+
+Mapiproxy output consists of a mix of single-line informational messages and
+multi-line frames that display the captured data. For example, in the
+single-line message
+
+```plain
+‣ #10 UPSTREAM client stopped sending
+```
+we see a connection id, a direction marker (UPSTREAM for client to server and
+DOWNSTREAM for server to client), and an informational message.
+
+Frames display the data either as a hex dump, like this,
+```plain
+┌ #10 UPSTREAM binary, message, 13 bytes
+│ 73 73 65 6c  65 63 74 20   34 32 0a 3b  0a __ __ __     sselect·42↵;↵
+└
+```
+or they display it as text, like this:
+```plain
+┌ #10 UPSTREAM text, message, 13 bytes
+│sselect 42↵
+│;↵
+└
+```
+Mapi generally prefers text output over binary, but will render a frame in
+binary when at least one of the following conditions holds true:
+
+* the data is not valid UTF-8 encoded text
+
+* the data contains control characters other than newline and tab
+
+* Mapiproxy is running in `--raw` mode
+
+* binary output is forced using the `-B` or `--binary` flag
+
+When Mapiproxy is in `--messages` mode, it collects all bytes of whole message
+and prints them at in a single frame, skipping the block headers. In `--blocks`
+mode, it collects whole blocks and prints one block per frame, also without the
+block header.
+
+In `--raw` mode, bytes are printed as they are received, including the block
+headers. This means that a single printed chunk may contain parts of multiple
+blocks and even parts of multiple messages. Conversely, a block or message may
+be spread across multiple chunks of received bytes.
+
+The `--raw` mode is the only mode in which block headers are included in the
+output. They are marked with angle brackets '⟨' and '⟩'. For example:
+
+```plain
+┌ #20 DOWNSTREAM 42 bytes
+│⟨51 00⟩5e 6d  61 70 69 3a   6d 65 72 6f  76 69 6e 67     Q░^mapi:meroving
+│ 69 61 6e 3a  2f 2f 70 72   6f 78 79 3f  64 61 74 61     ian://proxy?data
+│ 62 61 73 65  3d 64 65 6d   6f 0a __ __  __ __ __ __     base=demo↵
+└
+```
+
+Special characters and color escapes
+------------------------------------
+
+Mapiproxy output contains some non-ASCII Unicode characters, which currently
+cannot be disabled. While this is typically not an issue on Linux and MacOS,
+users on Windows platforms may encounter issues. Please share you experiences.
+
+When writing to a terminal or when explicitly enabled with `--color=always`,
+Mapiproxy uses VT-100/ANSI color escape sequences for enhanced readability,
+especially of the hex dumps. This behavior can be disabled by passing the flag
+`--color=never`.

src/usage.txt

@@ -2,18 +2,18 @@ Usage: mapiproxy [OPTIONS] LISTEN_ADDR FORWARD_ADDR
        mapiproxy [OPTIONS] --pcap PCAP_FILE
 
 LISTEN_ADDR and FORWARD_ADDR:
-    port, for example, 50000
-    host:port, for example, localhost:50000 or 127.0.0.1:50000
+    PORT, for example, 50000
+    HOST:PORT, for example, localhost:50000 or 127.0.0.1:50000
     /path/to/unixsock, for example, /tmp/.s.monetdb.50000
 
 Options:
-    -m --messages       Dump whole messages
-    -b --blocks         Dump individual blocks
-    -r --raw            Dump bytes as they come in
-    -B --binary         Force dumping as binary
-    --color=WHEN        Colorize output, one of 'always', 'auto' or 'never'
-    --help              This help
-    --version           Version info
+    -m, --messages       Dump whole messages
+    -b, --blocks         Dump individual blocks
+    -r, --raw            Dump bytes as they come in
+    -B, --binary         Force dumping as binary
+    --color=WHEN         Colorize output (Options: 'always', 'auto', 'never')
+    --help               Display this help message
+    --version            Show version information
 
 Experimental options:
-    --pcap=FILE         Read network capture data from FILE, use - for stdin
+    --pcap=FILE          Read network capture data from FILE (use '-' for stdin)