diff --git a/CHANGELOG.md b/CHANGELOG.md index 980cfa7..de6c21f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,10 @@ What changed in mapiproxy, per version - No longer default to --messages. It's not clear what the default should be so for the time being it's better to not have a default at all. +- Add experimental `--pcap=FILE` option to read captured network traffic files + written by for example, [tcpdump](https://www.tcpdump.org/). + So far tqhis has seen very limited testing. + ## mapiproxy 0.5.2 - 2024-02-21 diff --git a/README.md b/README.md index 0672e3f..a6c5846 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,134 @@ MAPI PROXY ========== -Proxy to inspect [MonetDB] network traffic. - -[MonetDB]: https://www.monetdb.org/ +Mapiproxy is a tool for inspecting [MonetDB](https://www.monetdb.org/) +network traffic by acting as a proxy between clients and servers. The MonetDB network protocol is called MAPI, hence the name. -The proxy can listen on and connect to TCP sockets, Unix Domain sockets or both. -The MAPI protocol is slightly different between the two, the proxy will -automatically adjust it when proxying between the two. +The proxy can work with TCP sockets, Unix Domain sockets or both at the same +time. The MAPI protocol is slightly different between TCP and Unix sockets, the +proxy will automatically adjust it when proxying between the two. + +Additionally, Mapiproxy offers experimental support for reading PCAP files: +captured network traffic files, often generated by tools like +[tcpdump](https://www.tcpdump.org/) and [Wireshark](https://www.wireshark.org/). +However, this feature is still in its early stages and may not be fully +functional in practice. + +## Usage -Usage: +The following is a summary of Mapiproxy's usage: ```plain Usage: mapiproxy [OPTIONS] LISTEN_ADDR FORWARD_ADDR mapiproxy [OPTIONS] --pcap PCAP_FILE LISTEN_ADDR and FORWARD_ADDR: - port, for example, 50000 - host:port, for example, localhost:50000 or 127.0.0.1:50000 + PORT, for example, 50000 + HOST:PORT, for example, localhost:50000 or 127.0.0.1:50000 /path/to/unixsock, for example, /tmp/.s.monetdb.50000 Options: - -m --messages Dump whole messages - -b --blocks Dump individual blocks - -r --raw Dump bytes as they come in - -B --binary Force dumping as binary - --color=WHEN Colorize output, one of 'always', 'auto' or 'never' - --help This help - --version Version info + -m, --messages Dump whole messages + -b, --blocks Dump individual blocks + -r, --raw Dump bytes as they come in + -B, --binary Force dumping as binary + --color=WHEN Colorize output (Options: 'always', 'auto', 'never') + --help Display this help message + --version Show version information Experimental options: - --pcap=FILE Read network capture data from FILE, use - for stdin + --pcap=FILE Read network capture data from FILE (use '-' for stdin) ``` + +## Installation + +Source code is available on [GitHub](https://github.com/MonetDBSolutions/mapiproxy). + +Pre-built binaries for x86-64 Linux, MacOS, and Windows are available on the +[Releases page](https://github.com/MonetDBSolutions/mapiproxy/releases/) of the +GitHub repository. + +Mapiproxy is written in Rust. If you want to build from source and don't have +the Rust compiler installed, you can obtain it from +[rustup.rs](https://rustup.rs/). + +To build from source, simply execute `cargo install mapiproxy` or clone the +repository from GitHub and build using `cargo build --release`. + + +Output Modes +------------ + +Mapiproxy offers three output modes, corresponding to the layers of the MAPI +protocol. In a nutshell, MAPI communication consists of a series of messages. +Each message is sent as of one or more blocks. Each block is preceded by a +two-byte block header. + +Mapiproxy output consists of a mix of single-line informational messages and +multi-line frames that display the captured data. For example, in the +single-line message + +```plain +‣ #10 UPSTREAM client stopped sending +``` +we see a connection id, a direction marker (UPSTREAM for client to server and +DOWNSTREAM for server to client), and an informational message. + +Frames display the data either as a hex dump, like this, +```plain +┌ #10 UPSTREAM binary, message, 13 bytes +│ 73 73 65 6c 65 63 74 20 34 32 0a 3b 0a __ __ __ sselect·42↵;↵ +└ +``` +or they display it as text, like this: +```plain +┌ #10 UPSTREAM text, message, 13 bytes +│sselect 42↵ +│;↵ +└ +``` +Mapi generally prefers text output over binary, but will render a frame in +binary when at least one of the following conditions holds true: + +* the data is not valid UTF-8 encoded text + +* the data contains control characters other than newline and tab + +* Mapiproxy is running in `--raw` mode + +* binary output is forced using the `-B` or `--binary` flag + +When Mapiproxy is in `--messages` mode, it collects all bytes of whole message +and prints them at in a single frame, skipping the block headers. In `--blocks` +mode, it collects whole blocks and prints one block per frame, also without the +block header. + +In `--raw` mode, bytes are printed as they are received, including the block +headers. This means that a single printed chunk may contain parts of multiple +blocks and even parts of multiple messages. Conversely, a block or message may +be spread across multiple chunks of received bytes. + +The `--raw` mode is the only mode in which block headers are included in the +output. They are marked with angle brackets '⟨' and '⟩'. For example: + +```plain +┌ #20 DOWNSTREAM 42 bytes +│⟨51 00⟩5e 6d 61 70 69 3a 6d 65 72 6f 76 69 6e 67 Q░^mapi:meroving +│ 69 61 6e 3a 2f 2f 70 72 6f 78 79 3f 64 61 74 61 ian://proxy?data +│ 62 61 73 65 3d 64 65 6d 6f 0a __ __ __ __ __ __ base=demo↵ +└ +``` + +Special characters and color escapes +------------------------------------ + +Mapiproxy output contains some non-ASCII Unicode characters, which currently +cannot be disabled. While this is typically not an issue on Linux and MacOS, +users on Windows platforms may encounter issues. Please share you experiences. + +When writing to a terminal or when explicitly enabled with `--color=always`, +Mapiproxy uses VT-100/ANSI color escape sequences for enhanced readability, +especially of the hex dumps. This behavior can be disabled by passing the flag +`--color=never`. \ No newline at end of file diff --git a/src/usage.txt b/src/usage.txt index 153d32f..c2b4bbe 100644 --- a/src/usage.txt +++ b/src/usage.txt @@ -2,18 +2,18 @@ Usage: mapiproxy [OPTIONS] LISTEN_ADDR FORWARD_ADDR mapiproxy [OPTIONS] --pcap PCAP_FILE LISTEN_ADDR and FORWARD_ADDR: - port, for example, 50000 - host:port, for example, localhost:50000 or 127.0.0.1:50000 + PORT, for example, 50000 + HOST:PORT, for example, localhost:50000 or 127.0.0.1:50000 /path/to/unixsock, for example, /tmp/.s.monetdb.50000 Options: - -m --messages Dump whole messages - -b --blocks Dump individual blocks - -r --raw Dump bytes as they come in - -B --binary Force dumping as binary - --color=WHEN Colorize output, one of 'always', 'auto' or 'never' - --help This help - --version Version info + -m, --messages Dump whole messages + -b, --blocks Dump individual blocks + -r, --raw Dump bytes as they come in + -B, --binary Force dumping as binary + --color=WHEN Colorize output (Options: 'always', 'auto', 'never') + --help Display this help message + --version Show version information Experimental options: - --pcap=FILE Read network capture data from FILE, use - for stdin + --pcap=FILE Read network capture data from FILE (use '-' for stdin)