docs: Ported docs over from old sentry docs (#189)

Author: mitsuhiko

docs/options.md

@@ -0,0 +1,252 @@
+# Configuration Options
+
+The base configuration for Relay lives in the file `.semaphore/config.yml`.  All keys are `snake_case`.
+
+## Relay
+
+General relay settings.
+
+`relay.mode`
+
+: *string, default: `managed`*
+
+  Controls the basic communication and configuration mode for this relay.
+  Allowed values are:
+
+  - `managed` *(default)*: Project configurations are managed by Sentry, unless
+    they are statically overridden via the file system. This requires
+    credentials to be set up and white listed in Sentry.
+  - `static`: Projects must be statically configured on the file system. If
+    configured, PII stripping is also performed on those events. Events for
+    unknown projects are automatically rejected.
+  - `proxy`: Relay acts as a proxy for all requests and events. It will not load
+    project configurations from the upstream or perform PII stripping. All
+    events are accepted unless overridden on the file system.
+
+  For more information on providing or overriding project configurations on the
+  file system, please refer to [Project Configuration](project-config.md) and
+  [PII Configuration](pii-config/index.md).
+
+`relay.upstream`
+
+: *string, default: `https://ingest.sentry.io`*
+
+  The upstream relay or sentry instance.
+
+  **Important**: Relay does not check for cycles. Ensure this option is not set
+  to an endpoint that will cause events to be cycled back here.
+
+`relay.host`
+
+: *string, default: `127.0.0.1`*
+
+  The host the relay should bind to (network interface).  Example: `0.0.0.0`
+
+`relay.port`
+
+: *integer, default: `3000`*
+
+  The port to bind for the unencrypted relay HTTP server.  Example: `3000`
+
+`relay.tls_port`
+
+: *integer, optional*
+
+  Optional port to bind for the encrypted relay HTTPS server.  Example: `3001`
+  
+  This is in addition to the `port` option: If you set up a HTTPS server at
+  `tls_port`, the HTTP server at `port` still exists.
+
+`relay.tls_identity_path`
+
+: *string, optional*
+
+  The filesystem path to the identity (DER-encoded PKCS12) to use for the HTTPS
+  server.  Example: `relay_dev.pfx`
+
+`relay.tls_identity_password`
+
+: *string, optional*
+
+  Password for the PKCS12 archive in `tls_identity_path`.
+
+## HTTP
+
+Set various network-related settings.
+
+`http.timeout`
+
+: *integer, default: `5`*
+
+  Timeout for upstream requests in seconds.
+
+`http.max_retry_interval`
+
+: *integer, default: `60`*
+
+  Maximum interval between failed request retries in seconds.
+
+## Caching
+
+Fine-tune caching of project state.
+
+`cache.project_expiry`
+
+: *integer, default: `300` (5 minutes)*
+
+  The cache timeout for project configurations in seconds.  Irrelevant if you
+  use the "simple proxy mode", where your project config is stored in a local
+  file.
+
+`cache.relay_expiry`
+
+: *integer, default: `3600` (1 hour)*
+
+  The cache timeout for downstream relay info (public keys) in seconds.
+
+`cache.event_expiry`
+
+: *integer, default: `600` (10 minutes)*
+
+  The cache timeout for events (store) before dropping them.
+
+`cache.miss_expiry`
+
+: *integer, default: `60` (1 minute)*
+
+  The cache timeout for non-existing entries.
+
+`cache.batch_interval`
+
+: *integer, default: `100` (100 milliseconds)*
+
+  The buffer timeout for batched queries before sending them upstream **in
+  milliseconds**.
+
+`cache.file_interval`
+
+: *integer, default: `10` (10 seconds)*
+
+  Interval for watching local cache override files in seconds.
+
+`cache.event_buffer_size`
+
+: *integer, default: `1000`*
+
+  The maximum number of events that are buffered in case of network issues or
+  high rates of incoming events.
+
+## Size Limits
+
+Controls various HTTP-related limits.  All values are human-readable strings of a number and a human-readable unit, such as:
+
+- `1KiB`
+- `1MB`
+- `1MiB`
+- `1MiB`
+- `1025B`
+
+`limits.max_concurrent_requests`
+
+: *integer, default: `100`*
+
+  The maximum number of concurrent connections to the upstream.
+
+`limits.max_event_payload_size`
+
+: *string, default: `256KB`*
+
+  The maximum payload size for events.
+
+`limits.max_api_payload_size`
+
+: *string, default: `20MB`*
+
+  The maximum payload size for general API requests.
+
+`limits.max_api_file_upload_size`
+
+: *string, default: `40MB`*
+
+  The maximum payload size for file uploads and chunks.
+
+`limits.max_api_chunk_upload_size`
+
+: *string, default: `100MB`*
+
+  The maximum payload size for chunks
+
+## Logging
+
+`logging.level`
+
+: *string, default: `info`*
+
+  The log level for the relay. One of:
+  
+  - `off`
+  - `error`
+  - `warn`
+  - `info`
+  - `debug`
+  - `trace`
+
+`logging.log_failed_payloads`
+
+: *boolean, default: `false`*
+
+  If set to true this emits log messages for failed event payloads.
+
+`logging.format`
+
+: *string, default: `auto`*
+
+  Controls the log format. One of:
+  
+  - `auto`: Auto detect (pretty for tty, simplified for other)
+  - `pretty`: With colors
+  - `simplified`: Simplified log output
+  - `json`: Dump out JSON lines
+
+`logging.enable_backtraces`
+
+: *boolean, default: `true`*
+
+  When set to true, backtraces are forced on.
+
+## Statsd Metrics
+
+`metrics.statsd`
+
+: *string, optional*
+
+  If set to a host/port string then metrics will be reported to this statsd
+  instance.
+
+`metrics.prefix`
+
+: *string, default: `sentry.relay`*
+
+  The prefix that should be added to all metrics.
+
+## Internal Error Reporting
+
+Configures error reporting for errors happening within Sentry. Disabled by
+default.
+
+`sentry.enabled`
+
+: *boolean, default: `false`*
+
+  Whether to report internal errors to a separate DSN. `false` means no internal
+  errors are sent (just logged).
+
+`sentry.dsn`
+
+: *string, optional*
+
+  DSN to report internal relay failures to.
+
+  It is not a good idea to set this to a value that will make the relay send
+  errors to itself. Ideally this should just send errors to Sentry directly, not
+  another relay.

docs/pii-config/index.md

@@ -0,0 +1,38 @@
+# PII Configuration
+
+The following document explores the syntax and semantics of PII configs for Relay. To get started with PII configs, it's recommended to use [_Piinguin_](https://getsentry.github.io/piinguin) and refer back to this document when needed.
+
+## A basic example
+
+Say you have an exception message which, unfortunately, contains IP addresses which are not supposed to be there. You'd write:
+
+```json
+{
+  "applications": {
+    "text": ["@ip:replace"]
+  }
+}
+```
+
+It reads as "apply rule `@ip:replace` to all `text` fields.
+
+## Rules
+
+The following rules exist by default:
+
+- `@ip:replace` and `@ip:hash` for pattern-matching IP addresses
+- `@imei:replace` and `@imei:hash` for pattern-matching IMEIs
+- `@mac:replace`, `@mac:mask` and `@mac:hash` for pattern-matching MAC addresses
+- `@email:mask`, `@email:replace` and `@email:hash` for pattern-matching email addresses
+- `@creditcard:mask`, `@creditcard:replace` and `@creditcard:hash` for pattern-matching creditcard numbers
+- `@userpath:replace` and `@userpath:hash` for pattern-matching local paths (e.g. `C:/Users/foo/`)
+- `@password:remove` for pattern matching passwords. In this case we're pattern matching against the field's key, whether it contains `password`, `credentials` or similar strings.
+
+### Writing your own rules
+
+Rules generally consist of two parts:
+
+- *Rule types* describe what to match. See [_PII Rule Types_](types.md) for an exhaustive list.
+- *Rule redaction methods* describe what to do with the match. See [_PII Rule Redaction Methods_](methods.md) for a list.  
+
+Each page comes with examples. Try those examples out by pasting them into the "PII config" column of [_Piinguin_](https://getsentry.github.io/piinguin) and clicking on fields to get suggestions.

docs/pii-config/methods.md

@@ -0,0 +1,90 @@
+# PII Rule Redaction Methods
+
+#### remove
+
+Remove the entire field. Relay may choose to either set it to `null` or to remove it entirely.
+  
+```json
+{
+  "rules": {
+    "remove_ip": {
+      "type": "ip",
+      "redaction": {
+        "method": "remove"
+      }
+    }
+  },
+  "applications": {
+    "text": ["remove_ip"]
+  }
+}
+```
+
+#### replace
+
+Replace the key with a static string.
+
+```json
+{
+  "rules": {
+    "replace_ip": {
+      "type": "ip",
+      "redaction": {
+        "method": "replace",
+        "text": "[censored]"
+      }
+    }
+  },
+  "applications": {
+    "text": ["replace_ip"]
+  }
+}
+```
+
+#### mask
+
+Replace every character of the matched string with a "masking" char. Compared to `replace` this preserves the length of the original string.
+
+```javascript
+{
+  "rules": {
+    "mask_ip": {
+      "type": "ip",
+      "redaction": {
+        "method": "mask",
+        "mask_char": "0",        // The character used for masking. Optional, default "*"
+        "chars_to_ignore": ".",  // Which characters to ignore. Optional, default empty
+        "range": [0, -1]         // Which range of the string to replace. Optional, defaults to full range. Negative indices count from the matches' end.
+      }
+    }
+  },
+  "applications": {
+    "text": ["mask_ip"]
+  }
+}
+```
+
+#### hash
+
+Replace the string with a hashed version of itself. Equal strings will produce the same hash, so if you, for example, decide to hash the user ID instead of replacing or removing it, you will still have an accurate count of users affected.
+
+```javascript
+{
+  "rules": {
+    "hash_ip": {
+      "type": "ip",
+      "redaction": {
+        "method": "hash",
+        "algorithm": "HMAC-SHA1", // One of "HMAC-SHA1", "HMAC-SHA256", "HMAC-SHA512"
+        "key": "myOverriddenKey"  // A key to salt the hash with. Defaults to the default key set in "vars"
+      }
+    }
+  },
+  "vars": {
+    "hashKey": "myDefaultKey"    // The default key to use
+  }
+  "applications": {
+    "text": ["mask_ip"]
+  }
+}
+```