Skip to content
This repository has been archived by the owner on Jul 23, 2023. It is now read-only.

Latest commit

 

History

History
147 lines (109 loc) · 5.01 KB

README.md

File metadata and controls

147 lines (109 loc) · 5.01 KB

railway-chord

Vector log egress for Railway.app projects.

railway-chord pipes the log stream from Railway's GraphQL API into Vector. This gives you project-wide centralized logging on Railway. For each Railway project you configure, railway-chord dumps the log stream of the project's deployed services and plugins (e.g. Postgres, Redis, etc.) in all environments into Vector.

Usage

⚠️ railway-chord depends on Railway's API. You must be a part of the Priority Boarding program to use Railway's API. You can join the program easily by following the instructions in the link.

Deploy on Railway

The button above will deploy railway-chord on Railway. You will be prompted to set some required environment variables; there is no additional configuration required beyond this. Log sinks are automatically configured based on the presence of required environment variables of each provider, i.e. setting a DATADOG_TOKEN will enable Datadog, and so on.

Datadog

To enable Datadog logs, set the DATADOG_TOKEN service variable.

There is an optional DATADOG_SITE setting if your Datadog account is hosted on a different Datadog instance (defaults to datadoghq.com).

Logtail

To enable Logtail, set the LOGTAIL_TOKEN service variable.

stdout

To enable logging to stdout, set ENABLE_STDOUT=true. This will output all enabled project's logs into railway-chord's stdout.

Project IDs

The RAILWAY_PROJECT_IDS variable requires a comma-separated list of Railway Project IDs to enable railway-chord for. You can find your Project ID under Railway's Dashboard -> Project -> Settings -> General.

Example:

# Separate each Project ID with a comma!
RAILWAY_PROJECT_IDS="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX,XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"

Log Enrichment

railway-chord will inject a railway object into the logs sent to the provider containing the deployment/plugin ID and name.

Example:

{
  "railway": {
    "id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
    "name": "app.up.railway.app",
    "type": "DEPLOYMENT"
  }
}

Railway API Rate Limit

Railway's Public API currently has a rate limit of 1,000 requests daily. railway-chord will make ~100 requests every 24 hours for each project enabled. Work is underway to reduce the amount of requests railway-chord is making (see railway-chord#3).

Be careful about this if you're using Railway's API for something else - railway-chord will eat into your rate limit!

Adding Vector sinks

Request for new sinks

To request for a new Vector sink, please open a GitHub issue.

Add your own sink

See this pull request for an example.

Before adding a new Vector sink, check the authentication mechanism of the provider you're using. There is usually an API key required.

  1. In src/main.ts, pass the required API key/token into configureVector():

    const PROVIDER_TOKEN = process.env.PROVIDER_TOKEN ?? null
    const vectorCfg = configureVector(
      ...,
      PROVIDER_TOKEN
    )
  2. Create a new Vector sink configuration in src/vector/sinks.ts (TOML format).

    const PROVIDER = (token: string) => `
    [sinks.PROVIDER]
    ...
    token = "${token}"
    `
  3. In src/vector/configure.ts, import and append the newly-created config created above, passing the required API key into it:

    import { PROVIDER } from './sinks'
    const configure = ( ..., providerToken: string | null ) => {
        ...
        if (providerToken !== null) {
            enabled.push('provider')
            cfg += PROVIDER(providerToken)
        }
        ...
    }

Configuration

Name Value
RAILWAY_API_TOKEN Required. Railway API token.
RAILWAY_PROJECT_IDS Required. A comma-separated list of Railway Project IDs.
LOGTAIL_TOKEN Optional. Logtail token.
DATADOG_TOKEN Optional. Datadog API token.
DATADOG_SITE Optional. Datadog site setting. Defaults to datadoghq.com.
ENABLE_STDOUT Optional. Enable Vector logging to stdout.
RAILWAY_API_HTTP_ENDPOINT Optional. Railway's HTTP GQL Endpoint. Defaults to https://backboard.railway.app/graphql/v2.
RAILWAY_WS_HTTP_ENDPOINT Optional. Railway's WebSockets GQL Endpoint. Defaults to wss://backboard.railway.app/graphql/v2.
VECTOR_BIN_PATH Optional. Path to Vector binary. Defaults to path defined in Docker build. You do not need to set this.

License

MIT