Skip to content

Latest commit

 

History

History
652 lines (498 loc) · 35.3 KB

README.md

File metadata and controls

652 lines (498 loc) · 35.3 KB
Raw

BX-bot

Gradle CI Maven CI Sonarcloud Status Contributor Covenant Join the chat at https://gitter.im/BX-bot/Lobby

What is BX-bot?

BX-bot (Bex) is a simple Bitcoin trading bot written in Java for trading on cryptocurrency exchanges.

The project contains the basic infrastructure to trade on a cryptocurrency exchange... except for the trading strategies - you'll need to write those yourself! A simple ExampleScalpingStrategy is included to get you started with the Trading API - see Ta4j for more ideas.

Exchange Adapters for using Bitstamp, Bitfinex, Kraken, and Gemini are included. Feel free to improve these or contribute new adapters to the project; that would be shiny!

A TryModeExchangeAdapter is configured by default to delegate public API calls to Bitstamp, but it simulates the private API (order management) calls; it's good for testing your initial setup and paper trading without actually sending orders to the exchange.

The Trading API provides support for limit orders traded at the spot price. If you're looking for something more sophisticated with a much richer Trading API, take a look at XChange.

Warning: Trading cryptocurrency carries significant financial risk; you could lose money. This software is provided 'as is' and released under the MIT license.

Architecture

bxbot-core-architecture.png

  • Trading Engine - the execution unit. It provides a framework for integrating Exchange Adapters and executing Trading Strategies.
  • Exchange Adapters - the data stream unit. They provide access to a given exchange.
  • Trading Strategies - the decision or strategy unit. This is where the trading decisions happen.
  • Trading API - Trading Strategies use this API to make trades. Exchange Adapters implement this to provide access to a given exchange.
  • Strategy API - Trading Strategies implement this so the Trading Engine can execute them.

Trading Strategies and Exchange Adapters are injected by the Trading Engine on startup. The bot uses a simple YAML backed dependency injection framework to achieve this; the long term goal is to convert it into a fully configurable Spring Boot app.

The bot was designed to fail hard and fast if any unexpected errors occur in the Exchange Adapters or Trading Strategies: it will log the error, send an email alert (if configured), and then shut down.

Installation Guide

The bot runs on Linux, macOS, and Windows.

BX-bot is supported on the current JDK 21 LTS. You'll need the JDK (e.g. OpenJDK 21 or Oracle JDK 21) installed on the machine you are going to use to build and run the bot. Be mindful of Oracle's recent licensing changes and how you intend to use the bot.

You can use Gradle or Maven to build the bot. The instructions below are for Linux/macOS, but equivalent Windows scripts are included.

Download the latest Release and unzip the bot.

Maven

  1. If you plan on using your own Trading Strategies/Exchange Adapters packaged in separate jar files, you'll need to add the dependency in the bxbot-app/pom.xml - see the commented out dependency examples inside it.
  2. From the project root, run ./mvnw clean package to produce the distribution artifacts bxbot-app-<version>-dist.tar.gz and bxbot-app-<version>-dist.zip in the ./bxbot-app/target folder.
  3. Copy either the bxbot-app-<version>-dist.tar.gz or the bxbot-app-<version>-dist.zip onto the machine you want to run the bot and unzip it someplace.
  4. Configure the bot as required - see the main Configuration section. The bot's default configuration uses the ExampleScalpingStrategy, but you'll probably want to code your own! The TryModeExchangeAdapter is configured out of the box to simulate trading with Bitstamp.
  5. Usage: ./bxbot.sh [start|stop|status]

Gradle

  1. If you plan on using your own Trading Strategies/Exchange Adapters packaged in separate jar files, you'll need to add the dependency in the bxbot-app/build.gradle - see the commented out dependency examples inside it.
  2. From the project root, run ./gradlew clean build to build the bot.
  3. Then run ./gradlew buildTarGzipDist or ./gradlew buildZipDist to build the distribution artifact: either bxbot-app-<version>.tar.gz or bxbot-app-<version>.zip respectively. It will be placed in the ./build/distributions folder.
  4. Copy the artifact onto the machine you want to run the bot and unzip it someplace.
  5. Configure the bot as described in step 4 of the previous Maven section.
  6. Usage: ./bxbot.sh [start|stop|status]

Docker

If you want to just play around with the ExampleScalpingStrategy and evaluate the bot, Docker is the way to go.

  1. Install Docker on the machine you want to run the bot.
  2. Fetch the BX-bot image from Docker Hub: docker pull gazbert/bxbot:2.2.1
  3. Run the Docker container: docker container run --publish=8080:8080 --name bxbot-2.2.1 -it gazbert/bxbot:2.2.1 bash
  4. Change into the bot's directory: cd bxbot*
  5. Configure the bot as described in step 4 of the previous Maven section.
  6. Usage: ./bxbot.sh [start|stop|status]
  7. You can detach from the container and leave the bot running using the CTRL-p CTRL-q key sequence.
  8. To re-attach to the Docker container, run docker container ls to get the CONTAINER ID. Then run: docker container attach <CONTAINER ID>

You could modify the Dockerfile to containerise the bot and deploy to Kubernetes. The config can be externalised using a Docker volume mount.

Build Guide

If you plan on developing the bot, you'll need JDK 21 installed on your dev box.

You can use Gradle or Maven to build the bot and pull down the dependencies.

The instructions below are for Linux/macOS, but equivalent Windows scripts are included.

Clone the repo locally (main branch).

Maven

  1. From the project root, run ./mvnw clean install. If you want to run the exchange integration tests, use ./mvnw clean install -Pint. To execute both unit and integration tests, use ./mvnw clean install -Pall.
  2. Take a look at the Javadoc in the ./target/apidocs folders of the bxbot-trading-api, bxbot-strategy-api, and bxbot-exchange-api modules after the build completes.

Gradle

  1. From the project root, run ./gradlew build. If you want to run the exchange integration tests, use ./gradlew integrationTests. To execute both unit and integration tests, use ./gradlew build integrationTests.
  2. To generate the Javadoc, run ./gradlew javadoc and look in the ./build/docs/javadoc folders of the bxbot-trading-api, bxbot-strategy-api, and bxbot-exchange-api modules.

Issue & Change Management

Issues and new features are managed using the project Issue Tracker - submit bugs here.

You are welcome to take on new features or fix bugs! See here for how to get involved.

For help and general questions about BX-bot, check out the Gitter channel.

Testing

The bot has undergone basic unit testing on a best-effort basis.

There is a CI build running on GitHub Actions.

The latest stable build can always be found on the Releases page. The SNAPSHOT builds on main are active development builds, but the tests should always pass and the bot should always be deployable.

User Guide

"Change your opinions, keep to your principles; change your leaves, keep intact your roots." - Victor Hugo

Configuration

The bot provides a simple plugin framework for:

  • Exchanges to integrate with.
  • Markets to trade on.
  • Trading Strategies to execute.

It uses YAML configuration files. These live in the config folder. Any config changes require a restart of the bot to take effect.

Sample configurations for running on different exchanges can be found in the config/samplesfolder.

Engine

The engine.yaml file is used to configure the Trading Engine.

engine:
  botId: my-bitstamp-bot_1
  botName: Bitstamp Bot
  emergencyStopCurrency: BTC
  emergencyStopBalance: 1.0
  tradeCycleInterval: 30

All fields are mandatory.

  • The botId value is a unique identifier for the bot. Value must be an alphanumeric string. Underscores and dashes are also permitted.

  • The botName is a friendly name for the bot. Value must be an alphanumeric string. Spaces are allowed.

  • The emergencyStopCurrency value must be set to prevent catastrophic loss on the exchange. This is normally the currency you intend to hold a long position in. It should be set to the currency short code for the wallet, e.g. BTC, LTC, USD. This value can be case-sensitive for some exchanges - check the Exchange Adapter documentation.

  • The emergencyStopBalance value must be set to prevent catastrophic loss on the exchange. The Trading Engine checks this value at the start of every trade cycle: if your emergencyStopCurrency wallet balance on the exchange drops below this value, the Trading Engine will log it, send an Email Alert (if configured) and then shut down. If you set this value to 0, the bot will bypass the check - be careful.

  • The tradeCycleInterval value is the interval in seconds that the Trading Engine will wait/sleep before executing each trade cycle. The minimum value is 1 second. Some exchanges allow you to hit them harder than others. However, while their API documentation might say one thing, the reality is you might get socket timeouts and 5xx responses if you hit it too hard. You'll need to experiment with the trade cycle interval for different exchanges.

Exchange Adapters

You specify the Exchange Adapter you want BX-bot to use in the exchange.yaml file.

BX-bot supports 1 exchange per bot. This keeps things simple and helps minimise risk: problems on one exchange should not impact trading on another.

The TryModeExchangeAdapter is configured by default to delegate public API calls to the BitstampExchangeAdapter. It simulates the private API (order management) calls; it's good for testing your initial setup and paper trading, but you'll eventually want to send live orders to the exchange!

The configuration below shows how to live-trade with Bitstamp:

exchange:
  name: Bitstamp
  adapter: com.gazbert.bxbot.exchanges.BitstampExchangeAdapter
  
  authenticationConfig:
    clientId: your-client-id
    key: your-api-key
    secret: your-secret-key
           
  networkConfig:
    connectionTimeout: 15
    nonFatalErrorCodes: [502, 503, 520, 522, 525]            
    nonFatalErrorMessages:
      - Connection reset
      - Connection refused
      - Remote host closed connection during handshake
      - Unexpected end of file from server
      
  otherConfig:
    not-needed-on-bitstamp-1: here for illustration purposes only
    not-needed-on-bitstamp-2: here for illustration purposes again

All fields are mandatory unless stated otherwise.

  • The name value is a friendly name for the Exchange. It is used in log statements to display the Exchange's name. Value must be an alphanumeric string. Spaces are allowed.

  • For the adapter value, you must specify the fully qualified name of the Exchange Adapter class for the Trading Engine to inject on startup. The class must be on the runtime classpath. See the How do I write my own Exchange Adapter? section for more details.

  • The authenticationConfig section is used by the inbuilt Exchange Adapters to configure their exchange trading API credentials - see the sample exchange.yaml config files for details.

  • The networkConfig section is optional. It is used by the inbuilt Exchange Adapters to set their network configuration as detailed below:

    • The connectionTimeout field is optional. This is the timeout value that the exchange adapter will wait on socket connect/socket read when communicating with the exchange. Once this threshold has been breached, the exchange adapter will give up and throw an ExchangeNetworkException. The sample Exchange Adapters are single threaded: if a request gets blocked, it will block all subsequent requests from getting to the exchange. This timeout value prevents an indefinite block. If not set, it defaults to 30 seconds.

    • The nonFatalErrorCodes field is optional. It contains a list of HTTP status codes that will trigger the adapter to throw a non-fatal ExchangeNetworkException. This allows the bot to recover from temporary network issues. See the sample exchange.yaml config files for status codes to use.

    • The nonFatalErrorMessages field is optional. It contains a list of java.io Exception message content that will trigger the adapter to throw a non-fatal ExchangeNetworkException. This allows the bot to recover from temporary network issues. See the sample exchange.yaml config files for messages to use.

  • The otherConfig section is optional. It is not needed for Bitstamp, but shown above for illustration purposes. If present, at least 1 item must be set - these are repeating key/value String pairs. This section is used by the inbuilt Exchange Adapters to set any additional config, e.g. buy/sell fees.

Markets

You specify which markets you want to trade on in the markets.yaml file.

  markets:            
    - id: btcusd    
      name: BTC/USD        
      baseCurrency: BTC
      counterCurrency: USD
      enabled: true
      tradingStrategyId: scalping-strategy
  
    - id: ltcusd
      name: LTC/BTC
      baseCurrency: LTC[CONTRIBUTING.md](CONTRIBUTING.md)
      counterCurrency: BTC
      enabled: false
      tradingStrategyId: scalping-strategy

All fields are mandatory unless stated otherwise.

  • The id value is the market id as defined on the exchange. E.g. the BTC/USD market id is btcusd on Bitstamp - see currency_pair values.

  • The name value is a friendly name for the market. The is used in the logs to display the market's name. Value must be an alphanumeric string.

  • The baseCurrency value is the currency short code for the base currency in the currency pair. When you buy or sell a currency pair, you are performing that action on the base currency. The base currency is the commodity you are buying or selling. E.g. in a BTC/USD market, the first currency (BTC) is the base currency and the second currency (USD) is the counter currency.

  • The counterCurrency value is the currency short code for the counter currency in the currency pair. This is also known as the quote currency.

  • The enabled value allows you to toggle trading on the market. Remember, config changes are only applied on startup.

  • The tradingStrategyId value must match a strategy id defined in your strategies.yaml config. Currently, BX-bot only supports 1 strategy per market.

Strategies

You specify the Trading Strategies you wish to use in the strategies.yaml file.

strategies:
  - id: scalping-strategy
    name: Basic Scalping Strat
    description: >
      A simple scalper that buys at the current BID price, holds until current market price has 
      reached a configurable minimum percentage gain, then sells at current ASK price, thereby
      taking profit from the spread.       
    # This strategy is injected using the bot's custom injection framework using its className
    className: com.gazbert.bxbot.strategies.ExampleScalpingStrategy
    configItems:
      counter-currency-buy-order-amount: 20                        
      minimum-percentage-gain: 2
            
  - id: macd-strategy
    name: MACD Based Strat
    description: Strat uses MACD data to take long position in USD.    
    # This strategy is injected using a Spring beanName
    beanName: yourMacdStrategyBean
    configItems:
      counter-currency-buy-order-amount: 20      
      shortEmaInterval: 12            
      longEmaInterval: 26

All fields are mandatory unless stated otherwise.

  • The id value is a unique identifier for the strategy. The markets.yaml tradingStrategyId entries cross-reference this. Value must be an alphanumeric string. Underscores and dashes are also permitted.

  • The name value is a friendly name for the strategy. The is used in the logs to display the strategy's name. Value must be an alphanumeric string. Spaces are allowed.

  • The description value is optional.

You configure the loading of your strategy using either a className or a beanName; you cannot specify both.

  • For the className value, you must specify the fully qualified name of your Strategy class for the Trading Engine to load and execute. This will use the bot's custom injection framework. The class must be on the runtime classpath. If you set this value to load your strategy, you cannot set the beanName value.

  • For the beanName value, you must specify the Spring bean name of you Strategy component class for the Trading Engine to load and execute. You will also need to annotate your strategy class with @Component("yourMacdStrategyBean") - see the ExampleScalpingStrategy. This results in Spring injecting the bean. If you set this value to load your strategy, you cannot set the className value.

  • The configItems section is optional. It allows you to set key/value pair config items. This config is passed to your Trading Strategy when the bot starts up; see the How do I write my own Trading Strategy? section.

Email Alerts

You specify the Email Alerts config in the email-alerts.yaml file.

This config is used to send email alerts when the bot is forced to shut down due to an unexpected error occurring in the Trading Strategies or Exchange Adapters. The email is sent to the SMTP host using TLS.

emailAlerts:
  enabled: false
  smtpConfig:
    host: smtp.gmail.com
    tlsPort: 587
    accountUsername: [email protected]
    accountPassword: your.account.password
    fromAddress: [email protected]
    toAddress: [email protected]

All fields are mandatory unless stated otherwise.

  • If enabled is set to true, the bot will send email alerts to the toAddress if it needs to shut down due to a critical error.

  • The smtpConfig config is optional and only required if enabled is set to true. Sample SMTP config for using a Gmail account is shown above - all elements within smtpConfig are mandatory.

How do I write my own Trading Strategy?

"I was seldom able to see an opportunity until it had ceased to be one." - Mark Twain

The best place to start is with the ExampleScalpingStrategy - more ideas can be found in the excellent Ta4j project. There is also a Trading Strategy specific channel on Gitter.

Your strategy must implement the TradingStrategy interface. This allows the Trading Engine to:

  • Inject your strategy on startup.
  • Pass any configuration (set in the strategies.yaml) to your strategy.
  • Invoke your strategy at each trade cycle.

You load your strategy using either className or beanName in the strategies.yaml file - see the Strategies Configuration section for full details. The choice is yours, but beanName is the way to go if you want to use other Spring features in your strategy, e.g. a Repository to store your trade data.

The Trading Engine will only send 1 thread through your Trading Strategy; you do not have to code for concurrency.

Making Trades

You use the TradingApi to make trades etc. The API is passed to your Trading Strategy implementation init method when the bot starts up. See the Javadoc for full details of the API.

Error Handling

Your Trading Strategy implementation should throw a StrategyException whenever it 'breaks'. BX-bot's error handling policy is designed to fail hard and fast; it will log the error, send an Email Alert (if configured), and shut down.

Note that the inbuilt Exchange Adapters will (some more often than others!) throw an ExchangeNetworkException if they encounter network issues connecting with the exchange. Your strategy should always catch this exception and choose what to do next, e.g. retry the previous Trading API call, or 'swallow' the exception and wait until the Trading Engine invokes the strategy again at the next trade cycle.

Configuration

You specify the Trading Strategies you wish to use in the strategies.yaml file - see the Strategies Configuration section for full details.

The configItems section in the strategies.yaml allows you to set key/value pair config items to pass to your Trading Strategy implementation. On startup, the Trading Engine will pass the config to your Trading Strategy's init(TradingApi tradingApi, Market market, StrategyConfig config) method.

Dependencies

Your Trading Strategy implementation has a compile-time dependency on the Strategy API and the Trading API.

The inbuilt ExampleScalpingStrategy also has a compile-time dependency on log4j and Google Guava.

Packaging & Deployment

To get going fast, you can code your Trading Strategy and place it in the bxbot-strategies module alongside the example strategy. When you build the project, your Trading Strategy will be included in the BX-bot jar. You can also create your own jar for your strats, e.g. my-strats.jar, and include it on BX-bot's runtime classpath - see the Installation Guide for how to do this.

How do I write my own Exchange Adapter?

"Battle not with monsters, lest ye become a monster, and if you gaze into the abyss, the abyss gazes also into you." - Friedrich Nietzsche

It's not easy, and can be frustrating at times, but a good place to start is with one of the inbuilt Exchange Adapters - see the latest BitstampExchangeAdapter for example. There is also an Exchange Adapter specific channel on Gitter.

Your adapter must implement the TradingApi and the ExchangeAdapter interfaces. This allows the:

  • Trading Engine to inject your adapter on startup.
  • Trading Engine to pass any configuration (set in the exchange.yaml) to your adapter.
  • Trading Strategies to invoke your adapter's implementation of the TradingApi at each trade cycle.

AbstractExchangeAdapter is a handy base class that all the inbuilt Exchange Adapters extend - it could be useful.

The Trading Engine will only send 1 thread through your Exchange Adapter; you do not have to code for concurrency.

Error Handling

Your Exchange Adapter implementation should throw a TradingApiException whenever it breaks; the Trading Strategies should catch this and decide how they want to proceed.

The Trading API provides an ExchangeNetworkException for adapters to throw when they cannot connect to the exchange to make Trading API calls. This allows for Trading Strategies to recover from temporary network failures. The exchange.yaml config file has an optional networkConfig section, which contains nonFatalErrorCodes and nonFatalErrorMessages elements - these can be used to tell the adapter when to throw the exception.

The first release of the bot is single-threaded for simplicity. The downside to this is that if an API call to the exchange gets blocked on IO, BX-bot will get stuck until your Exchange Adapter frees the block. The Trading API provides an ExchangeNetworkException for your adapter to throw if it times-out connecting to the exchange. It is your responsibility to free up any blocked connections - see the AbstractExchangeAdapter for an example how to do this.

The Trading Engine will also call your adapter directly when performing the Emergency Stop check to see if the emergencyStopCurrency wallet balance on the exchange drops below the configured emergencyStopBalance value. If this call to the TradingApi getBalanceInfo() fails and is not due to a ExchangeNetworkException, the Trading Engine will log the error, send an Email Alert (if configured), and shut down. If the API call failed due to an ExchangeNetworkException, the Trading Engine will log the error and sleep until the next trade cycle.

Configuration

You provide your Exchange Adapter details in the exchange.yaml file - see the Exchange Adapters Configuration section for full details.

The otherConfig section in the exchange.yaml allows you to set key/value pair config items to pass to your Exchange Adapter implementation. On startup, the Trading Engine will pass the config to your Exchange Adapter's init(ExchangeConfig config) method.

Dependencies

Your Exchange Adapter implementation has a compile-time dependency on the Trading API.

The inbuilt Exchange Adapters also have compile-time dependencies on log4j, Google Gson, and Google Guava.

Packaging & Deployment

To get going fast, you can code your Exchange Adapter and place it in the bxbot-exchanges module alongside the other inbuilt adapters. When you build the project, your Exchange Adapter will be included in the BX-bot jar. You can also create your own jar for your adapters, e.g. my-adapters.jar, and include it on BX-bot's runtime classpath - see the Installation Guide for how to do this.

Logging

Logging for the bot is provided by log4j. The log file is written to logs/bxbot.log using a rolling policy. When a log file size reaches 100 MB or a new day is started, it is archived and a new log file is created. BX-bot will create up to 7 archives on the same day; these are stored in a directory based on the current year and month. Only the last 90 archives are kept. Each archive is compressed using gzip. The logging level is set at info. You can change this default logging configuration in the config/log4j2.xml file.

We recommend running at info level, as debug level logging will produce a lot of output from the Exchange Adapters; it's very handy for debugging, but not so good for your disk space!

REST API

"Enlightenment means taking full responsibility for your life." - William Blake

The bot has a REST API that allows you to remotely:

  • View and update Engine, Exchange, Markets, Strategy, and Email Alerts config.
  • View and download the log file.
  • Restart the bot - this is necessary for any config changes to take effect.

It has role based access control (RBAC): Users can view config and the logs, but only administrators can update config and restart the bot.

It is secured using JWT and has TLS support for Production environments.

You can view the Springdocs at: http://localhost:8080/swagger-ui.html once you've configured and started the bot.

Configuration

The REST API listens for plain HTTP traffic on port 8080 by default - you can change the server.port in the ./config/application.properties file.

The bot must be configured to use TLS if you plan on accessing the REST API over a public network - see the TLS section below.

You must also change the bxbot.restapi.jwt.secret value in the ./config/application.properties before using the REST API over a public network. This value should be a long random alphanumeric string (100+ chars). It is used to sign your web tokens using the HS512 algorithm.

Other interesting configuration in the ./config/application.properties includes:

  • bxbot.restapi.maxLogfileLines - the maximum number of lines to be returned in a view log file request. (For a head request, the end of the file is truncated; for a tail request the start of the file is truncated).

  • bxbot.restapi.maxLogfileDownloadSize - the maximum size of the logfile to download. If the size of the logfile exceeds this limit, the end of the file will be truncated.

  • bxbot.restapi.jwt.expiration - the expires time of the JWT. Set to 10 mins. Be sure you know the risks if you decide to extend the expiry time.

Users

You must change the PASSWORD values in the ./bxbot-rest-api/src/main/resources/import.sql before using the REST API over a public network - see instructions in the file on how to bcrypt your passwords.

2 users have been set up out of the box: user and admin. These users have user and admin roles respectively. Passwords are the same as the usernames - remember to change these :-)

When the bot starts up, Spring Boot will load the import.sql file and store the users and their access rights in its H2 in-memory database.

Authentication

The REST API endpoints require a valid JWT to be passed in the Authorization header of any requests.

To obtain a JWT, your REST client needs to call the /api/token endpoint with a valid username/password contained in the import.sql file. See the Authentication Springdocs for how to do this.

The returned JWT expires after 10 mins. Your client should call the /api/refresh endpoint with the JWT before it expires in order to get a new one. Alternatively, you can re-authenticate using the /api/token endpoint.

TLS

The REST API must be configured to use TLS before accessing it over a public network.

You will need to create a keystore - the command to create a PKCS12 self-signed certificate is shown below:

keytool -genkeypair -alias rest-api-keystore -keyalg RSA -keysize 2048 -storetype PKCS12 -keystore keystore.p12 -validity 3650

The keystore must be on the app's classpath - you can put it in the ./bxbot-rest-api/src/main/resources and re-build the app to get up and running fast. For a Production system, you'll want to replace the self-signed certificate with a CA signed certificate.

The 'TLS Configuration' section in the ./config/application.properties file needs the following properties set:

# Spring Boot profile for REST API.
# Must use https profile in Production environment.
spring.profiles.active=https

# SSL (TLS) configuration to secure the REST API.
# Must be enabled in Production environment.
server.port=8443
security.require-ssl=true
server.ssl.key-store=classpath:keystore.p12
server.ssl.key-store-password=secret
server.ssl.key-store-type=PKCS12

Coming Soon... (Definitely Maybe)

BX-bot UI - it will consume the REST API.

See the Project Board for timescales and progress.