Skip to content

Latest commit

 

History

History
218 lines (173 loc) · 11.2 KB

README.md

File metadata and controls

218 lines (173 loc) · 11.2 KB

NineChronicles DataProvider

  • NineChronicles.DataProvider is an off-chain service that stores NineChronicles game action data to a database mainly for game analysis.
  • Currently, this service only supports MySQL database.

Table of Contents

Pre-requisite

Run

  • Before running the program, please refer to the option values in the latest official 9c-launcher-config.json and fill out the variables in appsettings.json.
  • In appsettings.json, AppProtocolVersionToken, StorePath, PeerStrings, MySqlConnectionString properties MUST be filled to run the program.
  • To setup the NineChronicles mainnet blockchain store to use in the StorePath, download and extract the latest mainnet snapshot to a desired location.
$ dotnet run --project ./NineChronicles.DataProvider.Executable/ -- 

Development Guide

  • This section lays out the steps in how to log a new action in the database.
  • The TransferAsset action is used as an example in this guide.

1. Setup Database

dotnet ef database update -- [Connection String]
- Connection String example: "server=localhost;database=data_provider;port=3306;uid=root;pwd=root;"

2. Create Model

namespace NineChronicles.DataProvider.Store.Models
{
    using System;
    using System.ComponentModel.DataAnnotations;

    public class TransferAssetModel
    {
        [Key]
        public string? TxId { get; set; }

        public long BlockIndex { get; set; }

        public string? Sender { get; set; }

        public string? Recipient { get; set; }

        public decimal Amount { get; set; }

        public DateOnly Date { get; set; }

        public DateTimeOffset TimeStamp { get; set; }
    }
}

// Table for storing TransferAsset actions
public DbSet<TransferAssetModel> TransferAssets => Set<TransferAssetModel>();

2. Create Store Method

public void StoreTransferAsset(TransferAssetModel model)
{
    using NineChroniclesContext ctx = _dbContextFactory.CreateDbContext();
    TransferAssetModel? prevModel =
        ctx.TransferAssets.FirstOrDefault(r => r.TxId == model.TxId);
    if (prevModel is null)
    {
        ctx.TransferAssets.Add(model);
    }
    else
    {
        prevModel.BlockIndex = model.BlockIndex;
        prevModel.Sender = model.Sender;
        prevModel.Recipient = model.Recipient;
        prevModel.Amount = model..Amount;
        prevModel.Date = model.Date;
        prevModel.TimeStamp = model.TimeStamp;
        ctx.TransferAssets.Update(prevModel);
    }

    ctx.SaveChanges();
}

3. (Optional) Add Data Getter

In some cases, you need to handle state to get data to make model.
To do this easily, you can make your own data getter inside NineChronicles.DataProvider/DataRendering/.

4. Render & Store Action Data

_actionRenderer.EveryRender<TransferAsset>()
    .Subscribe(ev =>
    {
        try
        {
            if (ev.Exception is null && ev.Action is { } transferAsset)
            {
                var model = new TransferAssetModel()
                {
                    TxId = transferAsset.TxId,
                    BlockIndex = transferAsset.BlockIndex,
                    Sender = transferAsset.Sender,
                    Recipient = transferAsset.Recipient,
                    Amount = Convert.ToDecimal(transferAsset.Amount.GetQuantityString()),
                    Date = DateOnly.FromDateTime(_blockTimeOffset.DateTime),
                    TimeStamp = _blockTimeOffset,
                };
                MySqlStore.StoreTransferAsset(model);
            }
        }
        catch (Exception e)
        {
            Console.WriteLine(e);
        }

5. Add Database Migration

dotnet ef migrations add AddTransferAsset -- [Connection String]
- Connection String example: "server=localhost;database=data_provider;port=3306;uid=root;pwd=root;"

Current Table Descriptions

Migrating Past Chain Data to MySQL Database

Migrate All Action Data

  • This command migrates all action data based on DataRendering to the designated MySQL database.
  • Options such as offset and limit are provided to specify which block data to migrate.
  • IMPORTANT) This migration tool requires you to have the necessary state data in the blocks you want to migrate (If your chain store lacks the state data, this tool will not work).
Usage: NineChronicles.DataProvider.Executable mysql-migration [--store-path <String>] [--mysql-server <String>] [--mysql-port <UInt32>] [--mysql-username <String>] [--mysql-password <String>] [--mysql-database <String>] [--offset <I
nt32>] [--limit <Int32>] [--help]

Migrate action data in rocksdb store to mysql db.

Options:
  -o, --store-path <String>    Rocksdb path to migrate. (Required)
  --mysql-server <String>      A hostname of MySQL server. (Required)
  --mysql-port <UInt32>        A port of MySQL server. (Required)
  --mysql-username <String>    The name of MySQL user. (Required)
  --mysql-password <String>    The password of MySQL user. (Required)
  --mysql-database <String>    The name of MySQL database to use. (Required)
  --offset <Int32>             offset of block index (no entry will migrate from the genesis block).
  --limit <Int32>              limit of block count (no entry will migrate to the chain tip).
  -h, --help                   Show help message

Migrate Battle Arena Ranking Data

  • This command calculates the battle arena ranking of all participants at a specific block index (migration-block-index) and inserts the data to a designated mysql database.
Usage: NineChronicles.DataProvider.Executable battle-arena-ranking-migration [--store-path <String>] [--mysql-server <String>] [--mysql-port <UInt32>] [--mysql-username <String>] [--mysql-password <String>] [--mysql-database <Stri
ng>] [--migration-block-index <Int64>] [--help]

Migrate battle arena ranking data at a specific block index to a mysql database.

Options:
  -o, --store-path <String>          RocksDB store path to migrate. (Required)
  --mysql-server <String>            Hostname of MySQL server. (Required)
  --mysql-port <UInt32>              Port of MySQL server. (Required)
  --mysql-username <String>          Name of MySQL user. (Required)
  --mysql-password <String>          Password of MySQL user. (Required)
  --mysql-database <String>          Name of MySQL database. (Required)
  --migration-block-index <Int64>    Block index to migrate.
  -h, --help                         Show help message

Migrate User Staking Data

  • This command calculates the NCG staking amount of all participants at a specific block index (migration-block-index) and inserts the data to a designated mysql database.
  • Currently, the slack-token and slack-channel options for sending the data in csv format are required, however, these will be changed to optional in the near future.
Usage: NineChronicles.DataProvider.Executable user-staking-migration [--store-path <String>] [--mysql-server <String>] [--mysql-port <UInt32>] [--mysql-username <String>] [--mysql-password <String>] [--mysql-database <String>] [--
slack-token <String>] [--slack-channel <String>] [--migration-block-index <Int64>] [--help]

Migrate staking amounts of users at a specific block index to a mysql database.

Options:
  -o, --store-path <String>          Rocksdb store path to migrate. (Required)
  --mysql-server <String>            Hostname of MySQL server. (Required)
  --mysql-port <UInt32>              Port of MySQL server. (Required)
  --mysql-username <String>          Name of MySQL user. (Required)
  --mysql-password <String>          Password of MySQL user. (Required)
  --mysql-database <String>          Name of MySQL database to use. (Required)
  --slack-token <String>             slack token to send the migration data. (Required)
  --slack-channel <String>           slack channel that receives the migration data. (Required)
  --migration-block-index <Int64>    Block index to migrate.
  -h, --help                         Show help message