Skip to content

Commit

Permalink
chore: merge develop to evm (#611)
Browse files Browse the repository at this point in the history
* deps: clean dependendies and add check script (#450)

* Ignore eslint in scripts

* Use async calls

* Ignore tests

* Show testonly dependencies

* Calculate deps

* Add docs

* Implement methods on Package

* Cleanup

* Rename deps to imports

* Fix exceptions

* Pass method

* Go trough all packages

* Improve logs

* Rename file

* Remove old scripts

* Exit with 1 if failing

* Fix validator-set-vote-weighted

* Validator set static

* Fix validator

* Fix utils

* Fix transaction pool

* Fix test-framework

* Fix packages

* Add deps check

* style: resolve style guide violations

* Fix ci

* Fix api

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(bootstrap): compare stored genesis block with config (#451)

* Check stored commit

* Empty commit

* refactor(processor): log verification errors (#452)

* Remove console logs

* Remove forger errors

* Remove UnexpectedError

* Remove extra exceptions

* Add ValidatorExceptions

* Handler throws errors

* Improve logs

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* chore: change banner format to jpeg (#453)

* Add banner jpeg

* Remove png version

* Update readme

* release: 0.0.1-alpha.8 (#454)

* release: 0.0.1-alpha.8

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix(validator-set-vote-weighted): active validators is subset of registered validators (#466)

* Mak method private

* Build validators

* Local variable

* Use for of loop

* Check active validators

* Fix tests

* fix(processor): `validatorVoteBalance` calculation (#467)

* Add pg:reset

* Decrease vote balance

* Include fee

* Remove unused line

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* chore: docker validator support (#468)

* release: 0.0.1-alpha.9 (#469)

* release: 0.0.1-alpha.9

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* chore: fix publish GitHub workflow (#470)

* Run on pull request

* No git checks

* set token

* Revert to original build conditions

---------

Co-authored-by: oXtxNt9U <[email protected]>

* refactor(transactions): move amount deduction into transaction handlers (#472)

* refactor transfer amount deduction

* cleanup

* update schema tests

* feat: add `env:paths:clear` commands (#473)

* Clear plugins & exports

* Clear data

* Remove variables

* Add paths to contracts

* Remove config paths

* Cache cache, log, temp

* Cleanup

* Add command to api

* Add clear all

* Fix case

* release: 0.0.1-alpha.10 (#474)

* release: 0.0.1-alpha.10

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(crypto-transaction-username-registration): limit one transaction by sender in pool (#475)

* Rename variable

* Fix text

* Check transaction already in the pool

* feat(validator): log loaded validators (#477)

* Inject

* Print method

* Print on bootstrap

* Rename method

* Execute after state check

* Improve text

* Log 0 validators

* Determine activa & standby

* Wallet to string

* chore: fix typo in transaction fee errors (#476)

* feat(state): implements `sizeOfIndex` on WalletRepository (#479)

* Add sizeOfIndex

* Add initial implementation

* Wallet repository clone

* Implement on wallet repo clone

* Add missing attribute

* FIx validator resignation

* release: 0.0.1-alpha.11 (#480)

* release: 0.0.1-alpha.11

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat: port to ES modules (#478)

* initial browser support

* add package link script

* port api-common

* port api-database

* port api-development

* port api-http

* port api-sync

* port api-transaction-pool

* port api

* port bootstrap

* port cli

* port configuration-generator

* port consensus

* port webhooks

* port validator

* port validator-set-vote-weighted

* port validator-set-static

* port validation

* port utils

* port transactions

* port transaction-pool

* port test-framework

* port state

* port serializer

* port proposer

* port processor

* port p2p

* port networking-ntp

* port networking-dns

* port logger-winston

* port logger-pino

* port kernel

* port fees

* port fees-static

* port fees-managed

* port fees-burn

* port database

* port crypto-worker

* port crypto-wif

* port crypto-validation

* port crypto-transaction

* port crypto-transaction-vote

* port crypto-transaction-validator-resignation

* port crypto-transaction-validator-registration

* port crypto-transaction-username-resignation

* port crypto-transaction-username-registration

* port crypto-transaction-transfer

* port crypto-transaction-multi-signature-registration

* port crypto-transaction-multi-payment

* port crypto-signature-schnorr

* port crypto-signature-schnorr-secp256k1

* port crypto-signature-ed25519

* port crypto-signature-ecdsa

* port crypto-signature-bls12-381

* port crypto-messages

* port crypto-key-pair-schnorr

* port crypto-key-pair-ed25519

* port crypto-key-pair-ecdsa

* port crypto-key-pair-bls12-381

* port crypto-hash-wasm

* port crypto-hash-noble

* port crypto-hash-bcrypto

* port crypto-consensus-bls12-381

* port crypto-config

* port crypto-commit

* port crypto-block

* port crypto-address-ss58

* port crypto-address-keccak256

* port crypto-address-bech32m

* port crypto-address-bech32

* port crypto-address-base58

* port core

* port contracts

* port container

* update tsconfig to nodenext

* fix kleur/color

* update core/bin/run.js

* command fixes

* lmdb import fixes

* p2p fixes

* ajv fix

* get __dirname

* update pnpm-lock.yaml

* remove unused @pm2/io dependency

* replace fileURLToPath

* use filesystem service inside kernel

* cleanup

* update TypeScript to 5.4.2

* move __dirname into local scope

* replace fast-memoize with memoizee

* test-runner packacge

* re-export test-runner

* tests: utils

* tests: validation

* tests: crypto-config

* tests: container

* tests: kernel

* tests: transactions

* style: resolve style guide violations

* fix more tests

* fix transaction dependency lookup

* fix p2p tests

* more fixes

* fix cli and configuration-generator

* cli fixes

* kernel fixes

* transaction-pool

* all pass

* commit dirty

* style: resolve style guide violations

* fix api-http integration tests

* fix deps:check

* runtime fixes

---------

Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: sebastijankuzner <[email protected]>

* fix(api): broken entrypoint (#481)

* fix api entrypoint

* fix more commands

* refactor(transaction-pool): return tx indexes as integers (#483)

* return tx indexes as integers

* fix compile error

* chore: update dependencies (#482)

* update dependencies

* Empyt commit

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix wallet response (#485)

* fix(configuration-generator): create genesis block script (#484)

* fix create genesis block script

* Empty commit

---------

Co-authored-by: sebastijankuzner <[email protected]>

* refactor(core): rename run script to have explicit file extension (#486)

* ensure run script has explicit file extension

fixes ERR_UNKNOWN_FILE_EXTENSION

* empty commit

* fix: use .js extension in `start` commands (#487)

* Fix start commands

* Fix package.json

* Fix tests

* release: 0.0.1-alpha.12 (#488)

* release: 0.0.1-alpha.12

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix(core): resolve correct plugin `package.json` path (#489)

* Fix package manifest

* Skip tests

* release: 0.0.1-alpha.13 (#490)

* release: 0.0.1-alpha.13

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(cli): determine correct entrypoint paths (#491)

* Add setup class

* Fix setup

* Bind

* Fixes

* Use setup paths

* Use in api

* Remove logs

* Fix tests

* Cleanup tests

* fix(test-framework): list project packages in dependencies (#492)

* Use direct dependencies

* Empty commit

* Fix dependencies

* Fix dependencies

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* exclude genesis transactions from fee statistics calculation (#494)

* fix(crypto): manage static fees via milestones (#493)

* add static fees to milestones

* update tests

* style: resolve style guide violations

* initialize static fees only in a single place

---------

Co-authored-by: oXtxNt9U <[email protected]>

* fix(api): sync active milestones on change (#495)

* sync active milestones on change

* fix test

* fix(api-http): add missing fields to transaction resource (#496)

* improve error logging

* populate all transaction fields

* review

* remove type cast

* add comment

* simplify more

* rename block model too and add assert

* fix paths according to the latest paths naming convention (#497)

* refactor(p2p): replace ip with `url` in api nodes (#498)

* api node urls

* update protobuf definitions

* update test fixtures

* cleanup

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* fix(transaction-pool): invalid tx error reporting (#500)

* fix tx pool error reporting

* empty commit

* test: use `tsx` TypeScript loader (#499)

* Add tsx

* Fix runner

* Fix test-runner scripts

* Update tsconfig

* Fix tests

* Use test config

* Fix api tests

* api-common

* Api-database

* api-development

* api-http

* api-sync

* api-transaction-pool

* bootstrap

* cli partial

* Fix cli tests

* Runner extends contracts

* Configuration generator

* Consensus

* Container

* Contracts

* Core

* Crypto address

* Crypto block

* Cypto commit

* Crypto config

* Crypto

* Crypto messages

* Crypto signature

* Crypto transaction

* Crypto

* Database

* Fees

* Kernel

* Logger pino

* Logger winston

* Networking

* Server

* Processor, proposer, serializer

* State

* Transaction pool

* Transactions

* Utils

* Validation

* Validator

* Validator set

* Webhooks

* Remove tsm

* Fix integration tests

* Update lint CI

* Fix dependencies

* Prettier

* Lint

* Perttier

* style: resolve style guide violations

* refactor: remove `__dirname` usage (#501)

* Remove __dirname

* Fix tests

* Factory require config

* test(functional/consensus): add skeleton for functional tests (#503)

* Prepare base structure

* Add test runner

* Base test

* Add base packages

* Register base packages

* Add config

* Add pacakges

* Export bootstrap

* Export individual bootstrappers

* Register bootstrappers

* Extract bootstrapper itnerface

* Test

* Use tsm

* Pass url to PluginManifest

* Register service provider

* Setup up to validator package

* Register all necessary packages

* Add missing packages

* Boot

* Register transactions

* Process genesis

* Run consensus

* Add worker

* Fix worker

* Fix broadcaster

* refactor(consensus-storage): extract storage logic into package (#502)

* Add consensus storage package

* Prepare service provider

* Remove storage from consensus

* Add dependencies

* Move root into consensus storage

* Rename identifier

* Fix identifiers

* Edit contracts

* Fix deps

* Update packages/consensus-storage/README.md

Co-authored-by: oXtxNt9U <[email protected]>

---------

Co-authored-by: oXtxNt9U <[email protected]>

* milestone generator add static fees (#504)

* test(functional/transaction-pool-api): initial boilerplate and tests (#505)

* update gitignore

* add tx pool api boilerplate and basic broadcast

* add signWithKeyPair to tx builder

* update functional consensus milestones

* wait for block and assert committed transaction

* invalid transfer test

* assert invalid transactions are not committed

* test with invalid / malformed signature

* fix(test-runner): rethrow error in runHook (#506)

* bubble up error thrown in runHook

* Empty

---------

Co-authored-by: sebastijankuzner <[email protected]>

* refactor(database): return commits instead blocks (#507)

* getBlock to getCommit

* Fix tests

* getLastBlock -> getLastCommit

* Fix tests

* Implement isEmpty

* Fix tests

* feat(transaction-pool): support `in-memory` database (#510)

* Support im memory database

* Deps

* Test

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* test(functional/consensus): prepare package replacements, asserts and p2p layer (#508)

* Bind consensus storage

* Mock database

* Add contracts

* Use relative paths

* Prepare 2 setups

* Run 2 nodes

* Make log protected

* Make MemoryLogger fields protected

* Prepare custom logger

* Prepare p2p layer

* Fix p2p

* MemoryDatabase

* Load crypto

* Read validators

* Prepare validators

* Use common config

* Add snooze for one block

* Stop node

* Snooze for block

* Prepare smaller config

* Asserts

* Fix and check TS code

* Add script for check & test

* Use strict true

* Support plugin options

* Assert validators

* refactor(kernel): create bootstraper for shutdown signals  (#511)

* Register error handler first

* Move listen to shutdown signals into bootstrapper

* test(functional/transaction-pool-api): test all transaction types (#509)

* missing dependency

* transfer

* vote

* username-registration

* username-resignation

* validator-registration

* validator-resignation

* multipayment

* fix default payload

* update utils

* multi signature test plus fixes

* fix compile error

* test transfer with multisig

* rename tests for clarity

* add wallets to context

* tsc strict

* add snapshot based balance checks

* reduce block time

* review

* test(functional/transaction-pool): increase vote coverage (#512)

* votes

* empty commit

* test switch vote and unvote for non voted validator

* test(functional/transaction-pool): increase validator registration coverage (#513)

* votes

* empty commit

* validator registration

* empty commit

* test(functional/transaction-pool): increase validator resignation coverage (#514)

* votes

* empty commit

* validator resignation

* empty commit

* test(functional/transaction-pool): increase username registration coverage (#515)

* votes

* empty commit

* username registration

* empty commit

* test(functional/transaction-pool): increase username resignation coverage (#516)

* votes

* empty commit

* username resignation

* empty commit

* fix(kernel): minority and majority calculation (#520)

* Fix majority, minority

* Tests

* Add extra tests

* fix(api-http): allow 0 nonce in schema (#521)

* Allow nonce 0

* Empty commit

* fix(api-http): implement missing `/transaction/fees` route (#523)

* implement missing route /transaction/fees

* add test

* style: resolve style guide violations

* remove unused query param

---------

Co-authored-by: oXtxNt9U <[email protected]>

* test(functional/transaction-pool): increase multi sig registration coverage  (#517)

* more multi signature tests

* empty commit

* update min participants

* test missing participant signature

* test(functional/transaction-pool): add more misc coverage  (#518)

* more multi signature tests

* empty commit

* add misc tests

* empty commit

* release: 0.0.1-alpha.14 (#522)

* release: 0.0.1-alpha.14

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(consensus): emit events (#524)

* Add bootstrapped

* New round

* Add events

* Dispatch events

* Fix tests

* test(functional/transaction-pool): extract transaction builders into separate package (#525)

* introduce test-transaction-builders package

* use test transactions package

* include type when logging rejected tx

* style: resolve style guide violations

* fix deps:check

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* test(functional/consensus): test proposal scenarios (#526)

* Add initial test

* Rename base test

* Optional blockId

* Full test

* Bind custom proposer selector

* Check generatorPublic key

* Make consensus methods protected

* Expose boot and bootstrap methods

* Skip error handler registration

* Make consensus methods public

* Assert blockRound

* Implement *Many helpers

* Fix base test

* Force async to p2p

* Test missing propose

* Simulate double propose

* Simulate 50:50 split

* Remove assertCommitValidators

* Record messages

* Asserts

* majority minority split

* SnozeForBlock with height

* Assert next block is fine

* Check multiple rounds

* Check invalid proposer

* Add CI for functional

* Rename tests to proposal

* Rename tests

* fix(api-http): delegate search (#527)

* Fix delegate search in API

* Initial support for sorting on jsonb fields

* Update tests

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* refactor(p2p): log route on codec error (#528)

* Log error

* Allow empty

* test(functional/transaction-pool): add .gitkeep (#530)

* add .gitkeep

* empty commit

* test(functional/consensus): test prevote & precommit scenarios (#529)

* Check minority not prevoting

* Test null prevotes

* Extract methods to utils

* Check random block

* Check with multiple prevotes

* Lint

* Fix ts errors

* Disable unicorn/no-useless-undefined

* Check missing precommits

* Test precommits

* Check multiple messages

* fix(configuration-generator): update export paths (#531)

* Update paths

* Empty

* feat(api-development): add `peers/banned` endpoint (#532)

* Disposer returns list of banned peers

* Add endpoint

* Add pagiantion

* Format timeout

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* test(functional): update ci workflow (#534)

* run all functional tests in ci

* test

* style: resolve style guide violations

* test

* revert me

* style: resolve style guide violations

* include matrix name

* revert trigger

---------

Co-authored-by: oXtxNt9U <[email protected]>

* test(functional/consensus): test rejection of invalid block proposals (#533)

* custom funded wallet provider

* REVERT ME: debug logs

* export Proposer

* custom block proposal

* emit event on invalid block

* add assertInvalidBlock

* extract custom proposal function

* ensure wallet exists on each node

* invalid proposals

* seed with randomBytes

* log error instance on mismatch

* lint

* Revert "REVERT ME: debug logs"

This reverts commit defdf7e.

* deps: update lmdb (#535)

* Update lmdb

* Log if commit cannot be read

* Await transaction flushed

* Await flush

* test(functional): fix `MaxListenersExceeded` warning (#537)

* fix MaxListenersExceededWarning

* empty commit

* feat(api-http): support sorting on jsonb attributes (#538)

* support sort on wallet attributes

* NULLS LAST

* tests

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* fix(p2p): peer ban and validate ip plugin (#539)

* Skip header check if peer is not in repo

* Validate peer ip

* Remove repo check

* Validate ip and data separately

* Fix test

* Update default peer ban time

* feat(cli): add --overwrite flag to `config:publish:custom` (#536)

* expose `skip` callback

* update publish:custom command

* style: resolve style guide violations

* Make fields optional

* Require flags error

---------

Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: sebastijankuzner <[email protected]>

* feat(crypto): add browser support for bls12-381 packages (#540)

* add bls12-381 browser support

* update test

* style: resolve style guide violations

* dep check exception

---------

Co-authored-by: oXtxNt9U <[email protected]>

* release: 0.0.1-alpha.15 (#542)

* release: 0.0.1-alpha.15

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix(p2p): postTransactions response schema and proto (#543)

* Update proto

* Build proto

* Fix schema

* Fix tests

* refactor(api): show `meta` field in response at top (#545)

* show `meta` field in response at top

* style: resolve style guide violations

* eslint: disable key sort

---------

Co-authored-by: oXtxNt9U <[email protected]>

* fix(api-http): change default sort for `/delegates` (#544)

* sort /delegates by rank by default

* empty commit

* refactor(block-processor): log commit round (#546)

* Log block round

* Empty commit

* perf(consensus): verify proposal signatures on main thread (#548)

* Verify signature on main thread

* Empty commit

* refactor(contracts): remove Consensus prefix from contracts (#549)

* Rename contracts

* Empty commit

* refactor(consensus): remove hasValidBlockGenerator check (#550)

* Remove hasValidBlockGenerator check

* Empty commit

* fix(api-http): sort delegates by rank asc by default (#552)

* sort delegates by rank asc by default

* empty commit

* test(e2e): add e2e tests and workflow (#547)

* network with 3 nodes

* add check for target block height

* workflow

* revert me

* exclude e2e directory

* style: resolve style guide violations

* docker fixes

* test

* add timeout

* test

* wait for checks to finish

* run detached

* show logs

* style: resolve style guide violations

* increase sleep

* increase target height

* revert trigger condition

* revert changes to defaults

* style: resolve style guide violations

* reply with 200

* Update tests/e2e/consensus/checks/index.mjs

Co-authored-by: Sebastijan K. <[email protected]>

* reduce timeout for now

---------

Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: Sebastijan K. <[email protected]>

* release: 0.0.1-alpha.16 (#553)

* release: 0.0.1-alpha.16

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(validator): support secret encryption (#554)

* remove skipValidation flag

* write validator keystore

* support validator with encrypted passphrase

* update commands

* update tests

* optional keystore

* style: resolve style guide violations

* support --password

* review

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* perf: deserialize proposal in steps (#551)

* Rename ProposedBlock to ProposedData

* Remove ProposedBlockJson

* Rename proposal.block to proposal.data

* Fix consensus tests

* Fix crypto-message tests

* Add getData and isDataDeserialized

* Fix consensus tests

* Test proposal

* Rename method

* Make makeProposedDataFromBytes public

* Make proposal injectable

* Fix tests

* Add deserialize to proposal

* Check status before returning data

* Include deserializer

* Fix tests

* Pass only serialized data to proposal

* Deserialize in proposal processor

* Deserialize in handler

* Remove todo

* Fix tests

* Fix header

* Verify lock proof after deserialization

* Fix functional tests

* Fix proposal types

* Fix proposal creation

* perf(consensus): run pending RoundStates only once (#555)

* Set pending jobs

* Remove key

* Reduce complexity

* fix: dispatch block events with expected payload (#558)

* dispatch BlockEvent.Forged again

* dispatch only block instead of whole commit

* fix

* test

* update e2e check

* chore: upgrade pnpm v9 (#556)

* upgrade pnpm v9

* empty commit

* support webhook condition on nested key (#557)

* improve stability (#564)

* fix(webhooks): restore webhooks on boot (#562)

* restore webhooks on boot

* skip events during bootstrap

* skip all but kernel events

* style: resolve style guide violations

* review

---------

Co-authored-by: oXtxNt9U <[email protected]>

* chore(ci): add workflow to enforce minimum commit count (#565)

* check minimum commit count

* style: resolve style guide violations

* rename

---------

Co-authored-by: oXtxNt9U <[email protected]>

* release: 0.0.1-alpha.17 (#566)

* release: 0.0.1-alpha.17

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix(api-sync): upsert version on bootstrap (#567)

* upsert version on bootstrap

* empty commit

* feat: add `blockPrepareTime` to milestones (#568)

* Add blockPrepareTime

* Nest

* Scheduler

* Sync

* Fix generator

* Test config generator

* Test crypto config

* Fix scheduler

* Add test

* Fix functional tests

* stageTime to stageTimeout

* Fix api-http tests

* refactor(api-sync): add `syncInterval` option (#569)

* Implement getSyncIntervalMs on abstractListener

* Add syncInterval defaults

* Use configuration

* fix(api-sync): process transactions in chunks to support very large blocks (#572)

* sync transactions in chunks

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* refactor(consensus): prepare block after commit (#571)

* Log on round start

* Propose after commit

* Use timestamp

* Rename methods

* Fix tests

* Rename method

* Fix tests

* Deps

* Fix consensus tests

* Fix tests

* Update packages/validator/source/validator.ts

Co-authored-by: oXtxNt9U <[email protected]>

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: sebastijankuzner <[email protected]>

* refactor(consensus):  dispatch *any events only once per round (#574)

* Return scheduler status

* Test scheduler

* Skip multiple dispatches

* Tests

* fix(api-http): support query criteria based on wallet attributes (#573)

* wallet attributes criteria

* delegate attributes criteria

* return BadRequest if request results in invalid query

* test for attribute query

* style: resolve style guide violations

* rename

---------

Co-authored-by: oXtxNt9U <[email protected]>

* feat(api-http): query round by number (#591)

* fix 0 vote balance

* add round parameter

* update tests

* rename route

* feat(crypto-transaction-validator-registration): accept registration from multisig wallet (#593)

* remove check

* add tests

* cleanup obsolete exception class

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* feat(api-http): expose validator round votes (#598)

* add column

* sync round votes

* return votes from controller

* update test

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* release: 0.0.1 alpha.18 (#599)

* release: 0.0.1-alpha.18

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix(api-sync): make bootstrap less verbose (#600)

* reduce verbosity of bootstrap

* empty commit

* feat: run transaction-pool in worker thread (#610)

* refactor(transaction-pool-service): rename package (#575)

* Rename package

* Empty commit

* Fix deeps check

* Fix p2p tests

* Fix e2e tests

* Fix workflow

* feat(transaction-pool): create CLI package (#576)

* Clone API package

* Use mainsail-tx-pool process name

* Neutral tests names

* Rename api: to tx-pool:

* Rename commands

* Scripts

* Deps

* Generate unit workflow

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(api-common): add JSON-RPC support (#577)

* Bind rpc processor

* Extract RPC contracts

* Add getRPCProcessor

* Validate schema

* Fix validation

* Add method

* Add call action

* Register action

* Handle methodNotFound

* Validate params & handler error

* Move plugin

* Use common utils

* Remove console.logs

* Extract prepareRpcError handler

* Fix imports and deps

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(api-common): use AJV for JSON_RPC schema validation (#580)

* Add ajv

* Add schema

* Export validation

* Add Validator.hasSchema

* Register schema

* Fix schema

* Verify payload

* Use ajv

* style: resolve style guide violations

* Fix deps

* Fix api-http tests

---------

Co-authored-by: sebastijankuzner <[email protected]>

* refactor(p2p): remove `postTransactions` route (#581)

* Remove postTransactions route

* Remove postTransactions

* Remove post transactions utils

* Remove broadcaster

* Rebuild proto

* Fix p2p tests

* Fix tests

* feat: implement `transaction-pool-server` & `transaction-pool-client` packages (#583)

* Add packages & crypto

* Add transaction pool api

* Add deps

* Add transaction pool server

* Add defaults

* Test controller

* Boot server

* Add client package

* Register client

* Add contract

* Register action

* Add base handler

* Remove test transactions

* Client getTx

* Replace TX pool service with client

* Fix schema validation

* Remove transactions from api-development

* Remove from bootstrapper

* Remove from processor and validator

* Remove api-transaction-pool from core

* Use optional query on TX

* Log errors

* style: resolve style guide violations

* Fix validator tests

* Fix consensus tests

* Temporary disable transaction-pool-api tests

* Update e2e app.json

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(state): supports state sync using delta changes (#584)

* Implement changesToJson

* Register onCommit

* Add address

* Add changesToJson on WalletRepository

* Return indexes

* Store to json

* Add interface

* Add commit action

* Apply changes to repostiory

* Apply wallet changes

* Apply changes

* Set and forget attributes

* Skip unknown attributes

* Errors

* style: resolve style guide violations

* Fix tests

* Fix

* Comment

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(transaction-pool-api): add transactions endpoints (#585)

* Rename route

* /unconfirmed

* /unconfirmed/{id}

* /types

* /unconfirmed

* Remove unused tag

* feat(transaction-pool): implement `get_transactions` action (#586)

* Collate transactions

* Set config height

* Transaction bytes return buffer

* Move transaction validator

* Validate transactions

* Add crypto block

* Simplify validator

* Test get transactions

* Remove collator

* Rename identifiers

* Rename contract

* Set failed transactions

* style: resolve style guide violations

* Empty commit

* Fix tests

* Fix consensus tests

---------

Co-authored-by: sebastijankuzner <[email protected]>

* refactor(crypto-transaction): make `id` mandatory (#587)

* Make id mandatory

* Make id mandatory

* Make type group mandatory

* Make network mandatory

* feat(transaction-pool): sync block and transactions (#588)

* Post failed transactions

* Commit block

* Increase payload size

* Reset failed transactions

* Set last block

* feat(transaction-pool): list & import snapshots (#589)

* Export state

* Add snapshot service

* Add action

* list snapshots

* Add import snapshot

* Call import snapshot

* Rename field

* Export from snapshot service

* Extract paths

* Use async fs methods

* Add remove snapshots

* List snapshots as promise

* Use list snapshots

* Implement import

* File changes

* Fix restore snapshots

* Restore snapshots

* Handle import snapshot

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* feat(state): add `skipUnknownAttributes` option to StateRepository (#592)

* Log errors

* Fix error messages

* Allow unknown attributes

* Make wallet injectable

* Inject attribute repository

* Rename repository to state repository

* StateRepositoryFactory

* Rename defaults

* Read from defaults

* Add to options

* Extract stateRepositoryFactory

* Fix wallet factory

* fix tests

* Fix tests

* feat(transaction-pool): implement `get_status` JSON_RCP endpoint (#595)

* Implement get status

* Log status

* Endpoint returns data

* Disable e2e

* feat(transaction-pool): add  request and response schemas for JSON_RPC actions (#596)

* Add commit interfaces

* Add commit schema

* Get Status contracts

* GetTransactions contracts

* ImportSnapshot contracts

* ListSnapshots contracts

* Validate response

* Validate JsonRPC response

* Move schemas to common

* refactor(state): recreate state on snapshot restore (#597)

* Recreate base store

* Fix bootstrapper

* refactor(kernel): replace child processes with worker threads (#601)

* Use workers

* Limit crypto worker count

* Remove signal for shutdown

* feat(transaction-pool): run transaction pool in worker thread (#602)

* Remove transaction-pool

* Add tx pool plugins

* Add core and tx-pool entries

* Load core

* Set thread name

* Add thread name

* Load crypto worker

* Remove requiredByWorker

* Clone

* Cleanup

* Extract contracts

* Update contracts

* Determine if is worker

* Extract worker package

* Register package

* Update contracts

* Fix bindings

* Boot

* Fix

* Add app.json

* Fix app

* Fix functional tests

* CI Unit

* Remove workerLoggingEnabled

* fix(transaction-pool-worker): update local module import workaround (#605)

* fix workaround for local module import with custom launch config

* remove try catch

* refactor(transaction-pool): disable snapshot export (#604)

* TxPoolClient is mandatory

* Check snapshots enabled

* Fix factory

* refactor(transaction-pool-worker): implement IPC actions (#606)

* Import snapshot

* Improve handler

* Implement commit

* Fix contract

* Extract commit handler

* Implement get transactions

* Extract import snapshot

* Remove server client packages

* Remove client instances

* Remove workerPool

* Remove old contracts

* Fix tests

* Regenerate unit CI

* chore: merge develop to tx-pool (#607)

* fix(api-http): support query criteria based on wallet attributes (#573)

* wallet attributes criteria

* delegate attributes criteria

* return BadRequest if request results in invalid query

* test for attribute query

* style: resolve style guide violations

* rename

---------

Co-authored-by: oXtxNt9U <[email protected]>

* feat(api-http): query round by number (#591)

* fix 0 vote balance

* add round parameter

* update tests

* rename route

* feat(crypto-transaction-validator-registration): accept registration from multisig wallet (#593)

* remove check

* add tests

* cleanup obsolete exception class

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* feat(api-http): expose validator round votes (#598)

* add column

* sync round votes

* return votes from controller

* update test

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>

* release: 0.0.1 alpha.18 (#599)

* release: 0.0.1-alpha.18

* style: resolve style guide violations

---------

Co-authored-by: sebastijankuzner <[email protected]>

* fix(api-sync): make bootstrap less verbose (#600)

* reduce verbosity of bootstrap

* empty commit

* Increase version

---------

Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: sebastijankuzner <[email protected]>

* test(functional): fix transaction-pool-api tests (#609)

* Add custom pool worker

* Empty

* Enable tests

* test(ci): fix e2e tests (#608)

* Enable E2E tests

* Empty commit

* Update app.json

* Fix

* enable hapi debug logs

* fix invalid config lookup

* style: resolve style guide violations

---------

Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: oXtxNt9U <[email protected]>

---------

Co-authored-by: sebastijankuzner <[email protected]>
Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: oXtxNt9U <[email protected]>

* Changes

* Fix unit tests

* Fix consensus tests

* Fix transaction-pool-api tests

* Update e2e tests

* style: resolve style guide violations

* Fix dependencies

* Fix integration tests

* update validator secrets

* merge fix

---------

Co-authored-by: sebastijankuzner <[email protected]>
Co-authored-by: Adrian Kerchev <[email protected]>
Co-authored-by: oXtxNt9U <[email protected]>
Co-authored-by: Edgar Goetzendorff <[email protected]>
Co-authored-by: oXtxNt9U <[email protected]>
  • Loading branch information
6 people authored Jun 5, 2024
1 parent 6e6e1b8 commit 62f2428
Show file tree
Hide file tree
Showing 337 changed files with 9,988 additions and 9,509 deletions.
18 changes: 14 additions & 4 deletions .github/workflows/unit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ jobs:
run: cd packages/configuration-generator && pnpm run test
- name: Test consensus
run: cd packages/consensus && pnpm run test
- name: Test consensus-storage
run: cd packages/consensus-storage && pnpm run test
- name: Test container
run: cd packages/container && pnpm run test
- name: Test contracts
Expand All @@ -76,6 +78,8 @@ jobs:
run: cd packages/crypto-address-ss58 && pnpm run test
- name: Test crypto-block
run: cd packages/crypto-block && pnpm run test
- name: Test crypto-commit
run: cd packages/crypto-commit && pnpm run test
- name: Test crypto-config
run: cd packages/crypto-config && pnpm run test
- name: Test crypto-consensus-bls12-381
Expand All @@ -96,8 +100,6 @@ jobs:
run: cd packages/crypto-key-pair-schnorr && pnpm run test
- name: Test crypto-messages
run: cd packages/crypto-messages && pnpm run test
- name: Test crypto-commit
run: cd packages/crypto-commit && pnpm run test
- name: Test crypto-signature-bls12-381
run: cd packages/crypto-signature-bls12-381 && pnpm run test
- name: Test crypto-signature-ecdsa
Expand Down Expand Up @@ -133,6 +135,8 @@ jobs:
run: cd packages/crypto-validation && pnpm run test
- name: Test crypto-wif
run: cd packages/crypto-wif && pnpm run test
- name: Test crypto-worker
run: cd packages/crypto-worker && pnpm run test
- name: Test database
run: cd packages/database && pnpm run test
- name: Test evm
Expand Down Expand Up @@ -169,8 +173,14 @@ jobs:
run: cd packages/test-runner && pnpm run test
- name: Test test-framework
run: cd packages/test-framework && pnpm run test
- name: Test transaction-pool
run: cd packages/transaction-pool && pnpm run test
- name: Test test-runner
run: cd packages/test-runner && pnpm run test
- name: Test test-transaction-builders
run: cd packages/test-transaction-builders && pnpm run test
- name: Test transaction-pool-service
run: cd packages/transaction-pool-service && pnpm run test
- name: Test transaction-pool-worker
run: cd packages/transaction-pool-worker && pnpm run test
- name: Test transactions
run: cd packages/transactions && pnpm run test
- name: Test utils
Expand Down
2 changes: 1 addition & 1 deletion lerna.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"$schema": "node_modules/lerna/schemas/lerna-schema.json",
"useWorkspaces": true,
"version": "0.0.1-alpha.17"
"version": "0.0.1-alpha.18"
}
2 changes: 1 addition & 1 deletion packages/api-common/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@mainsail/api-common",
"version": "0.0.1-alpha.17",
"version": "0.0.1-alpha.18",
"description": "Common API building blocks for the Mainsail blockchain",
"license": "GPL-3.0-only",
"contributors": [],
Expand Down
1 change: 1 addition & 0 deletions packages/api-common/source/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ export * as Plugins from "./plugins/index.js";
export * as Schemas from "./schemas.js";
export * from "./server.js";
export * from "./service-provider.js";
export * as Validation from "./validation/index.js";
19 changes: 3 additions & 16 deletions packages/api-common/source/rcp/processor.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import Hapi from "@hapi/hapi";
import { inject, injectable } from "@mainsail/container";
import { Contracts, Exceptions, Identifiers } from "@mainsail/contracts";
import { Contracts, Identifiers } from "@mainsail/contracts";

import { getRcpId, prepareRcpError } from "./utils.js";

Expand Down Expand Up @@ -33,24 +33,11 @@ export class Processor implements Contracts.Api.RPC.Processor {

try {
return {
jsonrpc: "2.0",
// eslint-disable-next-line sort-keys-fix/sort-keys-fix
id: getRcpId(request),
jsonrpc: "2.0",
result: await action.handle(payload.params),
};
} catch (error) {
if (error instanceof Exceptions.RpcError) {
return {
jsonrpc: "2.0",
// eslint-disable-next-line sort-keys-fix/sort-keys-fix
id: getRcpId(request),
// eslint-disable-next-line sort-keys-fix/sort-keys-fix
error: {
code: error.code,
message: error.message,
},
};
}
} catch {
return prepareRcpError(getRcpId(request), Contracts.Api.RPC.ErrorCode.InternalError);
}
}
Expand Down
6 changes: 2 additions & 4 deletions packages/api-common/source/rcp/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,10 @@ export const prepareRcpError = (
id: Contracts.Api.RPC.Id,
errorCode: Contracts.Api.RPC.ErrorCode,
): Contracts.Api.RPC.Error => ({
jsonrpc: "2.0",
// eslint-disable-next-line sort-keys-fix/sort-keys-fix
id,
// eslint-disable-next-line sort-keys-fix/sort-keys-fix
error: {
code: errorCode,
message: errorMessageMap[errorCode],
},
id,
jsonrpc: "2.0",
});
12 changes: 10 additions & 2 deletions packages/api-common/source/server.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import Boom from "@hapi/boom";
import { Server as HapiServer, ServerInjectOptions, ServerInjectResponse, ServerRoute } from "@hapi/hapi";
import { inject, injectable } from "@mainsail/container";
import { Contracts, Identifiers } from "@mainsail/contracts";
Expand Down Expand Up @@ -48,8 +49,15 @@ export abstract class AbstractServer {

this.server.ext("onPreResponse", (request, h) => {
if ("isBoom" in request.response && request.response.isBoom && request.response.isServer) {
// @ts-ignore
this.logger.error(`${request.path} - ${request.response.stack ?? request.response.message}`);
if (request.response.name === "QueryFailedError") {
const message = `${request.response.name} ${request.response.message}`;
// @ts-ignore
this.logger.warning(`${request.path} - ${message}`);
request.response = Boom.badRequest(message);
} else {
// @ts-ignore
this.logger.error(`${request.path} - ${request.response.stack ?? request.response.message}`);
}
}
return h.continue;
});
Expand Down
49 changes: 47 additions & 2 deletions packages/api-common/source/validation/schemas.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,58 @@
import { SchemaObject } from "ajv";

const id: SchemaObject = {
anyOf: [{ type: "string" }, { type: "integer" }, { type: "null" }],
};

const jsonrpc: SchemaObject = {
const: "2.0",
type: "string",
};

const jsonRpcError: SchemaObject = {
additionalProperties: false,
properties: {
error: {
additionalProperties: false,
properties: {
code: { type: "integer" },
data: {},
message: { type: "string" },
},
required: ["code", "message"],
type: "object",
},
id: id,
jsonrpc: jsonrpc,
},
required: ["jsonrpc", "error", "id"],
type: "object",
};

const jsonRpcResult: SchemaObject = {
additionalProperties: false,
properties: {
id,
jsonrpc,
result: {},
},
required: ["jsonrpc", "result", "id"],
type: "object",
};

export const jsonRpcResponseSchema: SchemaObject = {
$id: "jsonRpcResponse",
oneOf: [jsonRpcError, jsonRpcResult],
};

export const jsonRpcPayloadSchema: SchemaObject = {
$id: "jsonRpcPayload",

additionalProperties: false,

properties: {
id: { anyOf: [{ type: "string" }, { type: "integer" }, { type: "null" }] },
jsonrpc: { const: "2.0" },
id,
jsonrpc,
method: { type: "string" },
params: {},
},
Expand Down
3 changes: 2 additions & 1 deletion packages/api-database/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@mainsail/api-database",
"version": "0.0.1-alpha.17",
"version": "0.0.1-alpha.18",
"description": "API database for the Mainsail blockchain",
"license": "GPL-3.0-only",
"contributors": [],
Expand All @@ -26,6 +26,7 @@
"@mainsail/container": "workspace:*",
"@mainsail/contracts": "workspace:*",
"@mainsail/kernel": "workspace:*",
"@mainsail/utils": "workspace:*",
"dayjs": "1.11.10",
"pg": "8.11.3",
"typeorm": "0.3.20"
Expand Down
6 changes: 6 additions & 0 deletions packages/api-database/source/models/validator-round.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,10 @@ export class ValidatorRound {
type: "jsonb",
})
public readonly validators!: string[];

@Column({
nullable: false,
type: "jsonb",
})
public readonly votes!: string[];
}
2 changes: 2 additions & 0 deletions packages/api-database/source/search/criteria.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,8 @@ export type DelegateCriteria = {
forged?: DelegateForged;
production?: DelegateProduction;
blocks?: DelegateBlocks;

attributes?: OrContainsCriteria<Record<string, any>>;
};

export type OrDelegateCriteria = OrCriteria<DelegateCriteria>;
Expand Down
2 changes: 1 addition & 1 deletion packages/api-database/source/search/expressions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ export type Expression<TEntity> =
| JsonbAttributeExists<TEntity>;

export type JsonFieldOperator = "->>";
export type JsonFieldCastType = "bigint";
export type JsonFieldCastType = "bigint" | "numeric";
export type JsonFieldAccessor = {
operator: JsonFieldOperator;
fieldName: string;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { ApiNode } from "../../models/index.js";
import { ApiNodeCriteria, OrApiNodeCriteria } from "../criteria.js";
import { Expression } from "../expressions.js";
import { handleAndCriteria, handleNumericCriteria, handleOrCriteria, optimizeExpression } from "../search.js";
import { handleAndCriteria, handleComparisonCriteria, handleOrCriteria, optimizeExpression } from "../search.js";

export class ApiNodeFilter {
public static async getExpression(...criteria: OrApiNodeCriteria[]): Promise<Expression<ApiNode>> {
Expand All @@ -21,7 +21,7 @@ export class ApiNodeFilter {
case "version": {
return handleOrCriteria(criteria.version, async (c) =>
// @ts-ignore
handleNumericCriteria("version", c),
handleComparisonCriteria("version", c),
);
}
default: {
Expand Down
18 changes: 9 additions & 9 deletions packages/api-database/source/search/filters/block-filter.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { Block } from "../../models/block.js";
import { BlockCriteria, OrBlockCriteria } from "../criteria.js";
import { Expression } from "../expressions.js";
import { handleAndCriteria, handleNumericCriteria, handleOrCriteria, optimizeExpression } from "../search.js";
import { handleAndCriteria, handleComparisonCriteria, handleOrCriteria, optimizeExpression } from "../search.js";

export class BlockFilter {
public static async getExpression(...criteria: OrBlockCriteria[]): Promise<Expression<Block>> {
Expand All @@ -28,7 +28,7 @@ export class BlockFilter {
case "timestamp": {
return handleOrCriteria(criteria.timestamp, async (c) =>
// @ts-ignore
handleNumericCriteria("timestamp", c),
handleComparisonCriteria("timestamp", c),
);
}
case "previousBlock": {
Expand All @@ -41,45 +41,45 @@ export class BlockFilter {
case "height": {
return handleOrCriteria(criteria.height, async (c) =>
// @ts-ignore
handleNumericCriteria("height", c),
handleComparisonCriteria("height", c),
);
}

case "round": {
return handleOrCriteria(criteria.round, async (c) =>
// @ts-ignore
handleNumericCriteria("round", c),
handleComparisonCriteria("round", c),
);
}

case "numberOfTransactions": {
return handleOrCriteria(criteria.numberOfTransactions, async (c) =>
// @ts-ignore
handleNumericCriteria("numberOfTransactions", c),
handleComparisonCriteria("numberOfTransactions", c),
);
}
case "totalAmount": {
return handleOrCriteria(criteria.totalAmount, async (c) =>
// @ts-ignore
handleNumericCriteria("totalAmount", c),
handleComparisonCriteria("totalAmount", c),
);
}
case "totalFee": {
return handleOrCriteria(criteria.totalFee, async (c) =>
// @ts-ignore
handleNumericCriteria("totalFee", c),
handleComparisonCriteria("totalFee", c),
);
}
case "reward": {
return handleOrCriteria(criteria.reward, async (c) =>
// @ts-ignore
handleNumericCriteria("reward", c),
handleComparisonCriteria("reward", c),
);
}
case "payloadLength": {
return handleOrCriteria(criteria.payloadLength, async (c) =>
// @ts-ignore
handleNumericCriteria("payloadLength", c),
handleComparisonCriteria("payloadLength", c),
);
}
case "payloadHash": {
Expand Down
Loading

0 comments on commit 62f2428

Please sign in to comment.