Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NFFL FastNear indexer: CI test #322

Merged
merged 46 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
a333b92
FastNEAR indexer, by @firatsertgoz
Fly-Style Nov 22, 2024
b60eb74
Merge branch 'main' into fastnear-indexer
Fly-Style Nov 22, 2024
b5d38be
Finish fastnear setup for indexer
Fly-Style Nov 25, 2024
bd7a4ab
Update nearcore to v2.4.0-rc1
Fly-Style Nov 20, 2024
9a78477
Correct docker build
Fly-Style Nov 25, 2024
18ec0c2
Rename for better recognition
Fly-Style Nov 25, 2024
3a5c329
Setup compose file for fast Near indexer
Fly-Style Nov 25, 2024
3997ea4
Clean DCE'd code
Fly-Style Nov 25, 2024
f922c05
Update forgotten test dependency
Fly-Style Nov 25, 2024
6719a49
Fix typo and allow dead code temporarily
Fly-Style Nov 25, 2024
822ae74
Enhance docker compose liveness test
Fly-Style Nov 25, 2024
73a78a8
Mark 'unused' imports
Fly-Style Nov 25, 2024
ab574d4
Even more test enhancements
Fly-Style Nov 25, 2024
598dbb8
Revert upgrade to latest nearcore
Fly-Style Nov 25, 2024
077014b
Introduce CI action for FastNear
Fly-Style Nov 25, 2024
3ae6ec8
Separate default and fastnear indexers
Fly-Style Nov 25, 2024
499f325
Please work
Fly-Style Nov 25, 2024
342d8f1
Please work-2
Fly-Style Nov 25, 2024
0d09358
Update integration_test.go
Fly-Style Nov 25, 2024
a0d7873
Please work-3
Fly-Style Nov 26, 2024
8dfca70
Remove healthcheck from fast indexer docker image, upgrade version wh…
Fly-Style Nov 26, 2024
3720235
Please work-4
Fly-Style Nov 26, 2024
bed8ef2
Update compose-fastnear-test.yml
Fly-Style Nov 26, 2024
24c2f25
Try now?
Fly-Style Nov 26, 2024
1dcadcd
Modify healthcheck
Fly-Style Nov 26, 2024
73a6faa
Tweak the code
Fly-Style Nov 26, 2024
62e7a99
Revert code changes
Fly-Style Nov 26, 2024
fc0d6cd
Try to revert?
Fly-Style Nov 26, 2024
50ba61b
Merge
Fly-Style Nov 27, 2024
2a795e7
WIP
Fly-Style Nov 27, 2024
53c94df
Try to test on CI
Fly-Style Nov 27, 2024
2fa94e8
Fix fastnear docker compose
Fly-Style Nov 27, 2024
dd9b90c
Fix fastnear docker compose-2
Fly-Style Nov 27, 2024
59af8d1
Self-review, AI review comments and temporarily remove CI & IT test
Fly-Style Nov 28, 2024
f6ffc25
Unit tests
Fly-Style Nov 28, 2024
05bda85
Fix CI test
Fly-Style Nov 28, 2024
287f2a8
WIP
Fly-Style Nov 28, 2024
3f89150
Merge branch 'fastnear-indexer' into fastnear-indexer-it
Fly-Style Nov 28, 2024
a75e538
Use correct approach for neard initialization
Fly-Style Nov 30, 2024
e671faf
Merge branch 'main' into fastnear-indexer-it
Fly-Style Nov 30, 2024
bff2158
Initialize nearcore correctly via environmental variables, remove obs…
Fly-Style Nov 30, 2024
664f546
Have a correct CI test and prepare integration test
Fly-Style Dec 2, 2024
b7185a1
Update ctr_availability_test.rs
Fly-Style Dec 2, 2024
fd85b5b
Update .github/workflows/compose-fastnear-test.yml
Fly-Style Dec 2, 2024
fa06d3a
Update Makefile
Fly-Style Dec 2, 2024
5b1f77b
Revert IT tests
Fly-Style Dec 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions .github/workflows/compose-fastnear-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
name: Docker Compose Fastnear Liveness Test

on:
push:
branches:
- main
pull_request:
paths:
- 'indexer/**'

jobs:
run-docker-compose-with-fastnear:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Set up Docker Compose
run: |
sudo apt-get update
sudo apt-get install docker-compose -y

Fly-Style marked this conversation as resolved.
Show resolved Hide resolved
- name: Build fastnear indexer container
uses: docker/build-push-action@v5
with:
load: true
context: .
file: ./indexer/FastIndexer.dockerfile
tags: nffl-fast-indexer
cache-from: type=gha,scope=indexer
cache-to: type=gha,mode=max,scope=indexer
build-args: |
- TARGET="debug"
- COMPILATION_MODE=""
Fly-Style marked this conversation as resolved.
Show resolved Hide resolved

- name: Build relayer container
uses: docker/build-push-action@v5
with:
load: true
context: .
file: ./relayer/cmd/Dockerfile
tags: nffl-test-relayer
cache-from: type=gha,scope=relayer
cache-to: type=gha,mode=max,scope=relayer

- name: Build aggregator container
uses: docker/build-push-action@v5
with:
load: true
context: .
file: ./aggregator/cmd/Dockerfile
tags: nffl-aggregator
cache-from: type=gha,scope=aggregator
cache-to: type=gha,mode=max,scope=aggregator

- name: Build operator container
uses: docker/build-push-action@v5
with:
load: true
context: .
file: ./operator/cmd/Dockerfile
tags: nffl-operator
cache-from: type=gha,scope=operator
cache-to: type=gha,mode=max,scope=operator

- name: Build and start Docker Compose services
run: docker-compose -f ./docker-compose-fastnear.yml up --build -d

- name: Wait for a minute
run: sleep 60

- name: List running containers
run: |
echo "Running containers:"
docker ps --format "{{.Names}}"

- name: Ensure containers are running
run: |
containers=(
"nffl_operator1-health_1"
"nffl-operator1"
"nffl_operator0-health_1"
"nffl-operator0"
"rollup1-relayer"
"rollup0-relayer"
"nffl-indexer"
"nffl-aggregator"
"nffl_aggregator-health_1"
"mainnet-anvil"
"rollup0-anvil"
"rmq"
"rollup1-anvil"
) # List of container names
for container in "${containers[@]}"; do
if docker inspect -f '{{.State.Running}}' "$container" >/dev/null 2>&1; then
echo "$container is running"
else
echo "$container is not running"
exit 1
fi
done


- name: Check health status
run: |
services=(
"nffl_operator1-health_1"
"nffl_operator0-health_1"
"nffl_aggregator-health_1"
"rollup1-anvil"
"rollup0-anvil"
"mainnet-anvil"
"rmq"
) # List of services names
for service in "${services[@]}"; do
status=$(docker inspect -f '{{.State.Health.Status}}' $service)
if [ "$status" != "healthy" ]; then
echo "Service $service is not healthy"
docker-compose -f ./docker-compose-fastnear.yml logs $service
exit 1
else
echo "Service $service is healthy"
fi
done
Fly-Style marked this conversation as resolved.
Show resolved Hide resolved

- name: Tear down Docker Compose services
run: docker-compose -f ./docker-compose-fastnear.yml down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -143,8 +143,8 @@ near-da-rpc-sys:
tests-integration: ## runs all integration tests
go test ./tests/integration/integration_test.go -v -count=1
#go test ./tests/integration/registration_test.go -v -race -count=1
#test-integration-fastnear: ## runs fastnear integration test
# go test ./tests/integration/fastnear_test.go -v -count=1
# test-integration-fastnear: ## runs fastnear integration test
# go test ./tests/integration/fastnear/fastnear_integration_test.go -v -count=1

## runs slither for solidity files
## You can install Slither by following the guide at https://github.com/crytic/slither/tree/master?tab=readme-ov-file#how-to-install
Expand Down
6 changes: 6 additions & 0 deletions config-files/relayer1-docker-compose-fastnear.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
production: false
rpc_url: ws://rollup1-anvil:8547
da_account_id: da3.test.near
key_path: /root/.near-credentials/localnet/da3.test.near.json
network: http://nearcore:3030
metrics_ip_port_address: rollup1-relayer:9091
Loading