Skip to content

Commit

Permalink
feat[test]: parametrize CI over EVM versions (vyperlang#3842)
Browse files Browse the repository at this point in the history
this commit parametrizes the github actions rules over evm versions. it
uses the github actions matrix syntax, but because github actions
apparently does not allow additional matrices in the `include` rule, the
additional rules must be written out manually.

misc:
- fix for py-evm logger name
- refactor tests which were previously manually parametrized over evm
  versions
- remove the tox environment for running tests in favor of the gh
  actions script
  • Loading branch information
charles-cooper authored Mar 15, 2024
1 parent a9ee641 commit 73925d2
Show file tree
Hide file tree
Showing 11 changed files with 118 additions and 118 deletions.
50 changes: 41 additions & 9 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,17 +62,42 @@ jobs:
strategy:
matrix:
python-version: [["3.11", "311"]]
# run in modes: --optimize [gas, none, codesize]
opt-mode: ["gas", "none", "codesize"]
evm-version: [shanghai]
debug: [true, false]
# run across other python versions.# we don't really need to run all
# modes across all python versions - one is enough

# https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#expanding-or-adding-matrix-configurations
include:
# test default settings with 3.11 across all supported evm versions
- python-version: ["3.11", "311"]
debug: false
opt-mode: gas
evm-version: london
- python-version: ["3.11", "311"]
debug: false
opt-mode: gas
evm-version: paris
- python-version: ["3.11", "311"]
debug: false
opt-mode: gas
evm-version: shanghai
# enable when py-evm makes it work:
#- python-version: ["3.11", "311"]
# debug: false
# opt-mode: gas
# evm-version: cancun

# run across other python versions. we don't really need to run all
# modes across all python versions - one is enough
- python-version: ["3.10", "310"]
opt-mode: gas
debug: false
evm-version: shanghai

# TODO 3.12 doesn't work yet, investigate - may be hypothesis issue
#- python-version: ["3.12", "312"]

name: py${{ matrix.python-version[1] }}-opt-${{ matrix.opt-mode }}${{ matrix.debug && '-debug' || '' }}
name: py${{ matrix.python-version[1] }}-opt-${{ matrix.opt-mode }}${{ matrix.debug && '-debug' || '' }}-${{ matrix.evm-version }}

steps:
- uses: actions/checkout@v4
Expand All @@ -86,11 +111,18 @@ jobs:
python-version: ${{ matrix.python-version[0] }}
cache: "pip"

- name: Install Tox
run: pip install tox

- name: Run Tox
run: TOXENV=py${{ matrix.python-version[1] }} tox -r -- --optimize ${{ matrix.opt-mode }} ${{ matrix.debug && '--enable-compiler-debug-mode' || '' }} -r aR tests/
- name: Install dependencies
run: pip install .[test]

- name: Run tests
run: |
pytest \
-m "not fuzzing" \
--optimize ${{ matrix.opt-mode }} \
--evm-version ${{ matrix.evm-version }} \
${{ matrix.debug && '--enable-compiler-debug-mode' || '' }} \
--showlocals -r aR \
tests/
- name: Upload Coverage
uses: codecov/codecov-action@v4
Expand Down
24 changes: 21 additions & 3 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from web3.contract import Contract
from web3.providers.eth_tester import EthereumTesterProvider

import vyper.evm.opcodes as evm
from tests.utils import working_directory
from vyper import compiler
from vyper.ast.grammar import parse_vyper_source
Expand All @@ -38,7 +39,7 @@


def set_evm_verbose_logging():
logger = logging.getLogger("eth.vm.computation.Computation")
logger = logging.getLogger("eth.vm.computation.BaseComputation")
setup_DEBUG2_logging()
logger.setLevel("DEBUG2")

Expand All @@ -59,6 +60,13 @@ def pytest_addoption(parser):
)
parser.addoption("--enable-compiler-debug-mode", action="store_true")

parser.addoption(
"--evm-version",
choices=list(evm.EVM_VERSIONS.keys()),
default="shanghai",
help="set evm version",
)


@pytest.fixture(scope="module")
def output_formats():
Expand All @@ -81,6 +89,18 @@ def debug(pytestconfig):
_set_debug_mode(debug)


@pytest.fixture(scope="session", autouse=True)
def evm_version(pytestconfig):
# note: we configure the evm version that we emit code for,
# but eth-tester is only configured with the latest mainnet
# version.
evm_version_str = pytestconfig.getoption("evm_version")
evm.DEFAULT_EVM_VERSION = evm_version_str
# this should get overridden by anchor_evm_version,
# but set it anyway
evm.active_evm_version = evm.EVM_VERSIONS[evm_version_str]


@pytest.fixture
def chdir_tmp_path(tmp_path):
# this is useful for when you want imports to have relpaths
Expand Down Expand Up @@ -308,7 +328,6 @@ def _get_contract(
**kwargs,
):
settings = Settings()
settings.evm_version = kwargs.pop("evm_version", None)
settings.optimize = override_opt_level or optimize
out = compiler.compile_code(
source_code,
Expand Down Expand Up @@ -383,7 +402,6 @@ def _deploy_blueprint_for(
w3, source_code, optimize, output_formats, initcode_prefix=ERC5202_PREFIX, **kwargs
):
settings = Settings()
settings.evm_version = kwargs.pop("evm_version", None)
settings.optimize = optimize
out = compiler.compile_code(
source_code,
Expand Down
Loading

0 comments on commit 73925d2

Please sign in to comment.