Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor: Remove Cython - Reduce tech debt #346

Merged
merged 105 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from 85 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
ff0f2b2
uncythonize, pytest update
eddiebergman Dec 18, 2023
e915964
tests passing; fix add_conditions
eddiebergman Dec 19, 2023
2a29120
Fixup tests
eddiebergman Dec 19, 2023
79decbd
refactor: UniformFloat
eddiebergman Dec 19, 2023
add84b5
optim: UniformFloat get neighbors
eddiebergman Dec 19, 2023
a3f4297
refactor: deprecate UniformFloat is_legal_vector
eddiebergman Dec 19, 2023
60dc4db
big overhaul
eddiebergman Mar 6, 2024
279cac2
fix: depercation wraning of `np.integer` for dtype
eddiebergman Mar 6, 2024
3bc359f
refactor: Big internal rewrite, passing tests
eddiebergman Mar 12, 2024
b7f5d9d
test passing checkpoint
eddiebergman Mar 15, 2024
fb368c4
pythonize conditionals
eddiebergman Mar 16, 2024
e4064cf
optim: Topological sort and vectorize cond/forbiddens
eddiebergman Mar 25, 2024
5ebd21f
optim: Further small optimizations
eddiebergman Mar 25, 2024
a4916be
optim: Transpose sampling matrix for cache locality
eddiebergman Mar 26, 2024
4ab1c80
optim: Add test script for neighbor sampling
eddiebergman Mar 26, 2024
584b8ef
test: Repass tests
eddiebergman Mar 26, 2024
87eb45d
tmpcommmit - lets pray
eddiebergman Mar 26, 2024
fceffa6
refactor: Factor out DAG structure into dedicated class
eddiebergman Mar 27, 2024
ba1cb59
refactor: Move things out of `c_util`
eddiebergman Mar 27, 2024
26430fd
fix: Minimum spanning
eddiebergman Mar 27, 2024
5fb53f6
fix: min-cond-span sorted by min(depth of all children)
eddiebergman Mar 28, 2024
82e3fab
refactor: All tests and benchmarks pass
eddiebergman Apr 8, 2024
147e6ec
Merge branch 'main' into typing-cython-3
eddiebergman Apr 11, 2024
07687f2
refactor: Change deprecated test_util
eddiebergman Apr 8, 2024
99cd93b
Revert "refactor: Change deprecated test_util"
eddiebergman Apr 11, 2024
0df82ec
doc: Remove mention of cython
eddiebergman Apr 11, 2024
8da216d
build: Remove the use "build" in deps
eddiebergman Apr 11, 2024
55b9190
ci: Update test workflow
eddiebergman Apr 11, 2024
fb3fc42
ci: Remove conda tests since we no longer build w/ cython
eddiebergman Apr 11, 2024
68a0fbb
fix(typing): Use typing extension for future TypeAlias
eddiebergman Apr 11, 2024
a75708e
ci: Use caching for env across tests
eddiebergman Apr 11, 2024
ef96506
fix(typing): Use `Union` instead of 3.10 syntax
eddiebergman Apr 11, 2024
5af481f
ci: Remove git porcelain check
eddiebergman Apr 11, 2024
83cb54c
fix(typing): Use typing for inheritance to support <= 3.9
eddiebergman Apr 11, 2024
0a11215
fix: Remove `strict=True` from `zip` to support <= 3.10
eddiebergman Apr 11, 2024
b699216
fix(typing): Using `typing.Mapping` for inheritance
eddiebergman Apr 11, 2024
940c403
fix: Move benchmarked functions as main functions to use
eddiebergman Apr 11, 2024
f58a387
style: Update ruff to use numpy syntax
eddiebergman Apr 11, 2024
cb335b9
fix(functional): `linspace_chunked` acts more like `np.linspace`
eddiebergman Apr 11, 2024
7e2f97f
doc: Run `ruff --fix`
eddiebergman Apr 11, 2024
fab4b07
style: Fix ruff checks
eddiebergman Apr 11, 2024
f913bc2
refactor: Convert to `src` layout
eddiebergman Apr 11, 2024
cf2dd67
ci: Ignore use of `eval` in test
eddiebergman Apr 11, 2024
0bbd7a9
style: Fix remaning ruff errors
eddiebergman Apr 11, 2024
60791ab
refactor: Json encoding
eddiebergman Apr 15, 2024
2a7cbdd
refactor: Sorry, big dump
eddiebergman Apr 15, 2024
f9b0cf7
optim: Further speedups to one exchange
eddiebergman Apr 15, 2024
631a1e5
optim: Further speedups
eddiebergman Apr 16, 2024
4fb2787
fix: Hyperparameter attributes are simple types
eddiebergman Apr 16, 2024
9fe50d9
fix: Sampling from empty configuration spaces
eddiebergman Apr 16, 2024
79682ea
fix: Revert changes from different branch
eddiebergman Apr 16, 2024
ba65910
fix: Test for default_value type
eddiebergman Apr 16, 2024
574d355
fix: Deserialization with `"r"`
eddiebergman Apr 16, 2024
4ab87f3
fix: Conjunctions match clauses for equality #334
eddiebergman Apr 16, 2024
ac906a4
fix: Import TypeAlias from typing_extensions
eddiebergman Apr 16, 2024
d88c6bd
fix: Remove unreliable __hash__ from ConfigurationSpace
eddiebergman Apr 16, 2024
945df4c
fix: Ruff errors
eddiebergman Apr 19, 2024
693d64d
typing: Soft deprecate `pcs` and `pcs_new`
eddiebergman Apr 19, 2024
451766f
typing: Fix decoder type for json read and write
eddiebergman Apr 19, 2024
4515a8a
ci: Move to ruff for linting
eddiebergman Apr 19, 2024
3b54021
fix: Better condition cloning in replace hyperparameters
eddiebergman Apr 19, 2024
3fe243b
fix: Use 3.8 compatable isinstance checks
eddiebergman Apr 19, 2024
cb74fde
typing: `configuration_space.py`
eddiebergman Apr 19, 2024
470071b
typing: `dictionary.py`
eddiebergman Apr 19, 2024
77522df
typing: `_condition_tree.py`
eddiebergman Apr 19, 2024
222b097
typing: `forbidden.py`
eddiebergman Apr 19, 2024
5f65e5a
ci: Add `scipy.*` to list of mypy ignore missing import
eddiebergman Apr 19, 2024
636f0b6
typing: `conditions.py`
eddiebergman Apr 19, 2024
9c8a0c9
typing: Fix hyperparameter related issues
eddiebergman Apr 19, 2024
560ada3
test: Fixups
eddiebergman Apr 19, 2024
08b8cf9
refactor: Remove `nx` vendored lib
eddiebergman Apr 19, 2024
ad79b5c
test: Use new `add` as old was deprecated
eddiebergman Apr 19, 2024
4cdb5b3
refactor: Remove deprecations
eddiebergman Apr 19, 2024
9d66f56
fix: Use `isinstance(..., (type1, type2))`
eddiebergman Apr 19, 2024
d144026
fix: Better casting for reading from pcs
eddiebergman Apr 19, 2024
a4467f4
debug: Check windows issue of not legal
eddiebergman Apr 24, 2024
f773910
ci: Don't use cache for installation
eddiebergman Apr 24, 2024
215256a
fix: Change error type to IllegalVectorizedValue
eddiebergman Apr 24, 2024
01c68c6
debug: More debug prints for ci
eddiebergman Apr 24, 2024
e94825d
ci: Exclude 3.8 3.9 mac
eddiebergman Apr 24, 2024
05fbe4b
debug: More ci debugging statements
eddiebergman Apr 24, 2024
8a9d51f
fix: UnitScaler promotes to 64bit to prevent overflow on windows
eddiebergman Apr 24, 2024
f6d1343
fix: Remove non assigned type
eddiebergman Apr 26, 2024
dbaab3f
fix: Address minor comments
eddiebergman Apr 30, 2024
f84cbd4
fix: Minor review comments
eddiebergman Apr 30, 2024
5b65f45
doc: Overhaul to mkdocs, partially complete
eddiebergman Jun 20, 2024
669b108
doc: Fix missing comma
eddiebergman Jun 20, 2024
107204b
doc: Typo fixes
eddiebergman Jun 20, 2024
2f495d1
doc: typo fix again...
eddiebergman Jun 20, 2024
8a4ac0b
doc: Major doc overhaul to mkdocs
eddiebergman Jun 27, 2024
369803c
fix: Add backwards compatible `check_forbidden`
eddiebergman Jul 1, 2024
5152f9e
fix: Allow backwards compat `"default"` key in encoded spaces
eddiebergman Jul 1, 2024
cc5fd78
fix: Backwards compatible private methods
eddiebergman Jul 1, 2024
cf011bf
refactor: Use `check_` instead of `is_` as method raises
eddiebergman Jul 1, 2024
bc3afcd
doc: Forbiddens
eddiebergman Jul 1, 2024
2a48d08
test: fix test funcion name
eddiebergman Jul 1, 2024
995c417
fix(Forbidden): Uncomment value checking on `ForbiddenEqualsClaus`
eddiebergman Jul 1, 2024
09beefd
ci: Update doc and test workflows
eddiebergman Jul 1, 2024
debe675
test: Remove stray function
eddiebergman Jul 12, 2024
897bec7
doc: Type fix
eddiebergman Jul 12, 2024
b1f1fb9
doc: Add note to `_unsafe` code in `hp_components`
eddiebergman Jul 12, 2024
80e2c7c
doc: Typo fix
eddiebergman Jul 12, 2024
d23879d
doc: Improve code doc attribute clarity
eddiebergman Jul 12, 2024
de0b36e
doc: Change `note` to `todo`
eddiebergman Jul 12, 2024
82d66fd
doc: Fix documentation for `continuous_neighborhood`
eddiebergman Jul 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 12 additions & 144 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
name: Tests
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

on:
# Allow to manually trigger through github API
Expand All @@ -8,13 +11,11 @@ on:
push:
branches:
- main
- development

# Triggers with push to a pr aimed at master
pull_request:
branches:
- main
- development

schedule:
# Every day at 7AM UTC
Expand All @@ -31,16 +32,6 @@ env:
--durations=20
-v

# Version to run code-cov on
# NOTE: These are only acessible inside a jobs **steps** and not in the job setup,
# Hence, some of these varialbes are copied and are just here for reference
#
code-cov-active: true # Copied in job setup
code-cov-os: ubuntu-latest # Copied in job setup
code-cov-python-version: "3.8"
code-cov-args: >-
--cov=ConfigSpace
--cov-report=xml

jobs:

Expand All @@ -56,15 +47,21 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
exclude:
# These don't seem to be available anymore with setup-python?
- python-version: "3.8"
os: "macos-latest"
- python-version: "3.9"
os: "macos-latest"

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

Expand All @@ -73,136 +70,7 @@ jobs:
python -m pip install --upgrade pip
python -m pip install -e ".${{ env.extra-requires }}"

- name: Store git status
id: status-before
shell: bash
run: |
echo "::set-output name=BEFORE::$(git status --porcelain -b)"

- name: Tests
timeout-minutes: 45
run: |
pytest ${{ env.pytest-args }} ${{ env.test-dir }}

- name: Check for files left behind by test
run: |
before="${{ steps.status-before.outputs.BEFORE }}"
mfeurer marked this conversation as resolved.
Show resolved Hide resolved
after="$(git status --porcelain -b)"
if [[ "$before" != "$after" ]]; then
echo "git status from before: $before"
echo "git status from after: $after"
echo "Not all generated files have been deleted!"
exit 1
fi

# Testing with conda
conda-tests:
mfeurer marked this conversation as resolved.
Show resolved Hide resolved
name: conda-${{ matrix.python-version }}-${{ matrix.os }}
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash -l {0} # Default to using bash on all and load (-l) .bashrc which miniconda uses

strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
os: ["ubuntu-latest", "macos-latest", "windows-latest"]

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Conda install
uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
python-version: ${{ matrix.python-version }}

- name: Install ${{ env.package-name }}
run: |
python -V
python -m pip install --upgrade pip
python -m pip install wheel
python -m pip install -e ".${{ env.extra-requires }}"

- name: Tests
timeout-minutes: 45
run: |
pytest ${{ env.pytest-args }} ${{ env.test-dir }}
#
# Testing a dist install
dist-test:
name: dist-${{ matrix.python-version }}-${{ matrix.os }}

runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash

strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
os: ["ubuntu-latest", "macos-latest", "windows-latest"]

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Create sdist
id: sdist
run: |
python -m pip install --upgrade pip build
python -m build --sdist
echo "sdist_name=$(ls -t dist/${{ env.package-name }}-*.tar.gz | head -n 1)" >> "$GITHUB_ENV"

- name: Install ${{ env.package-name }}
run: |
python -m pip install ${{ env.sdist_name }}${{ env.extra-requires }}

- name: Tests
timeout-minutes: 45
run: |
pytest ${{ env.pytest-args }} ${{ env.test-dir }}


# Testing with codecov coverage uploaded
codecov-test:
name: codecov-test

runs-on: "ubuntu-latest"
defaults:
run:
shell: bash

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ env.code-cov-python-version }}

- name: Install ${{ env.package-name }}
run: |
python -m pip install --upgrade pip
python -m pip install wheel
python -m pip install -e ".${{ env.extra-requires }}"

- name: Tests
timeout-minutes: 45
run: |
pytest ${{ env.pytest-args }} ${{ env.code-cov-args }} ${{ env.test-dir }}

- name: Upload coverage
uses: codecov/codecov-action@v2
with:
fail_ci_if_error: true
verbose: true
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ docs/examples/*
# C extensions
*.so
*.c
*.jukit

# Packages
*.egg
Expand Down Expand Up @@ -74,4 +75,5 @@ prof/
.venv

# Running pre-commit seems to generate these
.mypy_cache
.mypy_cache
*.speedscope
27 changes: 5 additions & 22 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,32 +38,15 @@ repos:
hooks:
- id: mypy
files: '^ConfigSpace/.*\.py$'
additional_dependencies:
- "types-pyyaml"
args:
- "--no-warn-return-any" # Disable this because it doesn't know about 3rd party imports
- "--ignore-missing-imports"
- "--show-traceback"
- id: mypy
mfeurer marked this conversation as resolved.
Show resolved Hide resolved
files: '^test/.*\.py$'
args:
- "--no-warn-return-any" # Disable this because it doesn't know about 3rd party imports
- "--ignore-missing-imports"
- "--show-traceback"
- "--disable-error-code"
- "no-untyped-def"

- repo: https://github.com/psf/black
rev: 23.3.0
hooks:
- id: black
args: ["--config=pyproject.toml"]
- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.0.263
rev: v0.4.0
hooks:
- id: ruff
args: ["--fix", "ConfigSpace", "test"]
- repo: https://github.com/MarcoGorelli/cython-lint
rev: v0.15.0
hooks:
- id: cython-lint
args: [--ignore=E501]
- id: double-quote-cython-strings
args: [--fix, --exit-non-zero-on-fix, --no-cache]
- id: ruff-format
Loading
Loading