Skip to content

Commit 1be1105

Browse files
smackeseyclairelin135
authored andcommitted
Skip all but default python version on docs_snippets, automation tests (#25467)
## Summary & Motivation The `docs_snippets`, `docs_snippets_beta`, and `automation` packages all install large numbers of dagster packages in the same environment. This can sometimes cause second-order dependency collisions, particularly in older python versions. We reduce the likelihood of this occurring by only testing these packages against the default python version. - In the case of `automation`, this is not a problem because `automation` code only runs internally. - For `docs_snippets` and `docs_snippets_beta`, the loss in coverage is minimal since all the dagster code actually targeted by our snippets is tested on all (compatible) versions. ## How I Tested These Changes Existing test suite, with `test-all` to make sure the full suite (all specified versions) are tested in the branch build.
1 parent af04302 commit 1be1105

File tree

2 files changed

+20
-7
lines changed

2 files changed

+20
-7
lines changed

.buildkite/dagster-buildkite/dagster_buildkite/python_version.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,12 @@ def get_all(cls) -> List["AvailablePythonVersion"]:
2121
def get_default(cls) -> "AvailablePythonVersion":
2222
return cls["V3_11"]
2323

24+
# Useful for providing to `PackageSpec.unsupported_python_versions` when you only want to test
25+
# the default version.
26+
@classmethod
27+
def get_all_except_default(cls) -> List["AvailablePythonVersion"]:
28+
return [v for v in cls.get_all() if v != cls.get_default()]
29+
2430
@classmethod
2531
def get_pytest_defaults(cls) -> List["AvailablePythonVersion"]:
2632
branch_name = safe_getenv("BUILDKITE_BRANCH")

.buildkite/dagster-buildkite/dagster_buildkite/steps/packages.py

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -288,15 +288,19 @@ def k8s_extra_cmds(version: str, _) -> List[str]:
288288
PackageSpec(
289289
"examples/docs_snippets",
290290
pytest_extra_cmds=docs_snippets_extra_cmds,
291-
unsupported_python_versions=[
292-
# dependency on 3.9-incompatible extension libs
293-
AvailablePythonVersion.V3_9,
294-
# dagster-airflow dep
295-
AvailablePythonVersion.V3_12,
296-
],
291+
# The docs_snippets test suite also installs a ton of packages in the same environment,
292+
# which is liable to cause dependency collisions. It's not necessary to test all these
293+
# snippets in all python versions since we are testing the core code exercised by the
294+
# snippets against all supported python versions.
295+
unsupported_python_versions=AvailablePythonVersion.get_all_except_default(),
297296
),
298297
PackageSpec(
299298
"examples/docs_beta_snippets",
299+
# The docs_snippets test suite also installs a ton of packages in the same environment,
300+
# which is liable to cause dependency collisions. It's not necessary to test all these
301+
# snippets in all python versions since we are testing the core code exercised by the
302+
# snippets against all supported python versions.
303+
unsupported_python_versions=AvailablePythonVersion.get_all_except_default(),
300304
pytest_tox_factors=["all", "integrations"],
301305
),
302306
PackageSpec(
@@ -418,7 +422,10 @@ def tox_factors_for_folder(tests_folder_name: str) -> List[str]:
418422
LIBRARY_PACKAGES_WITH_CUSTOM_CONFIG: List[PackageSpec] = [
419423
PackageSpec(
420424
"python_modules/automation",
421-
unsupported_python_versions=[AvailablePythonVersion.V3_12],
425+
# automation is internal code that doesn't need to be tested in every python version. The
426+
# test suite also installs a ton of packages in the same environment, which is liable to
427+
# cause dependency collisions.
428+
unsupported_python_versions=AvailablePythonVersion.get_all_except_default(),
422429
),
423430
PackageSpec("python_modules/dagster-webserver", pytest_extra_cmds=ui_extra_cmds),
424431
PackageSpec(

0 commit comments

Comments
 (0)