[SYCL][E2E] Add functionality to split build and run of e2e tests #16016
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@uditagarwal97 @sarnex @aelovikov-intel Closed #15728 and reopened in this pr so that it is not a sycl-devops-pr and runs unnecessary checks. Going to leave this marked as draft until you guys review and are ok with the changes in format.py and lit.cfg.py, then mark as open so that it pings the tests owners after https://github.com/intel/llvm/actions/runs/11725963631 is the latest dry-run of build only |
but are marked as XFAIL in different ways
Added a run-line to one test as well
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds functionality to split the compilation and execution of e2e tests across separate machines.
This functionality is enabled via two new lit features
run-modeandbuild-mode. By default both of these are enabled and tests run just as they have before. We can call lit with the flag--param split-mode=with eitherbuildorrunto set these features. The featureunsplit-modeis added for convenience for when both build and run mode are set.When using
split-mode=buildonly thebuild-modefeature is added. In this mode all tests that can be built on the system will be built. When running a lit test in this mode two key things change:RUN:lines will be executed unless they contain%{run},%{run-unfiltered-devices}or%if run-mode.unsplit-modeorTEMPORARY_DISABLEDin a requires statement.When using
split-mode=runonly therun-modeis set. In this case tests are not built, they are only executed. Deciding whether a test is supported in this mode works the same way as when thesplit-modeparameter is not set. To only execute the tests we ignore all litRUN:lines unless they contain%{run},%{run-unfiltered-devices}or%if run-mode. Since we do not build the tests in this mode, for any test to pass we must have ran the tests in eitherbuildor non-split mode previously.Some notes/current limitations:
XFAILS: run-modeso that it does notXPASSwhen in build only mode.SYCL E2Eaction, selecting the "Linux, build" runner, and adding--param split-mode=buildtoLIT_OPTS. This pr does not have the needed CI changes to be able to use the run only mode.