From 65659aca8a6acb02035b0b863619dcb1374c9291 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Mon, 3 Feb 2025 09:56:52 -0800 Subject: [PATCH 1/9] Document changes to `build-only` --- sycl/test-e2e/README.md | 58 +++++++++++++++++++++++------------------ 1 file changed, 33 insertions(+), 25 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index b60d4b1940e4a..2e07b822b9a51 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -408,20 +408,37 @@ The modes work as follow: * `--param test-mode=build-only` - This mode can be used to compile all test binaries. To do this all - `UNSUPPORTED` and `REQUIRES` statements are ignored unless they contain - `UNSUPPORTED: true` or `REQUIRES: build-and-run-mode`. All `RUN:` lines within - a test are ran in this mode unless they contain the following expansions: - `%{run}`, `%{run-unfiltered-devices}`, or `%if run-mode`. - - Currently, the only triple supported for `build-only` mode is `spir64`. - - * `build-only` future work. - - Note, the fact that `build-only` ignores general `UNSUPPORTED`/`REQUIRES` - statements is a current limitation. The logic for taking into account the - features that affect compilation, and ignoring those that are only relevant - to the execution of the program is currently being worked on. + This mode can be used to compile all test binaries that can be built on the + system. To do this the `REQUIRES`/`UNSUPPORTED` statements are handled + differently to accommodate for the fact that in `build-only` mode we do not + have any devices, and as a result no device-specific features. Instead of + considering these features as missing, we assign a third "unknown" boolean + value to them. When evaluating An expressions it will result in an unknown + value if its result could be changed by setting the unknown features to either + true or false. i.e., `false || unknown = unknown` but `true || unknown = true`. + If an expression's final value is unknown we consider it to have met the + requirements. The list of features that are not unknown in `build-only` is + found in the `E2EExpr.py` file. + + The triples to build for in this mode are selected by evaluating the + `REQUIRES`/`UNSUPPORTED` statements with a build-target feature added to the + list of available features. The triple for each corresponding passing + build-target will be built for. If no build-targets fulfilled the requirements + the test is reported as unsupported. + + The set of build-targets that can be selected for tests is determined via the + `sycl_build_targets` parameter. Valid build targets are: `spir`,`nvidia`, + `amd`, `native_cpu`. Each build target should be separated with a semicolon. + This parameter is set to just `spir` by default. + + Note: the `target-*` features should be used in place of the backend features + if the intent is to mark something as requiring, or expectedly failing at + the compilation stage. On the other hand, backend features should be used when + we are considering the run stage. + + When executing the test in `build-only`, all `RUN:` lines are ran unless they + contain the following expansions: `%{run}`, `%{run-unfiltered-devices}`, or + `%if run-mode`. * `--param test-mode=run-only` @@ -449,8 +466,8 @@ system. execution, need to be marked as `XFAIL` with a feature that is device agnostic, or with `XFAIL: *`. Device agnostic features are those which are added added through a method other than processing the output of sycl-ls, for - example the OS, or the presence of a library. This needs to be done because - sycl-ls is not ran in `build-only` mode. + example the OS, a build target, or a library. This needs to be done because + in `build-only` we have no device-specific features. * If the expected failure occurs during run-time we will need to mark the test with `XFAIL` on a device specific feature (A feature that we add through processing sycl-ls output), or if its expected to always fail on run-time we @@ -474,15 +491,6 @@ otherwise it is classified as a build line. situation usually appears when we need to run a command in response to the execution of the test binary. -* Currently the `build-only` mode does not support logic to properly assess the -features in `REQUIRES`/`UNSUPPORTED` to know if a test can be built in the -system environment, or for `spir64`. Only tests that are marked with `REQUIRES: -build-and-run-mode` or `UNSUPPORTED: true` are skipped. Thus if a test will fail -building for the build environment we have on CI or for `spir64` we will need to -mark this as `REQUIRES: build-and-run-mode`. This is only temporary solution, -until further work is done to properly mark tests as unsupported on `build-only` -based on features. - * CPU and FPGA AOT tests are currently expected to fail when compiling and executing on separate machines. These failures occur on the `run-only` side, because during compilation the host machine's CPU architecture is targeted, From 819500a893c4d1d5d95f55984a7274bec1266112 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Thu, 20 Feb 2025 11:00:46 -0800 Subject: [PATCH 2/9] Update README --- sycl/test-e2e/README.md | 137 ++++++++++++++-------------------------- 1 file changed, 48 insertions(+), 89 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index 6440447e41e96..7625fe7bafb5f 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -385,7 +385,6 @@ project must keep using `` provided by the SYCL2020 specification.** ### Compiling and executing tests on separate systems - The execution of e2e tests can be separated into compilation and execution stages via the `test-mode` lit parameter. This allows us to reduce testing time by compiling tests on more powerful systems and reusing the binaries on other @@ -393,25 +392,25 @@ machines. By default the `test-mode` parameter is set to `full`, indicating that both stages will run. This parameter can be set to `build-only`, or `run-only`, to only run the compilation stage, or the execution stage respectively. -**NOTE:** This feature is a work-in-progress and current limitations are -expected to be addressed in the near future. - -The modes work as follow: - -* `--param test-mode=full` - - This is the default mode tests run in. Tests are marked as unsupported if no - device on the machine can fulfill the `REQUIRES`/`UNSUPPORTED` statements. In - this mode all `RUN:` lines are executed normally, and two extra features are - added: the `build-and-run-mode` and `run-mode`. +* `--param test-mode=run-only` - To make a test only run in `full` mode add a `REQUIRES: build-and-run-mode` - line. + In this mode, tests will not be compiled, they will only run. To do this only + the `RUN:` lines that contain a "run" expansion will be executed (`%{run}`, + `%{run-unfiltered-devices}`, or `%{run-aux}`). Since tests are not compiled in + this mode, for any test to pass the test binaries should already be in the + `test_exec_root` directory, either by having ran `full` or `build-only` modes + previously on the system, or having transferred the test binaries into that + directory. To mark a test as expected to fail only at run-time we can either + use the `run-mode` feature, which is only available in `run-only` and `full` + modes, or we can mark the test as `XFAIL` for a device-specific feature. + + `%{run-aux}` is an empty expansion and executes a line as is, without + expanding for each selected device and without using the run_launcher. * `--param test-mode=build-only` This mode can be used to compile all test binaries that can be built on the - system. To do this the `REQUIRES`/`UNSUPPORTED` statements are handled + system. To do this `REQUIRES`, and `UNSUPPORTED` statements are handled differently to accommodate for the fact that in `build-only` mode we do not have any devices, and as a result no device-specific features. Instead of considering these features as missing, we assign a third "unknown" boolean @@ -419,39 +418,21 @@ The modes work as follow: value if its result could be changed by setting the unknown features to either true or false. i.e., `false || unknown = unknown` but `true || unknown = true`. If an expression's final value is unknown we consider it to have met the - requirements. The list of features that are not unknown in `build-only` is - found in the `E2EExpr.py` file. - - The triples to build for in this mode are selected by evaluating the - `REQUIRES`/`UNSUPPORTED` statements with a build-target feature added to the - list of available features. The triple for each corresponding passing - build-target will be built for. If no build-targets fulfilled the requirements - the test is reported as unsupported. - - The set of build-targets that can be selected for tests is determined via the - `sycl_build_targets` parameter. Valid build targets are: `spir`,`nvidia`, - `amd`, `native_cpu`. Each build target should be separated with a semicolon. - This parameter is set to just `spir` by default. - - Note: the `target-*` features should be used in place of the backend features - if the intent is to mark something as requiring, or expectedly failing at - the compilation stage. On the other hand, backend features should be used when - we are considering the run stage. - - When executing the test in `build-only`, all `RUN:` lines are ran unless they - contain the following expansions: `%{run}`, `%{run-unfiltered-devices}`, or - `%if run-mode`. - -* `--param test-mode=run-only` - - In this mode, tests will not be compiled, they will only run. To do this only - the `RUN:` lines that contain `%{run}`, `%{run-unfiltered-devices}` or `%if - run-mode` are executed. Tests are marked as unsupported in the same manner as - `full` mode. Since tests are not compiled in this mode, for any test to pass - the test binaries should already be in the `test_exec_root` directory, either - by having ran `full` or `build-only` modes previously on the system, or having - transferred the test binaries into that directory. The `run-mode` feature is - added when in this mode. + requirements. The list of device-agnostic features that are not considered + unknown in `build-only` is found in the `E2EExpr.py`. + + The triples to compile for in this mode are set via the `sycl_build_targets` + lit parameter. Valid build targets are: `spir`,`nvidia`, `amd`, `native_cpu`. + These correspond to `spir64`, `nvptx64-nvidia-cuda`, `amdgcn-amd-amdhsa`, and + `native_cpu` triples respectively. Each build target should be separated with + a semicolon. This parameter is set to just `spir` by default. A test can be + marked as requiring, or not supporting a particular triple via the `target-*` + features. Note that we test if each build target is supported separately from + each other so statements like `REQUIRES: target-spir && target-nvidia` will + never be met. Instead we can use `any-target-is-*` features in this case. + + When executing the test in `build-only`, all `RUN:` lines that do not have a + run expansion will execute. #### Resolving common Issues with separate compilation and execution @@ -465,45 +446,23 @@ system. * Two scenarios need to be considered for tests that expectedly fail: * Tests that are expected to fail on compilation, and thus also during - execution, need to be marked as `XFAIL` with a feature that is device - agnostic, or with `XFAIL: *`. Device agnostic features are those which are - added added through a method other than processing the output of sycl-ls, for - example the OS, a build target, or a library. This needs to be done because - in `build-only` we have no device-specific features. + execution, need to be marked as `XFAIL` for a device-agnostic feature, or + with `XFAIL: *`. This is due to the fact that there are no devices in + `build-only` mode. For example if a test cannot compile for a triple, then it + should be marked as `XFAIL` for the corresponding build target feature, rather + than a backend feature. * If the expected failure occurs during run-time we will need to mark the test - with `XFAIL` on a device specific feature (A feature that we add through - processing sycl-ls output), or if its expected to always fail on run-time we - can use `XFAIL: run-mode`. This is because otherwise the test would compile - and pass on `build-only` mode and be reported as an `XPASS`. - -* To separate compilation and execution of tests, we classify `RUN:` directives -as being either build or run lines. If a line contains `%{run}`, -`%{run-unfiltered-devices}` or `%if run-mode` it is classified as a run line, -otherwise it is classified as a build line. - * All `RUN:` lines that execute test binaries should be marked with either - `%{run}` or `%{run-unfiltered-devices}`. Otherwise they will be incorrectly - marked as a build line, likely causing a failure at the `build-only` stage as - we try to execute the program without having the appropriate devices. - * The vast majority of `RUN:` lines that do not execute the test binaries are - needed to either set up files prior to compilation, or to compile the binary, - as such `RUN:` lines are by default considered as build lines. In the case - that we need to run a line on the `run-only` system, and it does not make - sense to mark them with `%{run}` or `%{run-unfiltered-devices}`, we can mark a - line with `%if run-mode` to specifically make the line a run line. This - situation usually appears when we need to run a command in response to the - execution of the test binary. - -* CPU and FPGA AOT tests are currently expected to fail when compiling and -executing on separate machines. These failures occur on the `run-only` side, -because during compilation the host machine's CPU architecture is targeted, -which may be different than that of the running machine. These tests are marked -as `REQUIRES: build-and-run-mode` as a result, until they can be refactored to -compile for the architectures that will be used on the run side. - -#### Falling back to `full` testing mode on `run-only` - -To not lose coverage of tests marked as `REQUIRES: build-and-run-mode` when -using `run-only` mode, lit can be called using -`--param fallback-to-build-if-requires-build-and-run=True`. When this option is -enabled in `run-only` mode, tests marked as requiring `build-and-run-mode` will -fallback to running on `full` mode, instead of being reported as unsupported. + with `XFAIL` on a device specific feature. If it is expected to fail for any + device at run-time we can use `XFAIL: run-mode`, this is because otherwise the + test would compile and pass on `build-only` mode and be reported as an `XPASS`. + +* To separate compilation and execution of tests, `RUN:` lines are filtered in +`build-only` and `run-only` mode based on the presence of "run" expansions. + * Any line that is meant to execute the test binary should be marked with + `%{run}` or `%{run-unfiltered-devices}` so that it is not ran in `build-only`. + * The `%{run-aux}` expansion can be used if a `RUN:` line that does not + execute a test binary needs to be ran in `run-only`. + +* Compiling for CPU AOT will target the ISA of the host CPU, thus compiling +these tests on a different system will lead to failures if the build system and +run system support different ISAs. From 363bc58d4bb1308d008a3a7746957f136290cd21 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Fri, 21 Feb 2025 11:25:55 -0800 Subject: [PATCH 3/9] Mention `build-mode` feature --- sycl/test-e2e/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index 7625fe7bafb5f..5b2d6ea5b1307 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -434,6 +434,8 @@ to only run the compilation stage, or the execution stage respectively. When executing the test in `build-only`, all `RUN:` lines that do not have a run expansion will execute. + The `build-mode` feature is added when in this mode. + #### Resolving common Issues with separate compilation and execution A number of extra considerations need to be taken to write tests that are able From c105449bfb1f211e61f5040ace6b1c3d8dea2d49 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Fri, 21 Feb 2025 12:21:05 -0800 Subject: [PATCH 4/9] Add table of content links --- sycl/test-e2e/README.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index 5b2d6ea5b1307..d6ba93e86ca9f 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -12,6 +12,10 @@ * [Marking tests as expected to fail](#marking-tests-as-expected-to-fail) * [Marking tests as unsupported](#marking-tests-as-unsupported) * [SYCL core header file](#sycl-core-header-file) +* [Compiling and executing tests on separate systems](#separate-build-and-run) + * [Run only mode](#run-only-mode) + * [Build only mode](#build-only-mode) + * [Common Issues with separate build and run](#common-build-and-run-issues) ## Overview @@ -392,7 +396,9 @@ machines. By default the `test-mode` parameter is set to `full`, indicating that both stages will run. This parameter can be set to `build-only`, or `run-only`, to only run the compilation stage, or the execution stage respectively. -* `--param test-mode=run-only` +#### Run only mode + + Set with: `--param test-mode=run-only` In this mode, tests will not be compiled, they will only run. To do this only the `RUN:` lines that contain a "run" expansion will be executed (`%{run}`, @@ -407,7 +413,9 @@ to only run the compilation stage, or the execution stage respectively. `%{run-aux}` is an empty expansion and executes a line as is, without expanding for each selected device and without using the run_launcher. -* `--param test-mode=build-only` +#### Build only mode + + Set with: `--param test-mode=build-only` This mode can be used to compile all test binaries that can be built on the system. To do this `REQUIRES`, and `UNSUPPORTED` statements are handled @@ -436,7 +444,7 @@ to only run the compilation stage, or the execution stage respectively. The `build-mode` feature is added when in this mode. -#### Resolving common Issues with separate compilation and execution +#### Common Issues with separate build and run A number of extra considerations need to be taken to write tests that are able to be compiled and executed on separate machines. From eb755bd923c90031941902972df2396c4337c981 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Fri, 21 Feb 2025 12:26:57 -0800 Subject: [PATCH 5/9] Fix links --- sycl/test-e2e/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index d6ba93e86ca9f..7207dd021a388 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -12,10 +12,10 @@ * [Marking tests as expected to fail](#marking-tests-as-expected-to-fail) * [Marking tests as unsupported](#marking-tests-as-unsupported) * [SYCL core header file](#sycl-core-header-file) -* [Compiling and executing tests on separate systems](#separate-build-and-run) +* [Compiling and executing tests on separate systems](#compiling-and-executing-tests-on-separate-systems) * [Run only mode](#run-only-mode) * [Build only mode](#build-only-mode) - * [Common Issues with separate build and run](#common-build-and-run-issues) + * [Common Issues with separate build and run](#common-issues-with-separate-build-and-run) ## Overview From 1a05dc08f15292be40fd12c5fda3204243d14049 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Fri, 21 Feb 2025 14:23:31 -0800 Subject: [PATCH 6/9] Address review comments --- sycl/test-e2e/README.md | 60 ++++++++++++++++++++++++++++------------- 1 file changed, 42 insertions(+), 18 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index 7207dd021a388..a0e8ed4f569bf 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -406,12 +406,12 @@ to only run the compilation stage, or the execution stage respectively. this mode, for any test to pass the test binaries should already be in the `test_exec_root` directory, either by having ran `full` or `build-only` modes previously on the system, or having transferred the test binaries into that - directory. To mark a test as expected to fail only at run-time we can either - use the `run-mode` feature, which is only available in `run-only` and `full` - modes, or we can mark the test as `XFAIL` for a device-specific feature. + directory. To mark a test as expected to fail at run-time the `XFAIL` + expression should use runtime features, such as `run-mode` or device-specific + features. `%{run-aux}` is an empty expansion and executes a line as is, without - expanding for each selected device and without using the run_launcher. + expanding for each selected device and without using the `run_launcher`. #### Build only mode @@ -421,11 +421,11 @@ to only run the compilation stage, or the execution stage respectively. system. To do this `REQUIRES`, and `UNSUPPORTED` statements are handled differently to accommodate for the fact that in `build-only` mode we do not have any devices, and as a result no device-specific features. Instead of - considering these features as missing, we assign a third "unknown" boolean - value to them. When evaluating An expressions it will result in an unknown - value if its result could be changed by setting the unknown features to either - true or false. i.e., `false || unknown = unknown` but `true || unknown = true`. - If an expression's final value is unknown we consider it to have met the + considering these features as missing, we assign a third "unknown" value to + them. When evaluating an expression it will result in an unknown value if its + result could be changed by setting the unknown features to either true or + false. i.e., `false || unknown = unknown` but `true || unknown = true`. If an + expression's final value is unknown we consider it to have met the requirements. The list of device-agnostic features that are not considered unknown in `build-only` is found in the `E2EExpr.py`. @@ -435,15 +435,36 @@ to only run the compilation stage, or the execution stage respectively. `native_cpu` triples respectively. Each build target should be separated with a semicolon. This parameter is set to just `spir` by default. A test can be marked as requiring, or not supporting a particular triple via the `target-*` - features. Note that we test if each build target is supported separately from - each other so statements like `REQUIRES: target-spir && target-nvidia` will - never be met. Instead we can use `any-target-is-*` features in this case. + features. Build targets are selected if they are able to pass the test's + requirements independent of the availability of other build targets. This is + done to avoid having to deal with a boolean satisfiability problem. For + example, `REQUIRES: target-spir && target-nvidia` will always be marked as + unsupported since it requires multiple targets simultaneously. Instead we can + use `any-target-is-*` features in this case, to check if a target is available + in the current lit configuration. When executing the test in `build-only`, all `RUN:` lines that do not have a run expansion will execute. The `build-mode` feature is added when in this mode. + Some examples of `REQUIRES`/`UNSUPPORTED` in build-only. If `linux` and `zstd` + are available, and `sycl_build_targets` is set to `spir;amd;nvidia` + * `REQUIRES: linux && zstd`: This would be supported, this is treated normally + since both features are device-agnostic. + * `REQUIRES: linux && sg-32`: Despite the `sg-32` feature not being available, + this would be supported. Since the `sg-32` is a device-specific feature it is + evaluated as unknown in this expression. + * `REQUIRES: windows && sg-32`: This would be unsupported. `sg-32` would be + evaluated as unknown, and `windows` would evaluate as false. The fact that we + have an unknown value does not affect the end result, since the result of an + `&&` expression where one sub-expression is false is always false. + * `REQUIRES: windows || sg-32`: this would be supported. Here because the + result of the `||` expression would change if we considered `sg-32` to be + either true or false the overall expression evaluates to unknown. + * `UNSUPPORTED: !sg-32`: this would be supported. `sg-32` is evaluated as + unknown, and the negation of unknown is also unknown. + #### Common Issues with separate build and run A number of extra considerations need to be taken to write tests that are able @@ -462,17 +483,20 @@ system. should be marked as `XFAIL` for the corresponding build target feature, rather than a backend feature. * If the expected failure occurs during run-time we will need to mark the test - with `XFAIL` on a device specific feature. If it is expected to fail for any - device at run-time we can use `XFAIL: run-mode`, this is because otherwise the - test would compile and pass on `build-only` mode and be reported as an `XPASS`. + with `XFAIL` with an expression dependent on runtime features. If it is + expected to fail for any device at run-time we can use `XFAIL: run-mode`, + This must be done because otherwise the test would compile and pass on + `build-only` mode and be reported as an `XPASS`. * To separate compilation and execution of tests, `RUN:` lines are filtered in `build-only` and `run-only` mode based on the presence of "run" expansions. * Any line that is meant to execute the test binary should be marked with - `%{run}` or `%{run-unfiltered-devices}` so that it is not ran in `build-only`. + `%{run}` or `%{run-unfiltered-devices}` so that it is not ran in `build-only`, + and the `run_launcher` substitution is properly employed. * The `%{run-aux}` expansion can be used if a `RUN:` line that does not execute a test binary needs to be ran in `run-only`. -* Compiling for CPU AOT will target the ISA of the host CPU, thus compiling +* CPU AOT compilation will target the ISA of the host CPU, thus compiling these tests on a different system will lead to failures if the build system and -run system support different ISAs. +run system support different ISAs. To accommodate this, these compilations +should be delayed to the "run" stage by using the `%{run-aux}` markup. From 2f52a3eca4ad726b214e805d4218ca5eda704e98 Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Mon, 24 Feb 2025 06:22:16 -0800 Subject: [PATCH 7/9] Add example for output file naming --- sycl/test-e2e/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index a0e8ed4f569bf..a2a679e86870a 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -473,7 +473,8 @@ to be compiled and executed on separate machines. * Tests that build and execute multiple binaries need to be written such that the output of each compilation has a different name. This way no files are overwritten, and all the necessary binaries can be transferred to the running -system. +system. For example, instead of setting the output of all compilation steps to +a file named `%t.out`, we can number them `%t1.out`, `%t2.out`, and so on. * Two scenarios need to be considered for tests that expectedly fail: * Tests that are expected to fail on compilation, and thus also during From 71a1636bd5e888b019e91c49c151f7418e0e839b Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Tue, 25 Feb 2025 06:15:25 -0800 Subject: [PATCH 8/9] Make note of build target autodetection --- sycl/test-e2e/README.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index a2a679e86870a..8d9ba3e8f5820 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -433,15 +433,16 @@ to only run the compilation stage, or the execution stage respectively. lit parameter. Valid build targets are: `spir`,`nvidia`, `amd`, `native_cpu`. These correspond to `spir64`, `nvptx64-nvidia-cuda`, `amdgcn-amd-amdhsa`, and `native_cpu` triples respectively. Each build target should be separated with - a semicolon. This parameter is set to just `spir` by default. A test can be - marked as requiring, or not supporting a particular triple via the `target-*` - features. Build targets are selected if they are able to pass the test's - requirements independent of the availability of other build targets. This is - done to avoid having to deal with a boolean satisfiability problem. For - example, `REQUIRES: target-spir && target-nvidia` will always be marked as - unsupported since it requires multiple targets simultaneously. Instead we can - use `any-target-is-*` features in this case, to check if a target is available - in the current lit configuration. + a semicolon. This parameter is set to `all` by default, which enables + autodetection for the available build targets. A test can be marked as + requiring, or not supporting a particular triple via the `target-*` features. + Build targets are selected if they are able to pass the test's requirements + independent of the availability of other build targets. This is done to avoid + having to deal with a boolean satisfiability problem. For example, + `REQUIRES: target-spir && target-nvidia` will always be marked as unsupported + since it requires multiple targets simultaneously. Instead we can use + `any-target-is-*` features in this case, to check if a target is available in + the current lit configuration. When executing the test in `build-only`, all `RUN:` lines that do not have a run expansion will execute. From 1e7e52a7bc5f0c499e5df4239348d6c55ffa5a3b Mon Sep 17 00:00:00 2001 From: "Garcia Orozco, David" Date: Tue, 4 Mar 2025 12:45:16 -0800 Subject: [PATCH 9/9] Address comments + add 2 more examples related to triple selection --- sycl/test-e2e/README.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/sycl/test-e2e/README.md b/sycl/test-e2e/README.md index 8d9ba3e8f5820..59044dec9ec79 100644 --- a/sycl/test-e2e/README.md +++ b/sycl/test-e2e/README.md @@ -389,6 +389,7 @@ project must keep using `` provided by the SYCL2020 specification.** ### Compiling and executing tests on separate systems + The execution of e2e tests can be separated into compilation and execution stages via the `test-mode` lit parameter. This allows us to reduce testing time by compiling tests on more powerful systems and reusing the binaries on other @@ -398,7 +399,7 @@ to only run the compilation stage, or the execution stage respectively. #### Run only mode - Set with: `--param test-mode=run-only` + Pass: `--param test-mode=run-only` In this mode, tests will not be compiled, they will only run. To do this only the `RUN:` lines that contain a "run" expansion will be executed (`%{run}`, @@ -415,7 +416,7 @@ to only run the compilation stage, or the execution stage respectively. #### Build only mode - Set with: `--param test-mode=build-only` + Pass: `--param test-mode=build-only` This mode can be used to compile all test binaries that can be built on the system. To do this `REQUIRES`, and `UNSUPPORTED` statements are handled @@ -429,8 +430,8 @@ to only run the compilation stage, or the execution stage respectively. requirements. The list of device-agnostic features that are not considered unknown in `build-only` is found in the `E2EExpr.py`. - The triples to compile for in this mode are set via the `sycl_build_targets` - lit parameter. Valid build targets are: `spir`,`nvidia`, `amd`, `native_cpu`. + The triples to compile in this mode are set via the `sycl_build_targets` lit + parameter. Valid build targets are: `spir`,`nvidia`, `amd`, `native_cpu`. These correspond to `spir64`, `nvptx64-nvidia-cuda`, `amdgcn-amd-amdhsa`, and `native_cpu` triples respectively. Each build target should be separated with a semicolon. This parameter is set to `all` by default, which enables @@ -449,8 +450,9 @@ to only run the compilation stage, or the execution stage respectively. The `build-mode` feature is added when in this mode. - Some examples of `REQUIRES`/`UNSUPPORTED` in build-only. If `linux` and `zstd` - are available, and `sycl_build_targets` is set to `spir;amd;nvidia` + Some examples of `REQUIRES`/`UNSUPPORTED` in build-only: + If `linux` and `zstd` are available, and `sycl_build_targets` is set to + `spir;amd` * `REQUIRES: linux && zstd`: This would be supported, this is treated normally since both features are device-agnostic. * `REQUIRES: linux && sg-32`: Despite the `sg-32` feature not being available, @@ -465,6 +467,13 @@ to only run the compilation stage, or the execution stage respectively. either true or false the overall expression evaluates to unknown. * `UNSUPPORTED: !sg-32`: this would be supported. `sg-32` is evaluated as unknown, and the negation of unknown is also unknown. + * `REQUIRES: target-spir`: This will be supported, and only the `spir64` + triple will be selected. + * `REQUIRES: target-spir && target-amd`: This will not be supported. When + checking if `target-spir` should be selected, `target-amd` will not be an + available feature, and vice versa when checking `target-amd`. + * `REQUIRES: target-spir && any-target-is-amd`: This will be supported, but + only the `spir64` triple will be selected. #### Common Issues with separate build and run