[SYCL][E2E] Add test to check REQUIRES #16019
Conversation
This test checks the content of every "REQUIRES" to avoid typos and non-existing requirements. This PR also updates some tests requirements found during test development.
KornevNikita
had a problem deploying
to
WindowsCILock
GitHub Actions
Failure
|
The test will fail in pre-commit as we have |
KornevNikita
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
|
|
||
|
|
||
| def parse_requirements(input_data_path, sycl_include_dir_path): | ||
| available_features = { |
There was a problem hiding this comment.
We must import features from llvm/sycl/test-e2e/lit.cfg.py instead of building a manually maintained copy of this list.
There was a problem hiding this comment.
We must import features from llvm/sycl/test-e2e/lit.cfg.py instead of building a manually maintained copy of this list.
Can we really do that in a reliable manner? Those features depend on the environment (OS, available tools, build configuration, enabled projects, etc.).
I.e. it is expected that REQUIRES may reference features which are not registered. But some of those could be known and therefore legal, whilst others could be a result of a typo and therefore should be fixed.
There was a problem hiding this comment.
We must import features from llvm/sycl/test-e2e/lit.cfg.py instead of building a manually maintained copy of this list.
+1 to Alexey's question. lit.cfg.py generates the set of available features "online" according to the specific env. Do you know if it's possible to get the whole set regardless of env? I agree it's not the best idea to maintain this set manually, but I'm not sure if we have another options.
There was a problem hiding this comment.
I have an idea, but it requires refactoring of llvm/sycl/test-e2e/lit.cfg.py.
We can have a dictionary of all features with callbacks like this:
features = {
'x': available, # 'available' function just returns true, i.e. make 'x' always available.
'y': is_y_available, # 'is_y_available' function returns true if feature 'y' is available.
'z': available, # 'available' function just returns true
...
}- To get the list of all features that can be used by e2e tests we just need to get the keys (i.e.
features.keys()) - To build the list of available features, we just go over all the items in the dictionary and use functions to decide which features are available.
The benefit of this implementation, you don't need to update your test when new feature is added.
sycl/test/e2e_test_requirements/check-correctness-of-requires.cpp
Outdated
Show resolved
Hide resolved
1. restore modified tests 2. use llvm-lit instead of grep 3. update a set of available features to match everything from lit.cfg.py
KornevNikita
had a problem deploying
to
WindowsCILock
GitHub Actions
Failure
KornevNikita
temporarily deployed
to
WindowsCILock
GitHub Actions
Inactive
This test checks the content of every "REQUIRES" to avoid typos and non-existing requirements.
This PR also updates some tests requirements found during test development.