Allow #[test_proc] to expect specific assertion failures #2109
Conversation
|
/cc @cdleary who originally commented on the preferred syntax: #1071 (comment) |
|
curious if that's something that regular SystemVerilog/UVM support, and what this would ultimately map to if we were generating SV test benches from DSLX tests. |
I believe there is no option to ignore the |
rw1nkler
force-pushed
the
76623-expected-fail
branch
from
2b6d318 to
48f880d
Compare
|
@richmckeever mind having a look at this as it touches on the frontend. The feature is very nice to have. |
rw1nkler
force-pushed
the
76623-expected-fail
branch
from
48f880d to
533c6ef
Compare
rw1nkler
force-pushed
the
76623-expected-fail
branch
3 times, most recently
from
fd95467 to
0cf4893
Compare
|
I also modified the DSLX formater accordingly |
| @@ -533,6 +533,68 @@ proc doomed { | |||
| IsTestResult(TestResult::kParseOrTypecheckError, 0, 0, 0)); | |||
| } | |||
|
|
|||
| TEST_P(RunRoutinesTest, TestProcExpectedToFailOnAssert) { | |||
There was a problem hiding this comment.
Let's add one more test, where we assert with an unexpected label and the test result is a failure.
There was a problem hiding this comment.
I added separate tests for both assert! and fail! macros that check whether the failure on incorrect label is treated correctly.
rw1nkler
force-pushed
the
76623-expected-fail
branch
from
0cf4893 to
14ab842
Compare
Enables writing tests that are expected to fail via `assert!` or `fail!`. The attribute value must match the label provided in the failing assertion. Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
rw1nkler
force-pushed
the
76623-expected-fail
branch
from
14ab842 to
d731ed2
Compare
| absl::StatusOr<TestProc*> ParseTestProc(Bindings& bindings); | ||
| absl::StatusOr<TestProc*> ParseTestProc( | ||
| Bindings& bindings, | ||
| std::optional<std::string> expected_fail_label = std::nullopt); |
There was a problem hiding this comment.
Optional: = std::nullopt isn't needed, since all the uses already specify the fail label.
This commit introduces the ability to specify that a test proc is expected to fail on a particular
assert!orfail!in the DSLX interpreter. The#[test_proc]attribute accepts an optionalexpected_failparameter. with the label marking the failure. If a failure occurs with the matching label, the test is treated as successful.The choice to implement this as an attribute, rather than a built-in, as discussed in #1071 (comment) was motivated by cases where there's no clear location to “catch” the failure. For example, a test that spawns a proc containing only a single
fail!statement.The functionality is implemented by storing the
expected_failvalue in theTestProcobject. When anassert!orfail!is handled in the DSLX interpreter, the label of the failure is attached to the error using anabsl::Statuspayload. This is then compared inRunDslxTestProcagainst the stored label in theTestProc.Resolves #1071