Skip to content

Validation tests don't match reality #1407

Open
@ghaith

Description

@ghaith

Describe the bug
Our validation tests don't use the build pipeline, and thus don't stop on parser critical paths.
Furthermore, they sometimes show errors that the user does not actually see because they are not reported correctly, other times, validations are duplicated.
This means we think a section in our code is being validated while the user does not see a relevant error for that.

Solution
Use the pipeline for validation tests, this ensures the validations are the same as what the user will see.
This means that all tests that are failing need to be re-checked and missing validations we were expecting has to be reported correctly.
Anything deleted from the insta snapshot means the user never saw this before. Any additional line to the insta snapshot means the user was seeing this error but we were not taking it into account.

Side effect
All validation tests will be integration tests since we need the full pipeline, but this should be fine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions