Skip to content

Correctness tests framework reusing upstream PromQL test #6086

Open
@yeya24

Description

@yeya24

Is your feature request related to a problem? Please describe.

Cortex already has some tests for query correctness. For example, Cortex has promqlsmith which is a random query generator to allow comparing query results between Cortex and Prometheus. However, the query random generator mostly focuses on different query strings. It doesn't have good coverage for different data types because that means the test input data needs to be randomly generated as well.

Prometheus uses promqltest which is a list of test cases for correctness verification. It would be nice if Cortex can have a way to consume those test cases and verify its correctness, especially against some newly added data types such as native histograms and native histogram custom buckets.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions