Skip to content

Commit 45c95d8

Browse files
sasselahasura-bot
authored andcommitted
server/tests: Add some detail to the hspec test style guide
[rendered](https://github.com/hasura/graphql-engine-mono/blob/hspec-docs/server/tests-hspec/README.md) This PR adds a little detail to the style guide, from design discussions whilst writing the RFC and reviewing some recent PRs. PR-URL: hasura/graphql-engine-mono#3815 GitOrigin-RevId: f6ead6b3327eed022b8b51083ab6b9508d89f3d0
1 parent 661bfeb commit 45c95d8

File tree

2 files changed

+37
-17
lines changed

2 files changed

+37
-17
lines changed

rfcs/hspec-test-suite.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,8 @@ Acceptance criteria
6969

7070
Checkpoints
7171
-----------
72+
[This GitHub issue is kept up to date with progress.](https://github.com/hasura/graphql-engine/issues/7760)
73+
7274
*Each checkpoint does not necessarily correspond to single PR. Organise PRs as you judge sensible, but prefer smaller PRs where they are functional and self-contained.*
7375

7476
We will scope this to something very simple for now: basic queries (where, order by, filter and offset) but aim to be as comprehensive and deliberate as possible. Incidentally, this will also be the first set of features we will want to be able to test for any new backend. We will extend this subset of features in the same order as that in which we prioritize work on new backends.
@@ -189,7 +191,7 @@ Related work
189191
[DB-to-DB Joins Test Suite tracking GitHub issue](https://github.com/hasura/graphql-engine-mono/issues/2528)
190192

191193

192-
[This PR](https://github.com/hasura/graphql-engine-mono/pull/2403) unblocks testing of DB-to-DB joins, which was untenable with our existing Python test suite. Although the PR includes exploratory work and overlaps somewhat with this RFC’s proposal, it’s minimally-scoped to support testing of DB-to-DB joins as soon as possible. Some or all of this work may be refactored or deleted depending on the outcome of this RFC and related work.
194+
[This change](https://github.com/hasura/graphql-engine/commit/67b4e1cc5b1351fd0e9951f003ed081fa65297e7) unblocks testing of DB-to-DB joins, which was untenable with our existing Python test suite. Although the PR includes exploratory work and overlaps somewhat with this RFC’s proposal, it’s minimally-scoped to support testing of DB-to-DB joins as soon as possible. Some or all of this work may be refactored or deleted depending on the outcome of this RFC and related work.
193195

194196
Within that PR, [Vamshi described](https://github.com/hasura/graphql-engine-mono/pull/2403#issuecomment-933630333) the expected DB-to-DB relationship behaviour and a proposal for designing new DB-to-DB joins tests.
195197

server/tests-hspec/README.md

Lines changed: 34 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,21 @@ For motivation, rationale, and more, see the [test suite rfc](../../rfcs/hspec-t
66

77
**Table of Contents**
88

9-
- [Running the test suite](#running-the-test-suite)
10-
- [Test suite structure](#test-suite-structure)
9+
- [tests-hspec](#tests-hspec)
10+
- [Running the test suite](#running-the-test-suite)
11+
- [Test suite structure](#test-suite-structure)
1112
- [Harness](#harness)
1213
- [Test](#test)
13-
- [Adding a new test](#adding-a-new-test)
14-
- [Specifying backends](#specifying-backends)
15-
- [Setup action](#setup-action)
16-
- [Teardown action](#teardown-action)
14+
- [Adding a new test](#adding-a-new-test)
15+
- [Specifying contexts](#specifying-contexts)
16+
- [Make local state action](#make-local-state-action)
17+
- [Setup action](#setup-action)
18+
- [Teardown action](#teardown-action)
1719
- [Writing tests](#writing-tests)
18-
- [Style guide](#style-guide)
20+
- [Style guide](#style-guide)
21+
- [Stick to Simple Haskell](#stick-to-simple-haskell)
22+
- [Write small, atomic, autonomous specs](#write-small-atomic-autonomous-specs)
23+
- [Use the `Harness.*` hierarchy for common functions](#use-the-harness-hierarchy-for-common-functions)
1924

2025
## Running the test suite
2126

@@ -68,7 +73,7 @@ For motivation, rationale, and more, see the [test suite rfc](../../rfcs/hspec-t
6873
cabal run tests-hspec -- --help
6974
```
7075

71-
3. The local databases presist even after shutting down docker-compose.
76+
3. The local databases persist even after shutting down docker-compose.
7277
If this is undesirable, delete the databases using the following command:
7378

7479
```sh
@@ -230,12 +235,25 @@ data:
230235
__Note__: these quasi-quoter can also perform string interpolation. See the relevant modules
231236
under the [Harness.Quoter](Harness/Quoter) namespace.
232237

233-
### Style guide
238+
## Style guide
234239

235-
- Test suite should be very easy to read for a junior or intermediate Haskell developer.
236-
Be mindful of advanced feature use and abstractions!
237-
- Prefer self-contained specs, even if there's some query duplication.
238-
- Avoid functions or types in tests, other than calls to the `Harness.*` API.
239-
Any supporting code should be in the `Harness.*` hierarchy and apply broadly to the test suites
240-
overall.
241-
- Consider reorganising tests if modules get much longer than 1/2 pagescrolls (~200-300 LOC?).
240+
### Stick to [Simple Haskell](https://www.simplehaskell.org/)
241+
This test suite should remain accessible to contributors who are new to Haskell and/or the GraphQL engine codebase. Consider the [power-to-weight](https://www.snoyman.com/blog/2019/11/boring-haskell-manifesto/#power-to-weight-ratio) ratio of features, language extensions or abstractions before you introduce them. For example, try to fully leverage Haskell '98 or 2010 features before making use of more advanced ones.
242+
243+
### Write small, atomic, autonomous specs
244+
Small: Keep specs short and succinct wherever possible. Consider reorganising modules that grow much longer than ~200-300 lines of code.
245+
246+
*For example: The [`TestGraphQLQueryBasic*` pytest class](../tests-py/test_graphql_queries.py#L251) was ported to the hspec suite as separate `BasicFields`, `LimitOffset`, `Where`, `Ordering`, `Directives` and `Views` specs.*
247+
248+
Atomic: Each spec should test only one feature against the backends (or contexts) that support it. Each module should contain only the context setup and teardown, and the tests themselves. The database schema, test data, and feature expectations should be easy to reason about without navigating to different module.
249+
250+
*For example: [`BasicFieldsSpec.hs`](Test/BasicFieldsSpec.hs)*
251+
252+
Autonomous: Each test should run independently of other tests, and not be dependent on the results of a previous test. Shared test state, where unavoidable, should be made explicit.
253+
254+
*For example: [Remote relationship tests](Test/RemoteRelationship/) explicitly require shared state.*
255+
256+
### Use the `Harness.*` hierarchy for common functions
257+
Avoid functions or types in tests, other than calls to the `Harness.*` API.
258+
259+
Any supporting code should be in the `Harness.*` hierarchy and apply broadly to the test suites overall.

0 commit comments

Comments
 (0)