[Spec] CCK Report: Tool/workflow

[bbros-dev]

Feature:
  As a Cucumber implementation
  We want to monitor compatibility with upstream
  In order to ensure feature files can be shared across languages
  And ensure Testers, BA's and PM's don't need retraining if Rust is used by Programmers

  Background:
     Given Cucumber Compatibility-Kit data is updated in Gherkin-Rust

  Scenario: Generate a Cucumber Compatibility-Kit Report
    When committing or merging changes
    Then CCK summary report data is generated

Existing implementations provide a guide:

• Specifically: https://github.com/cucumber/cucumber-js/blob/master/compatibility/cck_spec.ts

This comment from the upstream lead provides helpful guidance:

This corpus of files is used for Approval Testing of other Cucumber implementations.

Other Cucumber Implementation can use the CCK to verify its own message output with that of the CCK. This
CCK test suite also provides platform-specific implementations of the step definitions + hooks in the CCK.

It is not required that each implementation produce exactly the same output, but it should be equivalent.
We don't have a rigorous definition of what equivalent means, but we check the following (to the best of
my memory): The number of steps are the same, the number of step definitions are the same, the results are the same, etc.

And believe it or not, we also do some manual scream testing by feeding the generated messages through
@cucumber/html-formatter, which is based on @cucumber/react. For this we use StoryBook.
Using the StoryBook while you implement your Cucumber implementation's message output is actually a nice workflow
with very useful and quick feedback.

[luke-hill]

Hi @bbros-dev - Sorry for commenting on an old discussion. But I have recently ported the ruby tests in the CCK out a la JS so now we have 2 functionally equivalent CCK implementations. I've also done a load of work reconciling things there. So in terms of you getting something that is CCK conformant you could look at the work I did recently in the CCK for Ruby and look to work off that.

What you will need is 3 directories inside the CCK repo proper for rust which will be as follows

rust/features -> Phantom directory with no direct files. Will sync and copy over files from the devkit as/when they're generated / re-generated. Your rust crate will then contain these feature files. The contents are as follows

• Feature file proper
• Miscellaneous files such as attachments / images
• Run instructions (Only needed for retry feature)

rust/lib (Or whatever the convention is) -> Helper directory with code that will help you get at the feature files. I've managed to super diet down the ruby implementation to one file which has 3 methods. #gherkin returning all gherkin examples. #markdown returning all markdown examples and #feature_code_for which returns the directory of a specific example i.e. https://github.com/cucumber/compatibility-kit/tree/main/devkit/samples/attachments (But in ruby land)

rust/spec (Or whatever the convention is) -> Any testing to validate your helper code works

Hope this helps

Luke - Cucumber TLC + Ruby Lead