Versioning and stability guarantees #1880
Conversation
codebien
requested review from
mstoykov and
inancgumus
and removed request for
a team
|
|
||
| The k6 command-line interface (CLI), including commands, subcommands, and flags, is also part of the stable and supported API. These commands are essential for initiating and managing test executions locally and in the cloud. | ||
|
|
||
| ### JavaScript interpreter |
There was a problem hiding this comment.
| ### JavaScript interpreter | |
| ### JavaScript interpreter and TypeScript compiler |
Should we include TypeScript?
There was a problem hiding this comment.
I'm tempted to not include it. My reasoning being that while we own Sobek, and maintain it, Typescript support is offered thanks to esbuild which is a completely external tool we have no interactions with, nor any influence whatsoever.
I'm happy to be convinced to be convinced otherwise though 🙇🏻
|
|
||
| This includes the [go.k6.io/k6/js/modules](https://pkg.go.dev/go.k6.io/[email protected]/js/modules) Go package that internal and external developers have available to build module extensions, as well as the [go.k6.io/k6/output](https://pkg.go.dev/go.k6.io/[email protected]/output) that can be used to build custom outputs for k6. | ||
|
|
||
| It also includes public k6 Go packages that are exposed to developers to facilitate the development of these extensions (e.g., the current [go.k6.io/k6/lib.State](https://pkg.go.dev/go.k6.io/[email protected]/lib#State) package). |
There was a problem hiding this comment.
I feel this is not clear enough. Any suggestion how to improve? We have defined the policy as an allowlist, should we have here the full list of the supported packages?
There was a problem hiding this comment.
Not specific to this sentence, but as I gave the document a read I wondered if we should maybe state at the beggining something along the lines of "anything that is not explicitly covered by this document is not considered stable or supported". Which, I think was pretty much the general approach we had as we wrote it. What do you think?
For this specific case though, now that we have moved internal code in the internal package, would it be good enough to say that "anything that's not in the internal package" is considered public, stable, covered by this policy, and can be used by extensions in respect to k6's versioning scheme"?
codebien
requested review from
oleiade,
olegbespalov,
joanlopez,
andrewslotin,
dgzlopes and
pablochacin
| It also includes public k6 Go packages that are exposed to developers to facilitate the development of these extensions (e.g., the current [go.k6.io/k6/lib.State](https://pkg.go.dev/go.k6.io/[email protected]/lib#State) package). | ||
| ### Output | ||
|
|
||
| We guarantee the stability and support of metrics, logs, traces, and profiling outputs that we fully control in an end-to-end flow. This includes the JSON, CSV, JSON summary exports, cloud metrics, logs and traces outputs, and OpenTelemetry formats while excluding outputs dependent on third-party platforms and protocols. |
There was a problem hiding this comment.
If I recall correctly, the new JSON format for the summary output will not be introduced in v1.0.0. Therefore, according to this, it won't be introduced until at least v2.0.0 or we simply consider the existing JSON format deprecated and therefore changing it is no a "breaking"
There was a problem hiding this comment.
Yeah, good catch. This was added mostly because in our intentions but we haven't accomplished it. I am dropping it.
@joanlopez @oleiade do we introduce any guarantee for the new End-of-Test summary? I guess we should.
There was a problem hiding this comment.
@codebien we have had a couple discussions about this in the past. My stance is the end-of-test summary (both human readable and machine readable) is explicitly excluded from our stability policies and guarantees in v1.0.0. I expect v1.0.0 to explicitly mark the existing json end of test summary as deprecated.
As we go through the development cycle towards v2.0.0, we may ship a new, supported this time, machine readable end of test summary. At this point, then and only then, do I suggest we add the machine readable summary to this document, with the option to extend it to the new summary/report code and any potential summary extension API we might come up with, at that point in time.
Any strong objections?
What?
It introduces a new page for documenting k6 versioning and stability guarantees. Note, this is expected to be merged only after k6 v1.0.0 release.
Checklist
npm startcommand locally and verified that the changes look good.docs/sources/k6/nextfolder of the documentation.Related PR(s)/Issue(s)