Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add md-k6 script + new workflow #1828

Merged
merged 6 commits into from
Jan 6, 2025
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/run-code-blocks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Run Updated Code Blocks (Scripts)

on:
pull_request:
branches:
- main
paths:
- 'docs/sources/k6/next/**'

jobs:
run-code-blocks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Get Changed files
id: changed-files
uses: tj-actions/changed-files@v45
with:
files: |
**.md
- uses: grafana/setup-k6-action@v1
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- name: Run Updated Code Blocks
env:
ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
run: |
for file in ${ALL_CHANGED_FILES}; do
python -u scripts/md-k6.py "$file"
echo
done
46 changes: 46 additions & 0 deletions CONTRIBUTING/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ When you contribute to the docs, it helps to know how things work.
- [Use the `apply-patch` script](#use-the-apply-patch-script)
- [Style Guides](#style-guides)
- [Shortcodes](#shortcodes)
- [Shortcodes](#shortcodes)
- [Code snippets and ESLint](#code-snippets-and-eslint)
- [Code snippets evaluation](#code-snippets-evaluation)
- [Deploy](#deploy)
- [Create a new release](#create-a-new-release)

Expand Down Expand Up @@ -164,6 +167,49 @@ export default async function () {
```
````

### Code snippets evaluation

In addition to linting code snippets, we also actually run the snippets using k6 OSS. This is done automatically for all newly opened PRs, only for Markdown files that have been changed (when compared to `main`) in the `docs/sources/next` directory. See the `scripts/md-k6.py` script for details on how this works internally.
federicotdn marked this conversation as resolved.
Show resolved Hide resolved

Code snippets are run using the `-w` k6 OSS flag. If the code snippet causes k6 to exit with a nonzero status, then the script (and therefore the workflow) will fail. If any error is logged by k6 (for example, because an exception was raised), this will also fail the execution.
federicotdn marked this conversation as resolved.
Show resolved Hide resolved

You can control the behaviour of `md-k6.py` via magic `md-k6` HTML comments placed above the code snippets. The format is the following:

```text
<!-- md-k6:opt1,opt2,... -->
```

That is, `md-k6:` followed by a comma-separated list of options.

Currently, the only option that exists is `skip`, which will cause `md-k6.py` to ignore the code snippet completely (i.e. `<!-- md-k6:skip -->`). This is useful for code snippets that only showcase a very specific aspect of k6 scripting and do not contain an actually fully working script.

> [!TIP]
> You can combine both `md-k6.py` and ESLint skip directives by placing the `md-k6.py` directive first:
>
> ````Markdown
> <!-- md-k6:skip -->
> <!-- eslint-skip -->
>
> ```javascript
> export default async function () {
> const browser = chromium.launch({ headless: false });
> const page = browser.newPage();
> }
> ```
> ````

To run the `md-k6.py` script locally, invoke it using Python. For example:

```bash
python3 scripts/md-k6.py docs/sources/k6/next/examples/functional-testing.md
```

You can also read the usage information:

```bash
python3 scripts/md-k6.py --help
```

## Deploy

Once a PR is merged to the main branch, if there are any changes made to the `docs/sources` folder, the GitHub Action [`publish-technical-documentation.yml`](https://github.com/grafana/k6-docs/blob/main/.github/workflows/publish-technical-documentation.yml) will sync the changes with the grafana/website repository, and the changes will be deployed to production soon after.
Expand Down
148 changes: 148 additions & 0 deletions scripts/md-k6.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# md-k6.py
# Description: A script for running k6 scripts within Markdown files.
# Requires: Python 3.11+ (no external dependencies).
# Usage:
# python3 md-k6.py <file>

import os
import re
import json
import hashlib
import argparse
import subprocess
import textwrap
import tempfile
from collections import namedtuple

Script = namedtuple("Script", ["text", "options"])


def run_k6(script: Script) -> None:
script_file = tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".js")
script_file.write(script.text)
script_file.close()

logs_file = tempfile.NamedTemporaryFile(delete=False, suffix=".json")
logs_file.close()

k6 = os.getenv("K6_PATH", "k6")

result = subprocess.run(
[
k6,
"run",
script_file.name,
"--log-format=json",
f"--log-output=file={logs_file.name}",
"-w",
],
)

if result.returncode:
print("k6 returned non-zero status:", result.returncode)
exit(1)

with open(logs_file.name) as f:
lines = f.readlines()

for line in lines:
line = line.strip()
parsed = json.loads(line)
if parsed["level"] == "error":
print("error in k6 script execution:", line)
exit(1)


def main() -> None:
print("Starting md-k6 script.")

parser = argparse.ArgumentParser(
description="Run k6 scripts within Markdown files."
)
parser.add_argument("file", help="Path to Markdown file.", type=argparse.FileType())
parser.add_argument(
"--blocks",
default=":",
help="Python-like range of code blocks to run (0, 1, 2, 0:2, 3:, etc.).",
)
parser.add_argument("--lang", default="javascript", help="Code block language.")
args = parser.parse_args()

print("Reading from file:", args.file.name)

lang = args.lang
text = args.file.read()

# A somewhat complicated regex in order to make parsing of the code block
# easier. Essentially, takes this:
#
# <!-- md-k6:opt1,opt2 -->
# ```javascript
# (JS code)
# ```
#
# And converts it into:
#
# ```javascript$opt1,opt2
# (JS code)
# ```
#
# This is done for the entire Markdown file.
# After that's done, we can split the text by "```javascript", and parse
# each part separately. If a part's first line starts with "$", then we
# know one or more options were specified by the user (such as "skip").
#
# Additionally, we also skip over any "<!-- eslint-skip -->" comments, to
# allow developers to use both md-k6 *and* ESLint skip directives in code
# blocks.

text = re.sub(
"<!-- *md-k6:([^ -]+) *-->\n+(<!-- eslint-skip -->\n+)?```" + lang,
"```" + lang + "$" + r"\1",
text,
)

scripts = []
blocks = [block.strip() for block in text.split("```")[1::2]]
for b in blocks:
lines = b.splitlines()
if not lines[0].startswith(lang):
continue

if "$" in lines[0]:
options = [opt.strip() for opt in lines[0].split("$")[-1].split(",")]
else:
options = []

if "skip" in options:
continue

scripts.append(Script(text="\n".join(lines[1:]), options=options))

range_parts = args.blocks.split(":")
try:
start = int(range_parts[0]) if range_parts[0] else 0
end = (
int(range_parts[1])
if len(range_parts) > 1 and range_parts[1]
else len(scripts)
)
except ValueError:
print("Invalid range.")
exit(1)

print("Number of code blocks (scripts) read:", len(scripts))
print("Number of code blocks (scripts) to run:", len(scripts[start:end]))

for i, script in enumerate(scripts[start:end]):
script_hash = hashlib.sha256(script.text.encode("utf-8")).hexdigest()[:16]
print(
f"Running script #{i} (hash: {script_hash}, options: {script.options}):\n"
)
print(textwrap.indent(script.text, " "))
run_k6(script)
print()


if __name__ == "__main__":
main()
Loading