Skip to content

Latest commit

 

History

History
380 lines (295 loc) · 12.5 KB

01-user-guide.adoc

File metadata and controls

380 lines (295 loc) · 12.5 KB
Raw

User Guide

Table of Contents

Audience

You want to verify that the Clojure/ClojureScript code examples in your docs work as expected.

Prerequisites

You can generate tests with Clojure v1.9 or above.

You can run generated tests with Clojure v1.8 and above (if supported by your test runner).

You’ll use your test runner of choice: Clojure test-runner, cljs-test-runner, kaocha, or some lein test runner.

Introduction

Test-doc-blocks might (or might not; see limitations and interesting alternatives) be of interest for your Clojure/ClojureScript project.

Test-doc-blocks recognizes:

  • CommonMark code blocks found in Clojure sources docstrings, .md files and .adoc files.

  • AsciiDoc code blocks found in .adoc files.

An AsciiDoc code block looks like this:

[source,clojure]
----
;; Some valid Clojure code here.
----

A CommonMark (sometimes also referred to as GitHub Markdown) code block looks like this:

```Clojure
;; Some valid Clojure code here.
```

If the code in your code blocks are REPL style:

user=> (* 6 7)
42

Or editor style:

(* 6 7)
;; => 42

And don’t rely on any special unspecified setup, then test-doc-blocks will generate tests that verify that, yes, indeed:

(* 6 7) evaluates to 42 via test assertion (is (= 42 (* 6 7))).

Usage

Clojure CLI

An example deps.edn alias:

  :gen-doc-tests {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                                 com.github.lread/test-doc-blocks {:mvn/version "1.1.20"}}
                  ;; for -X syntax support specify exec-fn
                  :exec-fn lread.test-doc-blocks/gen-tests
                  ;; for -M syntax support specify main-opts
                  :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"]}

We use :replace-deps to keep deps to the minimum while generating our tests.

The most basic usage is:

clojure -X:gen-doc-tests

Or:

clojure -M:gen-doc-tests

This generates Clojure tests for code blocks in your README.md file to the target/test-doc-blocks/test directory. Any existing tests under target/test-doc-blocks will be replaced.

The generated tests have no dependency on test-doc-blocks. You can run them with your preferred test runner.

Tip

Windows users: if trying to escape -X args on the command line is leading to frustration, consider:

  • using the -M syntax

  • or specifying options under :exec-args or :main-opts in your :gen-doc-tests alias in your deps.edn.

deps.edn Examples

The test-doc-blocks deps.edn has some aliases that might serve as useful examples for running generated tests:

  • :isolated/cljs-test-runner - runs generated tests under ClojureScript using cljs-test-runner
    Invoke for this project via: clj -M:isolated/cljs-test-runner

  • :isolated/kaocha - runs generated tests under Clojure using kaocha
    Invoke for this project via: clj -M:isolated/kaoacha generated.

    Important
    		 Review the kaocha tests.edn config which tells koacha to run tests in order (rather than randomized order).

  • :isolated/clj-test-runner - runs generated tests under Clojure using Cognitect test-runner
    Invoke for this project via: clj -M:isolated/clj-test-runner

Leiningen

An example subset of a project.clj:

(defproject my-lovely-library "0.0.0"
  ;; ...
  :profiles {:gen-doc-tests {:test-paths   ["target/test-doc-blocks/test"]
                             :dependencies [[com.github.lread/test-doc-blocks "1.1.20"]]}}
  :aliases  {"run-doc-tests" ^{:doc "Generate, then run, tests from doc code blocks"}
             ["with-profile" "gen-doc-tests" "do"
              ["run" "-m" "lread.test-doc-blocks" "gen-tests"
               ;; change gen-tests options as appropriate for your project
               "--platform" "clj"
               "src/**.clj" "doc/example.md"]
              ["test"]]})

Running

lein run-doc-tests

  1. Generates clj tests for all code blocks found in:

    • all Clojure source files under src

    • doc/example.md

  2. Runs the generated tests.

Detailed Doc Examples with Inline Options

For detailed doc examples that include inline options, you will want to read:

Command Line Options

Tip

After you experiment with which options are appropriate for your project, you’ll likely incorporate options directly into your deps.edn or project.clj file.
Tip

Leiningen users should focus on the -M syntax and apply it to the leiningen example.

docs

By default, tests are generated for README.md only.

-X syntax
If you want to specify a different vector of files, you can do so via :docs

clojure -X:gen-doc-tests :docs '["README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]'

Moving this to a deps.edn alias would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.1.20"}}
   :exec-fn lread.test-doc-blocks/gen-tests
   :exec-args {:docs ["README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]}}

-M syntax
Equivalent -M syntax is:

clojure -M:gen-doc-tests README.adoc doc/example.adoc doc/example.md doc/example.cljc

Moving this to a deps.edn alias would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.1.20"}}
   :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"
                    "README.adoc" "doc/example.adoc" "doc/example.md" "doc/example.cljc"]}

globs
The files you specify can include glob syntax. For example, the following is equivalent to the last example:

clojure -M:gen-doc-tests README.adoc doc/example.{adoc,md,cljc}
Tip

When running from a terminal shell, be sure to use any necessary quoting should you want to prevent your shell from interpreting wildcard glob characters, for example, from a bash shell:

clojure -M:gen-doc-tests 'src/**.clj' 'doc/*.md'

Any special quoting should not be necessary when specifying options directly in a deps.edn or project.clj file.

target-root

By default, test-doc-blocks generates tests to ./target.

-X syntax
Override via :target-root when using the -X syntax:

clojure -X:gen-doc-tests :target-root '"./someplace/else"'

Expressed within a deps.edn alias, this would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.0.146-alpha"}}
   :exec-fn lread.test-doc-blocks/gen-tests
   :exec-args {:target-root "./someplace/else"}}

-M syntax
Use --target-root when using the -M syntax:

clojure -M:gen-doc-tests --target-root ./someplace/else

Expressed within a deps.edn alias, this would look like:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.1.20"}}
   :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"
                    "--target-root" "./someplace/else"]}
Note

Test-doc-blocks will delete and recreate the test-docs-block/test dir under the target root.
Warning

Keep the target location in mind when figuring out where to point your test runner.
If you get the location wrong for your test runner, it will likely not complain; many test runners will happily pass a test run that finds 0 tests.

platform

The platform governs what Clojure file types test-doc-blocks generates for tests.

Specify:

  • :clj for Clojure, generates .clj files

  • :cljs for ClojureScript, generates .cljs files

  • :cljc for mixed, generates .cljc files

The default is :cljc.

-X Syntax Example platform override using -X syntax:

clojure -X:gen-doc-tests :platform :clj

The same, expressed within a deps.edn alias:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.1.20"}}
   :exec-fn lread.test-doc-blocks/gen-tests
   :exec-args {:platform :clj}}

-M Syntax Same override but using -M syntax:

clojure -M:gen-doc-tests --platform clj

The same, expressed within a deps.edn alias:

  :gen-doc-tests
  {:replace-deps {org.clojure/clojure {:mvn/version "1.10.3"}
                  com.github.lread/test-doc-blocks {:mvn/version "1.1.20"}}
   :main-opts ["-m" "lread.test-doc-blocks" "gen-tests"
                    "--platform" "clj"]}
Tip

You can override the platform for code blocks via inline options within your docs.
Note

Test-doc-blocks makes no platform assumptions when generating tests from doc blocks found in Clojure source files. Specify what makes sense for your tests.

Limitations

Some limitations that we might entertain addressing:

  • Test-doc-blocks will automatically handle inline (require …​) and (import …​) appearing in code blocks, but not in any complex expressions of these forms.

  • Parsing adoc and md files is on the naive side but should handle most common cases. If we’ve overlooked a common syntax, let us know.

Some limitations we have no current plans to address:

  • Code blocks using ns or in-ns will not work with test-doc-blocks.

  • For REPL style code blocks, we only look for user⇒ prompts and no other ns prompts.

  • It is possible to embed HTML into your docs. If you express code or headings in embedded HTML within your doc, test-doc-blocks won’t find them.

Interesting Alternatives

Other options and related projects that I am currently aware of (in alphabetical order):

  • alc.x-as-tests - Runs code in (comment …​) blocks as tests.

  • clj-concordian - "A BDD / Specification by Example tool somewhat similar to Cucumber but far simpler", according to its author, go check it out!

  • kaocha - Kaocha supports running Cucumber tests. It uses this support in tests for some of its documentation. A .feature document describes the feature and includes given, when, then scenarios that are both run and shown in the documentation. You can use step definitions to hide any gritty details.

  • notebook - Someday notebook-type tools might serve both as tests and docs. Until that golden day, test-doc-blocks and similar tools are worthy of consideration.

  • RCF - Turns your Rich Comment Forms into tests

  • readme (archived) - Generates tests for code blocks found in .md files and then runs them. It is simpler but also has fewer features. This project was the inspiration for test-doc-blocks. Note: This project is now archived and recommends existing users consider test-doc-blocks instead.

  • testdoc - Tests code blocks in docstrings and external docs.