Skip to content

Commit 73f047a

Browse files
committed
Added the ability to extract doctests from arbitrary documentation (not just special header file comments). Allowed for multiple comment prefix styles, and multiple file types.
1 parent 872fd95 commit 73f047a

46 files changed

Lines changed: 1562 additions & 326 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# CHANGELOG
2+
3+
We will document notable changes in this file.
4+
5+
## v1.0.0
6+
7+
Initial release.
8+
9+
## 2.0.0
10+
11+
The initial release only scanned C++ header files for embedded doctests in comments where all the lines were prefixed by `///`.
12+
This version looks for doctests in any passed file and can handle doctests with any well known comment prefix.
13+
This means it can now handle doctests in Javadoc style comment blocks, as well as no-prefix comments in Markdown files, etc.

README.md

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -2,35 +2,35 @@
22

33
# Introduction
44

5-
**Doxytest** is a tool for generating C++ test programs from code embedded in header file comments.
5+
**Doxytest** is a tool for generating C++ test programs from documentation, for example, from code embedded in header file comments.
66

77
Its inspiration is a [Rust][] feature called [doctests][].
88

9-
A doctest is a snippet of sample code in the documentation block above a function or type definition.
9+
A typical doctest is a snippet of sample code in the documentation block above a function or type definition.
1010
In Rust, the example becomes part of the documentation generated by the `cargo doc` command.
1111

1212
However, in Rust, doctests are not just part of the documentation; they are also used to generate test programs.
13-
The `cargo test` command collects doctests from all the project's modules by looking for comments containing triple backtick fenced code blocks. The extracted code is then compiled and run as a test program.
13+
The `cargo test` command collects doctests from all the project's modules by looking for comments containing fenced code blocks delimited by triple backticks. The extracted code is then compiled and run as a test program.
1414

15-
Looking through the source code of Rust crates, you will see _lots_ of embedded doctests. Code in `Examples` sections is usually crafted as tests using Rust's `assert!` and `assert_eq!` macros.
15+
Looking through the source code of Rust crates, you will see _lots_ of embedded doctests. Examples are
1616

17-
After using this feature in Rust for a while, I wanted to do the same thing in C++. I decided to write a Python script that would extract the code snippets from the comments in C++ header files and use them to generate standalone C++ test programs.
17+
After using this feature in Rust for a while, I wanted to do the same thing in C++. I decided to write a Python script that would extract the code snippets from various sources and use them to generate standalone C++ test programs.
1818

19-
The Doxytest script, [`doxytest.py`][] looks for comment lines in C++ header files that start with `///` and which contain a fenced code block --- a _doctest_. The script extracts the doctests, wraps them in `try` blocks to catch any failures, and then embeds them in a standalone test program.
19+
The Doxytest script, [`doxytest.py`][], looks for fenced code blocks—_doctests_. The script extracts the doctests, wraps them in `try` blocks to catch any failures, and then embeds them in a standalone test program.
2020

21-
Of course, for this to be useful, doctest code must be formulated as a test. To that end, Doxytest also supplies [assertions][] that you can use in your doctests. These are relatively simple macros that capture the values of the arguments passed to an assertion along with some other helpful information. They throw a particular exception if the assertion fails, and the test program captures and processes that exception. The assertion macros are automatically defined and included in every test program generated by `doxytest.py`.
21+
Of course, for this to be useful, doctest code must be formulated as a test. To that end, Doxytest also supplies [assertions][] that you can use in your doctests. These are relatively simple macros that capture the values of the arguments passed to an assertion along with some other helpful information. They throw a custom exception when the assertion fails, and the test program captures and processes it. The assertion macros are automatically defined and included in every test program generated by `doxytest.py`.
2222

23-
Doxytest also supplies [`doxytest.cmake`][], a CMake module that automates the process of extracting tests from comments in header files and adding build targets for the resulting test programs. It defines a single CMake function called `doxytest` which is a wrapper around the `doxytest.py` script.
23+
Doxytest also supplies [`doxytest.cmake`][], a CMake module that automates the process of extracting tests from documentation and adding build targets for the resulting test programs. It defines a single CMake function, `doxytest`, which is a wrapper for the `doxytest.py` script.
2424

2525
## Installation
2626

27-
The main script file is `doxytest.py`, which you can copy and use on a standalone basis.
27+
The main script file is `doxytest.py`, which you can copy and run standalone.
2828

29-
If you use CMake then copy both `doxytest.cmake` and `doxytest.py` to the same directory.
30-
By default, `doxytest.cmake` expects that the Python script is located in the same directory it is.
29+
If you use CMake, then copy both `doxytest.cmake` and `doxytest.py` to the same directory.
30+
By default, `doxytest.cmake` expects the Python script to be in the same directory as it is.
3131
The `doxytest` function defined in the module has an option to change that default.
3232

33-
Typical CMake projects have a top-level `cmake/` subdirectory for their CMake modules which is a good place to store `doxytest.cmake` and `doxytest.py` .
33+
Typical CMake projects have a top-level `cmake/` subdirectory for their CMake modules, which is a good place to store `doxytest.cmake` and `doxytest.py`.
3434

3535
## Documentation
3636

@@ -51,11 +51,11 @@ Here is a super complicated C++ header file `add.h` with a comment block contain
5151
constexpr int add(int a, int b) { return a + b; }
5252
````
5353
54-
The header comment is the thing you might pass to [Doxygen][] to generate documentation for the function. Even if you don't use Doxygen (I prefer not to myself), most code editors will happily consume documentation like this and show it in a nicely formatted _tool tip_ if a user hovers over the `add` function in any code that uses it. This is similar in spirit to using the `crate doc` command in Rust.
54+
The header comment is the thing you might pass to [Doxygen][] to generate documentation for the function. Even if you don't use Doxygen, most code editors will happily consume documentation like this and show it in a nicely formatted _tool tip_ if a user hovers over the `add` function in any code that uses it. This is similar in spirit to using the `crate doc` command in Rust.
5555
5656
The comment contains a fenced code block (wrapped in triple backticks) that illustrates how you might use the `add` method. That, by itself, is a valuable piece of documentation.
5757
58-
However, it is more than that, as the example is a test using assertions. Doxytest supplies the `assert_eq` macro you can use to assert that two values are equal. On failure, it prints the values of the arguments and may terminate the program. All `doxytest.py` generated test source files have the macro definition.
58+
However, it is more than that, as the example is a test using assertions. Doxytest supplies the `assert_eq` macro you can use to assert that two values are equal. On failure, it prints the argument values and may terminate the program. All `doxytest.py` generated test source files have the macro definition.
5959
6060
You can turn the _doctest_ into the source for an actual test program by invoking the `doxytest.py` script on the header file:
6161
@@ -87,33 +87,33 @@ test 1/1 (add.h:4) ... pass
8787
8888
Why bother with documentation tests?
8989
90-
Rust has a published set of [documentation conventions][] and those suggestions are followed closely by the Rust community. This result is a very consistent documentation style for the many thousands of crates on the [crate docs][] site.
90+
Rust's [documentation conventions][] are followed closely by the Rust community. The result is a very consistent documentation style for the many thousands of crates on the [crate docs][] site.
9191
92-
The conventions make sense for any coding language. One suggestion is always to include an "Examples" section in the comments with working code examples. Using Markdown, the code is enclosed in a triple-backtick-delimited block, allowing users to cut and paste to get a practical taste of how to use a particular type or function.
92+
The conventions make sense for any coding language. One suggestion is always to include an "Examples" section in the comments with working code examples. Using Markdown, the code is enclosed in a triple-backtick-delimited block, allowing users to cut and paste a snippet to get a practical taste of how to use a particular type or function.
9393
9494
An examples section is obviously a great idea. Indeed, in C++, it is a key feature of [cppreference][], which is the reference everyone uses for the standard library.
9595
9696
It took a Rust brainwave to realise that it makes good pedagogical sense to format examples as tests! You get standardised documentation _and_ some tests for the price of one ticket!
9797
9898
Rust has some standard macros for making assertions, which makes it easy to write documentation examples as tests. For the most part, you can write an awful lot of practical test examples using `assert!` and `assert_eq!` (in Rust, macro names end in an exclamation mark). The two macros are nothing special and work precisely as you'd expect.
9999
100-
C++ has a rather rudimentary `assert` macro, which it inherited from the early days of C. That macro is not very useful for testing because it does not print the values of the arguments that failed the assertion. For this reason, in the `doxytest` generated code, we overwrite the existing' assert' with our own version and also provide the `assert_eq` macro, as used in our example. The two macros are automatically defined and included in the test programs generated by `doxytest.py`. They only have an effect in test code extracted from triple backtick fenced comment blocks.
100+
C++ has a rudimentary `assert` macro, which it inherited from the early days of C. That macro is not helpful for testing because it does not print the values of the arguments that failed the assertion. For this reason, in the `doxytest`-generated code, we overwrite the existing' assert' with our own version and also provide the `assert_eq`macro, as used in the example above. The two macros are automatically defined and included in the test programs generated by`doxytest.py`. They only have an effect in test code extracted from triple backtick fenced comment blocks.
101101
102102
## Scope
103103
104-
Doxytest is a simple tool for generating C++ test programs from code embedded in header file comments.
104+
Doxytest is a simple tool for generating C++ test programs from documentation.
105105
It isn't a replacement for a full-blown testing framework, such as [`Catch2`][] or [`Google Test`][].
106106
107107
Doctests are typically just a few lines of code that primarily illustrate how to use a function or class and are crafted as tests. You're unlikely to write a lot of complicated edge case code as comments in a header file.
108108
109109
On the other hand, once you get used to the idea, you tend to write a doctest for almost every function or class you write.
110-
So, while the depth of test coverage may not be as high as that of a full-blown testing framework, the breadth of coverage is impressive.
110+
So, while the depth of test coverage may not be as high as that of a full-blown testing framework, the breadth of coverage is excellent.
111111
112-
The breadth of coverage is very valuable when adding new features or fixing bugs. Just compiling all the doctests can serve as a quick sanity check to see if you have inadvertently broken something else. And, of course, running the tests will help you catch at least basic regression errors.
112+
That breadth of coverage is very valuable when adding new features or fixing bugs. Compiling all the doctests can serve as a quick sanity check to ensure you haven't inadvertently broken anything else. And, of course, running the tests will help you catch at least basic regression errors.
113113
114114
If you are using Rust in an IDE like [VSCode][], you can run the doctest for an individual method by clicking a discrete "Run" above the method in the IDE. There is also a "Run" button above a type definition that runs all the doctests for that type.
115115
116-
We haven't implemented a `doxytest` extension for VSCode yet. However, we do have a CMake module, [`doxytest.cmake`][], that can automate the process of extracting those tests and adding build targets for each resulting test program. This is already very useful and, if you are using the CMake Tools extension for [VSCode][], it will let you easily run doctests at the click of a button.
116+
We haven't implemented a `doxytest` extension for VSCode yet. However, we do have a CMake module, [`doxytest.cmake`][], that can automate the extraction of those tests and the addition of build targets for each resulting program. This is already very useful and, if you are using the CMake Tools extension for [VSCode][], it lets you run doctests with a single click.
117117
118118
### Contact
119119

docs/_quarto.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ website:
3838
image: "images/logo.svg"
3939
favicon: "images/logo.svg"
4040
open-graph: true
41+
google-analytics: G-62CQE8TY0X
4142
site-url: "https://nessan.github.io/doxytest"
4243
repo-url: "https://github.com/nessan/doxytest"
4344
repo-subdir: docs

docs/pages/cmake/index.qmd

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: "`doxytest.cmake`"
44

55
## Introduction
66

7-
`doxytest.cmake` is a CMake module that automates the process of extracting tests from comments in header files and adding build targets for the resulting test programs.
7+
`doxytest.cmake` is a CMake module that automates the process of extracting tests from documentation and adding build targets for the resulting test programs.
88
It defines a single CMake function called `doxytest` which is a wrapper around the {doxytest.script} script.
99

1010
## Usage
@@ -25,8 +25,9 @@ Assuming everything goes well, CMake builds will generate two test source files
2525
- `doxytests/doxy_foo.cpp`
2626
- `doxytests/doxy_bar.cpp`
2727

28-
Note that the default output directory for the test source files is `doxytests/`. That script creates the directory if necessary.
29-
Also note that, by default, each test source file has a name that has a prefix (by default, `doxy_`) followed by the base name of the corresponding header file.
28+
Note that the default output directory for the test source files is `doxytests/`.
29+
The script creates the directory if necessary.
30+
Also note that, by default, each test source file has a name that has a prefix (by default, `doxy_`) followed by the base name of the source file for the doctests.
3031

3132
The `doxy_foo.cpp` source file will have an include directive for the `foo.h` header file, and the `doxy_bar.cpp` source file will have an include directive for the `bar.h` header file.
3233
The path to the header files is relative to the location of the test source file, so something like:
@@ -94,7 +95,7 @@ The directory where our script will write the test source files, which by defaul
9495
```cmake
9596
doxytest(include/foo.h DOXY_DIR tests/)
9697
```
97-
This will generate test files in a top-level subdirectory called `tests/`.
98+
This will generate test files in the top-level `tests/` subdirectory.
9899
This directory is relative to the `CMAKE_SOURCE_DIR`.
99100

100101
If necessary, the script creates the directory; otherwise, the module will error out if the operation fails or if the directory is not writeable.
@@ -127,7 +128,7 @@ WARNING: All included headers should have header guards or a `#pragma once` dire
127128

128129
### `LIBRARIES <libs>`
129130

130-
This is another commonly used option.
131+
Another commonly used option.
131132
It allows you to specify a list of libraries to link to the test executables.
132133
For example, if you are testing a library called `my_lib`, you may want to link to the `my_lib` library in the test executables.
133134
The `LIBRARIES` option achieves this:
@@ -142,7 +143,7 @@ The `LIBRARIES` option can be given a list of multiple libraries to link to.
142143

143144
### `DOXY_MODE <mode>`
144145

145-
This option controls the type of output from the script.
146+
This option controls the script's output type.
146147
By default, the `doxytest` function generates _both_ individual test files and a combined test file.
147148
You can change this by passing the `DOXY_MODE` option to the `doxytest` function:
148149

@@ -223,7 +224,7 @@ Here's what it looks like when the project is open in {vscode}:
223224
![Editing the project in VSCode](VSCode.png)
224225

225226

226-
The `CMakeLists.txt` file in the root of the project uses the `doxytest.cmake` module to extract the doctests from the headers in the `include/calc/` directory and package them as test source code it puts in the `doxytests/` directory.
227+
The `CMakeLists.txt` file at the root of the project uses the `doxytest.cmake` module to extract the doctests from the headers in the `include/calc/` directory and package them as test source code, placing them in the `doxytests/` directory.
227228
Targets to build and run those tests are then made available. That includes a target for a combined test program `doxytests`.
228229

229230
The {cmake_tools} extension for VSCode add several buttons to bottom of the editor window to build, run, and debug the project.

0 commit comments

Comments
 (0)