Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docfx and documentation style guide #103

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8db2115
initial draft of updated readme
banchan86 Mar 18, 2024
d7b527d
updated formatting
banchan86 Mar 18, 2024
1ea57b0
updated readme with how to setup a new docfx website
banchan86 Mar 19, 2024
7c18b4f
edited code blocks
banchan86 Mar 19, 2024
d835e6e
added section on adding markdown tables, mermaid graphs, and publishi…
banchan86 Mar 22, 2024
092f1e0
added section on testing unpublished packages and other edits to cont…
banchan86 Mar 28, 2024
5bd73f6
minor formatting edits to fix examples not showing
banchan86 Mar 28, 2024
2521838
added docs and TOC organization section, alerts section
banchan86 Mar 29, 2024
6ffb7ef
added individual operator articles (apidoc)
banchan86 Mar 29, 2024
a0fd365
Expanded docfx.json configuration options, edits to enhance imperativ…
banchan86 Mar 29, 2024
4825f06
reorganised sections, added instructions on amending config.yml to ad…
banchan86 Mar 29, 2024
921ff9e
added tutorial submodule section and other minor formatting/grammar e…
banchan86 Mar 29, 2024
bed6dfd
minor formatting and spelling corrections
banchan86 Mar 29, 2024
cd821c4
Trim README, move content to new package documentation article
banchan86 Sep 9, 2024
049f3b2
Add documentation alert to create package article
banchan86 Sep 9, 2024
97a4136
Split doc article, update TOC
banchan86 Sep 10, 2024
e8cc69a
Clean up docfx article, add directory trees
banchan86 Sep 11, 2024
70ec67c
Minor edits to TOC, and docfx doc article
banchan86 Sep 11, 2024
a9eb2c0
Expand documentation style article
banchan86 Sep 12, 2024
73cf304
Update doc style guide with cross-referencing
banchan86 Sep 16, 2024
4d13261
Fix broken section anchor in docfx doc guide
banchan86 Sep 16, 2024
1388d0b
Minor edits for clarity and grammar
banchan86 Sep 19, 2024
a846242
Minor edits to clarity and formatting
banchan86 Sep 19, 2024
8b28c16
Clarify build.ps1 purpose
banchan86 Sep 21, 2024
5bb3b85
Apply suggestions from code review
banchan86 Nov 9, 2024
83c57d4
Further edits to spelling and formatting based on code review
banchan86 Nov 9, 2024
10d4a8e
Edit README to add contribution section and clarify external links
banchan86 Nov 9, 2024
2567ef1
Apply suggestions to change absolute to relative links
banchan86 Nov 12, 2024
fb42b2c
Change absolute links to relative links in remaining articles
banchan86 Nov 12, 2024
4457090
Minor edits to wording clarity and formatting
glopesdev Nov 15, 2024
7bc3fe8
Edit wording and embed assets directly in the text
glopesdev Nov 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 69 additions & 36 deletions articles/documentation-docfx.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,24 +28,24 @@ dotnet new install Bonsai.Templates
dotnet new bonsaienv
```

- `.github/workflows/` - The `docfx` website is published to [GitHub Pages](https://pages.github.com/) using a [GitHub Actions](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions) workflow called `docs.yml`. Download this folder from the [docfx-assets](https://github.com/bonsai-rx/docfx-assets) repository and amend `Bonsai.PackageName` to point to your package source code in `docs.yml`.
- `.github/workflows/` - The `docfx` website is often automatically published to [GitHub Pages](https://pages.github.com/) using a [GitHub Actions](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions) workflow specified in the `docs.yml` file. Unfortunately these workflows tend to be very project specific and we do not yet have a standardized pipeline. For a minimal example and inspiration you can check [the workflow for this repository](https://github.com/bonsai-rx/docs/blob/main/.github/workflows/build.yml).
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am ok with pointing to a sample repository for the GitHub Actions recipe, my main concern with pointing to the bonsai docs repo 'build.yml' is that compared to most of the other packages, it's missing the parts of the pipeline for building and executing the docfx-tools pipeline for SVG export on GitHub Actions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point, although unfortunately most of the other repos are also using deprecated actions which I would prefer not to use as a reference going forward. I was wondering maybe the best middle ground would be to upgrade one of the old repos to try out the upgraded pipeline.

I am reviewing bonsai-rx/zeromq#22 and will try to upgrade everything to fit correctly. Once we are done we can point to that, does that sound good?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me!


## Setting up docfx
## Setup docfx

1) Setup a local installation of `docfx` in your package repository with the following commands (executed in the root directory):
You can setup a local installation of `docfx` directly in your package repository with the following commands (executed in the root directory):

```powershell
dotnet new tool-manifest
dotnet tool install --local docfx
```

2) Navigate to the `docs` folder and initialize a new `docfx` website with the following command.
Then navigate to the `docs` folder and initialize a new `docfx` website with the following command:

```powershell
dotnet docfx init
```

3) Select these options for the new website.
We recommend the following options for new websites (make sure to match the name to your package):

```powershell
Name (mysite): Bonsai - Package Name
Expand Down Expand Up @@ -101,18 +101,46 @@ To update the submodule to the latest commit:
git submodule update --remote
```

- `template/` - this custom CSS template patches in `docfx-tools` and adds a GitHub link to the top right of the navigation bar. Download this folder from the [docfx-assets](https://github.com/bonsai-rx/docfx-assets) repository and amend the GitHub link in `main.css` to your repository.
- `template/` - the files in this `docfx` template extend the base template in `docfx-tools`. Below are templates for `main.css` and `main.js` which add a GitHub link to the top right of the navigation bar and include all styles for workflow containers.

- `build.ps1` - this custom script exports Bonsai workflow images using `docfx-tools` and then calls the `docfx` build process. Download the file from the [docfx-assets](https://github.com/bonsai-rx/docfx-assets) repository and amend the line that specifies the package name and source location.
###### main.css
```css
@import "workflow.css";
```

###### main.js
```js
import WorkflowContainer from "./workflow.js"

export default {
defaultTheme: 'auto',
iconLinks: [{
icon: 'github',
href: 'https://github.com/org-name/PackageName',
title: 'GitHub'
}],
start: () => {
WorkflowContainer.init();
}
}
```

- `build.ps1` - this custom script exports Bonsai workflow images using `docfx-tools` and then calls the `docfx` build process. Make sure to change the line to point to your package output location:

```powershell
.\bonsai\modules\Export-Image.ps1 "..\src\Bonsai.PackageName\bin\Release\net462"
dotnet docfx @args
```

- `logo.svg` and `favicon.ico` - for official Bonsai packages these can be downloaded from the [docfx-assets](https://github.com/bonsai-rx/docfx-assets) repository.


### Configuring docfx

The `docfx.json` file in the `docs` folder hosts the configuration options for the website and needs to be modified to fully utilize the available features. If in doubt, refer to a Bonsai package repository that has been recently updated (for instance https://bonsai-rx.org/machinelearning/) or the [docfx documentation](https://dotnet.github.io/docfx/) for more information. These steps are listed in order of appearance in `docfx.json`.
The `docfx.json` file in the `docs` folder specifies the configuration options for the website and needs to be modified to make use of all available features. If in doubt, refer to a Bonsai package repository that has been recently updated, e.g. [bonsai-rx/machinelearning](https://bonsai-rx.org/machinelearning/), or the [docfx documentation](https://dotnet.github.io/docfx/) for more information. These steps are listed in order of appearance in `docfx.json`.

1) Add a `filter` attribute to [filter](https://dotnet.github.io/docfx/docs/dotnet-api-docs.html#filter-apis) obsolete operators in the package. In the process of upgrading packages we want to avoid documenting operators that are no longer supported (but are still included for compatibility purposes for old workflows). You can also hide private classes that are not supposed to be shown to the end user.
#### Hide obsolete operators
Add a [`filter`](https://dotnet.github.io/docfx/docs/dotnet-api-docs.html#filter-apis) attribute to hide obsolete operators from the package docs. In the process of upgrading packages we want to avoid documenting operators that are no longer supported (but are still included for compatibility purposes for old workflows). You can also hide private classes that are not supposed to be shown to the end user.

```yml
"metadata": [
Expand All @@ -122,7 +150,7 @@ The `docfx.json` file in the `docs` folder hosts the configuration options for t
]
```

Make a file called `filter.yml` with the following lines and place in the `docs` directory. You can also specify other [uids](https://dotnet.github.io/docfx/docs/dotnet-api-docs.html#filter-apis) here.
Make a file called `filter.yml` with the following lines and place it in the `docs` directory. You can also specify other [uids](https://dotnet.github.io/docfx/docs/dotnet-api-docs.html#filter-apis) here.

```yml
apiRules:
Expand All @@ -131,7 +159,7 @@ apiRules:
uid: System.ObsoleteAttribute
```

2) Exclude these files from content to avoid duplicate errors.
Exclude these files from content to avoid duplicate errors.

```json
"build": {
Expand All @@ -145,7 +173,8 @@ apiRules:
}
```

3) Add the `overwrite` attribute to enable [individual operator articles](./documentation-style-guide.md) stored in the `apidoc` folder to be included in articles and API docs.
#### Enable operator articles
Add the `overwrite` attribute to ensure [individual operator articles](./documentation-style-guide.md) stored in the `apidoc` folder are included in both articles and API docs.

```json
"build": {
Expand All @@ -163,7 +192,8 @@ apiRules:
}
```

4) Modify the `resource` attribute to include files that need to be imported.
#### Add resource files
Modify the `resource` attribute to include files that need to be imported.

```json
"resource": {
Expand All @@ -176,7 +206,8 @@ apiRules:
}
```

5) Modify the `template` attribute to use the modern template and apply the `docfx-tools` custom templates to enable workflow containers.
#### Apply modern template
Modify the `template` attribute to use the modern template and apply the `docfx-tools` custom templates to enable workflow containers.

```json
"template": [
Expand All @@ -187,7 +218,8 @@ apiRules:
]
```

6) Modify the `globalMetadata` attribute to change package name and add the `_appFooter` attribute and description.
#### Update global metadata
Modify the `globalMetadata` attribute to change package name and add the `_appFooter` attribute and description.

```json
"globalMetadata": {
Expand All @@ -197,7 +229,8 @@ apiRules:
}
```

7) Add the `markdownEngineProperties` attribute to enable [markdig](https://github.com/xoofx/markdig)(docfx markdown processor) extensions for additional markdown functionality.
#### Enable custom extensions and xref
Add the `markdownEngineProperties` attribute to enable [markdig](https://github.com/xoofx/markdig)(docfx markdown processor) extensions for additional markdown functionality.

```json
"build": {
Expand All @@ -210,7 +243,7 @@ apiRules:
}
```

8) Add the `xref` attribute to cross reference classes from the Bonsai library (if you use them to build your package). If you have used other libraries with `docfx` documentation, add their `xrefmap.yml` here as well.
Add the `xref` attribute to cross reference classes from the Bonsai library (if you use them to build your package). If you have used other libraries with `docfx` documentation, add their `xrefmap.yml` here as well.

```json
"build": {
Expand All @@ -224,12 +257,12 @@ apiRules:

Once you are satisfied with the website, publish to [GitHub Pages](https://docs.github.com/en/pages/getting-started-with-github-pages/creating-a-github-pages-site). A GitHub Actions workflow should already be in place if you follow the standard folder structure. Make sure `docs.yml` is in the `.github/workflows` [folder](#repository-organization).

1) Setup a new branch called `gh-pages` on your fork of the repository.
2) Go to your repo settings > `Pages` > `Build and deployment` > under `Source` select `Deploy from a branch` and make sure `gh-pages` is selected.
3) Commit your edits and push them online.
4) Under the `Actions` tab of your GitHub repo, trigger the `Build docs` workflow manually with the `Run workflow` button on the branch you committed your edits to. This will build the docs site on the `gh-pages` branch.
5) Once the `Build docs` workflow has been completed, the `pages-build-deployment` workflow will run and publish your forked repo automatically.
7) The URL for the site can be found in the `Pages` section of your repository settings.
1. Setup a new branch called `gh-pages` on your fork of the repository.
2. Go to your repo settings > `Pages` > `Build and deployment` > under `Source` select `Deploy from a branch` and make sure `gh-pages` is selected.
3. Commit your edits and push them online.
4. Under the `Actions` tab of your GitHub repo, trigger the `Build docs` workflow manually with the `Run workflow` button on the branch you committed your edits to. This will build the docs site on the `gh-pages` branch.
5. Once the `Build docs` workflow has been completed, the `pages-build-deployment` workflow will run and publish your forked repo automatically.
7. The URL for the site can be found in the `Pages` section of your repository settings.

> [!NOTE]
> To ensure that the Bonsai Editor can link to the reference documentation for package operators, make sure that the `PackageProjectUrl` in either the `.csproj` file or the `Directory.Build.props` file points to the URL for the documentation website.
Expand All @@ -250,32 +283,32 @@ To keep your online GitHub repository clean, you can use `.gitignore` files to i
...
```

## Testing unpublished packages
## Creating example workflows
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I am not sure about the title change here, as it changes the emphasis and function of this section. For contributing to documentation for published packages there is no need to go through the visual studio pipeline. and a separate section on saving Bonsai workflows in the documentation style guide (./documentation-style-guide.md#bonsai-workflows).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about the duplicate section. I think I would rather move everything to one of the sections then. Perhaps this one, since including example workflows feels really part of the core documentation infrastructure and not simply a style recommendation. Does that sound good?

Maybe I am confused about the distinction between "published" and "unpublished" packages. I was imagining that documentation would always be done targeting the source code of an existing package, and so in a way you always treat it as "unpublished" since you might be documenting a new release.

Although perhaps you were thinking of cookbook style or recipe examples and tutorials which might live outside of any package and where you just describe in detail how to use existing packages to achieve a specific goal? In that case I can see the difference, but then the entire way of setting up docfx might be different anyway and potentially deserve a separate guide? For now maybe we just explain how to document custom packages?

Copy link
Contributor Author

@banchan86 banchan86 Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I am not sure how to answer this, but I think there are two separate issues to consider here:

  1. Unpublished vs published packages

I guess the distinction for me was writing documentation for new releases like Bonsai.PulsePal vs existing releases like BonVision.

It was a pain point for me when i started working on the Bonsai.PulsePal documentation because I initially did not know how to test the package out (which at that time was not released yet). I took what I learnt from the 'Creating a Package' guide and decided to write up this short section in case it helped other documentation contributors who were not package developers. However I do think that maybe it is a little out of place here and package developers writing documentation for their own packages would already have this knowledge. In that case maybe this section is unnecessary?

  1. Bonsai workflow examples

I think consolidating the instructions for how to generate Bonsai example workflows might be a good idea as I do think its part of the core documentation infrastruture. I guess I was just struggling with how to organise this content and chose to document the import of the docfx-tools submodule as part of the docfx infrastructure and the saving of the Bonsai examples workflows as part of the media/content recommendations.

In that case, I would probably make a new separate section in this article and title it "Bonsai workflow support" and pull in the content about the importing of the docfx tools submodule, where to save the Bonsai workflows, and how to use the 'build.ps1' script. That leaves the markdown instructions for how to include them in articles. It might still be a good idea to leave in the style guide, along with a link to this "Bonsai workflow support" section to ensure that people set up the infrastructure correctly. If thats ok with you, I could do a rewrite in a new commit (unless you want to organise it yourself differently?)


To write documentation for [new packages or releases](./create-package.md) that have not yet been published to the community, it is good practice to start by writing test workflows in Visual Studio.

1) Install [Visual Studio](https://www.visualstudio.com/) (the community edition can be installed for free for open-source projects).
2) Install Bonsai VS Extensions. Assuming Bonsai is already installed, from the Windows Start Menu, search for the "Install Bonsai VS Extensions" shortcut and run it.
3) In Visual Studio, open `src/PackageName.sln` in the repository.
4) Press F5 to open the Bonsai editor with the new package added.
5) From here, you can make Bonsai workflows and save them as per normal.
1. Install [Visual Studio](https://www.visualstudio.com/) (the community edition can be installed for free for open-source projects).
2. Install Bonsai VS Extensions. Assuming Bonsai is already installed, from the Windows Start Menu, search for the "Install Bonsai VS Extensions" shortcut and run it.
3. In Visual Studio, open `src/PackageName.sln` in the repository.
4. Press F5 to open the Bonsai editor with the new package added.
5. From here, you can make Bonsai workflows and save them to the `workflows` folder as examples.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be good to throw a link to the bonsai workflow container section and also change the text a little bit to reinforce the function of this section. Suggested change - "From here, you can test the package and save Bonsai workflows as examples."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I will rephrase.



## Troubleshooting

### File not found errors
#### File not found errors

These are usually due to folders not being in the right place, check the directory trees above or a repository that has been recently updated (for instance https://bonsai-rx.org/machinelearning/) for reference. Also pay attention to any scripts/css files that require modification of the package name/link/source location.
These are usually due to folders not being in the right place. Check the directory trees above or a repository that has been recently updated, e.g. [bonsai-rx/machinelearning](https://bonsai-rx.org/machinelearning/), for reference. Also pay attention to any scripts or CSS files that require modification to the package name or link/source location.

### Docfx local preview not updating
#### Docfx local preview not updating

If you are running the local preview `dotnet docfx --serve` and not seeing changes:

1) Make sure that any changes you have made in your code editor are saved.
2) Hard refresh pages in the browser using either `Ctrl`+`Shift`+`R` and `Ctrl`+`F5` or clear the cache to avoid cache issues.
3) As a last resort - check that the content updates online by [publishing your website](#publishing-to-github-pages).
1. Make sure that any changes you have made in your code editor are saved.
2. Hard refresh pages in the browser using either `Ctrl`+`Shift`+`R` and `Ctrl`+`F5` or clear the cache to avoid cache issues.
3. As a last resort - check that the content updates online by [publishing your website](#publishing-to-github-pages).

### Differences between local and online builds
#### Differences between local and online builds

If there are discrepancies between local and online builds and they persist, this could indicate a `docfx` version conflict. The online build process uses a local installation of `docfx` but it is also possible to install `docfx` globally (which we do not recommend). To check if this is the cause of the discrepancy:

Expand Down
Loading