Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Config reference documentation autogeneration #2033

Open
wants to merge 45 commits into
base: main
Choose a base branch
from

Conversation

ilyakuz-db
Copy link
Contributor

@ilyakuz-db ilyakuz-db commented Dec 18, 2024

Changes

Documentation autogeneration tool. This tool uses same annotations_*.yml files as in json-schema

Result will go there and there

Tests

Manually

@eng-dev-ecosystem-bot
Copy link
Collaborator

Test Details: go/deco-tests/12401106836

Copy link

github-actions bot commented Jan 3, 2025

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 2033
  • Commit SHA: c6703c13637c0d85e6754e4c802f38052304a5d6

Checks will be approved automatically on success.

@ilyakuz-db ilyakuz-db changed the title [WIP] feat: Config reference documentation autogeneration feat: Config reference documentation autogeneration Jan 13, 2025
md "github.com/nao1215/markdown"
)

const (
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These templates are in this repo temporary to simplify the process of docs updating

This can be further improved by using include declarations in doc repo so we could generate only meaningful part here and store the wrapper file in docs repo

@ilyakuz-db
Copy link
Contributor Author

@pietern @shreyas-goenka @andrewnester this one is ready for review if anyone want to take a look!

More content updates to annotations will be introduced in follow-up PRs

Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you consider checking in the Markdown output as well?

That can serve as both a check that the contents is OK and that it doesn't inadvertently drift when changes to the schema or annotations are made.

bundle/internal/annotation/main.go Outdated Show resolved Hide resolved
bundle/internal/annotation/main.go Outdated Show resolved Hide resolved
bundle/internal/docs/README.md Outdated Show resolved Hide resolved
bundle/internal/docs/markdown.go Outdated Show resolved Hide resolved
bundle/internal/docs/markdown.go Show resolved Hide resolved
return isCustomField
}

func extractNodes(prefix string, props map[string]*jsonschema.Schema, refs map[string]jsonschema.Schema, customFields map[string]bool) []rootProp {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

return true
}

func resolveAdditionaProperties(v *jsonschema.Schema) *jsonschema.Schema {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo.

bundle/internal/schema/annotations.go Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
return nodes[i].Title < nodes[j].Title
})
return nodes
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very hard to tell what this function does and how it is supposed to work. Please include a few more comments with how you expect it to work, and if possible even a testcase or two to make sure it does work the way you assume. As is we need to run it and check the output to confirm this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added few test cases and re-arranged the code with more comments to highlight important parts

@ilyakuz-db
Copy link
Contributor Author

ilyakuz-db commented Jan 15, 2025

Did you consider checking in the Markdown output as well?

That can serve as both a check that the contents is OK and that it doesn't inadvertently drift when changes to the schema or annotations are made.

You mean to remove from .gitignore, right?

schema:
go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

docs: vendor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs: vendor

We don't add this implicitly anymore to other targets. If you're on CI, you can always specify both with "make vendor docs".

BTW, do you intended to run it on CI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, do you intended to run it on CI?

Currently the only use case of using that tool is to checkout CLI repo and run locally

We don't add this implicitly anymore to other targets

By implicitly you mean using make deps or in general?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean the make entry should be:

docs:
  ...

no dependency on vendor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate little bit more on the reasoning behind "We don't add this implicitly anymore to other targets" so I can understand how to better fix it?

Is it just that we don't want to rely on make deps syntax and this is still allowed?

docs:
  make vendor
  go run ....

schema:
go run ./bundle/internal/schema ./bundle/internal/schema ./bundle/schema/jsonschema.json

docs: vendor
go run ./bundle/internal/docs ./bundle/internal/schema ./bundle/internal/docs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this part if /bundle/internal/ ?

internal is meant for code that is not meant to be used outside of the package, but this is clearly intended to be used outside the package, that's why it's in Makefile.

How about placing this at top level? /docsgen/ ?

I see you're following existing structure, so it's a more of a question for @pietern

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you're following existing structure,

Yes I reused same approach as with json schema generation which is used in .codegen.json along with other internal scripts

@ilyakuz-db ilyakuz-db requested a review from pietern January 16, 2025 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants