diff --git a/README.md b/README.md index 653f742..8174efc 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,9 @@ - [Expected Output](#expected-output) - [Index page example](#index-page-example) - [Issue page example](#issue-page-example) +- [Documentation Ticket Introduction](#documentation-ticket-introduction) + - [Labels](#labels) + - [Hosting Documentation Tickets in a Solo Repository](#hosting-documentation-tickets-in-a-solo-repository) - [Project Setup](#project-setup) - [Run Scripts Locally](#run-scripts-locally) - [Run Pylint Check Locally](#run-pylint-check-locally) @@ -28,13 +31,18 @@ - [License Information](#license-information) - [Contact or Support Information](#contact-or-support-information) -A tool designed to data-mine GitHub repositories for issues containing project documentation (e.g. tagged with feature-related labels). This tool automatically generates comprehensive living documentation in Markdown format, providing detailed feature overview pages and in-depth feature descriptions. +![vision.jpg](img/vision.jpg) + +A tool designed to data-mine GitHub repositories for [documentation tickets](#documentation-ticket-introduction) containing project documentation (e.g. tagged with feature-related labels). This tool automatically generates comprehensive living documentation in Markdown format, providing detailed feature overview pages and in-depth feature descriptions. ## Motivation + Addresses the need for continuously updated documentation accessible to all team members and stakeholders. Achieves this by extracting information directly from GitHub issues and integrating this functionality to deliver real-time, markdown-formatted output. Ensures everyone has the most current project details, fostering better communication and efficiency throughout development. ## Usage + ### Prerequisites + Before we begin, ensure you have a GitHub Token with permission to fetch repository data such as Issues and Pull Requests. ### Adding the Action to Your Workflow @@ -115,9 +123,11 @@ See the full example of action step definition (in example are used non-default ``` ## Action Configuration + Configure the action by customizing the following parameters based on your needs: ### Environment Variables + - **LIV_DOC_ACCESS_TOKEN** (required): - **Description**: GitHub access token for authentication, that has a permission to fetch from requested repositories. - **Usage**: Store it in the GitHub repository secrets and reference it in the workflow file using `${{ secrets.LIV_DOC_ACCESS_TOKEN }}`. @@ -128,6 +138,7 @@ Configure the action by customizing the following parameters based on your needs ``` #### How to Create a Token with Required Scope + 1. Go to your GitHub account settings. 2. Click on the `Developer settings` tab in the left sidebar. 3. In the left sidebar, click on `Personal access tokens` and choose `Tokens (classic)`. @@ -155,6 +166,7 @@ Configure the action by customizing the following parameters based on your needs 8. Authorize new token to organization you want to fetch from. #### How to Store Token as a Secret + 1. Go to the GitHub repository, from which you want to run the GitHub Action. 2. Click on the `Settings` tab in the top bar. 3. In the left sidebar, click on `Secrets and variables` > `Actions`. @@ -162,6 +174,7 @@ Configure the action by customizing the following parameters based on your needs 5. Name the token `LIV_DOC_ACCESS_TOKEN` and paste the token value. ### Inputs + - **repositories** (required) - **Description**: A JSON string defining the repositories to be included in the documentation generation. - **Usage**: List each repository with its organization name, repository name, query labels and attached projects you want to filter if any. Only projects with these titles will be considered. For no filtering projects, leave the list empty. @@ -191,6 +204,7 @@ Configure the action by customizing the following parameters based on your needs ``` ### Features De/Activation + - **project-state-mining** (optional, `default: false`) - **Description**: Enables or disables the mining of project state data from [GitHub Projects](https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects). - **Usage**: Set to true to activate. @@ -219,6 +233,7 @@ Configure the action by customizing the following parameters based on your needs ``` ## Action Outputs + The Living Documentation Generator action provides a key output that allows users to locate and access the generated documentation easily. This output can be utilized in various ways within your CI/CD pipeline to ensure the documentation is effectively distributed and accessible. The output-path can not be an empty string. It can not aim to the root and other project directories as well. @@ -235,10 +250,12 @@ The output-path can not be an empty string. It can not aim to the root and other ``` ## Expected Output + The Living Documentation Generator is designed to produce an Issue Summary page (index.md) along with multiple detailed single issue pages. ### Index Page Example + ```markdown # Issue Summary page @@ -287,15 +304,58 @@ Users often struggle to find specific books in a large catalog. An advanced sear ... ``` +## Documentation Ticket Introduction + +A **Documentation Ticket** is a small piece of documentation realised as GitHub Issue dedicated to project documentation. Unlike development-focused tickets, Documentation Ticket remain open continuously, evolving as updates are needed, and can be reopened or revised indefinitely. They are not directly tied to Pull Requests (PRs) but can be referenced for context. + +- **Content Rules**: + - **Non-technical Focus:** + - Keep the documentation body free of technical solution specifics. + - Technical insights should be accessible through linked PRs or Tickets within the development repository. + - **Independent Documentation:** + - Ensure the content remains independent of implementation details to allow a clear, high-level view of the feature or user story's purpose and functionality. + +### Labels + +To enhance clarity, the following label groups define and categorize each Documentation Issue: +- **Topic**: + - **{Name}Topic:** Designates the main focus area or theme relevant to the ticket, assigned by the editor for consistency across related documentation. + - Examples: `ReportingTopic`, `UserManagementTopic`, `SecurityTopic`. + - **noTopic:** Indicates that the ticket does not align with a specific topic, based on the editor's discretion. +- **Type**: + - **DocumentedUserStory:** Describes a user-centric functionality or process, highlighting its purpose and value. + - Encompasses multiple features, capturing the broader goal from a user perspective. + - **DocumentedFeature:** Details a specific feature, providing a breakdown of its components and intended outcomes. + - Built from various requirements and can relate to multiple User Stories, offering an in-depth look at functionality. + - **DocumentedRequirement:** Outlines individual requirements or enhancements tied to the feature or user story. +- **Issue States**: + - **Upcoming:** The feature, story, or requirement is planned but not yet implemented. + - **Implemented:** The feature or requirement has been completed and is in active use. + - **Deprecated:** The feature or requirement has been phased out or replaced and is no longer supported. + +**DocumentedUserStory** and **DocumentedFeature** serve as **Epics**, whereas **DocumentedRequirement** represents specific items similar to feature enhancements or individual requirements. + +### Hosting Documentation Tickets in a Solo Repository + +Using a dedicated repository solely for documentation tickets provides multiple advantages: +- **Streamlined Management:** This avoids cross-project conflicts and board exclusions and enables specialized templates solely for documentation purposes. +- **Focused Access Control:** This allows a small team to manage and edit documentation without interference, maintaining high-quality content. +- **Optimized Data Mining:** Supports easier and more efficient data extraction for feedback and review cycles through Release Notes. +- **Implementation Reflection:** Mirrors elements from the implementation repositories, providing a high-level knowledge source that is valuable for both business and technical teams. +- **Release Notes Integration:** Documentation can evolve based on insights from release notes, serving as a dynamic feedback loop back to the documentation repository. + ## Project Setup + If you need to build the action locally, follow these steps: ### Prepare the Environment + ```shell python3 --version ``` ### Set Up Python Environment + ```shell python3 -m venv venv source venv/bin/activate @@ -303,9 +363,11 @@ pip install -r requirements.txt ``` ## Run Scripts Locally + If you need to run the scripts locally, follow these steps: ### Create the Shell Script + Create the shell file in the root directory. We will use `run_script.sh`. ```shell touch run_script.sh @@ -316,6 +378,7 @@ Add the shebang line at the top of the sh script file. ``` ### Set the Environment Variables + Set the configuration environment variables in the shell script following the structure below. Also make sure that the INPUT_GITHUB_TOKEN is configured in your environment variables. INPUT_OUTPUT_PATH can not be an empty string. It can not aim to the root and other project directories as well. @@ -335,6 +398,7 @@ export INPUT_VERBOSE_LOGGING="true" ``` ### Running the script locally + For running the GitHub action incorporate these commands into the shell script and save it. ``` cd src || exit 1 @@ -345,17 +409,20 @@ cd .. || exit 1 ``` ### Make the Script Executable + From the terminal that is in the root of this project, make the script executable: ```shell chmod +x run_script.sh ``` ### Run the Script + ```shell ./run_script.sh ``` ## Run Pylint Check Locally + This project uses [Pylint](https://pypi.org/project/pylint/) tool for static code analysis. Pylint analyses your code without actually running it. It checks for errors, enforces, coding standards, looks for code smells etc. @@ -367,6 +434,7 @@ We are aiming to keep our code quality high above the score 9.5. Follow these steps to run Pylint check locally: ### Set Up Python Environment + From terminal in the root of the project, run the following command: ```shell @@ -378,6 +446,7 @@ pip install -r requirements.txt This command will also install a Pylint tool, since it is listed in the project requirements. ### Run Pylint + Run Pylint on all files that are currently tracked by Git in the project. ```shell pylint $(git ls-files '*.py') @@ -391,6 +460,7 @@ pylint living_documentation_generator/generator.py ``` ### Expected Output + This is the console expected output example after running the tool: ``` ************* Module main @@ -401,6 +471,7 @@ Your code has been rated at 9.41/10 (previous run: 8.82/10, +0.59) ``` ## Run Black Tool Locally + This project uses the [Black](https://github.com/psf/black) tool for code formatting. Black aims for consistency, generality, readability and reducing git diffs. The coding style used can be viewed as a strict subset of PEP 8. @@ -412,6 +483,7 @@ We also do exclude the `tests/` file from the black formatting. Follow these steps to format your code with Black locally: ### Set Up Python Environment + From terminal in the root of the project, run the following command: ```shell @@ -423,6 +495,7 @@ pip install -r requirements.txt This command will also install a Black tool, since it is listed in the project requirements. ### Run Black + Run Black on all files that are currently tracked by Git in the project. ```shell black $(git ls-files '*.py') @@ -436,6 +509,7 @@ black living_documentation_generator/generator.py ``` ### Expected Output + This is the console expected output example after running the tool: ``` All done! ✨ 🍰 ✨ @@ -473,6 +547,7 @@ htmlcov/index.html ``` ## Deployment + This project uses GitHub Actions for deployment draft creation. The deployment process is semi-automated by a workflow defined in `.github/workflows/release_draft.yml`. - **Trigger the workflow**: The `release_draft.yml` workflow is triggered on workflow_dispatch. @@ -538,6 +613,7 @@ This feature allows you to generate structured output for the living documentati We welcome contributions to the Living Documentation Generator! Whether you're fixing bugs, improving documentation, or proposing new features, your help is appreciated. #### How to Contribute + Before contributing, please review our [contribution guidelines](https://github.com/AbsaOSS/living-doc-generator/blob/master/CONTRIBUTING.md) for more detailed information. ### License Information diff --git a/VISION.md b/VISION.md new file mode 100644 index 0000000..79a23c8 --- /dev/null +++ b/VISION.md @@ -0,0 +1,242 @@ +# Living Documentation Generator + +- [Purpose of this Page](#purpose-of-this-page) +- [Regimes and Their Usage](#regimes-and-their-usage) + - [DocumentedUserStories](#documenteduserstories) + - [DocumentedFeatures](#documentedfeatures) + - [DocumentedRequirements](#documentedrequirements) + - [Coverage matrix](#coverage-matrix) + - [Repositor(y/ies) YML Files](#repository-yml-files) + - [Incidents](#incidents) + - [Release Notes](#release-notes) + - [User Guide](#user-guide) +- [Rationale for an Isolated GitHub Repository](#rationale-for-an-isolated-github-repository) +- [Proposed Issue Content Examples](#proposed-issue-content-examples) +- [Content Formatting for Documentation Types (US, Features, Req)](#content-formatting-for-documentation-types-us-features-req) +- [Examples](#examples) + - [DocumentedUserStory GH Issue](#documenteduserstory-gh-issue) + - [DocumentedFeature GH Issue](#documentedfeature-gh-issue) + - [DocumentedRequirement GH Issue](#documentedrequirement-gh-issue) + +## Purpose of this Page + +The goals of this document are to: +- Outline features yet to be implemented. +- Expand upon the content provided in README.md. + +## Regimes and Their Usage + +Outlined below are the various regimes, their intended purposes, and the specific inputs and outputs associated with each. +- Term **regime**: a User configurable set of features that can be enabled or disabled. Each regime controls the generation of a specific type of documentation. + +### DocumentedUserStories + +Produces documentation centered on user-oriented stories that outline workflows or features from the user’s perspective. +- Build on expectation that User Stories are one of the minimum unit of documentation to be created. +- When enabled with DocumentedFeatures regime and relation between User Stories and Features is established, both outputs can be connected by the links. + +### DocumentedFeatures + +Produces documentation specifically for project features. +- Build on expectation that Features are one of the minimum unit of documentation to be created. +- **Default** in current implementation. + +### DocumentedRequirements + +Produces documentation for granular requirements tied to specific features. +- Can be operated only with DocumentedFeatures regime enabled. + +### Coverage matrix + +Provides an overview of the test coverage for User Stories, Features, and Requirements, enabling teams to track documentation alignment with test validation. This feature visualizes test completeness by listing covered items across unit, integration, and E2E tests, helping to identify any gaps in the test suite for crucial documentation items. + +### Repositor(y/ies) YML Files + +Generates an organized summary of all .yml files within specified repositories, capturing key details such as events, jobs, and workflows. This overview can serve as a quick reference for repository configurations, supporting consistent CI/CD practices across projects. + +### Incidents + +Documents historical and active incidents related to the project, capturing details such as incident type, affected features or requirements, severity level, and resolution status. This feature enhances transparency and helps trace incident tickets back to related documentation for streamlined troubleshooting and future prevention. + +### Release Notes + +Generates consolidated release notes summarizing feature deployments, updates, and deprecations. Links back to related documentation tickets, features, and requirements provide context, enabling easy navigation from release highlights to detailed documentation. This feature supports alignment across development, testing, and business stakeholders by providing a clear view of project progress. + +### User Guide + +Uses **DocumentedUserStories** as the foundational data source to build a User Guide. +- **Content:** + - Each User Story should define a required input for User Guide generation. + - Each User Story should ideally include one end-to-end (E2E) test, with a preference for automation to ensure repeatable validation. + - E2E test should generate screenshots for User Guide inclusion. +- **Structure:** + - **Body:** Organizes content into well-defined, categorized chapters. + - **Steps:** Includes descriptive steps with illustrative images for better clarity and user comprehension. + +## Rationale for an Isolated GitHub Repository + +Hosting documentation tickets in a dedicated GitHub repository offers several key benefits: +- **Process Clarity and Separation:** Maintains a clear boundary between documentation tickets and active project tickets, reducing clutter and ensuring documentation is easily accessible without interference from active development. +- **Pilot Platform:** Serves as an initial testing ground for the documentation approach, enabling iterative improvements based on early feedback. +- **Controlled Access:** Limits editing permissions to a core group, ensuring content accuracy and consistency, while allowing broader read-only access for stakeholders and team members. + +## Proposed Issue Content Examples + +Here are structural suggestions for key documentation types, using examples to clarify content organization: +- **User Stories (US):** Outlines overarching user needs and goals, providing context for related features. +- **Features (Feat):** Describes specific aspects or components supporting the User Story, broken down into clear, actionable tasks. +- **Requirements (Req)**: Lists individual requirements essential for completing a feature, enhancing detail and traceability. + +**Using Task Lists:** Leverage task lists in both User Stories and Features to improve clarity and progress tracking. + +## Content Formatting for Documentation Types (US, Feat, Req) + +A consistent structure helps standardize documentation across different types: +- **Open Format:** + - **Information Collection:** Use an initial section to gather relevant information at the start. + - **Description:** Separate sections for high-level descriptions to serve both documentation and User Guide purposes (in case of US). +- **Body (Markdown):** + - **Versioning:** + - **Expected:** Specifies the target version for initial release. + - **Available:** Indicates the actual version released. + - **Deprecated from:** States the version from which the feature or requirement is no longer supported. + - Note: Version can link to Release Notes. + - **Dependencies:** + - No direct linkage to development tickets or PRs is required. +- **Labels:** + - **Topic:** + - Requirements do not need a topic. + - For Features, each issue should focus on a single topic (as a label). + - User Stories are broader in scope. + - **Type:** + - DocumentedUserStory, DocumentedFeature, DocumentedRequirement + - **State:** + - Upcoming, In Progress, Implemented, Deprecated + +This structure is intended to ensure consistency, clarity, and easy navigation across documentation types, fostering a cohesive understanding of project features and requirements. + +## Examples + +Mined information, not needed to repeat in body of the issue: +- **Unique-ID**: AbsaOSS/any-project-docs/4 +- **Type:** DocumentedUserStory, DocumentedFeature, DocumentedRequirement (from label) +- **Topic:** SearchingTopic (from label) +- **State:** Upcoming, Implemented, Deprecated (from label) + +### DocumentedUserStory GH Issue + +````markdown +# User Story +- Title: Advanced Search Functionality +- Version: + - Expected: v1.2 + - Available: v1.2.1 + - Deprecated from: v5.3 + +## Description + +As a user, I want to be able to perform advanced searches within the application, so I can quickly locate specific content based on multiple filters and criteria. + +## Acceptance Criteria + +- The user can apply multiple filters simultaneously (e.g., date range, category). +- The search results update dynamically based on selected filters. +- The search results display within 2 seconds. + +```[tasklist] +## Related Features +- [ ] #5 +- [ ] #6 +- [ ] #34 +``` + +## User Guide + +### Description + +This user story covers the advanced search functionality, enabling users to locate specific information efficiently by setting filters. Each step of the search experience is tailored for usability and speed. + +### End-to-End Test to Guide Steps + +- Automated: True +- Prerequisite: + - Ensure at least one searchable object is in the database and that the user has access to the advanced search page. +- Steps: + - Step 1: Open the search page. + - Step 2: Define the search criteria. (screenshot: step-2.png) + - Step 3: Apply multiple filters. + - Step 4: Verify the search results update correctly. (screenshot: step-4.png) + - Step 5: Verify the search results display within 2 seconds. + +## Change Log + +| Version | Date | Comment | +|---------|------------|----------------------------------| +| v1 | 2024-10-30 | Initial version of the User Story | +```` + +### DocumentedFeature GH Issue + +````markdown +# Feature + +- Title: Date Filter for Search +- Version: + - Expected: v1.2 + - Available: v1.2.1 + - Deprecated from: - + +## Description + +The Date Filter allows users to narrow down search results by selecting a specific start and end date. It enhances search precision, especially for time-sensitive data. + +```[tasklist] +## Related User Stories +- [ ] #2 +``` + +```[tasklist] +## Related Requirements +- [ ] #45 +- [ ] #46 +``` + +## Change Log + +| Version | Date | Comment | +|---------|------------|----------------------------------| +| v1 | 2024-10-30 | Initial version of the Feature | + +```` + +### DocumentedRequirement GH Issue + +````markdown +# Requirement + +- Title: Filter Search Results by Date Range +- Priority: High +- Version: + - Expected: v1.1 + - Available: v1.1.5 + +## Description (What User does) + +The user selects a start and end date to limit search results to items within that date range. + +## Rationale (What User may want) + +The user may want to focus on specific periods to find relevant information faster or analyze data from a particular timeframe. + +```[tasklist] +## Associated Feature +- [ ] #2 +``` + +## Change Log + +| Version | Date | Comment | +|---------|------------|----------------------------------| +| v1 | 2024-10-30 | Initial version of the Requirement | + +```` diff --git a/img/vision.jpg b/img/vision.jpg new file mode 100644 index 0000000..9b6bfb7 Binary files /dev/null and b/img/vision.jpg differ