[Feature] Sprint enhanced models & daily sprint-issue history (to build velocity report) (#136)

* Add sprint metrics

* remove unified rag

* add more sprint metrics

* Final updates for sprint metrics, add sprint story points models

* syntax

* syntax

* syntax

* syntax

* requirements.txt + yml

* requirements.txt + yml

* yml config

* yml config

* schema

* tests + docs

* requirements.txt

* Jira sprint report updates, new enhanced model with sprint_issue_history

* seed

* New sprint issue history and sprint enhanced models

* Updated logic for sprint issue history and sprint enhanced

* New updates

* reset dbt_project.yml

* Docs

* test databricks sql warehouse fix

* test databricks sql warehouse fix

* Apply suggestions from code review

Co-authored-by: Joe Markiewicz <[email protected]>

* PR review

* Add certifi, more PR review updates

* Final PR review notes

* Release review notes

* Update packages.yml

---------

Co-authored-by: Joe Markiewicz <[email protected]>

Author: fivetran-avinash

.gitignore

@@ -2,6 +2,6 @@ target/
 dbt_modules/
 logs/
 .DS_Store
-models/intermediate/int_jira__all_field_options.sql
+env/
 dbt_packages/
 package-lock.yml

.quickstart/quickstart.yml

@@ -21,7 +21,9 @@ destination_configurations:
 
 public_models: [
   "jira__daily_issue_field_history",
+  "jira__daily_sprint_issue_history",
   "jira__user_enhanced",
   "jira__issue_enhanced",
-  "jira__project_enhanced"
+  "jira__project_enhanced",
+  "jira__sprint_enhanced"
 ]

CHANGELOG.md

@@ -1,6 +1,35 @@
-# dbt_jira version.version
+# dbt_jira v0.20.0
+This release includes the following updates.
+
+**8 total changes • 6 possible breaking changes (requires a --full-refresh run after upgrading)**
+
+| Data Model                                    | Change Type | Old Name | New Name                                  | Notes                                                             |
+|---------------------------------------------------|-------------|----------|-------------------------------------------|-------------------------------------------------------------------|
+| [jira__daily_sprint_issue_history](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__daily_sprint_issue_history)        | New Model   |     |  | Each record represents a snapshot of a sprint and its assorted issues on a given day between the sprint start date and the most recent update to the sprint.               |
+| [jira__sprint_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__sprint_enhanced)        | New Model   |          |    |       This model provides enhanced sprint-level metrics in Jira, incorporating data from sprints, sprint-related issue history, and sprint-based story points tracking.        |
+| [jira__daily_issue_field_history](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__daily_issue_field_history)        | New Columns   |          |  `story_points`, `story_point_estimate`    |    Adding story point fields by default to persist to sprint/issue end models.     |
+| [jira__issue_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__issue_enhanced)        | New Columns   |          |  `story_points`, `story_point_estimate`    |    Adding story point fields by default to persist to sprint/issue end models.       |
+| [jira__user_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__user_enhanced)        | New Columns   |          |  `is_active`    |    Helps show the active user records.    |
+| [stg_jira__issue_multiselect_history](https://fivetran.github.io/dbt_jira_source/#!/model/model.jira_source.stg_jira__issue_multiselect_history)        | New Columns   |          |  `is_active`    |    Shows which of the field history values is the current one.       |
+| [stg_jira__issue_field_history](https://fivetran.github.io/dbt_jira_source/#!/model/model.jira_source.stg_jira__issue_field_history)        | New Columns   |          |  `is_active`    |    Shows which of the field history values is the current one.      |
+| [stg_jira__issue_user](https://fivetran.github.io/dbt_jira_source/#!/model/model.jira_source.stg_jira__issue_user)        | New Columns   |          |  `is_active`    |    Shows which of the field history values is the current one.      |
+
+## Breaking Changes
+- Added `story_points` and `story_point_estimate` as default fields in the `jira__daily_issue_field_history` and `jira__issue_enhanced` models. ([#136](https://github.com/fivetran/dbt_jira/pull/136))
+  - **Important**: These fields are now included by default. If your `issue_field_history_columns` variable already includes `story_points` or `story_point_estimate`, you must remove them and run a `--full-refresh` to avoid duplication errors.
+
+## Feature Updates: New Sprint Reports
+- Introduced two new models: `jira__daily_sprint_issue_history` to look at daily sprint issue snapshots, and `jira__sprint_enhanced` to examine high level sprint metrics, such as velocity, time tracking and story point completion. Customers should now be able to build valuable sprint visualizations, like their own velocity reports, estimation tracking, and goal metrics. ([#136](https://github.com/fivetran/dbt_jira/pull/136)) 
+- If users are not utilizing the `sprint` source, these models can be disabled by setting the variable `jira_using_sprints` to `false`. More instructions [are available in the README](https://github.com/fivetran/dbt_jira?tab=readme-ov-file#step-4-disable-models-for-non-existent-sources). ([#136](https://github.com/fivetran/dbt_jira/pull/136))
+
+## Under the Hood
+- Added consistency and integrity tests for the `jira__sprint_enhanced` model. ([#136](https://github.com/fivetran/dbt_jira/pull/136))
+- Added new `jira__daily_sprint_issue_history` and `jira__sprint_enhanced`  to public models in the `quickstart.yml` configuration. ([#136](https://github.com/fivetran/dbt_jira/pull/136))
+- Corrected formatting on intermediate and end models to conform with best practices and improve readability. ([#136](https://github.com/fivetran/dbt_jira/pull/136))
 
 ## Documentation
+- Updated yml documentation and README with new models, fields and descriptions. ([#136](https://github.com/fivetran/dbt_jira/pull/136))
+- [Created a DECISIONLOG](https://github.com/fivetran/dbt_jira/blob/main/DECISIONLOG.md#enhancing-jira-sprint-reporting-with-flexible-metrics) to explain why we bring in multiple metrics from current sprint data as well as historical sprint data for the `jira__sprint_enhanced` model. ([#136](https://github.com/fivetran/dbt_jira/pull/136))
 - Added Quickstart model counts to README. ([#135](https://github.com/fivetran/dbt_jira/pull/135))
 - Corrected references to connectors and connections in the README. ([#135](https://github.com/fivetran/dbt_jira/pull/135))
 

DECISIONLOG.md

@@ -0,0 +1,7 @@
+# Decision Log
+In creating this package, which is meant for a wide range of use cases, we had to take opinionated stances on a few different questions we came across during development. We've consolidated significant choices we made here, and will continue to update as the package evolves.
+
+## Enhancing Jira Sprint Reporting with Flexible Metrics
+To improve sprint reporting in the Jira dbt package, we introduced two new models, `jira__daily_sprint_issue_history` and `jira__sprint_enhanced`, designed to capture key sprint metrics such as velocity, time tracking, and story point completion.
+  
+If further refinements or reporting is needed for the sprint report, customers can submit a feature request by clicking the [New Issue button in our package issue page](https://github.com/fivetran/dbt_jira/issues).

LICENSE

@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright © 2025 Fivetran Inc.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,4 +1,6 @@
-<p align="center">
+# Jira Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_jira/))
+
+<p align="left">
     <a alt="License"
         href="https://github.com/fivetran/dbt_jira/blob/main/LICENSE">
         <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" /></a>
@@ -13,13 +15,12 @@
         <img src="https://img.shields.io/badge/Fivetran_Quickstart_Compatible%3F-yes-green.svg" /></a>
 </p>
 
-# Jira Transformation dbt Package ([Docs](https://fivetran.github.io/dbt_jira/))
 ## What does this dbt package do?
 - Produces modeled tables that leverage Jira data from [Fivetran's connector](https://fivetran.com/docs/applications/jira) in the format described by [this ERD](https://fivetran.com/docs/applications/jira#schemainformation) and builds off the output of our [Jira source package](https://github.com/fivetran/dbt_jira_source).
 - Enables you to better understand the workload, performance, and velocity of your team's work using Jira issues. It performs the following actions:
   - Creates a daily issue history table so you can quickly create agile reports, such as burndown charts, along any issue field.
   - Enriches the core issue table with relevant data regarding its workflow and current state.
-  - Aggregates bandwidth and issue velocity metrics along projects and users.
+  - Aggregates bandwidth and issue velocity metrics along projects, users and sprints.
 - Generates a comprehensive data dictionary of your source and modeled Jira data through the [dbt docs site](https://fivetran.github.io/dbt_jira/).
 
 <!--section="jira_transformation_model"-->
@@ -32,8 +33,11 @@ The following table provides a detailed list of all tables materialized within t
 | [jira__issue_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__issue_enhanced)            | Each record represents a Jira issue, enriched with data about its current assignee, reporter, sprint, epic, project, resolution, issue type, priority, and status. It also includes metrics reflecting assignments, sprint rollovers, and re-openings of the issue. Note that all epics are considered `issues` in Jira and are therefore included in this model (where `issue_type='epic'`). |
 | [jira__project_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__project_enhanced)            | Each record represents a project, enriched with data about the users involved, how many issues have been opened or closed, the velocity of work, and the breadth of the project (i.e., its components and epics). |
 | [jira__user_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__user_enhanced)            | Each record represents a user, enriched with metrics regarding their open issues, completed issues, the projects they work on, and the velocity of their work. |
+| [jira__sprint_enhanced](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__sprint_enhanced)            | Each record represents a sprint, enriched with metrics regarding issues created, brought into, and resolved in the sprint, the story points estimated and worked, and the velocity of work completed. |
+| [jira__daily_sprint_issue_history](https://fivetran.github.io/dbt_jira/#!/model/model.jira.jira__daily_sprint_issue_history)            | Each record represents a snapshot of a sprint and its assorted issues on a given day between the sprint start date and the most recent update to the sprint. |
+
 ### Materialized Models
-Each Quickstart transformation job run materializes 43 models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.
+Each Quickstart transformation job run materializes 44 models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.
 <!--section-end-->
 
 ## How do I use the dbt package?
@@ -68,7 +72,7 @@ Include the following jira package version in your `packages.yml` file:
 ```yaml
 packages:
   - package: fivetran/jira
-    version: [">=0.19.0", "<0.20.0"]
+    version: [">=0.20.0", "<0.21.0"]
 
 ```
 ### Step 3: Define database and schema variables
@@ -109,7 +113,7 @@ vars:
 ```
 
 #### Define daily issue field history columns
-The `jira__daily_issue_field_history` model generates historical data for the columns specified by the `issue_field_history_columns` variable. By default, the only columns tracked are `status`, `status_id`, and `sprint`, but all fields found in the Jira `FIELD` table's `field_name` column can be included in this model. The most recent value of any tracked column is also captured in `jira__issue_enhanced`.
+The `jira__daily_issue_field_history` model generates historical data for the columns specified by the `issue_field_history_columns` variable. By default, the only columns tracked are `status`, `status_id`,`sprint`, `story_points` and `story_point_estimate`, but all fields found in the Jira `FIELD` table's `field_name` column can be included in this model. The most recent value of any tracked column is also captured in `jira__issue_enhanced`.
 
 If you would like to change these columns, add the following configuration to your `dbt_project.yml` file. After adding the columns to your `dbt_project.yml` file, run the `dbt run --full-refresh` command to fully refresh any existing models:
 
@@ -184,7 +188,7 @@ This dbt package is dependent on the following dbt packages. These dependencies
 ```yml
 packages:
     - package: fivetran/jira_source
-      version: [">=0.7.0", "<0.8.0"]
+      version: [">=0.8.0", "<0.9.0"]
 
     - package: fivetran/fivetran_utils
       version: [">=0.4.0", "<0.5.0"]

dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'jira'
-version: '0.19.0'
+version: '0.20.0'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 vars: