Skip to content

Commit

Permalink
Address linting in pipeline adr
Browse files Browse the repository at this point in the history
  • Loading branch information
@dneed-nimble
dneed-nimble authored Nov 19, 2024
1 parent 3df187d commit 03fea16
Showing 1 changed file with 81 additions and 74 deletions.
155 changes: 81 additions & 74 deletions docs/adrs/0020-pipeline-academies-from-adb.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,105 +4,112 @@

## Status

Accepted
Accepted

## Context
## Context

The FIAT project currently relies on the Academies Database (ADB) and the FIAT database for all its data requirements. As we plan to introduce the "Pipeline Academies" functionality, we need to integrate data from additional applications within the RSD portfolio, including:
The FIAT project currently relies on the Academies Database (ADB) and the FIAT database for all its data requirements. As we plan to introduce the "Pipeline Academies" functionality, we need to integrate data from additional applications within the RSD portfolio, including:

- **Prepare Conversions**
- **Prepare Transfers**
- **Manage Free School Projects**
- **Complete**
- **Prepare Conversions**
- **Prepare Transfers**
- **Manage Free School Projects**
- **Complete**

Long term, the optimal solution is to consume this data through the APIs provided by each respective application. However, the timelines for implementing this feature make it impractical to expect these projects to deliver their APIs in time.
Long term, the optimal solution is to consume this data through the APIs provided by each respective application. However, the timelines for implementing this feature make it impractical to expect these projects to deliver their APIs in time.

To meet our deadlines, we propose an interim approach: leveraging the existing synchronisation mechanisms where these applications sync their data back to the Academies Database daily for reporting purposes. This allows us to utilise a consolidated and up-to-date data source for our needs while maintaining acceptable performance and reliability.
To meet our deadlines, we propose an interim approach: leveraging the existing synchronisation mechanisms where these applications sync their data back to the Academies Database daily for reporting purposes. This allows us to utilise a consolidated and up-to-date data source for our needs while maintaining acceptable performance and reliability.

## Decision
## Decision

For the interim, we will integrate data from the Academies Database (ADB) to support the "Pipeline Academies" feature. This approach involves querying specific tables within ADB, which aggregate data synced from the relevant applications.
For the interim, we will integrate data from the Academies Database (ADB) to support the "Pipeline Academies" feature. This approach involves querying specific tables within ADB, which aggregate data synced from the relevant applications.

In the medium term, we will transition to consuming this data via versioned APIs from the respective applications once they become available.
In the medium term, we will transition to consuming this data via versioned APIs from the respective applications once they become available.

## Reasons for the Decision
## Reasons for the Decision

1. **Access to Current Data**: Leveraging the ADB ensures we have access to data that is synced daily from the relevant applications, enabling us to work with up-to-date information.
1. **Access to Current Data**: Leveraging the ADB ensures we have access to data that is synced daily from the relevant applications, enabling us to work with up-to-date information.
2. **Simplicity and Feasibility**: This approach avoids dependencies on external teams delivering APIs within our tight timelines, reducing implementation risk.
3. **Performance**: Consolidating data into a single "super atomic" call from the ADB ensures our application remains performant, while allowing us to query the exact data we need.
4. **Maintainability**: While interim integration requires managing changes to ADB schema, transitioning to APIs later will improve maintainability with version control and documentation inherent to API usage.

2. **Simplicity and Feasibility**: This approach avoids dependencies on external teams delivering APIs within our tight timelines, reducing implementation risk.
## Consequences

3. **Performance**: Consolidating data into a single "super atomic" call from the ADB ensures our application remains performant, while allowing us to query the exact data we need.
### Short-Term Benefits

4. **Maintainability**: While interim integration requires managing changes to ADB schema, transitioning to APIs later will improve maintainability with version control and documentation inherent to API usage.
- **Rapid Development**: We can meet project deadlines without waiting for API availability.
- **Centralised Data Access**: ADB provides a unified source of truth for data from multiple applications.

## Consequences
### Drawbacks

### Short-Term Benefits
- **Rapid Development**: We can meet project deadlines without waiting for API availability.
- **Centralised Data Access**: ADB provides a unified source of truth for data from multiple applications.
1. **Dependence on ADB Schema**: Changes in the data model or structure of contributing applications may impact our queries and require maintenance.
2. **Limited Data Validation**: Relying on ADB means less control over the accuracy of the data, as it is aggregated and not fetched directly from source systems.

### Drawbacks
1. **Dependence on ADB Schema**: Changes in the data model or structure of contributing applications may impact our queries and require maintenance.
2. **Limited Data Validation**: Relying on ADB means less control over the accuracy of the data, as it is aggregated and not fetched directly from source systems.
### Medium-Term Considerations

### Medium-Term Considerations
- Transitioning to APIs will provide improved control, validation, and resilience against schema changes.
- Transitioning to APIs will provide improved control, validation, and resilience against schema changes.

## Trade-offs
## Trade-offs

### Simplified Implementation vs. Future Maintenance
This decision simplifies our immediate implementation but introduces potential maintenance overhead as ADB schemas evolve with changes in upstream applications.
### Simplified Implementation vs. Future Maintenance

### Interim vs. Long-Term Solutions
Our interim reliance on ADB data ensures we can deliver the feature on time but will require effort to migrate to API-based integration later.
This decision simplifies our immediate implementation but introduces potential maintenance overhead as ADB schemas evolve with changes in upstream applications.

## Requirements for API Integration (Medium-Term)
### Interim vs. Long-Term Solutions

While our analysis is ongoing, the following data fields are identified as necessary for the integration:
Our interim reliance on ADB data ensures we can deliver the feature on time but will require effort to migrate to API-based integration later.

### Prepare Conversions (Academisation API)
| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | School Name | [academisation].[Project] | [SchoolName] |
| URN | Unique Reference Number (URN) | [academisation].[Project] | [urn] |
| Project Type | Academy Type and Route | [academisation].[Project] | [AcademyTypeAndRoute] |
| Age Range | Age Range | [academisation].[Project] | [AgeRange] |
| Local Authority | Local Authority | [academisation].[Project] | [LocalAuthority] |
| Proposed Conversion/Transfer Date | Proposed Conversion Date | [academisation].[Project] | [ProposedAcademyOpeningDate] |
## Requirements for API Integration (Medium-Term)

### Prepare Transfers (Academisation API)
| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | School Name | [mis].[Establishment] | [EstablishmentName] |
| URN | Unique Reference Number (URN) | [mis].[Establishment] | [URN] |
| Project Type | Academy Type and Route | [academisation].[TransferProject] | [TypeOfTransfer] |
| Age Range | Age Range | [mis].[Establishment] | [StatutoryLowAge], [StatutoryHighAge] |
| Local Authority | Local Authority | [academisation].[TransferringAcademy] | [LocalAuthority] |
| Proposed Transfer Date | Proposed Transfer Date | [academisation].[TransferProject] | [TargetDateForTransfer] |
While our analysis is ongoing, the following data fields are identified as necessary for the integration:

### Complete (Complete API)
#### Conversion Endpoint
| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | Name | [mis].[Establishment] | [EstablishmentName] |
| URN | Academy URN | [complete].[projects] | [academy_urn] |
| Project Type | Type | [complete].[projects] | [type] |
| Age Range | Age Range | [mis].[Establishment] | [StatutoryLowAge], [StatutoryHighAge] |
| Conversion Date | Conversion Date | [complete].[projects] | [completed_at] |
| Local Authority | Local Authority | [complete].[projects] | [local_authority] |
### Prepare Conversions (Academisation API)

### Manage Free School Projects (MFSP API)
| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | Name | [mis].[Establishment] | [EstablishmentName] |
| URN | Academy URN | [complete].[projects] | [academy_urn] |
| Project Type | Type | [complete].[projects] | [type] |
| Age Range | Age Range | [mis].[Establishment] | [StatutoryLowAge], [StatutoryHighAge] |
| Conversion Date | Conversion Date | [complete].[projects] | [completed_at] |
| Local Authority | Local Authority | [complete].[projects] | [local_authority] |
| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | School Name | [academisation].[Project] | [SchoolName] |
| URN | Unique Reference Number (URN) | [academisation].[Project] | [urn] |
| Project Type | Academy Type and Route | [academisation].[Project] | [AcademyTypeAndRoute] |
| Age Range | Age Range | [academisation].[Project] | [AgeRange] |
| Local Authority | Local Authority | [academisation].[Project] | [LocalAuthority] |
| Proposed Conversion/Transfer Date | Proposed Conversion Date | [academisation].[Project] | [ProposedAcademyOpeningDate] |

## Future Considerations
### Prepare Transfers (Academisation API)

1. **Transition to APIs**: Once APIs are implemented by the respective applications, we will integrate them for better data accuracy, resilience to schema changes, and improved maintainability.
2. **Versioning and Stability**: API-based integration allows version control, reducing disruptions caused by changes in data models.
3. **Automated Validation**: Implement automated checks to ensure data integrity during the interim period.
| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | School Name | [mis].[Establishment] | [EstablishmentName] |
| URN | Unique Reference Number (URN) | [mis].[Establishment] | [URN] |
| Project Type | Academy Type and Route | [academisation].[TransferProject] | [TypeOfTransfer] |
| Age Range | Age Range | [mis].[Establishment] | [StatutoryLowAge], [StatutoryHighAge] |
| Local Authority | Local Authority | [academisation].[TransferringAcademy] | [LocalAuthority] |
| Proposed Transfer Date | Proposed Transfer Date | [academisation].[TransferProject] | [TargetDateForTransfer] |

### Complete (Complete API)

#### Conversion Endpoint

| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | Name | [mis].[Establishment] | [EstablishmentName] |
| URN | Academy URN | [complete].[projects] | [academy_urn] |
| Project Type | Type | [complete].[projects] | [type] |
| Age Range | Age Range | [mis].[Establishment] | [StatutoryLowAge], [StatutoryHighAge] |
| Conversion Date | Conversion Date | [complete].[projects] | [completed_at] |
| Local Authority | Local Authority | [complete].[projects] | [local_authority] |

### Manage Free School Projects (MFSP API)

| Column Name | Field Name | ADB Table Name | Column Name |
|---------------------------------|---------------------------------|-----------------------|---------------------------------|
| Name | Name | [mis].[Establishment] | [EstablishmentName] |
| URN | Academy URN | [complete].[projects] | [academy_urn] |
| Project Type | Type | [complete].[projects] | [type] |
| Age Range | Age Range | [mis].[Establishment] | [StatutoryLowAge], [StatutoryHighAge] |
| Conversion Date | Conversion Date | [complete].[projects] | [completed_at] |
| Local Authority | Local Authority | [complete].[projects] | [local_authority] |

## Future Considerations

1. **Transition to APIs**: Once APIs are implemented by the respective applications, we will integrate them for better data accuracy, resilience to schema changes, and improved maintainability.
2. **Versioning and Stability**: API-based integration allows version control, reducing disruptions caused by changes in data models.
3. **Automated Validation**: Implement automated checks to ensure data integrity during the interim period.

0 comments on commit 03fea16

Please sign in to comment.