Update NTD agency information

[erikamov]

Description

This PR is a result of the review NTD Annual Agency Information scrape process requested on issue #3497.

• Two new columns added on 2022 and 2023 files:

  • division_department
  • state_parent_ntd_id -> This new column was added as part of the unique key on mart table (year, ntd_id and state_parent_ntd_id)

• Renamed NTD mart table dim_annual_ntd_agency_information to dim_annual_agency_information, removing _ntd_ from name since it is on mart_ntd group, to keep the same name pattern as the other NTD mart tables (as requested on issue).

• I had to Bump upload-artifact version to v3 in order to fix deprecated error on build-docs:
  This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v2. Learn more: https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation

How has this been tested?

Tested locally.

Post-merge follow-ups

• No action required
• Actions required (specified below)

• Verify if the DAG runs successfully after merging and if creates dim_annual_agency_information.
• Run scrape to import data for new columns (2022 and 2023 files):
  > poetry run python scrape_ntd.py annual-database-agency-information 2023 https://www.transit.dot.gov/sites/fta.dot.gov/files/2024-10/2023%20Agency%20Information_0.xlsx
> poetry run python scrape_ntd.py annual-database-agency-information 2022 https://www.transit.dot.gov/sites/fta.dot.gov/files/2024-07/2022%20Agency%20Information_1-3_0.xlsx
• Alert all users about replacing the table on Metabase and Jupyter.
• Drop deprecated table dim_annual_ntd_agency_information.

[github-actions]

Warehouse report 📦

Checks/potential follow-ups

Checks indicate the following action items may be necessary.

• For new models, do they all have a surrogate primary key that is tested to be not-null and unique?

New models 🌱

calitp_warehouse.mart.ntd.dim_annual_agency_information

calitp_warehouse.staging.state_geoportal.stg_state_geoportal__state_highway_network_stops

DAG

Legend (in order of precedence)

Resource type	Indicator	Resolution
Large table-materialized model	Orange	Make the model incremental
Large model without partitioning or clustering	Orange	Add partitioning and/or clustering
View with more than one child	Yellow	Materialize as a table or incremental
Incremental	Light green
Table	Green
View	White

[github-actions]

Preview url: https://docs-data-infra-3559--cal-itp-previews.netlify.app