Automate town close QC reports

[jeancochrane]

Background

Currently, we need a staff member to manually run the town close QC export script in order to deliver the QC workbooks to Valuations. This is usually not too hard (see here for a detailed description of the process) but Valuations often likes to get started QCing very early in the morning, when Data team staff members are generally not available to run the script.

Goal

Add a new argument to the export_qc_town_close_reports.py script to read a configuration from an Excel workbook in order to determine which towns are eligible for export on the current day. Once we've merged this change, we can schedule the script to run daily on the VM and read from a config file on the O Drive in order to automatically generate any town close QC reports that Valuations wants to produce on a given day.

Scope

• Add a new --township-config-file argument to the export_qc_town_close_reports.py script that can optionally read a path to an Excel file
• Add a new function to the utils/townships.py module called get_townships_from_config_file() that takes the path to the config file as a string argument, reads the townships and configured date ranges from the file, and returns a list of the Township objects that are configured to run on the current day
  • See O:\CCAODATA\Valuations Town Close QC\Valuations Town Close QC Schedule.xlsx for an example of the config file schema
  • The function should raise an error if a township code is not in the TOWNSHIPS list, or if the township names do not match between the config file and the name as configured in TOWNSHIPS (compare lowercased versions of the strings, to avoid false positives)
• Update the block in export_qc_town_close_reports.py that parses the --township argument to follow new logic:
  1. If the --township-config-file argument is not empty, use get_townships_from_config_file() to get the list of townships to populate the townships variable
  2. If the --township-config-file argument is empty but --township is not empty, use --township to populate the townships variable (similar to the current logic)
  3. If both arguments are empty, fall back to TOWNSHIPS (similar to the current logic)
  4. In all conditional branches, make sure to add a log line to provide a hint as to which source we are drawing the townships variable from
• Add optional logging and alerting arguments to the export_qc_town_close_reports.py script so that we can log/alert from AWS in case of errors (see Add remote logging and error reporting to daily IC reference file export script #860 for the pattern that you can follow to implement these arguments)

[jeancochrane]

Waiting on approval from Valuations.