Skip to content

Commit

Permalink
[Issue 2665] Minor fix to Makefile and CLI to integrate GitHub export…
Browse files Browse the repository at this point in the history
… and import capabilities (#2744)

## Summary
Partially Fixes #2665

### Time to review: __1 min__

## Changes proposed
> What was added, updated, or removed in this PR.

Minor change to Makefile and CLI to integrate the export work done in
#2481 with the import work done in #2482.

## Context for reviewers
> Testing instructions, background context, more in-depth details of the
implementation, and anything else you'd like to call out or ask
reviewers. Explain how the changes were verified.

Issue #2481 added capability to export data from GitHub to a flat file
that can be ingested by the new import capabilities added by issue
#2482. However, in testing, I just noticed a filename mismatch in
`Makefile` that precludes e2e integration. This PR fixes the filename
mismatch.

## Additional information
> Screenshots, GIF demos, code examples or output to help show the
changes working as expected.
  • Loading branch information
DavidDudas-Intuitial authored Nov 5, 2024
1 parent c93c1cf commit e513579
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 6 deletions.
2 changes: 1 addition & 1 deletion analytics/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ gh-transform-and-load:
@echo "=> Transforming and loading GitHub data into the database"
@echo "====================================================="
$(POETRY) analytics etl transform_and_load \
--deliverable-file $(DELIVERY_FILE) \
--issue-file $(ISSUE_FILE) \
--effective-date $(EFFECTIVE_DATE)
@echo "====================================================="

Expand Down
5 changes: 2 additions & 3 deletions analytics/src/analytics/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@
STATUS_ARG = typer.Option(
help="Deliverable status to include in report, can be passed multiple times",
)
DELIVERABLE_FILE_ARG = typer.Option(help="Path to file with exported deliverable data")
EFFECTIVE_DATE_ARG = typer.Option(help="YYYY-MM-DD effective date to apply to each imported row")
# fmt: on

Expand Down Expand Up @@ -264,7 +263,7 @@ def initialize_database() -> None:

@etl_app.command(name="transform_and_load")
def transform_and_load(
deliverable_file: Annotated[str, DELIVERABLE_FILE_ARG],
issue_file: Annotated[str, ISSUE_FILE_ARG],
effective_date: Annotated[str, EFFECTIVE_DATE_ARG],
) -> None:
"""Transform and load etl data."""
Expand All @@ -282,7 +281,7 @@ def transform_and_load(
return

# hydrate a dataset instance from the input data
dataset = EtlDataset.load_from_json_file(file_path=deliverable_file)
dataset = EtlDataset.load_from_json_file(file_path=issue_file)

# sync data to db
etldb.sync_db(dataset, datestamp)
Expand Down
4 changes: 2 additions & 2 deletions analytics/tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ def test_transform_and_load_with_valid_parameters(self):
command = [
"etl",
"transform_and_load",
"--deliverable-file",
"--issue-file",
self.TEST_FILE_1,
"--effective-date",
str(self.EFFECTIVE_DATE),
Expand All @@ -328,7 +328,7 @@ def test_transform_and_load_with_malformed_effective_date_parameter(self):
command = [
"etl",
"transform_and_load",
"--deliverable-file",
"--issue-file",
self.TEST_FILE_1,
"--effective-date",
"2024-Oct-07",
Expand Down

0 comments on commit e513579

Please sign in to comment.