diff --git a/docs/flowcharts/00-processing-overview.md b/docs/flowcharts/00-processing-overview.md new file mode 100644 index 0000000000..e957d9e2be --- /dev/null +++ b/docs/flowcharts/00-processing-overview.md @@ -0,0 +1,26 @@ +```mermaid + +graph + +A["Orchestrator.prepare()"] --> B[start-manifest-processing] +B --> C["ReportDownloader()"] +C --> D["get manifest list\n(download manifest, insert manifest record)"] +D --> |"iterate over manifests (only GCP has more than 1)"| E["ensure each file of manifest has status record (weirdly done)"] +E --> |iterate over each file of manifest| F["generate new `report_context` dict from manifest copy"] +F --> G[create list of tasks to download report file] +G --> H["trigger task to download files (pass in report context and account info)"] + +H --> I[get_report_files] +I --> J[_get_report_files] +J --> K["ReportDownloader()"] +K --> L[download file] + + + + +``` + + +H --> I[trigger summary tasks] +H --> J[trigger hcs tasks] +H --> K[trigger subs tasks] diff --git a/docs/flowcharts/01-check-report-updates.md b/docs/flowcharts/01-check-report-updates.md new file mode 100644 index 0000000000..6fcdb4cde2 --- /dev/null +++ b/docs/flowcharts/01-check-report-updates.md @@ -0,0 +1,25 @@ +```mermaid + +graph + A["check_report_updates (scheduled task)"] -->|"masu.celery.tasks.check_report_updates (default queue)"| B["Orchestrator(scheduled=True).prepare()"] + B --> C[for provider in pollable_providers] + C --> D{providers left?} + D -->|yes| E[provider-type] + E -->|AWS/Azure| F[O.prepare_monthly_report_sources] + F --> G["report_months = O.get_reports\n(there is some masu API specific logic in\nhere. But also, this returns\nmonths to iterate over...)"] + G --> H["for month in report_months\n(list of datetime.date)"] + H --> I{months left?} + I -->|yes| J[" `account['report_month'] = month`\nself.start_manifest_processing\n(see 02-start-manifest-processing.md)"] + J -->|Success| K[label accounts] + K -->|Success/Exception| I + J -->|Exception| I + I -->|no| D + + + E -->|GCP/OCI| L[O.prepare_continuous_report_sources] + L --> M[" `account['report_month'] = start_date`\n`DateAccessor().get_billing_month_start(f'{self.bill_date}01')` (a datetime.date)\nor\nDateHelper().today (a datetime.datetime)"] + M --> N["self.start_manifest_processing\n(see 02-start-manifest-processing.md)"] + N -->|Success/Exception| D + + D -->|"no"| Z[Done] +``` diff --git a/docs/flowcharts/02-start-manifest-processing.md b/docs/flowcharts/02-start-manifest-processing.md new file mode 100644 index 0000000000..e69de29bb2