Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[COST-3814] Azure subs processing start 2 days prior to source creation #4836

Merged
merged 2 commits into from
Dec 14, 2023

Conversation

cgoodfred
Copy link
Contributor

@cgoodfred cgoodfred commented Dec 14, 2023

Jira Ticket

COST-3814

Description

This change will make Azure subs processing start 2 days prior to source creation instead of 1. Azure processing requires the day of data to be "complete" and when a source is created, yesterdays data is unlikely to be completed as the export for today would have to have run already. Using 2 days will process data upon source creation instead of waiting an extra day.

Testing

Easiest testing is to create an AWS SUBSDataExtractor() and an Azure SUBSDataExtractor() and see the different in time:

  1. Spin up koku and create a test customer, it doesn't need data. make create-test-customer
  2. Start with make shell
  3. Check the processing time on an Azure source (substitute your provider_uuid), expected output (today - 2 days): datetime.datetime(2023, 12, 12, 0, 0, tzinfo=<UTC>):
from subs.subs_data_extractor import SUBSDataExtractor
schema = "org1234567"
prov_type = "Azure-local"
prov_uuid = "fa6ec55c-470b-4f5d-b300-e5548c52f501"
context = {"schema": schema, "provider_type": prov_type, "provider_uuid": prov_uuid}
sde = SUBSDataExtractor('', context)
sde.creation_processing_time
  1. Check the AWS source (substitute your provider_uuid), expected output (today - 1 days): datetime.datetime(2023, 12, 13, 0, 0, tzinfo=<UTC>):
prov_type_2 = "AWS-local"
prov_uuid_2 = "2d1a54f5-d8bc-4374-a81b-d474245e97eb"
context_2 = {"schema": schema, "provider_type": prov_type_2, "provider_uuid": prov_uuid_2}
sde_2 = SUBSDataExtractor('', context_2)
sde_2.creation_processing_time

Notes

...

@cgoodfred cgoodfred added the aws-smoke-tests pr_check will build the image and run aws + ocp on aws smoke tests label Dec 14, 2023
@cgoodfred cgoodfred self-assigned this Dec 14, 2023
@cgoodfred cgoodfred requested review from a team as code owners December 14, 2023 14:42
esebesto
esebesto previously approved these changes Dec 14, 2023
Copy link

codecov bot commented Dec 14, 2023

Codecov Report

Merging #4836 (6abeefb) into main (9bb8c30) will decrease coverage by 0.0%.
The diff coverage is 100.0%.

Additional details and impacted files
@@           Coverage Diff           @@
##            main   #4836     +/-   ##
=======================================
- Coverage   94.0%   94.0%   -0.0%     
=======================================
  Files        364     364             
  Lines      30204   30206      +2     
  Branches    3598    3599      +1     
=======================================
+ Hits       28380   28381      +1     
  Misses      1164    1164             
- Partials     660     661      +1     

Copy link

sonarcloud bot commented Dec 14, 2023

@cgoodfred cgoodfred merged commit a7902d3 into main Dec 14, 2023
10 checks passed
@cgoodfred cgoodfred deleted the COST-3814-azure-two-days branch December 14, 2023 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aws-smoke-tests pr_check will build the image and run aws + ocp on aws smoke tests smokes-required
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants