Feature store refactoring

[kingman]

Description

• remove the need of reading the config.yaml file from the feature-store module, which causes the terraform to always replace the datasets and tables when re-applied.
• use for each construct to simplify the repeated resources creation of datasets and tables
• add new store procedures that aggregates predictions for all the use cases and take account that the prediction table maybe not existing

How has this been tested?

Tested in integration environment

Please explain how you have tested the new changes.
Due to the change of using foreach construct, the resources id has changed in terraform and when pulling this change into an existing deployment run the following mv command to move the existing resources before doing terraform apply:
terraform state mv 'module.feature_store[0].google_bigquery_dataset.audience_segmentation' 'module.feature_store[0].google_bigquery_dataset.datasets["audience_segmentation"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.auto_audience_segmentation' 'module.feature_store[0].google_bigquery_dataset.datasets["auto_audience_segmentation"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.churn_propensity' 'module.feature_store[0].google_bigquery_dataset.datasets["churn_propensity"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.customer_lifetime_value' 'module.feature_store[0].google_bigquery_dataset.datasets["customer_lifetime_value"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.feature_store' 'module.feature_store[0].google_bigquery_dataset.datasets["feature_store"]'
terraform state mv 'module.feature_store[0].google_bigquery_dataset.purchase_propensity' 'module.feature_store[0].google_bigquery_dataset.datasets["purchase_propensity"]'
terraform state mv 'module.feature_store[0].module.aggregated_predictions.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["aggregated_predictions"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["aggregated_vbb"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_dataset.main' 'module.feature_store[0].google_bigquery_dataset.datasets["gemini_insights"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.audience_segmentation_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["audience_segmentation_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.churn_propensity_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["churn_propensity_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.churn_propensity_label' 'module.feature_store[0].google_bigquery_table.tables["churn_propensity_label"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.customer_lifetime_value_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["customer_lifetime_value_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.customer_lifetime_value_label' 'module.feature_store[0].google_bigquery_table.tables["customer_lifetime_value_label"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.purchase_propensity_inference_preparation' 'module.feature_store[0].google_bigquery_table.tables["purchase_propensity_inference_preparation"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.purchase_propensity_label' 'module.feature_store[0].google_bigquery_table.tables["purchase_propensity_label"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_dimensions"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_lifetime_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_lifetime_dimensions"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_lookback_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_lookback_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_rolling_window_lifetime_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_rolling_window_lifetime_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_rolling_window_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_rolling_window_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_lifetime_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_lifetime_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_scoped_segmentation_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_scoped_segmentation_metrics"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_segmentation_dimensions' 'module.feature_store[0].google_bigquery_table.tables["user_segmentation_dimensions"]'
terraform state mv 'module.feature_store[0].google_bigquery_table.user_session_event_aggregated_metrics' 'module.feature_store[0].google_bigquery_table.tables["user_session_event_aggregated_metrics"]'
terraform state mv 'module.feature_store[0].module.aggregated_predictions.google_bigquery_table.main["latest"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_predictions_latest"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_correlation"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_correlation"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_volume_daily"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_volume_daily"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["aggregated_value_based_bidding_volume_weekly"]' 'module.feature_store[0].google_bigquery_table.tables["aggregated_value_based_bidding_volume_weekly"]'
terraform state mv 'module.feature_store[0].module.aggregated_vbb.google_bigquery_table.main["vbb_weights"]' 'module.feature_store[0].google_bigquery_table.tables["vbb_weights"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_daily"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_daily"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_monthly"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_monthly"]'
terraform state mv 'module.feature_store[0].module.gemini_insights.google_bigquery_table.main["user_behaviour_revenue_insights_weekly"]' 'module.feature_store[0].google_bigquery_table.tables["user_behaviour_revenue_insights_weekly"]'

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have successfully run the E2E tests, and have included the links to the pipeline runs below
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have updated any relevant documentation to reflect my changes
• I have assigned a reviewer and messaged them

Pipeline run links:

[chmstimoteo]

TO BE REVIEWED.