[COST-4852] Inserting attempts loop to Trino queries

[bacciotti]

Jira Ticket

COST-4852

Description

This change will implement an attempts' logic on Trino queries to avoid some Exceptions raisings.
We are removing the loop from the delete function and inserting in the functions that execute insertion queries.
I've set the attempts to three just for Azure for now. Let's monitor the Exceptions.

Testing

1. Checkout Branch
2. Restart Koku
3. Test local ingestion, specially with Azure
4. Run tox -e py39 -- hcs.test.database.test_report_db_accessor.TestHCSReportDBAccessor and tox -e py39 -- masu.test.database.test_azure_report_db_accessor.AzureReportDBAccessorTest

Release Notes

• Improving some Trino queries logic to avoid Exceptions.

* [COST-4582](https://issues.redhat.com/browse/COST-4582) Improving some Trino queries logic to avoid Exceptions.

[codecov]

Codecov Report

Attention: Patch coverage is 92.47312% with 7 lines in your changes missing coverage. Please review.

Project coverage is 94.1%. Comparing base (58ef74a) to head (268a7f1).

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #5210   +/-   ##
=====================================
  Coverage   94.1%   94.1%           
=====================================
  Files        375     375           
  Lines      31411   31422   +11     
  Branches    4627    4627           
=====================================
+ Hits       29545   29560   +15     
+ Misses      1190    1189    -1     
+ Partials     676     673    -3     

[bacciotti]

/retest

[maskarb]

Shouldn't we pull the changes in #5237 into this PR?

[samdoran]

In order to retry only on hive metastore errors, we'll need another exception object.

class TrnioMetastoreError(Exception):
    ...

@retry(retry_on=(TrinoNoSuchKeyException, TrinoHiveMetastoreError)
    try:
        ...
    except TrinoQueryError as trino_exc:
        exc_to_raise = TrinoStatementExecError(
            statement=stmt, statement_number=stmt_num, sql_params=s_params, trino_error=trino_exc
        )
        if "NoSuchKey" in str(trino_exc):
            exc_to_raise = TrinoNoSuchKeyException(
                message=trino_exc.message,
                query_id=trino_exc.query_id,
                error_code=trino_exc.error_code,
            )

        if trino_exc.error_name == "HIVE_METASTORE_ERROR":
            exc_to_raise = TrinoHiveMetastoreError(
                message=trino_exc.message,
                query_id=trino_exc.query_id,
                error_code=trino_exc.error_code,
            )

        LOG.warning(f"{exc_to_raise!s}")
        raise exc_to_raise from trino_exc

You could create an exception class hierarchy if you wanted for these custom exceptions.

[samdoran]

/retest

[bacciotti]

/retest