[COST-4852] feat: adding an exception to handle TrinoErrors.

[bacciotti]

Jira Ticket

COST-4852

Description

This change will handle the NoSuchKey error on Trino.

Testing

1. Checkout Branch
2. Restart Koku
3. Try to create and load sample data.

Release Notes

• proposed release note

* [COST-4852](https://issues.redhat.com/browse/COST-4852) Handling some erros during Trino SQL execution.

[codecov]

Codecov Report

Attention: Patch coverage is 12.50000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 94.1%. Comparing base (abd4261) to head (c70d717).

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #5177     +/-   ##
=======================================
- Coverage   94.2%   94.1%   -0.0%     
=======================================
  Files        376     376             
  Lines      31252   31260      +8     
  Branches    3734    3736      +2     
=======================================
+ Hits       29426   29427      +1     
- Misses      1163    1169      +6     
- Partials     663     664      +1     

[lcouzens]

It might just be me miss-understanding, but I don't fully grasp the changes. To me it looks like we added a new exception for TrinoExternalError which would already get collected by the generic Exception no?

Then within that new exception we have additional retry logic?? Is this necessary since we are already rerunning the queries until the retry count is 0 in the generic exception anyway?

The real change here I guess are these lines:
if "NoSuchKey" in str(ex) and attempts_left > 0:
LOG.warning(log_json(msg="file not found, retrying", log_ref=log_ref, context=ctx), exc_info=ex)

To which, would it not make more sense to just put this inside the generic exception?
For example:
except Exception as ex:
if attempts_left == 0:
# Additional if changing the log type based on "NoSuchKey" in str(ex)

[bacciotti]

Well, I think I didn't get where in the current code and in your suggestion the query would be re-run. We haven't a new call to _execute_trino_raw_sql_query_with_description function here:

        except Exception as ex:
            if attempts_left == 0:
                LOG.error(log_json(msg="failed trino sql execution", log_ref=log_ref, context=ctx), exc_info=ex)
            raise ex

That is, when is it going to be called again in this scope (if there are attempts left)?

I mean, unless I explicitly put a new call to _execute_trino_raw_sql_query_with_description recursively, it wouldn't call while there are attempts left.
(cc @lcouzens)