[COST-2231] Upgrade to Django 4

[djnakabaale]

Jira Ticket

COST-2231

Description

This change will:

• Upgrade Django from 3.2 to 4.0 and DRF to 3.14 - [COST-4514] Upgrade to django 4.0 #4839
• Upgrade Django from 4.0 to 4.1 - [COST-4515] Upgrade to Django 4.1 #4853
• Upgrade Django from 4.1 to 4.2 - [COST-4516] Upgrade to Django 4.2 #4893

Testing

1. Stop server containers

   docker compose stop koku-server masu-server worker
   

2. Update template schema and clone_schema SQL function

   python koku/manage.py migrate_serial_to_identity_columns --schema template0 --write
   

   Or update all schemas

   python koku/manage.py migrate_serial_to_identity_columns --write
   

3. Rebuild and start server contaiters

   make docker-up-min
   

4. Create test customer and load data. You may need to delete existing sources first.

   make create-test-customer
   make load-test-customer-data
   

5. Observe data processing

   make docker-logs
   

6. Verify sources were created and cost data was loaded

   http://localhost:8000/api/cost-management/v1/reports/oci/costs/
   http://localhost:8000/api/cost-management/v1/reports/azure/costs/
   http://localhost:8000/api/cost-management/v1/reports/aws/costs/
   # etc.
   

7. Verify template schema was correctly updated in database. Look for the ID column to be generated as identity. Do this on as many tables as you wish.

   psql --host $POSTGRES_SQL_SERVICE_HOST \
        --port $POSTGRES_SQL_SERVICE_PORT \
        --username $DATABASE_ADMIN \
        --password $DATABASE_PASSWORD
        
   \d template0.reporting_ocp_cost_category
   Table "template0.reporting_ocp_cost_category"
        Column     |  Type   | Collation | Nullable |             Default
   ----------------+---------+-----------+----------+----------------------------------
    id             | integer |           | not null | generated by default as identity
    ...
   

8. Verify the clone_schema SQL function was updated

   psql --host $POSTGRES_SQL_SERVICE_HOST \
    --port $POSTGRES_SQL_SERVICE_PORT \
    --username $DATABASE_ADMIN \
    --password $DATABASE_PASSWORD
    
    \sf clone_schema
    
    [compare the output to db_functions/clone_schema.sql]
   

9. Run tests

    tox -r -e py39
   

[codecov]

Codecov Report

Merging #4844 (7060573) into main (a7d38ed) will decrease coverage by 0.0%.
The diff coverage is 85.5%.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #4844     +/-   ##
=======================================
- Coverage   94.0%   94.0%   -0.0%     
=======================================
  Files        374     374             
  Lines      30838   30841      +3     
  Branches    3663    3663             
=======================================
  Hits       28978   28978             
- Misses      1196    1199      +3     
  Partials     664     664             

[sonarcloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
92.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[samdoran]

/retest

[maskarb]

I'm not sure I follow the thinking about adding this new license file. Yes, we added a file that uses the MIT license, but could we not just add it to the single file that uses it?

[samdoran]

It's not necessary but it makes it far easier to communicate the licenses used by the project. Otherwise it's necessary to inspect every file in the project to see how they are licensed.

[samdoran]

/retest