Migrate roles and grants to new table structure

[mikemountain]

This PR migrates the previous iam_role_grant_scopes table to new tables iam_role_global, iam_role_org, iam_role_project, iam_role_global_individual_org_grant_scope, iam_role_global_individual_project_grant_scope, and iam_role_org_individual_grant_scope, making db access and returns more performant.

A new table iam_grant exists, migrated from iam_role_grant for all distinct canonical_grants.

[mikemountain]

This PR does rely on code from #5733 to "repair" the broken migrations, so it will need to be rebased

[mikemountain]

This file is a large migration test and adds 30,000 roles to test performance

[bosorawis]

2 nitpicks and 1 question. Awesome work on all the tests! 🎉 Thank you for adding those

[bosorawis]

should we also drop the old tables and columns as a part of this cleanup script?

table:
iam_role_grant_scope

columns:
iam_role.version
iam_role.name
iam_role.description

[mikemountain]

I've got that in this separate PR because it was listed as a separate work item

[github-actions]

Database schema diff between llb-normalized-grants and mikemountain-grants-migrations @ e69b823

To understand how these diffs are generated and some limitations see the
documentation of the script.

Functions

diff --git a/.schema-diff/funcs_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/set_resource.sql b/.schema-diff/funcs_309566a0846660ef028ddc8d20e1eb4060e82a63/set_resource.sql
index 31ae58f06..4bd583854 100644
--- a/.schema-diff/funcs_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/set_resource.sql
+++ b/.schema-diff/funcs_309566a0846660ef028ddc8d20e1eb4060e82a63/set_resource.sql
@@ -41,7 +41,6 @@ create function public.set_resource() returns trigger
       into type_matches
     from kv
     where k = 'type';
-
     -- if there are multiple canonical grant types specified, throw an error.
     -- ensure that the canonical_grant type is only referencing a single resource
     if type_matches is not null and array_length(type_matches, 1) > 1 then

Tables

diff --git a/.schema-diff/tables_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/iam_role_org_individual_grant_scope.sql b/.schema-diff/tables_309566a0846660ef028ddc8d20e1eb4060e82a63/iam_role_org_individual_grant_scope.sql
index f96d2a86d..001154690 100644
--- a/.schema-diff/tables_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/iam_role_org_individual_grant_scope.sql
+++ b/.schema-diff/tables_309566a0846660ef028ddc8d20e1eb4060e82a63/iam_role_org_individual_grant_scope.sql
@@ -37,7 +37,7 @@ create table public.iam_role_org_individual_grant_scope (
 -- name: table iam_role_org_individual_grant_scope; type: comment; schema: public; owner: -
 --
 
-comment on table public.iam_role_org_individual_grant_scope is 'iam_role_org_individual_grant_scope is the subtype table for the org role with grant_scope as individual.';
+comment on table public.iam_role_org_individual_grant_scope is 'iam_role_org_individual_grant_scope is a list of individually granted project scope to org roles with grant_scope of individual';
 
 
 --

Views

Unchanged

Triggers

Unchanged

Indexes

diff --git a/.schema-diff/indexes_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/iam_role_grant_canonical_grant_ix.sql b/.schema-diff/indexes_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/iam_role_grant_canonical_grant_ix.sql
deleted file mode 100644
index 328c6abda..000000000
--- a/.schema-diff/indexes_0d43b4dc4ae9cd4eae423b9fcc71f04bdb585540/iam_role_grant_canonical_grant_ix.sql
+++ /dev/null
@@ -1,31 +0,0 @@
---
--- postgresql database dump
---
-
--- dumped from database version 13.21
--- dumped by pg_dump version 16.9 (ubuntu 16.9-1.pgdg24.04+1)
-
-set statement_timeout = 0;
-set lock_timeout = 0;
-set idle_in_transaction_session_timeout = 0;
-set client_encoding = 'utf8';
-set standard_conforming_strings = on;
-select pg_catalog.set_config('search_path', '', false);
-set check_function_bodies = false;
-set xmloption = content;
-set client_min_messages = warning;
-set row_security = off;
-
-set default_tablespace = '';
-
---
--- name: iam_role_grant_canonical_grant_ix; type: index; schema: public; owner: -
---
-
-create index iam_role_grant_canonical_grant_ix on public.iam_role_grant using btree (canonical_grant);
-
-
---
--- postgresql database dump complete
---
-

Constraints

Unchanged

Foreign Key Constraints

Unchanged