This commit introduces a user catalog table, percona_tde.pg_tde_key_provider,
within the percona_tde schema, as part of the pg_tde extension. The purpose of
this table is to store essential provider information. The catalog accommodates
various key providers, present and future, utilizing a JSON type
options field to capture provider-specific details.

To facilitate the creation of key providers,
the commit introduces new SQL interfaces:

- pg_tde_add_key_provider(provider_type VARCHAR(10),
        provider_name VARCHAR(128), options JSON)
- pg_tde_add_key_provider_file(provider_name VARCHAR(128),
        file_path TEXT)
- pg_tde_add_key_provider_vault_v2(provider_name VARCHAR(128),
        vault_token TEXT, vault_url TEXT,
        vault_mount_path TEXT, vault_ca_path TEXT)

Additionally, the commit implements the C interface for catalog
interaction, detailed in the 'tde_keyring.h' file.

These changes lay the foundation for implementing multi-tenancy in pg_tde by
eliminating the necessity of a 'keyring.json' file for configuring a
cluster-wide key provider. With this enhancement, each database can have its
dedicated key provider, added via SQL interface, removing the need
for DBA intervention in TDE setup."

Up until now, pg_tde relied on a hard-coded master key name, primarily for
proof-of-concept purposes. This commit introduces a more robust infrastructure
for configuring the master key and managing a dynamic shared memory-based
master-key cache to enhance accessibility.

For user interaction, a new SQL interface is provided:
- pg_tde_set_master_key(master_key_name VARCHAR(255), provider_name VARCHAR(255));
This interface enables users to set a master key for a specific database and make
further enhancements toward implementing the multi-tenancy.

In addition to the public SQL interface, the commit optimizes the internal
master-key API. It introduces straightforward Get and Set functions,
handling locking, retrieval, caching, and assignment of a master key for
a database seamlessly.

The commit also introduces a unified internal interface for requesting and
utilizing shared memory, contributing to a more cohesive and efficient
master key and cache management system.

This commit unifies the master-key and key-provider modules with the core of
pg_tde, marking a significant evolution in the architecture.
As part of this integration, the keyring API undergoes substantial changes
to enhance flexibility and remove unnecessary components such as the key cache.

As a result of the keyring refactoring, the file keyring is also rewritten,
offering a template for implementing additional key providers for the extension.
The modifications make the keyring API more pluggable, streamlining
interactions and paving the way for future enhancements.

…rements

This commit addresses PostgreSQL core's requirement for upfront information
regarding the number of locks needed by the extension. Given the connection
between locks and the shared memory interface, a new callback routine
is introduced. This routine allows modules to specify
the number of locks they require.

In addition to this functionality, the commit includes code cleanups
and adjustments to nomenclature for improved clarity and consistency.

Co-authored-by: Andrew Pogrebnoi <[email protected]>

Co-authored-by: Andrew Pogrebnoi <[email protected]>

Adjusting provider_name length in pg_tde_key_provider catalog
definition and adding a TODO note for implementing an eviction
policy for the master key cache.

This commit enhances the extension by adding a new mechanism to facilitate
cleanup or setup procedures when the extension is installed in a database.
The core addition is a function "pg_tde_extension_initialize" invoked upon
executing the database's 'CREATE EXTENSION' command.

To streamline future development and ensure extensibility,
the commit introduces a callback registration mechanism. This enables any module
to specify a callback function (registered using on_ext_install() ) to be
invoked during extension creation.

As of this commit, the callback functionality is explicitly utilized by the
master key module to handle the cleanup of the master key information file.
This file might persist in the database directory if the extension had been
previously deleted in the same database.

This enhancement paves the way for a more modular and maintainable extension
architecture, allowing individual modules to manage their specific
setup and cleanup tasks seamlessly."

Commit also contains the test case adjustments.

Commit also contains the fix for review comments and a test case
for Vault-V2 type key provider