Modernize usage and support for public key cryptography

[mattrm456]

This PR introduces a significant number of changes to modernize the usage and support for public key cryptography, as used by ntci::StreamSocket to allow a user to upgrade into and out of TLS sessions. This PR has several goals:

1. Support generation of more than just RSA private keys, namely, the SEC-endorsed, named elliptic curves such as "secp256r1", and the popular Edwards curves ED25519 and ED448.

2. Support user-defined validation of a peer's certificate. Practically speaking, to make any reasonable decision about the validity of a certificate, the structure and values in the certificate must be transparent. This PR adds value-semantic representations of certificates and private keys (ntca_encryptioncertificate and ntca_encryptionkey) and capability to encode and decode certificates and private keys according to the Distinguished Encoding Rules (DER). This required the introduction of a full, DER-compliant ASN.1 encoder and decoder in ntsa_abstract. Note that bdl, part of the BDE libraries, contains a BER encoder and decoder but cryptographic objects require the DER subset of BER, which the BDE libraries do not support. DER is principally distinguished by requiring definite lengths for each tag-length-value and requiring each value be encoded in the smallest possible number of octets. There are several other features required to correctly decode cryptographic objects, such as precise handling of date/time objects, that BDE also does not support. For completeness, thoroughness, and sanity, a self-contained complete DER encoder/decoder is provided in ntsa_abstract.

3. Support additional, built-in certificate validation parameters such as subject common name and subject alternative name matching.

4. Support TLS clients issuing a "server name indication" (SNI) when they initiate the TLS session handshake, to request a particular sub-configuration of the TLS server, and support TLS servers optionally specifying SNI-specific certificates and keys they should use when clients indicate the associated name. The motivation for this feature of TLS is to support a server running on a single machine whose address can be resolved multiple ways. If clients wish to verify the server's certificate to contain the name they expect, they need a channel to communicate to the server how the client resolved the transport address.

5. Support configuring TLS clients and TLS servers with certificates and keys in more than just ASN.1/PEM-encoded files. Support now includes ASN.1, PKCS7, PKCS8, PKCS12, and optionally subsequent wrapping in PEM of most of the formats. TLS clients and TLS server can now be configured by simply arming the configuration with one or more blobs of data, which can be in any supported format; the implementation takes care of registering the end-users certificate, private key, and trusted certificate authorities. This includes support for simply delivering all these necessary objects in one PKC12 (*.pfx) file, for example.

6. Support for a user-defined mechanism to resolve a shared secret on-demand, when we discover we are decoding something like a PKCS8 file that has been symmetrically encrypted.

Here is an overview of the new components, from lowest-level to highest.

ntsa_abstract: Abstract Syntax One (ASN.1) objects and encoders and decoders conforming to the Distinguished Encoding Rules (DER). Includes arbitry precision integers, bit strings, octet strings, and containers-of-any-value.

ntca_encryptionkey: Value-semantic representation of a private key, in either the structure of the original RSA key format, the non-standard but commonly used Elliptic Curve format, or the PKCS7 format.

ntca_encryptioncertificate: Value-semantic representation of an X.509 certificate, including its many sub-structures, such as extensible names, key usages, public key information, and most popular certificate extensions.

ntca_encryptionresourcetype: Enumeration of the supported certificate and key storage formats.

ntca_encryptionresourceoptions: Options that aid in the encoding and decoding of the various formats of how certificates and keys are stored.

ntca_encryptionresourcedescriptor: A union of representations of the source of certificates and keys: either file, literal encoded data, or as the value-semantic objects themselves.

ntca_encryptionresource: A resource's descriptor and options, bundled together.

ntca_encryptionkeytype: Enumeration of the well-known, cryptographically-endorsed private key algorithms.

ntca_encryptionoptions: Name-specific overrides of the configuration of a TLS session. Used to support Server Name Indication (SNI). The existing ntca_encryptionclientoptions and ntca_encryptionserveroptions have been extended to support a map of names to these options, which represent the effective, name-specific configuration that is hot-swapped in during the TLS handshake when SNI requested from the client.

ntca_encryptionvalidation: User-defined parameters to influence how certificates are verified, including an optional callback for the user to restrict certificate validity according to their own definitions. Note that the most significant, complex part of certificate verification (matching a trusted issuer and ensuring it's private key signed the certificate) is delegated to the encryption driver; these parameters allow the user to specified stronger constraints on this like the subject common name and the subject alternative names (which are typically domain names or IP addresses identifier the holder of the certificate, and usually correspond to the name or address of how the client establishes the transport-level connection to the server.)

ntca_encryptionsecret: A bag of bytes that represents a shared secret used for any necessary symmetric encryption and decryption. In practice, this is only relevant when decoding a PEM, PKC7 or PKC12 (i.e., those formats that store private keys) where the encoder choose to apply a symmetric cipher.

[smtrfnv]

It's unbuildable on Windows at least.

C:\bloomberg\ntf-core\groups\ntc\ntca\ntca_encryptioncertificate.cpp(2307): error C2065: 'NL_TEXTMAX': undeclared identifier

C:\bloomberg\ntf-core\groups\nts\ntsa\ntsa_abstract.cpp(788): error C2220: the following warning is treated as an error
C:\bloomberg\ntf-core\groups\nts\ntsa\ntsa_abstract.cpp(788): warning C4244: 'argument': conversion from 'const uint64_t' to 'uint8_t', possible loss of data
C:\bloomberg\ntf-core\groups\nts\ntsa\ntsa_abstract.cpp(5366): warning C4244: 'argument': conversion from 'uint64_t' to 'BloombergLP::ntsa::AbstractIntegerRepresentation::Block', possible loss of data

[smtrfnv]

Not sure if we care, but today is 2024 :)

[mattrm456]

Let's universally bump the year when we do the next release, and henceforth plan to do this on Jan 1 of 2025.

[smtrfnv]

Shouldn't this type contain a ptr to allocator aand use it for d_resourceVector, d_authorityDirectory and d_cipherSpec?

[mattrm456]

Strictly speaking, unless the implementation needs to retain the basic allocator as a member for later direct use to dynamically allocate memory consistent with the lifetime of the object, the policy is to simply forward basicAllocator to each allocator-aware member.

[mattrm456]

I fixed the warnings-as-errors on Windows with stricter static_casts.

[smtrfnv]

It's amazing how well structured and carefully designed this PR is. Let's merge it. Just don't forget to fix GitlabActions failure.

[mattrm456]

GitHub Actions are now all successful.