-
Notifications
You must be signed in to change notification settings - Fork 151
Implementation of XAES-256-GCM with EVP_CIPHER #2750
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: xaes-256-gcm
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang-tidy made some suggestions
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## xaes-256-gcm #2750 +/- ##
================================================
- Coverage 78.94% 78.81% -0.13%
================================================
Files 677 677
Lines 115525 114584 -941
Branches 16249 16258 +9
================================================
- Hits 91198 90314 -884
+ Misses 23530 23473 -57
Partials 797 797 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…to xaes-256-gcm
crypto/cipher_extra/cipher_test.cc
Outdated
| ASSERT_TRUE(ctx); | ||
| ASSERT_TRUE(EVP_CipherInit_ex(ctx.get(), EVP_xaes_256_gcm(), NULL, NULL, NULL, 1)); | ||
|
|
||
| // Invalid nonce size |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please write down the valid range for nonce size.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I'll follow your suggestion!
crypto/cipher_extra/cipher_test.cc
Outdated
| const uint8_t *plaintext = (const uint8_t *)"Hello, XAES-256-GCM!"; | ||
| size_t plaintext_len = strlen((const char *)plaintext); | ||
|
|
||
| ciphertext.resize(20); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll fix this
crypto/cipher_extra/cipher_test.cc
Outdated
| TEST(CipherTest, XAES_256_GCM_EVP_CIPHER) { | ||
| { | ||
| std::vector<uint8_t> ciphertext, tag, decrypted; | ||
| const uint8_t key[32] = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not std::vector<uint8_t>?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I copied the test case directly here https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM/openssl/openssl.c
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in aws-lc, we tend to use C++ style for unit tests so let's please convert to that.
crypto/fipsmodule/cipher/e_aes.c
Outdated
| struct xaes_256_gcm_ctx *xaes_ctx = | ||
| (struct xaes_256_gcm_ctx *)((uint8_t*)ctx->cipher_data + XAES_256_GCM_CTX_OFFSET); | ||
|
|
||
| uint8_t derived_key[(256 >> 3)]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
magic numbers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll fix this. 256 >> 3 = 256 / 2^3 = 32
crypto/fipsmodule/cipher/e_aes.c
Outdated
| return 1; | ||
| } | ||
|
|
||
| static const uint8_t kZeroIn[AES_BLOCK_SIZE] = {0}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can move inside the function it's using it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll fix this
crypto/fipsmodule/cipher/e_aes.c
Outdated
|
|
||
| static int xaes_256_gcm_ctx_init(struct xaes_256_gcm_ctx *xaes_ctx, | ||
| const uint8_t *key, const unsigned key_len) { | ||
| if ((key_len << 3) != 256) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
magic numbers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll fix this. n << 3 ~ n * 2^3.
crypto/fipsmodule/cipher/e_aes.c
Outdated
| ptr += (uintptr_t)ptr & 8; | ||
|
|
||
| OPENSSL_memcpy(ptr, gctx, sizeof(EVP_AES_GCM_CTX)); | ||
| OPENSSL_free(temp); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- cipher_data is allocated memory space of the same size as the context of AES-GCM. But we need to have space for storing xaes_256_gcm_ctx. I allocate another memory space to accommodate both the existing context of underlying AES-GCM and also struct xaes_256_gcm_ctx. The reason that I cannot change the context size of xaes_256_gcm_ctx right in the beginning is because of this: https://github.com/aws/aws-lc/blob/main/crypto/fipsmodule/cipher/e_aes.c#L356, which prevents the initialization of underlying AES-GCM when wrapping it inside XAES-256-GCM. Furthermore, EVP_CIPHER *cipher in ctx is declared as const (https://github.com/aws/aws-lc/blob/main/include/openssl/cipher.h#L622), so I cannot change ctx->cipher->ctx_size once it was initialized.
- The alignment part is for compatibility with this https://github.com/aws/aws-lc/blob/xaes-256-gcm/crypto/fipsmodule/cipher/e_aes.c#L360
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cipher_data is allocated memory space of the same size as the context of AES-GCM.
where is this allocated? why can't we allocate enough space in the first place?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I already explained it above:
"The reason that I cannot change the context size of xaes_256_gcm_ctx right in the beginning is because of this: https://github.com/aws/aws-lc/blob/main/crypto/fipsmodule/cipher/e_aes.c#L356, which prevents the initialization of underlying AES-GCM when wrapping it inside XAES-256-GCM."
crypto/fipsmodule/cipher/e_aes.c
Outdated
|
|
||
| static int xaes_256_gcm_init(EVP_CIPHER_CTX *ctx, const uint8_t *key, | ||
| const uint8_t *iv, int enc) { | ||
| if(key != NULL && !xaes_256_gcm_init_common(ctx, key, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a bug -- if key is NULL than you'll fall through and call xaes_256_gcm_set_gcm_key, but I don't think you want to do that, do you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- When key is NOT NULL, it invokes xaes_256_gcm_init_common() to initializes xaes_256_gcm_ctx with the main key.
- When key is NULL but nonce is NOT NULL, it invokes xaes_256_gcm_set_gcm_key() to derive a subkey, then invokes aes_gcm_init_key() to initialize the underlying AES-GCM with the subkey.
- Function aes_gcm_init_key() is called every time EVP_CipherInit_ex is called, and even when key is NULL. See this: https://github.com/aws/aws-lc/blob/main/crypto/fipsmodule/cipher/e_aes.c#L369, it only does nothing if both key and iv are NULL. And since the flag EVP_CIPH_ALWAYS_CALL_INIT is set for AES-GCM (see https://github.com/aws/aws-lc/blob/main/crypto/fipsmodule/cipher/e_aes.c#L971), it always calls this init function every time when EVP_CipherInit_ex is called (see https://github.com/aws/aws-lc/blob/main/crypto/fipsmodule/cipher/cipher.c#L232). The idea is to maintain the behaviors of XAES-256-GCM the same as those of underlying AES-GCM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure I completely understand the explanation but I would suggest we simplify the code and add comments explaining what is done when. We can start by adding this at the beginning of the function:
if (iv == NULL && key == NULL) {
return;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang-tidy made some suggestions
crypto/cipher_extra/cipher_test.cc
Outdated
|
|
||
| std::vector<uint8_t> key(32), nonce(24), plaintext(256); | ||
| std::vector<uint8_t> aad(256), ciphertext(256), tag(16); | ||
| uint8_t plaintext_len, aad_len; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
warning: variable 'aad_len' is not initialized [cppcoreguidelines-init-variables]
| uint8_t plaintext_len, aad_len; | |
| = 0 |
crypto/cipher_extra/cipher_test.cc
Outdated
|
|
||
| std::vector<uint8_t> key(32), nonce(24), plaintext(256); | ||
| std::vector<uint8_t> aad(256), ciphertext(256), tag(16); | ||
| uint8_t plaintext_len, aad_len; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
warning: variable 'plaintext_len' is not initialized [cppcoreguidelines-init-variables]
| uint8_t plaintext_len, aad_len; | |
| = 0 |
crypto/fipsmodule/cipher/e_aes.c
Outdated
| } | ||
|
|
||
| static XAES_256_GCM_CTX *xaes_256_gcm_from_cipher_ctx(EVP_CIPHER_CTX *ctx) { | ||
| // XAES_256_GCM_CTX data is put at the rear of ctx->cipher_data |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where? I can't find this. and why would we do it like that, why not store the context at ctx->cipher_data?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will fail a test case MAC non-fips 16. I cannot get a docker to debug the error as there is no available docker link in that test. I guest there needs to have some padding space after EVP_AES_GCM_CTX to pass that test. If I add a field just to add space for passing that test, there will be more confusions.
Why cannot do like that? What is the error that may occur by doing (XAES_256_GCM_CTX *)((uint8_t*)ctx->cipher_data + XAES_256_GCM_CTX_OFFSET);?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please show me where you "put data at the rear of ctx->cipher_data"? If storing the data where it should be stored (ctx->cipher_data) is causing a test failure then we should investigate what's the issue before implementing a work around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Line 1901, the context size is defined as:
out->ctx_size = sizeof(EVP_AES_GCM_CTX) + EVP_AES_GCM_CTX_PADDING + sizeof(XAES_256_GCM_CTX);
It means I would like to allocate more space for cipher_data to accommodate a struct XAES_256_GCM_CTX, apart from EVP_AES_GCM_CTX of standard AES-GCM.
I would like to reuse the function: aes_gcm_from_cipher_ctx, as this function is also called in many other places (cipher, ctrl, etc.). For this reason, XAES_256_GCM_CTX should be put at the end of cipher_data.
If I follow your code, I declare a struct that includes EVP_AES_GCM_CTX inside, then write another function similar to aes_gcm_from_cipher_ctx to specify the start position of EVP_AES_GCM_CTX. Then I also have to see why a test case failed? Is that too cumbersome?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the whole issue stems from the fact that you are trying to use an AES specific function for XAES. We should not do that, instead, we should define a new XAES specific function.
crypto/fipsmodule/cipher/e_aes.c
Outdated
| #define XAES_256_GCM_CTX_OFFSET (sizeof(EVP_AES_GCM_CTX) + EVP_AES_GCM_CTX_PADDING) | ||
| #define XAES_256_GCM_KEY_LENGTH (AES_BLOCK_SIZE * 2) | ||
| #define XAES_256_GCM_KEY_COMMIT_SIZE (AES_BLOCK_SIZE * 2) | ||
| #define XAES_256_GCM_CMAC_INPUT_SIZE (AES_BLOCK_SIZE * 2) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought this would be
| #define XAES_256_GCM_CMAC_INPUT_SIZE (AES_BLOCK_SIZE * 2) | |
| #define XAES_256_GCM_CMAC_INPUT_SIZE (AES_BLOCK_SIZE) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This const is no longer used as I removed the code using CMAC available in AWS-LC. I'll remove it.
crypto/fipsmodule/cipher/e_aes.c
Outdated
| OPENSSL_MSVC_PRAGMA(warning(pop)) | ||
|
|
||
| /* ---------------------------- XAES-256-GCM ---------------------------- | ||
| Specification: https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we're allowing the nonce size to vary, we need to reference the description that allows a nonce smaller than 24 bytes. I see you have it further below on l. 1849, but it would be good to add that reference here too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add it. It is a simple extension in the case without key commitment. We use the first 12 bytes of nonce for key derivation, and the bytes at [b-12: b] for IV in encryption, where 20 <= b <= 24 is the size of the input nonce.
| aes 47 : id-aes256-CCM : aes-256-ccm | ||
| aes 48 : id-aes256-wrap-pad | ||
| aes 48 : id-aes256-wrap-pad | ||
| aes 49 : id-xaes256-GCM : xaes-256-gcm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should add a comment here that this is a new OID we're introducing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! I'll add a comment like you suggested.
| // hexdump writes |msg| to |fp| followed by the hex encoding of |len| bytes | ||
| // from |in|. | ||
| void hexdump(FILE *fp, const char *msg, const void *in, size_t len); | ||
| bool ConvertToBytes(std::vector<uint8_t> *out, const std::string &value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, can you add this comment here with a TODO so we can decide later whether we want to make it public in file_test.h?
| int len = 0; | ||
| ASSERT_TRUE(EVP_CipherFinal_ex(ctx.get(), ciphertext.data() + ciphertext_len, &len)); | ||
| ciphertext_len += len; | ||
| ASSERT_TRUE(EVP_DigestUpdate(d.get(), ciphertext.data(), ciphertext_len)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this line for as one as l.1532? It would be good if the flow here has comments as it is not very clear to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I already added the reference. The source of this test is at:
https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM/go/XAES-256-GCM_test.go
I just rewrite it in C++. It uses hash function SHAKE128 to generate data. Also, verifying the ciphertext output is done by that hash function. It verifies the ciphertext output of each iteration, then compares the final digest output after 10K iterations, 1M iterations.
EVP_DigestUpdate(d.get(), ciphertext.data(), ciphertext_len);
It accumulates ciphertext.data() of ciphertext_len bytes into the digest output.
| # Source of test vectors: | ||
| # https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM.md | ||
| Cipher = XAES-256-GCM | ||
| Key = 0101010101010101010101010101010101010101010101010101010101010101 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume you ensured these test vectors are being used, e.g. by inserting an error in one of them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are being used in cipher_test.cc. I verified like what you said. I edited the test case to make it different and when I run test, it shows errors and fails the test.
| test(plaintext, strlen((const char *)plaintext), ciphertext, tag); | ||
| } | ||
|
|
||
| // Source of multi-loop tests: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Are there KATs (known-answer tests) that test the streaming API, i.e. calling CipherUpdate multiple times?
- are the shorter nonces tested at least with a self-consistency test, i.e. we can decrypt what we encrypt?
- what does multi-loop here stand for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
As you see in my implementation, all functions of aes-gcm are reused, except the init one (aes_gcm_init is replaced by xaes_256_gcm_init). Invoking CipherUpdate again will have the same behavior as the first time it is called. The KATs available with too small plaintext sizes, so I do not need to call it multiple times. Maybe speedtool tests that behavior, and it is still fine.
-
I do not have KATs with shorter nonces. I can write tests if required. I'm not sure whether that is necessary since it is a small change:
From:
aes_gcm_init_key(ctx, derived_key, nonce + AES_GCM_NONCE_LENGTH, enc);
To:
aes_gcm_init_key(ctx, derived_key, nonce + ivlen - AES_GCM_NONCE_LENGTH, enc); -
I tried to get all KATs available. Multi-loop tests are from: https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM/go/XAES-256-GCM_test.go. I just rewrite it in C/C++.
| assert(ctx->cipher->ctx_size == | ||
| sizeof(EVP_AES_GCM_CTX) + EVP_AES_GCM_CTX_PADDING); | ||
| break; | ||
| // not execute assert checking with xaes-256-gcm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still don't understand why would we call this function with XAES?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
aes_gcm_from_cipher_ctx() is to specify the position of EVP_AES_GCM_CTX. It is used in many other places: cipher, ctrl, etc.
In this function, there is check:
assert(ctx->cipher->ctx_size == sizeof(EVP_AES_GCM_CTX) + EVP_AES_GCM_CTX_PADDING);
The context size is no longer true with xaes-256-gcm, since we need more space for XAES_256_GCM_CTX, apart from EVP_AES_GCM_CTX and EVP_AES_GCM_CTX_PADDING.
So I rewrite the code like this to temporarily bypass this check for XAES-256-GCM, so that tests will not fail in CI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
my question is why do you use aes_gcm_from_cipher_ctx for XAES?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That function is used by aes_gcm_init_key, aes_gcm_cipher, and aes_gcm_ctrl. I do not use aes_gcm_from_cipher_ctx explicitly. I reuse aes_gcm_init_key, aes_gcm_cipher, aes_gcm_ctrl, and aes_gcm_cleanup for XAES-256-GCM.
crypto/fipsmodule/cipher/e_aes.c
Outdated
| out->ctx_size = sizeof(EVP_AES_GCM_CTX) + EVP_AES_GCM_CTX_PADDING | ||
| + sizeof(XAES_256_GCM_CTX); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this correct? why are we not storing everything in XAES_256_GCM_CTX?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is just style. Like I explained, I can declare a field of type EVP_AES_GCM_CTX inside XAES_256_GCM_CTX. Then, like in previous review, you require another function similar to aes_gcm_from_cipher_ctx to specify the start position of EVP_AES_GCM_CTX. Then I also have to see why a test case failed? Is that too cumbersome?
crypto/cipher_extra/cipher_test.cc
Outdated
| CHECK_ERROR(EVP_DecryptFinal(ctx.get(), out_vec.data(), &out_len), ERR_R_SHOULD_NOT_HAVE_BEEN_CALLED); | ||
| } | ||
|
|
||
| TEST(CipherTest, XAES_256_GCM_EVP_CIPHER) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please separate out these tests:
- Invalid nonce and key length
- deriving many subkeys from a main key but with different nonces
- multi-loop test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Issues:
Description of Changes
Implementation for API EVP_CIPHER of XAES-256-GCM is appended to e_aes.c, and the tests are appended to cipher_test.cc.
Testing
We use the test vectors provided here:
https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM.md
https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM/openssl/openssl.c
https://github.com/C2SP/C2SP/blob/main/XAES-256-GCM/go/XAES-256-GCM_test.go
Evaluation
Performance evaluation with speedtool:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.