remove cstruct from mirage-crypto #214
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
reynir
reviewed
Mar 18, 2024
| @@ -256,25 +253,38 @@ module Modes = struct | |||
|
|
|||
| let tag_size = GHASH.tagsize | |||
| let key_sizes, block_size = C.(key, block) | |||
| let z128, h = create block_size, create block_size | |||
There was a problem hiding this comment.
Maybe we don't need a global h anymore as bytes are cheaper to allocate?
Comment on lines
+265
to
+270
| let pack64s = | ||
| let _cs = Bytes.create 16 in | ||
| fun a b -> | ||
| Bytes.set_int64_be _cs 0 a; | ||
| Bytes.set_int64_be _cs 8 b; | ||
| Bytes.unsafe_to_string _cs |
There was a problem hiding this comment.
Suggested change
| let pack64s = | |
| let _cs = Bytes.create 16 in | |
| fun a b -> | |
| Bytes.set_int64_be _cs 0 a; | |
| Bytes.set_int64_be _cs 8 b; | |
| Bytes.unsafe_to_string _cs | |
| let pack64s a b = | |
| let b = Bytes.create 16 in | |
| Bytes.set_int64_be b 0 a; | |
| Bytes.set_int64_be b 8 b; | |
| Bytes.unsafe_to_string b |
There was a problem hiding this comment.
maybe we should evaluate this together with #186 in mind -- we have at various places global buffers (that are bad for OCaml 5, and eventually there's not much performance impact). I'd prefer to keep this "remove global buffers" out of this PR.
Co-authored-by: Reynir Björnsson <[email protected]>
|
thanks for the review, @reynir. indeed CCM could get another pass to reduce allocations and remove code duplications. |
|
I get 15 failures. If I comment out |
Ctr_of: avoid big copy
|
I did a bench between v0.11.3 and the recent commit here. Takeaways: eventually investigate -pk performance drop (~10%); symmetric ciphers with big blocks got slower (investigate!). If you want to post results, please take the time to reformat into tables (as below) to ensure that it is nice to read (maybe we should enhance speed.ml to output markdown directly). Mirage-crypto-pkrsa-generate
rsa-encrypt
rsa-decrypt
rsa-pkcs1-encrypt
rsa-pkcs1-decrypt
rsa-pkcs1-sign
rsa-pkcs1-verify
rsa-pss-sign
rsa-pss-verify
dsa-generate
dsa-sign
dsa-verify
dh-secret
dh-share
Mirage-crypto-ECecdsa-generate
ecdsa-sign
ecdsa-verify
ecdh-secret
ecdh-share
Mirage-cryptochacha20-poly1305
aes-128-ecb
aes-128-cbc-e
aes-128-cbc-d
aes-128-ctr
aes-128-gcm
aes-128-ghash
aes-128-ccm
aes-192-ecb
aes-256-ecb
d3des-ecb
Mirage-crypto-rngfortuna
|
Reduce allocations and string copying in CCM mode
hannesm
added a commit
to hannesm/opam-repository
that referenced
this pull request
Jun 29, 2024
CHANGES: ### Breaking changes * mirage-crypto: Poly1305 API now uses string (mirage/mirage-crypto#203 @hannesm) * mirage-crypto: Poly1305 no longer has type alias "type mac = string" (mirage/mirage-crypto#232 @hannesm) * mirage-crypto: the API uses string instead of cstruct (mirage/mirage-crypto#214 @reynir @hannesm) * mirage-crypto: Hash module has been removed. Use digestif if you need hash functions (mirage/mirage-crypto#213 @hannesm) * mirage-crypto: the Cipher_block and Cipher_stream modules have been removed, its contents is inlined: Mirage_crypto.Cipher_block.S -> Mirage_crypto.Block Mirage_crypto.Cipher_stream.S -> Mirage_crypto.Stream Mirage_crypto.Cipher_block.AES.CTR -> Mirage_crypto.AES.CTR (mirage/mirage-crypto#225 @hannesm, suggested in mirage/mirage-crypto#224 by @reynir) * mirage-crypto-pk: s-expression conversions for private and public keys (Dh, Dsa, Rsa) have been removed. You can use PKCS8 for encoding and decoding `X509.{Private,Public}_key.{en,de}code_{der,pem}` (mirage/mirage-crypto#208 @hannesm) * mirage-crypto-pk: in the API, Cstruct.t is no longer present. Instead, string is used (mirage/mirage-crypto#211 @reynir @hannesm) * mirage-crypto-rng: the API uses string instead of Cstruct.t. A new function `generate_into : ?g -> bytes -> ?off:int -> int -> unit` is provided (mirage/mirage-crypto#212 @hannesm @reynir) * mirage-crypto-ec: remove NIST P224 support (mirage/mirage-crypto#209 @hannesm @Firobe) * mirage-crypto: in Uncommon.xor_into the arguments ~src_off and ~dst_off are required now (mirage/mirage-crypto#232 @hannesm), renamed to unsafe_xor_into (98f01b14f5ebf98ba0e7e9c2ba97ec518f90fddc) * mirage-crypto-pk, mirage-crypto-rng: remove type alias "type bits = int" (mirage/mirage-crypto#236 @hannesm) ### Bugfixes * mirage-crypto (32 bit systems): CCM with long adata (mirage/mirage-crypto#207 @reynir) * mirage-crypto-ec: fix K_gen for bitlen mod 8 != 0 (reported in mirage/mirage-crypto#105 that P521 test vectors don't pass, re-reported mirage/mirage-crypto#228, fixed mirage/mirage-crypto#230 @Firobe) * mirage-crypto-ec: zero out bytes allocated for Field_element.zero (reported mirleft/ocaml-x509#167, fixed mirage/mirage-crypto#226 @dinosaure) ### Data race free * mirage-crypto (3DES): avoid global state in key derivation (mirage/mirage-crypto#223 @hannesm) * mirage-crypto-rng: use atomic instead of reference to be domain-safe (mirage/mirage-crypto#221 @dinosaure @reynir @hannesm) * mirage-crypto, mirage-crypto-rng, mirage-crypto-pk, mirage-crypto-ec: avoid global buffers, use freshly allocated strings/bytes instead, avoids data races (mirage/mirage-crypto#186 mirage/mirage-crypto#219 @dinosaure @reynir @hannesm) ### Other changes * mirage-crypto: add {de,en}crypt_into functions (and unsafe variants) to allow less buffer allocations (mirage/mirage-crypto#231 @hannesm) * mirage-crypto-rng-miou: new package which adds rng support with miou (mirage/mirage-crypto#227 @dinosaure) * PERFORMANCE mirage-crypto: ChaCha20/Poly1305 use string instead of Cstruct.t, ChaCha20 interface unchanged, performance improvement roughly 2x (mirage/mirage-crypto#203 @hannesm @reynir) * mirage-crypto-ec, mirage-crypto-pk, mirage-crypto-rng: use digestif for hashes (mirage/mirage-crypto#212 mirage/mirage-crypto#215 @reynir @hannesm) * mirage-crypto-rng: use a set for entropy sources instead of a list (mirage/mirage-crypto#218 @hannesm) * mirage-crypto-rng-mirage: provide a module type S (for use instead of mirage-random in mirage) (mirage/mirage-crypto#234 @hannesm)
avsm
pushed a commit
to avsm/opam-repository
that referenced
this pull request
Sep 5, 2024
CHANGES: ### Breaking changes * mirage-crypto: Poly1305 API now uses string (mirage/mirage-crypto#203 @hannesm) * mirage-crypto: Poly1305 no longer has type alias "type mac = string" (mirage/mirage-crypto#232 @hannesm) * mirage-crypto: the API uses string instead of cstruct (mirage/mirage-crypto#214 @reynir @hannesm) * mirage-crypto: Hash module has been removed. Use digestif if you need hash functions (mirage/mirage-crypto#213 @hannesm) * mirage-crypto: the Cipher_block and Cipher_stream modules have been removed, its contents is inlined: Mirage_crypto.Cipher_block.S -> Mirage_crypto.Block Mirage_crypto.Cipher_stream.S -> Mirage_crypto.Stream Mirage_crypto.Cipher_block.AES.CTR -> Mirage_crypto.AES.CTR (mirage/mirage-crypto#225 @hannesm, suggested in mirage/mirage-crypto#224 by @reynir) * mirage-crypto-pk: s-expression conversions for private and public keys (Dh, Dsa, Rsa) have been removed. You can use PKCS8 for encoding and decoding `X509.{Private,Public}_key.{en,de}code_{der,pem}` (mirage/mirage-crypto#208 @hannesm) * mirage-crypto-pk: in the API, Cstruct.t is no longer present. Instead, string is used (mirage/mirage-crypto#211 @reynir @hannesm) * mirage-crypto-rng: the API uses string instead of Cstruct.t. A new function `generate_into : ?g -> bytes -> ?off:int -> int -> unit` is provided (mirage/mirage-crypto#212 @hannesm @reynir) * mirage-crypto-ec: remove NIST P224 support (mirage/mirage-crypto#209 @hannesm @Firobe) * mirage-crypto: in Uncommon.xor_into the arguments ~src_off and ~dst_off are required now (mirage/mirage-crypto#232 @hannesm), renamed to unsafe_xor_into (98f01b14f5ebf98ba0e7e9c2ba97ec518f90fddc) * mirage-crypto-pk, mirage-crypto-rng: remove type alias "type bits = int" (mirage/mirage-crypto#236 @hannesm) ### Bugfixes * mirage-crypto (32 bit systems): CCM with long adata (mirage/mirage-crypto#207 @reynir) * mirage-crypto-ec: fix K_gen for bitlen mod 8 != 0 (reported in mirage/mirage-crypto#105 that P521 test vectors don't pass, re-reported mirage/mirage-crypto#228, fixed mirage/mirage-crypto#230 @Firobe) * mirage-crypto-ec: zero out bytes allocated for Field_element.zero (reported mirleft/ocaml-x509#167, fixed mirage/mirage-crypto#226 @dinosaure) ### Data race free * mirage-crypto (3DES): avoid global state in key derivation (mirage/mirage-crypto#223 @hannesm) * mirage-crypto-rng: use atomic instead of reference to be domain-safe (mirage/mirage-crypto#221 @dinosaure @reynir @hannesm) * mirage-crypto, mirage-crypto-rng, mirage-crypto-pk, mirage-crypto-ec: avoid global buffers, use freshly allocated strings/bytes instead, avoids data races (mirage/mirage-crypto#186 mirage/mirage-crypto#219 @dinosaure @reynir @hannesm) ### Other changes * mirage-crypto: add {de,en}crypt_into functions (and unsafe variants) to allow less buffer allocations (mirage/mirage-crypto#231 @hannesm) * mirage-crypto-rng-miou: new package which adds rng support with miou (mirage/mirage-crypto#227 @dinosaure) * PERFORMANCE mirage-crypto: ChaCha20/Poly1305 use string instead of Cstruct.t, ChaCha20 interface unchanged, performance improvement roughly 2x (mirage/mirage-crypto#203 @hannesm @reynir) * mirage-crypto-ec, mirage-crypto-pk, mirage-crypto-rng: use digestif for hashes (mirage/mirage-crypto#212 mirage/mirage-crypto#215 @reynir @hannesm) * mirage-crypto-rng: use a set for entropy sources instead of a list (mirage/mirage-crypto#218 @hannesm) * mirage-crypto-rng-mirage: provide a module type S (for use instead of mirage-random in mirage) (mirage/mirage-crypto#234 @hannesm)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
that's stlll WIP (compiles, but there are test failures) -- likely there's something wrong when I transcribed from cstruct to bytes/string.also should be reviewed in terms of where stuff is unecessarily copied and could be avoided to be copied
//cc @reynir who may be interested ;)