-
Notifications
You must be signed in to change notification settings - Fork 416
offer: make the merkle tree signature public #3892
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
offer: make the merkle tree signature public #3892
Conversation
👋 Thanks for assigning @valentinewallace as a reviewer! |
This is helpfull for the users that want to use the merkle tree signature in their own code, for example to verify the signature of bolt12 invoices or recreate it. Very useful for people that are building command line tools for the bolt12 offers. Signed-off-by: Vincenzo Palazzo <[email protected]>
0ffed61
to
be23002
Compare
I'm not personally opposed to this and it may be nice to expose it for the use cases you mention. I'm also not sure about any API guarantees for these methods. I don't think it's too much of a hassle to treat them the same way we treat any other public API. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3892 +/- ##
==========================================
- Coverage 89.66% 88.80% -0.87%
==========================================
Files 164 166 +2
Lines 134661 119416 -15245
Branches 134661 119416 -15245
==========================================
- Hits 120743 106045 -14698
+ Misses 11237 11034 -203
+ Partials 2681 2337 -344 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
lightning/src/offers/merkle.rs
Outdated
@@ -41,7 +41,7 @@ impl TaggedHash { | |||
/// Creates a tagged hash with the given parameters. | |||
/// | |||
/// Panics if `bytes` is not a well-formed TLV stream containing at least one TLV record. | |||
pub(super) fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self { | |||
pub fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have a user for this? Feels pretty low-level to be exposing. Echo dunxen's sentiments about improving the docs if we do want to expose it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, if we're gonna expose this we should really add a validating wrapper version that checks the stream is valid and can Err
instead of panicking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have a user for this? Feels pretty low-level to be exposing.
Yeah, there is. Everyone who wants to verify that an offer is linked to a bolt12 invoice, also if the big picture is that the bolt12 invoice will never be exposed, but for now that we do not have a better PoP a person should allow this kind of verification IMHO
A possible example of verification can be
/// Verify that an offer, invoice, and preimage are valid and consistent
pub fn verify_offer_payment(offer: &str, invoice: &str, preimage: &str) -> Result<()> {
log::info!("Starting verification process...");
log::info!("Offer: {}", offer);
log::info!("Invoice: {}", invoice);
log::info!("Preimage: {}", preimage);
let offer =
Offer::from_str(offer).map_err(|e| anyhow::anyhow!("Failed to parse offer: {:?}", e))?;
// Parse the bolt12 invoice preimage
let (hrp, data) = bech32::decode_without_checksum(invoice)?;
if hrp.as_str() != "lni" {
return Err(anyhow::anyhow!(
"Invalid HRP: expected 'lni', found '{}'",
hrp
));
}
let data = Vec::<u8>::from_base32(&data)?;
let invoice = Bolt12Invoice::try_from(data)
.map_err(|e| anyhow::anyhow!("Failed to parse BOLT12: {e:?}"))?;
let issuer_sign_pubkey = if let Some(issue_sign_pubkey) = offer.issuer_signing_pubkey() {
issue_sign_pubkey
} else {
let valid_signing_keys = offer
.paths()
.iter()
.filter_map(|path| path.blinded_hops().last())
.map(|hop| hop.blinded_node_id)
.collect::<Vec<_>>();
log::debug!("{:?}", valid_signing_keys);
let valid_signing_keys = valid_signing_keys
.first()
.ok_or(anyhow::anyhow!(
"Offer does not contain issuer signing pubkey"
))?
.clone();
valid_signing_keys
};
let inv_signing_pubkey = invoice.signing_pubkey();
if issuer_sign_pubkey != inv_signing_pubkey {
return Err(anyhow::anyhow!(
"Offer and invoice issuer signing pubkeys do not match"
));
}
let mut bytes = vec![];
invoice
.write(&mut bytes)
.map_err(|e| anyhow::anyhow!("Failed to serialize invoice: {:?}", e))?;
let tagged_hash = TaggedHash::from_valid_tlv_stream_bytes(SIGNATURE_TAG, &bytes);
if let Err(err) =
merkle::verify_signature(&invoice.signature(), &tagged_hash, issuer_sign_pubkey)
{
anyhow::bail!("Failed to verify invoice signature: {:?}", err);
}
let preimage =
hex::decode(preimage).map_err(|e| anyhow::anyhow!("Failed to decode preimage: {:?}", e))?;
let preimage: [u8; 32] = preimage
.try_into()
.map_err(|_| anyhow::anyhow!("Preimage must be 32 bytes"))?;
let preimage = PaymentPreimage(preimage);
// Validate the Proof of Payment for the invoice
let computed_hash = PaymentHash::from(preimage);
let payment_hash = invoice.payment_hash();
if computed_hash != payment_hash {
anyhow::bail!("Fail to perform PoP");
}
log::info!("Parsed offer: {:?}", offer);
log::info!("Parsed invoice: {:?}", invoice);
Ok(())
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm... we verify the invoice's signature when parsing it. So it's just a matter of matching that the Offer
given is the same one that is part of the Bolt12Invoice
. This should be possible by comparing Offer::id
-- which is just the tagged hash of the offer's merkle root -- against one obtained from the Bolt12Invoice
. We could possibly add Bolt12Invoice::offer_id
for this.
Note that while OfferId
uses an LDK-specific tag, it is computed over the TLV stream bytes so it doesn't matter if the offer wasn't created with LDK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok implemented it @jkczyz cfd6ad4 let me know what do you think.
Thanks, left some comments.
However, I still think that ldk should expose a way to verify the signature with the public key that is inside the offer, at least till we do not get a better PoP with bolt12
Given the signature is verified when parsing, shouldn't it be sufficient to check that Bolt12Invoice::signing_pubkey
matches the one in the offer? The parser is already doing this when calling check_invoice_signing_pubkey
, too, FWIW. Not sure I understand why we need to expose these for the user to piece together and call on their own.
At very least, there doesn't seem to be a need to expose TaggedHash::from_valid_tlv_stream_bytes
if we can just have a method on Bolt12Invoice
give the TaggedHash
for manual signature verification.
👋 The first review has been submitted! Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer. |
This commit addresses review feedback by adding a validating version of the merkle tree signature functions that returns proper error types instead of panicking on invalid input. The new from_tlv_stream_bytes function validates TLV streams before processing and returns a Result, making it safer for command line tools and external applications to use. Additionally, documentation has been improved to clarify that these are low-level functions for specific use cases, with clear guidance to use higher-level methods for production. Comprehensive tests have been added to ensure consistency between the validating and non-validating functions and prevent regressions. Signed-off-by: Vincenzo Palazzo <[email protected]>
pub fn from_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Result<Self, TlvStreamError> { | ||
// Validate the TLV stream first | ||
if bytes.is_empty() { | ||
return Err(TlvStreamError::EmptyStream); | ||
} | ||
|
||
// Try to parse the TLV stream to check validity | ||
let mut cursor = io::Cursor::new(bytes); | ||
let mut has_records = false; | ||
|
||
while cursor.position() < bytes.len() as u64 { | ||
// Try to read type | ||
let type_result = <BigSize as Readable>::read(&mut cursor); | ||
if type_result.is_err() { | ||
return Err(TlvStreamError::InvalidRecord); | ||
} | ||
|
||
// Try to read length | ||
let length_result = <BigSize as Readable>::read(&mut cursor); | ||
if length_result.is_err() { | ||
return Err(TlvStreamError::InvalidRecord); | ||
} | ||
|
||
let length = length_result.unwrap().0; | ||
let end_position = cursor.position() + length; | ||
|
||
// Check if the record extends beyond the buffer | ||
if end_position > bytes.len() as u64 { | ||
return Err(TlvStreamError::InvalidRecord); | ||
} | ||
|
||
// Skip the value | ||
cursor.set_position(end_position); | ||
has_records = true; | ||
} | ||
|
||
if !has_records { | ||
return Err(TlvStreamError::EmptyStream); | ||
} | ||
|
||
// If validation passes, create the tagged hash | ||
Ok(Self::from_valid_tlv_stream_bytes(tag, bytes)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TLV stream validation implementation is missing a critical requirement from the Lightning Network specification: TLV records must appear in ascending order by type.
To ensure full compliance, consider tracking the previous type value during iteration and verifying that each new type is greater than the previous one:
let mut previous_type: Option<u64> = None;
while cursor.position() < bytes.len() as u64 {
// Try to read type
let type_result = <BigSize as Readable>::read(&mut cursor);
if type_result.is_err() {
return Err(TlvStreamError::InvalidRecord);
}
let current_type = type_result.unwrap().0;
// Check ascending order
if let Some(prev) = previous_type {
if current_type <= prev {
return Err(TlvStreamError::InvalidOrder);
}
}
previous_type = Some(current_type);
// Rest of the validation...
}
This would require adding an InvalidOrder
variant to the TlvStreamError
enum.
pub fn from_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Result<Self, TlvStreamError> { | |
// Validate the TLV stream first | |
if bytes.is_empty() { | |
return Err(TlvStreamError::EmptyStream); | |
} | |
// Try to parse the TLV stream to check validity | |
let mut cursor = io::Cursor::new(bytes); | |
let mut has_records = false; | |
while cursor.position() < bytes.len() as u64 { | |
// Try to read type | |
let type_result = <BigSize as Readable>::read(&mut cursor); | |
if type_result.is_err() { | |
return Err(TlvStreamError::InvalidRecord); | |
} | |
// Try to read length | |
let length_result = <BigSize as Readable>::read(&mut cursor); | |
if length_result.is_err() { | |
return Err(TlvStreamError::InvalidRecord); | |
} | |
let length = length_result.unwrap().0; | |
let end_position = cursor.position() + length; | |
// Check if the record extends beyond the buffer | |
if end_position > bytes.len() as u64 { | |
return Err(TlvStreamError::InvalidRecord); | |
} | |
// Skip the value | |
cursor.set_position(end_position); | |
has_records = true; | |
} | |
if !has_records { | |
return Err(TlvStreamError::EmptyStream); | |
} | |
// If validation passes, create the tagged hash | |
Ok(Self::from_valid_tlv_stream_bytes(tag, bytes)) | |
pub fn from_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Result<Self, TlvStreamError> { | |
// Validate the TLV stream first | |
if bytes.is_empty() { | |
return Err(TlvStreamError::EmptyStream); | |
} | |
// Try to parse the TLV stream to check validity | |
let mut cursor = io::Cursor::new(bytes); | |
let mut has_records = false; | |
let mut previous_type: Option<u64> = None; | |
while cursor.position() < bytes.len() as u64 { | |
// Try to read type | |
let type_result = <BigSize as Readable>::read(&mut cursor); | |
if type_result.is_err() { | |
return Err(TlvStreamError::InvalidRecord); | |
} | |
let current_type = type_result.unwrap().0; | |
// Check ascending order | |
if let Some(prev) = previous_type { | |
if current_type <= prev { | |
return Err(TlvStreamError::InvalidOrder); | |
} | |
} | |
previous_type = Some(current_type); | |
// Try to read length | |
let length_result = <BigSize as Readable>::read(&mut cursor); | |
if length_result.is_err() { | |
return Err(TlvStreamError::InvalidRecord); | |
} | |
let length = length_result.unwrap().0; | |
let end_position = cursor.position() + length; | |
// Check if the record extends beyond the buffer | |
if end_position > bytes.len() as u64 { | |
return Err(TlvStreamError::InvalidRecord); | |
} | |
// Skip the value | |
cursor.set_position(end_position); | |
has_records = true; | |
} | |
if !has_records { | |
return Err(TlvStreamError::EmptyStream); | |
} | |
// If validation passes, create the tagged hash | |
Ok(Self::from_valid_tlv_stream_bytes(tag, bytes)) |
Spotted by Diamond
Is this helpful? React 👍 or 👎 to let us know.
🔔 1st Reminder Hey @TheBlueMatt @valentinewallace! This PR has been waiting for your review. |
1 similar comment
🔔 1st Reminder Hey @TheBlueMatt @valentinewallace! This PR has been waiting for your review. |
cc @jkczyz |
cfd6ad4
to
dd046d5
Compare
- Add an Option<OfferId> field to Bolt12Invoice to track the originating offer. - Compute the offer_id for invoices created from offers by extracting the offer TLV records and hashing them with the correct tag. - Expose a public offer_id() accessor on Bolt12Invoice. - Add tests to ensure the offer_id in the invoice matches the originating Offer, and that refund invoices have no offer_id. - All existing and new tests pass. This enables linking invoices to their originating offers in a robust and spec-compliant way. Signed-off-by: Vincenzo Palazzo <[email protected]>
dd046d5
to
29e5bd6
Compare
let offer_tlv_stream = TlvStream::new(&self.bytes).range(OFFER_TYPES); | ||
let experimental_offer_tlv_stream = TlvStream::new(&self.experimental_bytes).range(EXPERIMENTAL_OFFER_TYPES); | ||
let combined_tlv_stream = offer_tlv_stream.chain(experimental_offer_tlv_stream); | ||
let tagged_hash = TaggedHash::from_tlv_stream("LDK Offer ID", combined_tlv_stream); | ||
Some(OfferId(tagged_hash.to_bytes())) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can reuse OfferId::from_valid_invreq_tlv_stream
(or something with a more appropriate name) rather than duplicating that code.
let secp_ctx = Secp256k1::new(); | ||
let payment_id = PaymentId([1; 32]); | ||
|
||
// Create an offer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably don't need to the comments throughout these tests if they are just repeating what the code does.
@@ -967,6 +987,11 @@ impl Bolt12Invoice { | |||
self.tagged_hash.as_digest().as_ref().clone() | |||
} | |||
|
|||
/// Returns the offer ID if this invoice corresponds to an offer. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's link OfferId
and Offer
.
lightning/src/offers/merkle.rs
Outdated
@@ -41,7 +41,7 @@ impl TaggedHash { | |||
/// Creates a tagged hash with the given parameters. | |||
/// | |||
/// Panics if `bytes` is not a well-formed TLV stream containing at least one TLV record. | |||
pub(super) fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self { | |||
pub fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok implemented it @jkczyz cfd6ad4 let me know what do you think.
Thanks, left some comments.
However, I still think that ldk should expose a way to verify the signature with the public key that is inside the offer, at least till we do not get a better PoP with bolt12
Given the signature is verified when parsing, shouldn't it be sufficient to check that Bolt12Invoice::signing_pubkey
matches the one in the offer? The parser is already doing this when calling check_invoice_signing_pubkey
, too, FWIW. Not sure I understand why we need to expose these for the user to piece together and call on their own.
At very least, there doesn't seem to be a need to expose TaggedHash::from_valid_tlv_stream_bytes
if we can just have a method on Bolt12Invoice
give the TaggedHash
for manual signature verification.
This is helpful for users who want to use the Merkle tree signature in their own code, for example, to verify the signature of bolt12 invoices or recreate it.
Very useful for people who are building command line tools for the bolt12 offers.
I am opening this, but I do not know if it is something that you want to do, but at the same time, IMHO, this is very useful to expose because it allows to use LDK in command line tools and in learning tools.
However, this is not strictly necessary because
Bolt12Invoice::try_from
already verifies the signature, but I would open this to know your point of view.