Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/txindex #5661

Draft
wants to merge 3 commits into
base: develop
Choose a base branch
from
Draft

Feat/txindex #5661

wants to merge 3 commits into from

Conversation

rdeioris
Copy link
Contributor

@rdeioris rdeioris commented Jan 6, 2025

Description

This is a reengineering of #5484 based on feedbacks from both the initial code review and nakamoto meetings.

In this first draft it just covers the infrastructure for adding a new "txindex" (boolean) option in the node config and deprecation of the old STACKS_TRANSACTION_LOG environment variable.

This option injects into the nakamoto coordinator and adds a parameter to NakamotoChainState::process_next_nakamoto_block enabling/disabling the record of a transaction.

Note that enabling both txindex and STACKS_TRANSACTION_LOG will trigger an error on startup prevending the system to enable both.

Once this first draft is approved i will add back the rpc endpoints features (that should be way more simple now)

TODO/Questions:

  • Which transactions fields should we add to the transactions table ? (currently we have result, txid, index_block_hash, transaction body as hex) [proposed: fee, events, time the transaction has been in the mempool]
  • A new rpc endpoint for querying transactions is going to be added, do we want to supprt deserialization of the tx body on the fly ? (instead of returnign the raw hex)

Applicable issues

Additional info (benefits, drawbacks, caveats)

Checklist

  • Test coverage for new or modified code paths
  • Changelog is updated
  • Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
  • New clarity functions have corresponding PR in clarity-benchmarking repo
  • New integration test(s) added to bitcoin-tests.yml

@rdeioris rdeioris mentioned this pull request Jan 7, 2025
5 tasks
@aldur aldur requested a review from jcnelson January 8, 2025 16:08
@rdeioris rdeioris requested review from obycode and kantai January 28, 2025 19:16
@aldur aldur added this to the 3.1.0.0.6 milestone Feb 4, 2025
@@ -643,16 +643,8 @@ impl<'a> ChainstateTx<'a> {
events: &[StacksTransactionReceipt],
) {
if *TRANSACTION_LOG {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just get rid of this variable entirely I think

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, but also add a clear message about this change to the changelog.

Copy link
Member

@kantai kantai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, but I think we should drop the old TRANSACTION_LOG variable.

Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just some minor comments.

"INSERT INTO transactions (txid, index_block_hash, tx_hex, result) VALUES (?, ?, ?, ?)";
let txid = tx_receipt.transaction.txid();
let tx_hex = tx_receipt.transaction.serialize_to_dbstring();
let result = tx_receipt.result.to_string();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is just replicating the old behavior, but while we're changing things, do we know if anyone who wants to use this feature would want to save any other metadata besides just the result, e.g. the events emitted or the cost?

@@ -643,16 +643,8 @@ impl<'a> ChainstateTx<'a> {
events: &[StacksTransactionReceipt],
) {
if *TRANSACTION_LOG {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, but also add a clear message about this change to the changelog.

@@ -620,6 +620,7 @@ impl RunLoop {
require_affirmed_anchor_blocks: moved_config
.node
.require_affirmed_anchor_blocks,
txindex: false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why doesn't this get the value from the config?

@diwakergupta
Copy link
Member

@rdeioris have you done a genesis sync with this change? I'd be curious to understand the storage impact of enabling this feature.

@rdeioris
Copy link
Contributor Author

rdeioris commented Feb 7, 2025

@rdeioris have you done a genesis sync with this change? I'd be curious to understand the storage impact of enabling this feature.

Pretty huge, was about 20G on testnet with the older patch

@rdeioris
Copy link
Contributor Author

rdeioris commented Feb 7, 2025

@diwakergupta i think we can eventually avoid to store the tx body (as an option) and let the user retrieve it by the block itself using the index_block_hash

@jcnelson
Copy link
Member

jcnelson commented Feb 7, 2025

From the call: we're still waiting for feedback from the builders who will use this before it's ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Status: 💻 In Progress
Development

Successfully merging this pull request may close these issues.

6 participants