eth: implemented peer consensus based witness verification #1766

pratikspatil024 · 2025-09-15T11:31:07Z

Description

Implemented peer consensus based witness verification

Warning Threshold System
- Triggers verification when a peer reports # pages > allowed (calculated dynamically)
- Peers under threshold are assumed honest
Peer Consensus Verification
- Query 2 random peers for verification
- Use collective peer knowledge to detect malicious behaviour
Peer Management
- Drop peers immediately
- Cache verification results to avoid redundant checks

CheckWitnessPageCount() - Entry point for verification
verifyWitnessPageCount() - Core consensus logic
RequestWitnessesWithVerification() - Verification-enabled witness requests
getAllPeers() - Random peer selection support

//configs
const (
    witnessPageWarningThreshold = 10               // Trigger verification threshold
    witnessVerificationPeers    = 2                // Number of peers to query
    witnessVerificationTimeout  = 5 * time.Second  // Query timeout
    witnessVerificationCacheTTL = 10 * time.Minute // Cache duration
)

Verification Flow:


1. Node A requests witness 0xABC... from Peer B.
2. Downloads Page 0, learns there are 50 total pages.
3. Locks TotalPages = 50.
4. Calculates threshold → ceil(45MB / 15MB) = 3 pages.
5. Since 50 > 3, pauses to verify.
6. Downloads Pages 1, 2, and 3 (up to threshold).Bandwidth used so far: 48MB.
7. Cache check → MISS.
8. Selects two random peers → Peer C and Peer D.
9. Requests metadata (WIT2) from both peers.
10. Both report: TotalPages = 50, Size = 800MB, Block = 12345.
11. Bandwidth for metadata: 180 bytes.
12. Calculates consensus → majority agrees on 50 pages.
13. Peer B deemed honest.
14. Caches result {0xABC...: 50 pages, TTL = 10 min}.
15. Resumes download from Peer B (Pages 4–49).
16. Validates each page (total count, page index, limits).
17. Completes full download (50 pages = 800MB).
18. Reconstructs witness and imports it.

Changes

Bugfix (non-breaking change that solves an issue)
Hotfix (change that solves an urgent issue, and requires immediate attention)
New feature (non-breaking change that adds functionality)
Breaking change (change that is not backwards-compatible and/or changes current functionality)
Changes only for a subset of nodes

Breaking changes

Please complete this section if any breaking changes have been made, otherwise delete it

Nodes audience

In case this PR includes changes that must be applied only to a subset of nodes, please specify how you handled it (e.g. by adding a flag with a default value...)

Checklist

I have added at least 2 reviewer or the whole pos-v1 team
I have added sufficient documentation in code
I will be resolving comments - if any - by pushing each fix in a separate commit and linking the commit hash in the comment reply
Created a task in Jira and informed the team for implementation in Erigon client (if applicable)
Includes RPC methods changes, and the Notion documentation has been updated

Cross repository changes

This PR requires changes to heimdall
- In case link the PR here:
This PR requires changes to matic-cli
- In case link the PR here:

Testing

I have added unit tests
I have added tests to CI
I have tested this code manually on local environment
I have tested this code manually on remote devnet using express-cli
I have tested this code manually on amoy
I have created new e2e tests into express-cli

Manual tests

Please complete this section with the steps you performed if you ran manual tests for this functionality, otherwise delete it

Additional comments

Please post additional comments in this section if you have them, otherwise delete it

core/stateless/witness_test.go

lucca30 · 2025-09-15T12:45:12Z

eth/handler_eth.go

+						if witPacket, ok := res.Res.(*wit.WitnessPacketRLPPacket); ok && len(witPacket.WitnessPacketResponse) > 0 {
+							return witPacket.WitnessPacketResponse[0].TotalPages, nil
+						}


The res from resCh is not *wit.WitnessPacketRLPPacket but actually []*stateless.Witness.

Reference:

bor/eth/peer.go

Lines 256 to 276 in 8c7fa04

var witnesses []*stateless.Witness

var responseHashes []common.Hash

for hash, wit := range reconstructedWitness {

witnesses = append(witnesses, wit)

responseHashes = append(responseHashes, hash)

}

if len(witnesses) != len(hashes) {

p.witPeer.Peer.Log().Error("Not able to fetch all requests witnesses", "peer", p.ID(), "requestedHashes", hashes, "responseHashes", responseHashes)

}

doneCh := make(chan error)

go func() {

<-doneCh

}()

// Adapt wit.Response[] to eth.Response.

// We can only copy exported fields. The unexported fields (id, recv, code)

// will have zero values in the ethRes sent to the caller.

// Correlation still works via the Req field.

ethRes := &eth.Response{

Req: wrapperReq.Request, // Point back to the embedded shim request.

Res: witnesses,

A few things we should consider:

By requesting that way, we are downloading the whole witness, not a single page. (In this case >160MB)

Even requesting a single page, still a big amount of data (16MB for each peer)

We should consider implementing a new method to just retrieve the page count. (may be useful for future scenarios)

Thanks Lucca, makes sense

does this makes sense?

BTW, this does not add a separate wit protocol method, as it would need a protocol upgrade.
Do you think I should do that?
@lucca30 @cffls

Hi @pratikspatil024 , good question. Introducing a new method allows us reducing from 16MB in this packet to basically 0. So it seems like an interesting upgrade. We can maybe deliver this first version that way but keep the upgrade on backlog.

cffls · 2025-09-15T16:31:26Z

Since this isn't a necessary feature in the initial veblop release, let's use develop as the base branch.

cffls · 2025-09-23T22:36:46Z

core/stateless/witness_test.go

+func TestWitnessVerificationConstants(t *testing.T) {
+	// These constants should match the ones defined in eth/fetcher/witness_manager.go
+	const (
+		witnessPageWarningThreshold = 10


Might be better to make this a dynamic parameter based on the block gas limit.

…p-pos-2955

…est case

…s between 3 peers

…metadata

…p-pos-2955

cffls

Has this been tested in a devnet? It would be great if we can create some test scenarios for this new logic.

cffls · 2025-10-06T21:44:42Z

eth/handler_eth.go

-			// Request witnesses using the wit peer
-			return ethPeer.RequestWitnesses([]common.Hash{hash}, sink)
+			// Create verification callback for page count verification (returns true if honest)
+			verifyPageCount := func(hash common.Hash, pageCount uint64, peer string) bool {


Could this be moved to a standalone function instead of being an embedded function?

Moved, thanks!

cffls · 2025-10-06T21:50:39Z

eth/peer.go

+				mapsMu.Lock()
+				downloadPaused[page.Hash] = true
+				mapsMu.Unlock()
+				p.witPeer.Peer.Log().Warn("Peer failed verification, pausing witness download", "peer", p.ID(), "hash", page.Hash, "totalPages", page.TotalPages)


Should the node drop a peer here if the peer is dishonest? The block hash is added to downloadPaused but no further action is taken it seems.

Nice catch! Fixed, thanks!

cffls · 2025-10-06T21:58:51Z

eth/handler_eth.go

+				}
+
+				// Run synchronous verification and return result
+				return h.blockFetcher.GetWitnessManager().CheckWitnessPageCount(hash, pageCount, peer, getRandomPeers, getWitnessPageCount)


Should we call CheckWitnessPageCount first before sampling from peers? Because it would avoid sending unnecessary metadata request to peers in most cases.

This is what's happening right now, the getRandomPeers function is used at a later point. Added comments.

Let me know if this is not what you meant, thanks!

…p-pos-2955

sonarqubecloud · 2025-10-15T11:41:36Z

Quality Gate failed

Failed conditions
2 Security Hotspots
11.7% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

eth: implemented peer consensus based witness verification

7d8d855

ssandeep requested review from cffls, lucca30 and manav2401 September 15, 2025 12:12

lucca30 reviewed Sep 15, 2025

View reviewed changes

cffls reviewed Sep 23, 2025

View reviewed changes

pratikspatil024 added 2 commits September 25, 2025 15:29

eth/fetcher: update peer drop mechanism

fc81044

Merge branch 'develop' of https://github.com/maticnetwork/bor into ps…

96ab269

…p-pos-2955

pratikspatil024 changed the base branch from v2.3.0-candidate to develop September 25, 2025 10:20

pratikspatil024 added 2 commits September 30, 2025 11:44

eth: added dynamic page limit for witness based on gas limit

e94a428

eth: fix lint

9515bd1

pratikspatil024 force-pushed the psp-pos-2955 branch from 46a08b6 to 9515bd1 Compare September 30, 2025 09:10

pratikspatil024 added 8 commits September 30, 2025 17:13

eth: download witness within limit and block for verificatio

d327194

eth: fix lint

8723b42

eth, core: added a check to fix a potential vulnerability and added t…

83ab606

…est case

eth, core: include the original peer in the consenses to get consense…

3a6ab74

…s between 3 peers

eth: implemented RequestWitnessPageCount

439e927

eth: implemented wit/2 and added new message type to get the witness …

e77b139

…metadata

Merge branch 'develop' of https://github.com/maticnetwork/bor into ps…

a1703bb

…p-pos-2955

eth: updated mock file

9f1fd71

cffls reviewed Oct 6, 2025

View reviewed changes

pratikspatil024 added 4 commits October 15, 2025 15:55

eth: removed code duplication

d9f0371

eth: dropping peer immediately when page count verification fails

ba40cb6

eth: added more comments

5cb56b4

Merge branch 'develop' of https://github.com/maticnetwork/bor into ps…

6b8d178

…p-pos-2955

	var witnesses []*stateless.Witness
	var responseHashes []common.Hash
	for hash, wit := range reconstructedWitness {
	witnesses = append(witnesses, wit)
	responseHashes = append(responseHashes, hash)
	}
	if len(witnesses) != len(hashes) {
	p.witPeer.Peer.Log().Error("Not able to fetch all requests witnesses", "peer", p.ID(), "requestedHashes", hashes, "responseHashes", responseHashes)
	}
	doneCh := make(chan error)
	go func() {
	<-doneCh
	}()

	// Adapt wit.Response[] to eth.Response.
	// We can only copy exported fields. The unexported fields (id, recv, code)
	// will have zero values in the ethRes sent to the caller.
	// Correlation still works via the Req field.
	ethRes := &eth.Response{
	Req: wrapperReq.Request, // Point back to the embedded shim request.
	Res: witnesses,

eth: implemented peer consensus based witness verification #1766

Are you sure you want to change the base?

eth: implemented peer consensus based witness verification #1766

Uh oh!

Conversation

pratikspatil024 commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Breaking changes

Nodes audience

Checklist

Cross repository changes

Testing

Manual tests

Additional comments

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cffls commented Sep 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cffls left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Oct 15, 2025

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pratikspatil024 commented Sep 15, 2025 •

edited

Loading