Pin the missed mutants from overnight run 38686116770

TESTING.md

@@ -377,6 +377,8 @@ A cell that names a crate-local test describes the witness that exists today. It
 | `electrum_idle_timeout_disconnects_quiet_client` | Electrum | Idle timeout closes a quiet socket |
 | `esplora_broadcast_visible_in_rpc_and_electrum` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir: HTTP `sendrawtransaction` / `testmempoolaccept` (allowed, missing-or-spent, exact 100 sat/kvB min-relay accept + one-sat-under reject, RBF one-sat-short incremental reject + exact incremental accept); Esplora `POST /tx` parent and mempool child appear in `getrawmempool` and Electrum mempool/history (`fee` on unconfirmed, including child `height = -1`); Electrum `listunspent` of that child is `height=-1` and the parent UTXO drops; process `gettxout` / `getchaintips`; Esplora `POST /txs/package` 1p1c (including parent-alone below min-relay + paying child), 25-tx accept, 26-tx and over-weight `package too large`; serving-only `submitpackage` refuses (relay off); live `GET /mempool` / `/mempool/txids` / `/mempool/recent` / `/fee-estimates` answers 503 while flow is cold and the chain holds too little fee history (a thin live pool alone sets no rate); process `getmempoolancestors` / descendants / cluster / `gettxspendingprevout` / feerate diagram / verbose `getrawmempool` on that 1p1c; `waitforblockheight` timeout=0 while behind returns the live tip; GBT stale `longpollid` is immediate; current id / `waitfornewblock` / `waitforblockheight` wake on the pad `generate`; `getblockhash` tip ok / tip+1 `-8`; unknown `getblock` `-5`; verbosity 0 hex and 2 vin/vout; `GET /blocks` 10 newest, `/blocks/0` and `/blocks/:tip` (start past tip clamps); `/block/:hash/txs/:start` last page shorter than 25, one-past last page 404, unknown hash 404; `/block/:hash/txids` + coinbase merkle-proof + unspent `outspend/0`; `/tx/:id/outspends`; `/block` JSON/raw/status/`txid/0` (OOB 404); `/tx/:id/raw` vs hex; merkleblock-proof; `/block-height` (missing 404); `/block/:hash/header` 160 hex; `/tx/:id/status` + full JSON (`unknown` OP_TRUE type, coinbase vin) and missing-tx 404s; OP_TRUE scripthash info/summary/utxo/`txs/chain` cursor and combined `/txs`. No-hub: mempool and fee routes, and `POST /tx` 503. Header bytes match the wire header. `tx_status_json` matches the status route. Esplora `/tx/:id/status` confirms the package parent on generate; `generate` includes those txs (parent before child) then leaves IBD (relay on); `submitpackage` maxfeerate reject, 1p1c success, already-in-mempool continue, below-min-relay parent + paying child success, 26-tx / over-weight `package too large`; immature coinbase sendraw rejects. Package JSON errors. `testmempoolaccept` dry-run reports the orphan count. `gettxout` with `include_mempool`, on a disconnected tip, and on a leftover. `generate` selects the chained mempool parent first. A `submitpackage` child failure keeps the parent. Maxburn `submitpackage` rejects. Waiters return when the node stops. Unix `--rpc-socket` (mode 0660, no datadir `rpc.sock`) `getblockcount` without Authorization; TCP `GET`/`POST /internal/mempool/txs`; `GET /internal/block/:hash/txs` full list vs public 25/page; `POST /internal/txs/outspends/by-txid` same-length unknown `[]` slot; `GET /address-prefix/bc1` **404**; unauthenticated Core REST on the RPC listener (`chaininfo`, block hash/headers/block/tx, mempool info/contents, `getutxos`, `deploymentinfo`, basic `blockfilter` bin/hex/json) and `getblockfilter` with `--block-filter-index` |
 | `fee_history_backfills_from_the_chain_when_relay_starts` | Node + RPC | `run_p2p` on a mature regtest datadir, started twice; `generate` leaves IBD and turns relay on; the preload writes the fee history file and the restart extends it; with flow cold and too little history, `estimatesmartfee` answers Core's insufficient-data shape. Rates from a ready history are the hub's `far_horizon_follows_block_history_not_pool_tail`; the success object is `smart_fee_json` |
+| `asmap_loopback_peers_report_their_asn` | Node + RPC | One regtest `run_p2p`, no seeds and no outbound peer. `{datadir}/ip_asn.dat` is the loopback map: three BIP324 inbounds bound to `127.0.0.1`, `.2`, and `.3` show `getpeerinfo.mapped_as` 1, 2, and 9, and the startup line names that file's length and digest. A second start with relative `--asmap` of a lone return maps `127.0.0.1` to ASN 1 (length 3, digest `709e80c8`). |
+| `asmap_refused_file_uses_prefix_netgroups` | Node | One exit per bad file (`max_run_secs` 0, no peer): a second default, a trailing one bit, eight trailing zeros, two short matches, a match past the last IP bit, a jump off the file, a jump that leaves too few bits, the smallest jump alone, an empty file, and a one-byte file each log `Sanity check of asmap file` and do not log `Opened asmap file`. A missing `--asmap` path logs `Failed to open asmap file`. |
 | `node_listen_and_exit` | Node + Electrum + Esplora + RPC | One `run_p2p` datadir, restarted with its one `--connect` refusing (a pinned connect at genesis still enters tip mode): a taken `--health-listen` port stops the start before the store opens; a junk `peers` file and a missing `--asmap` start an empty book and exit, and the saved book records the refused connect; the next start loads that book and a valid `ip_asn.dat`, and Esplora, Electrum, and RPC answer at genesis until `stop`. On that start `/healthz` is 200 and `/progress` is 200 JSON while genesis `/readyz` is initial block download and `/metrics` reports the same; `generate` makes `/readyz` 200 and the scrape ready at height 1. An Electrum port and an RPC port another process holds warn, `/readyz` names both, and `/metrics` is absent without `--metrics`. Without `--connect` and with seeds on, regtest resolves none and the node exits short of tip mode; after a `--prune-seqsigwit` start, an unpruned start refuses. Live peers are `node_run_p2p_short`. The `/readyz` phase, lag, and tip-age table stays `readiness_reports_the_first_failing_gate`. A timed-out probe keeping its permit, the full-gate 503, and the routes answering again are `health_gate_caps` |
 | `tor_control_onion_lifecycle` | Node + RPC | One `run_p2p` datadir against a fake Tor control port and SAM bridge (live Tor and i2pd are overlay-functional). A cookie from another Tor (SAFECOOKIE server hash mismatch), a 2-byte cookie, and a Tor that offers only plain COOKIE each refuse the start, and none sends `AUTHENTICATE`. Password auth with `--listen-onion`, `--i2p-accept-incoming`, Electrum, and Esplora: `ADD_ONION NEW` per service with the P2P virtual port on the loopback bind, each key saved `0600` under `onion/`, each SAM destination under `i2p/`, `STREAM FORWARD` to each port, and `getnetworkinfo.localaddresses` lists the three onions and the I2P address. A SAFECOOKIE restart reuses every saved key and destination |
 | `enter_tip_mode_indexes` | Node + Electrum + RPC | One `run_p2p` datadir restarted with `--sh-index` off, on, off, on. Off with no index yet: RPC, tip follow, and Electrum listen, `blockchain.scripthash.get_history` fails closed, and `generateblock` mines three OP_TRUE coinbases. First start on: the index is collected from Class A before history answers, and the OP_TRUE history has three rows. Off again: Electrum still listens and the leftover watermark keeps those three history rows. On after a crash that left a collect run and a lagging include high-water mark: the durable index resumes under write-behind, so the run is discarded (not merged) and history answers, and the next block lands in history |

changelog.d/overnight-38686116770.md

@@ -0,0 +1,5 @@
+Changed
+
+- Overnight missed mutants are gone or pinned on a peer message, an RPC result, or a reopened asmap or mempool file. Dead helpers (asmap byte views, the block-cache locator, the dial warm locator, and the witness wtxid-count reject) are deleted instead of skipped.
+- A headers ask sends the walk locator, which stops at eight hashes, and still appends the confirmed tip. It no longer drops walk hashes once eight names were already queued.
+- The bound P2P port is written to `{datadir}/run/p2p.addr`, the same way the RPC and wallet listeners publish their addresses.

crates/rbitcoin-consensus/src/block/mod.rs

@@ -555,10 +555,8 @@ fn check_witness_commitment_with_wtxids(
     let mut committed = [0u8; 32];
     committed.copy_from_slice(&b[6..38]);
 
-    if precomputed_non_cb.len() != block.txdata.len().saturating_sub(1) {
-        return Err(ConsensusError::BadBlock("wtxid count mismatch"));
-    }
-    // Witness merkle: coinbase wtxid is 32 zero bytes.
+    // Witness merkle: coinbase wtxid is 32 zero bytes. The caller builds
+    // `precomputed_non_cb` as one wtxid per non-coinbase tx.
     let mut leaves = Vec::with_capacity(block.txdata.len());
     leaves.push([0u8; 32]);
     leaves.extend_from_slice(precomputed_non_cb);

crates/rbitcoin-mempool/src/store.rs

@@ -724,8 +724,7 @@ impl Mempool {
         let logical = body_logical_len(&self.body)?;
         let end = logical.saturating_add(need);
         if end > self.body.len() {
-            let new_len = (self.body.len().max(4096) * 2).max(end + 4096);
-            self.body.resize(new_len, 0);
+            self.body.resize(end, 0);
         }
         self.body[8..16].copy_from_slice(&(end as u64).to_le_bytes());
         Ok(logical as u64)
@@ -740,31 +739,20 @@ impl Mempool {
         let body_path = self.dir.join("tx.body");
         let logical = body_logical_len(&self.body)? as u64;
         let start = self.body_persisted_len.min(logical);
-        if start < BODY_HEADER as u64 {
+        if logical != start {
             self.body_file
-                .seek(SeekFrom::Start(0))
+                .seek(SeekFrom::Start(8))
                 .map_err(|e| MempoolError::io(&body_path, e))?;
             self.body_file
-                .write_all(&self.body[..logical as usize])
+                .write_all(&self.body[8..16])
                 .map_err(|e| MempoolError::io(&body_path, e))?;
-        } else {
-            if start > BODY_HEADER as u64 || logical != start {
-                self.body_file
-                    .seek(SeekFrom::Start(8))
-                    .map_err(|e| MempoolError::io(&body_path, e))?;
-                self.body_file
-                    .write_all(&self.body[8..16])
-                    .map_err(|e| MempoolError::io(&body_path, e))?;
-            }
-            if logical > start {
-                self.body_file
-                    .seek(SeekFrom::Start(start))
-                    .map_err(|e| MempoolError::io(&body_path, e))?;
-                self.body_file
-                    .write_all(&self.body[start as usize..logical as usize])
-                    .map_err(|e| MempoolError::io(&body_path, e))?;
-            }
         }
+        self.body_file
+            .seek(SeekFrom::Start(start))
+            .map_err(|e| MempoolError::io(&body_path, e))?;
+        self.body_file
+            .write_all(&self.body[start as usize..logical as usize])
+            .map_err(|e| MempoolError::io(&body_path, e))?;
         self.body_file
             .set_len(logical)
             .map_err(|e| MempoolError::io(&body_path, e))?;
@@ -1089,7 +1077,6 @@ fn open_or_init_slots(path: &Path, slot_cap: u32) -> Result<(File, Vec<u8>), Mem
 }
 
 fn open_or_init_body(path: &Path) -> Result<(File, Vec<u8>), MempoolError> {
-    let initial = BODY_HEADER + 64;
     if path.exists() {
         let mut file = OpenOptions::new()
             .read(true)
@@ -1116,10 +1103,6 @@ fn open_or_init_body(path: &Path) -> Result<(File, Vec<u8>), MempoolError> {
         if logical > len {
             return Err(MempoolError::Corrupt("body logical past file"));
         }
-        // Spare capacity in process for appends (not on disk until persist).
-        if buf.len() < initial {
-            buf.resize(initial, 0);
-        }
         Ok((file, buf))
     } else {
         let mut file = OpenOptions::new()
@@ -1128,11 +1111,11 @@ fn open_or_init_body(path: &Path) -> Result<(File, Vec<u8>), MempoolError> {
             .create_new(true)
             .open(path)
             .map_err(|e| MempoolError::io(path, e))?;
-        let mut buf = vec![0u8; initial];
+        let mut buf = vec![0u8; BODY_HEADER];
         buf[0..4].copy_from_slice(&MEM_MAGIC);
         buf[4..6].copy_from_slice(&MEM_SCHEMA.to_le_bytes());
         buf[8..16].copy_from_slice(&(BODY_HEADER as u64).to_le_bytes());
-        file.write_all(&buf[..BODY_HEADER])
+        file.write_all(&buf)
             .map_err(|e| MempoolError::io(path, e))?;
         file.flush().map_err(|e| MempoolError::io(path, e))?;
         Ok((file, buf))
@@ -1314,6 +1297,39 @@ pub(crate) mod tests {
             assert!(mp.dir().join("slots").exists());
             assert!(mp.dir().join("tx.body").exists());
         }
+        assert_eq!(
+            fs::metadata(dir.join("tx.body")).unwrap().len(),
+            BODY_HEADER as u64,
+            "an empty body file is the 16-byte header"
+        );
+        let slots_path = dir.join("slots");
+        let slots = fs::read(&slots_path).unwrap();
+        let mut longer = slots.clone();
+        longer.push(0xab);
+        fs::write(&slots_path, &longer).unwrap();
+        Mempool::open_or_create(&dir).expect("slots longer than the table still open");
+        fs::write(&slots_path, &slots[..slots.len() - 1]).unwrap();
+        let err = match Mempool::open_or_create(&dir) {
+            Err(e) => e,
+            Ok(_) => panic!("a short slots file must not open"),
+        };
+        assert!(
+            matches!(err, MempoolError::Corrupt("slots file short")),
+            "{err}"
+        );
+        fs::write(&slots_path, &slots).unwrap();
+        let body_path = dir.join("tx.body");
+        let body = fs::read(&body_path).unwrap();
+        assert_eq!(body.len(), BODY_HEADER);
+        fs::write(&body_path, &body[..BODY_HEADER - 1]).unwrap();
+        let err = match Mempool::open_or_create(&dir) {
+            Err(e) => e,
+            Ok(_) => panic!("a short body file must not open"),
+        };
+        assert!(
+            matches!(err, MempoolError::Corrupt("body too short")),
+            "{err}"
+        );
         let _ = fs::remove_dir_all(&dir);
     }
 
@@ -1461,6 +1477,16 @@ pub(crate) mod tests {
             .unwrap();
         let persisted = mp.body_persisted_len();
         mp.persist_slots_without_body().unwrap();
+        let on_disk = fs::read(dir.join("slots")).unwrap();
+        assert_eq!(
+            on_disk[SLOTS_HEADER], SLOT_FREE,
+            "a live row past the durable body is free on disk"
+        );
+        assert_eq!(
+            &on_disk[SLOTS_HEADER + 16..SLOTS_HEADER + 48],
+            t1.as_byte_array(),
+            "demotion keeps the txid and only clears LIVE"
+        );
         drop(mp);
         let mp = Mempool::open_or_create(&dir).unwrap();
         let live = mp
@@ -1738,9 +1764,19 @@ pub(crate) mod tests {
             "expected grow on full table, got {:?}",
             r.err().map(|e| e.to_string())
         );
-        assert!(mp.meta().slot_cap > tiny);
+        assert_eq!(mp.meta().slot_cap, 16_388);
         assert_eq!(mp.live_count(), tiny + 1);
-        // Must not be the old Corrupt message.
+        drop(mp);
+        assert_eq!(
+            fs::metadata(dir.join("slots")).unwrap().len(),
+            786_640,
+            "grown slots file is the 16-byte header plus 16388 records"
+        );
+        let mp = Mempool::open_or_create(&dir).unwrap();
+        assert_eq!(mp.meta().slot_cap, 16_388);
+        // Grow persists the four rows already in the table. The fifth slot is
+        // RAM until the next sidecar write, so a reopen keeps those four.
+        assert_eq!(mp.load_live_txs().unwrap().len(), 4);
         assert!(!format!("{:?}", MempoolError::Full).contains("corrupt"));
         let _ = fs::remove_dir_all(&dir);
     }