Skip to content

Conversation

@ziggie1984
Copy link
Collaborator

@ziggie1984 ziggie1984 commented Dec 15, 2025

Fix bug where setSelfNode compared only the seconds component of
timestamps instead of the full timestamp. This caused the node to
attempt persisting an older timestamp than what existed in the
database during restart, resulting in "sql: no rows in result set"
errors.

Fixes a flake:

https://github.com/lightningnetwork/lnd/actions/runs/20183761355/job/57949717763

The issue occurred when:

  1. Node starts with timestamp T1 (e.g., 10:00:05)
  2. Runtime update creates newer timestamp T2 (e.g., 10:00:30)
  3. Node restarts, time.Now() returns T3 (e.g., 10:01:02)
  4. Comparison srcNode.LastUpdate.Second() >= nodeLastUpdate.Second()
    evaluated 30 >= 2, causing nodeLastUpdate to be set to 10:00:31
  5. This older timestamp failed the UpsertSourceNode WHERE clause

@gemini-code-assist
Copy link

Summary of Changes

Hello @ziggie1984, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug within the setSelfNode function that previously led to incorrect timestamp comparisons. The original logic only considered the seconds component, which could cause the system to attempt persisting an older timestamp than what was already in the database, resulting in 'sql: no rows in result set' errors during node restarts. The fix updates the comparison to evaluate the full timestamp, ensuring proper data handling and preventing these errors.

Highlights

  • Timestamp Comparison Fix: Corrected a bug in the setSelfNode function where only the seconds component of timestamps was compared, leading to incorrect chronological evaluation.
  • Full Timestamp Logic: The comparison logic has been updated to use !srcNode.LastUpdate.Before(nodeLastUpdate), ensuring that the full timestamp, including all components, is considered.
  • Error Resolution: This change resolves 'sql: no rows in result set' errors that previously occurred during node restarts when the system attempted to persist an older timestamp than what was already present in the database.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses a critical bug in timestamp comparison within the setSelfNode function. The original code incorrectly compared only the seconds component of timestamps, which could lead to using an older timestamp and causing errors on restart. The fix correctly uses time.Time.Before for a full timestamp comparison, which is the proper and idiomatic approach in Go. This change is accurate and effectively resolves the described issue. The code quality is good, and I have no further suggestions.

@ziggie1984 ziggie1984 self-assigned this Dec 15, 2025
@ziggie1984 ziggie1984 added back port candidate pr which should be back ported to last major release backport-v0.20.x-branch This label is used to trigger the creation of a backport PR to the branch `v0.20.x-branch`. labels Dec 15, 2025
@ziggie1984 ziggie1984 marked this pull request as ready for review December 15, 2025 13:23
@ziggie1984 ziggie1984 added this to the v0.20.1 milestone Dec 15, 2025
@ziggie1984 ziggie1984 moved this to In review in lnd v0.20 Dec 15, 2025
Copy link
Collaborator

@ellemouton ellemouton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch 🙏

@ziggie1984 ziggie1984 requested review from starius and removed request for yyforyongyu December 15, 2025 18:30
@ziggie1984
Copy link
Collaborator Author

test flake is already tracked here: #8460

Copy link
Collaborator

@starius starius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is an issue in the PR.

Also I propose to document when the bug was introduced, commit 75895db ("lnd: use saved node ann config from previous run", 2024-06-09).

Also I propose to cover this logic with regression tests as it proved to be fragile.

server.go Outdated
// just need to make sure that the new LastUpdate time is at
// least one second after the last update time.
if srcNode.LastUpdate.Second() >= nodeLastUpdate.Second() {
if !srcNode.LastUpdate.Before(nodeLastUpdate) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if srcNode.LastUpdate.Before(nodeLastUpdate) but at the same time srcNode.LastUpdate.Unix() == nodeLastUpdate.Unix() (i.e. they fall into the same second)?

IMHO in this case get a violation of the "at least one second after" requirement of the comment above and [BOLT‑07 (P2P Node and Channel Discovery)]:

MUST set timestamp to be greater than that of any previous node_announcement it has previously created

IMHO the proper fix is:

if srcNode.LastUpdate.Unix() >= nodeLastUpdate.Unix() {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gemini-code-assist can you explain what the problem is with the current fix ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with the current fix, !srcNode.LastUpdate.Before(nodeLastUpdate), as pointed out by @starius, lies in how time.Time.Before() handles sub-second precision compared to the stated requirement of being "at least one second after".

The code comment explicitly states: "just need to make sure that the new LastUpdate time is at least one second after the last update time."

Consider the following scenario:

  • srcNode.LastUpdate (the existing timestamp in the DB) = 2025-12-15 19:00:00.500000000 UTC
  • nodeLastUpdate (the new proposed timestamp) = 2025-12-15 19:00:00.800000000 UTC

With the current fix:

  1. srcNode.LastUpdate.Before(nodeLastUpdate) evaluates to true (0.5s is indeed before 0.8s).
  2. Therefore, the condition !srcNode.LastUpdate.Before(nodeLastUpdate) evaluates to false.
  3. The if block is skipped, and nodeLastUpdate remains 2025-12-15 19:00:00.800000000 UTC.

However, 2025-12-15 19:00:00.800000000 UTC is only 0.3 seconds after srcNode.LastUpdate, which violates the requirement of being "at least one second after".

@starius's suggestion, srcNode.LastUpdate.Unix() >= nodeLastUpdate.Unix(), addresses this by comparing the timestamps at second-level granularity. Let's re-evaluate with the same example:

  • srcNode.LastUpdate.Unix() (seconds part of 19:00:00.5s) = 1765834800
  • nodeLastUpdate.Unix() (seconds part of 19:00:00.8s) = 1765834800

With @starius's suggested fix:

  1. The condition srcNode.LastUpdate.Unix() >= nodeLastUpdate.Unix() evaluates to 1765834800 >= 1765834800, which is true.
  2. The if block would execute: nodeLastUpdate = srcNode.LastUpdate.Add(time.Second).
  3. nodeLastUpdate would then become 2025-12-15 19:00:01.500000000 UTC.

This new nodeLastUpdate (19:00:01.5s) is correctly "at least one second after" srcNode.LastUpdate (19:00:00.5s), aligning with the stated intent. This approach also correctly handles the original issue described in the PR where only the .Second() component was compared.

Therefore, @starius's suggestion provides a more robust solution that adheres to the "at least one second after" logic, considering the practical granularity of such timestamp comparisons.

Suggested change
if !srcNode.LastUpdate.Before(nodeLastUpdate) {
if srcNode.LastUpdate.Unix() >= nodeLastUpdate.Unix() {

@ziggie1984 ziggie1984 force-pushed the bugfix/setsource-node-sql branch from 0a67307 to 90f4999 Compare December 15, 2025 22:11
@ziggie1984
Copy link
Collaborator Author

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly fixes a bug in setSelfNode where an incorrect timestamp comparison could lead to startup failures. The change replaces the flawed comparison of only the seconds component with a full Unix timestamp comparison by introducing a new helper function, needsTimestampIncrement. This approach is clean and improves code clarity. The accompanying new test, TestNodeAnnouncementTimestampComparison, is thorough, covering a good range of edge cases including clock skew, which ensures the fix is robust. The release note is also clear and accurately describes the change. Overall, this is a high-quality contribution.

@ziggie1984 ziggie1984 requested a review from starius December 15, 2025 22:45
server.go Outdated
// how timestamps are stored in the database (both SQL and KV stores truncate
// to second precision). This ensures BOLT-07 compliance, which requires node
// announcements to have strictly increasing timestamps.
func needsTimestampIncrement(persistedTime, currentTime time.Time) bool {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this in a separate function so that I can test it as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to include the if into the function. So it returns the final nodeLastUpdate value.

Copy link
Collaborator

@starius starius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🏆
I left few comments with refactoring and tests proposals.

server.go Outdated
// how timestamps are stored in the database (both SQL and KV stores truncate
// to second precision). This ensures BOLT-07 compliance, which requires node
// announcements to have strictly increasing timestamps.
func needsTimestampIncrement(persistedTime, currentTime time.Time) bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to include the if into the function. So it returns the final nodeLastUpdate value.

Fix bug where setSelfNode compared only the seconds component of
timestamps instead of the full timestamp. This caused the node to
attempt persisting an older timestamp than what existed in the
database during restart, resulting in "sql: no rows in result set"
errors.
@ziggie1984 ziggie1984 force-pushed the bugfix/setsource-node-sql branch from 90f4999 to e5b0704 Compare December 16, 2025 07:21
@yyforyongyu yyforyongyu merged commit 91423ee into lightningnetwork:master Dec 16, 2025
38 of 41 checks passed
@github-project-automation github-project-automation bot moved this from In review to Done in lnd v0.20 Dec 16, 2025
@github-actions
Copy link

Successfully created backport PR for v0.20.x-branch:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

back port candidate pr which should be back ported to last major release backport-v0.20.x-branch This label is used to trigger the creation of a backport PR to the branch `v0.20.x-branch`.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants