Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to send the largest amount from a single channel. #2

Open
wants to merge 135 commits into
base: master
Choose a base branch
from

Conversation

yaslama
Copy link
Member

@yaslama yaslama commented Sep 7, 2020

Try to send the largest amount from a single channel.

yyforyongyu and others added 14 commits July 22, 2020 22:50
This commit moves all localized instances of mock implementations of
the Signer interface to the lntest/mock package. This allows us to
remove a lot of code and have it housed under a single interface in
many cases.
With this commit we add a simple macaroon jar that can encrypt its
content with a user-provided password when being serialized to JSON.
A profile file can contain multiple profile entries. Each
entry has a name, a set of default options to use and an optional list
of macaroons in a jar. The profile file can be
serialized/deserialized to and from JSON.
We add a new 'profile' sub command to lncli to manage pre-defined
configuration profiles for all of lncli's CLI flags.
Roasbeef and others added 15 commits September 7, 2020 12:43
build: bump version of master branch to v0.11.99-beta
Original PR was written with 4 spaces instead of 8, do a once off fix
here rather than fixing bit-by bit in the subsequent commits and
cluttering them for review.
The current implementation of subscribe is difficult to mock because
the queue that we send updates on in unexported, so you cannot create
a subscribe.Client object and then add your own updates. While it is
possible to run a subscribe server in tests, subscribe servers will
shutdown before dispatching their udpates to all clients, which can be
flakey (and is difficult to workaround). In this commit, we add a
subscription interface so that these testing struggles can be addressed
with a mock.
As we add more elements to the chanfitness subsystem, we will require
more complex testing. The current tests are built around the inability
to mock subscriptions, which is remedied by addition of our own mock.
This context allows us to run the full store in a test, rather than
having to manually spin up the main goroutine. Mocking our subscriptions
is required so that we can block our subscribe updates on consumption,
using the real package provides us with no guarantee that the client
receives the update before shutdown, which produces test flakes.

This change also makes a move towards separating out the testing of our
event store from testing the underlying event logs to prepare for
further refactoring.
To get our uptime, we first filter our event log to get online periods.
This change updates this code to be tolerant of consecutive online or
offline events in the log. This will be required for rate limiting,
because we will not record every event for anti-dos reasons, so we could
record an online event, ignore an offline event and then record another
offline event. We could just ignore this duplicate event, but we will
also need this tolerance for when we persist uptime and our peers
can have their last event before restart as an online event and record
another online event when we come back up.
We currently query the store for uptime and lifespan individually. As
we add more fields, we will need to add more queries with this design.
This change combines requests into a single channel infor request so
that we do not need to add unnecessary boilerplate going forward.
When dealing with online events, we actually need to track our events
by peer, not by channel. All we need to track channels is to have a
set of online events for a peer which at least contain those events.
This change refactors chanfitness to track by peer.
In preparation for storing our flap count on disk, we start tracking
flap count per-peer.
To prevent flapping peers from endlessly dos-ing us with online and
offline events, we rate limit the number of events we will store per
period using their flap rate to determine how often we will add their
events to our in memory list of online events.

Since we are tracking online events, we need to track the aggregate
change over the rate limited period, otherwise we will lose track of
a peer's current state. For example, if we store an online event, then
do not store the subsequent offline event, we will believe that the
peer is online when they actually aren't. To address this, we "stage"
a single event which keeps track of all the events that occurred while
we were rate limiting the peer. At the end of the rate limting period,
we will store the last state for that peer, thereby ensureing that
we maintain our record of their most recent state.
Since we will use peer flap rate to determine how we rate limit, we
store this value on disk per peer per channel. This allows us to
restart with memory of our peers past behaviour, so we don't give badly
behaving peers have a fresh start on restart. Last flap timestamp is
stored with our flap count so that we can degrade this all time flap
count over time for peers that have not recently flapped.
Make field names consistent with the command line flag.
Since we store all-time flap count for a peer, we add a cooldown factor
which will discount poor flap counts in the past. This is only applied
to peers that have not flapped for at least a cooldown period, so that
we do not downgrade our rate limiting for badly behaved peers.
yyforyongyu and others added 26 commits September 17, 2020 14:52
This commit removes the duplicate tor.streamisolation option from `sample-lnd.conf` example config file (which was accidentally added in commit #104a9094980f31560ca269d3b01f000dd775778d)
To make it possible to request a Let's Encrypt certificate by using a
different IP address where the port 80 might still be free, we add the
IP part to its configuration as well instead of just the port.
This makes it possible to use an IPv6 address for the ACME request if
all available IPv4 addresses already have their port 80 occupied.
…-pubkey

signrpc: SharedKeyRequest accept raw pubkey
Extend the fee estimator to take into account parent transactions with
their weights and fees.

Do not try to cpfp parent transactions that have a higher fee rate than
the sweep tx fee rate.
For unconfirmed commit tx anchors, supply the sweeper with cpfp info and
a confirmation target fee estimate.

The sweeper will try to pay for the parent commit tx as long as the
current fee estimate exceeds the pre-signed commit tx fee rate.
docs: remove duplicate "tor.streamisolation" option from sample-lnd.conf
This commit adds commitQueue which is a lightweight contention manager
for STM transactions. The queue attempts to queue up transactions that
conflict for sequential execution, while leaving all "unblocked"
transactons to run freely in parallel.
This commit integrates an externally passed commitQueue instance with
the STM to reduce retries for conflicting transactions.
lnd+config: allow Let's Encrypt listen IP to be set
etcd: STM transaction queue to effectively reduce retries for conflicting transactions
This in prep for a bigger move in the next commit.
Otherwise, if we remove the build tags, then there's no default backend,
and compilation will fail.
…ld tag

In this commit, we modify our build tag set up to allow the main test
files to be buildable w/o the current rpctest tag. We do this so that
those of us that use extensions which will compile live files like
vim-go can once again fix compile errors as we go in our editors.

In order to do this, we now make an external `testsCases` variable, and
have two variants: one that's empty (no build tag), and one that's fully
populated with all our tests (build tag active). As a result, the main
file will now always build regardless of if the build tag is active or
not, but we'll only actually execute tests if the `testCases` variable
has been populated.

As sample run w/ the tag off:
```
=== RUN   TestLightningNetworkDaemon
--- PASS: TestLightningNetworkDaemon (0.00s)
PASS
ok  	github.com/lightningnetwork/lnd/lntest/itest	0.051s
```
We fix all linter issues except for the 'lostcontext' and 'unparam' ones
as those are too numerous and would increase the diff even more.
Therefore we silence them in the itest directory for now.
Because the linter is still not build tag aware, we also have to silence
the unused and deadcode sub linters to not get false positives.
To make it possible to compile the itests together with the other tests,
we don't want to use anything from the optional subservers.
There is a setting to control how often the garbage collector is run.
Apparently this is a tradeoff between CPU and memory usage. If we can
limit the memory being used in that way, this allows us to use multiple
worker again, so overall this shouldn't be much slower than before.
lntest: allow the main test files to be buildable w/o the rpctest build tag
…ice-txn

watchtower: conditionally reconstruct justice txns for anchor channels
@yaslama yaslama force-pushed the splitamoutperchannel branch 2 times, most recently from 85261f6 to 8d06ea2 Compare September 23, 2020 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.