Skip to content

staticaddr: support multiple receive and change addresses - #1218

Open
hieblmi wants to merge 34 commits into
masterfrom
multi-address-core
Open

hieblmi wants to merge 34 commits into
masterfrom
multi-address-core

Conversation

@hieblmi

@hieblmi hieblmi commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator

This is PR 1 of 3 in the Static Address multi-address stack.

Static Address previously treated one legacy/root address as the owner of every
deposit. This PR introduces fresh receive and operation-specific change
addresses while preserving the address parameters that own each deposit.

Loop-ins and withdrawals can consequently spend deposits received across
multiple derived addresses, with each input signed and proven using its actual
address parameters.

Key Changes

  • Derive fresh static receive and change addresses from dedicated key families.
  • Persist each deposit's owning static-address parameters and restore them
    across restarts.
  • Backfill pre-migration deposits and fractional loop-ins to the legacy/root
    address.
  • Create cooperative signing sessions from each deposit's actual keys.
  • Send per-deposit address proofs to the server for loop-ins and withdrawals.
  • Generate and persist operation-specific change addresses for fractional
    loop-ins and partial withdrawals.
  • Rebuild and maintain an active address index during startup.
  • Import only missing wallet scripts and narrowly recognize duplicate-import
    errors for the expected Taproot output key.
  • Resolve requested funding addresses through the active script index instead
    of reconstructing and scanning every persisted address.
  • Track expiry sweeps by the spent deposit outpoint so unrelated transactions
    paying the same destination script cannot finalize a deposit.
  • Add loop static deposit support for creating and optionally funding a fresh
    address through lnd SendCoins.
  • Include each deposit's receiving address in RPC and CLI listings.

RPC and Compatibility Changes

NewStaticAddress now derives a fresh receive address on every request and
never funds it. Callers must not assume repeated requests are idempotent or
return the same address.

Wallet funding is handled by the separate FundStaticAddress RPC, exposed as
POST /v1/staticaddr/fund. It creates and funds a fresh address when
send_coins_request.addr is empty, or funds a known existing static address
when it is set. The loop static deposit command uses this endpoint.

Funding requires wallet:fund in addition to swap:execute and loop:in,
or an explicit URI grant for FundStaticAddress. Existing execute/in and
NewStaticAddress URI macaroons cannot authorize funding. The default local
Loop macaroon is regenerated on startup when permissions change; distributed
or custom funding macaroons must be explicitly updated.

Because address creation mutates wallet and database state, the RPC permission
changes from swap:read to swap:execute. Operators using custom scoped
macaroons must rebake them accordingly.

StaticAddressSummaryResponse.static_address remains populated with the
legacy/root address for wire compatibility, but is deprecated and must not be
treated as the current receive address. Call NewStaticAddress to derive a
fresh address.

Database and Recovery

The database now records:

  • The static-address owner of every deposit.
  • The generated change address associated with a fractional loop-in.

Migrations backfill existing records using the legacy/root address. Store
reconstruction restores deposit ownership and loop-in and withdrawal change
metadata after restart.

Stacked PRs

  1. This PR — multi-address persistence, derivation, signing, RPC, and CLI core.
  2. [staticaddr/withdraw: harden replacement monitoring #1215 — withdrawal replacement hardening](https://github.com/lightninglabs/loop/
    pull/1215)
    validates and follows the transactions that actually replace multi-address
    withdrawals.
  3. #1217 — loop-in HTLC recovery
    persists the confirmed HTLC output so recovery can rebuild the exact
    server-published transaction.

The address startup and lookup hardening previously isolated in
#1214 has been folded into
this PR, and #1214 is now closed.

Testing

  • go test ./...
  • go vet ./...
  • go test -race ./staticaddr/address ./loopd
  • make docs-check
  • Commit-message lint across the complete PR range
  • git diff --check

Release Notes

Release notes document the new address behavior, RPC permission change,
deprecated summary field, migration compatibility, and address lookup
hardening.

@hieblmi hieblmi mentioned this pull request Aug 28, 2026
1 task done
@hieblmi hieblmi self-assigned this Aug 28, 2026
@hieblmi
hieblmi requested a review from starius August 28, 2026 13:56
@lightninglabs lightninglabs deleted a comment from lightninglabs-gateway Bot Aug 28, 2026
@hieblmi
hieblmi force-pushed the multi-address-core branch 4 times, most recently from a1b5ec5 to c74ac6c Compare August 31, 2026 10:37
@hieblmi
hieblmi force-pushed the multi-address-core branch 6 times, most recently from ee1bbba to 5b4bd5b Compare September 3, 2026 06:16
Comment thread docs/release-notes/release-notes-next.md Outdated
@hieblmi
hieblmi requested a review from starius September 3, 2026 08:09
@litbot-9000

Copy link
Copy Markdown

@starius: review reminder

Comment thread swap/keychain_test.go Outdated
Comment thread swap/keychain.go Outdated
Comment thread swap/keychain.go
Comment thread loopd/swapclient_server.go Outdated
Comment thread staticaddr/address/sql_store.go
Comment thread staticaddr/address/interface.go Outdated
Comment thread staticaddr/address/manager.go Outdated
Comment thread looprpc/perms.go
Comment thread loopd/swapclient_server.go Outdated
Comment thread loopd/swapclient_server.go Outdated
@hieblmi
hieblmi force-pushed the multi-address-core branch 2 times, most recently from 2f594a8 to 205077e Compare September 17, 2026 08:14
@hieblmi
hieblmi requested a review from starius September 17, 2026 08:35
@litbot-9000

Copy link
Copy Markdown

@starius: review reminder

Reserve separate key families for static receive and change addresses.
This keeps derived keys out of the legacy static-address and HTLC key
streams.
Associate every deposit with the static address parameters that created
it. This lets restored deposits recover the correct script and signing
keys instead of assuming the legacy root address.
Create receive and change addresses from locally derived client keys
while reusing the server key and expiry from the legacy seed. Persist,
import, and activate each script before returning it to callers.

Rebuild the active address index on startup and serialize issuance
without blocking address reads. Import only scripts missing from lnd,
and accept duplicate-import errors only when they identify the expected
Taproot output key.
Look up each newly discovered wallet UTXO by script and persist the
matching active-address parameters on the deposit. Reject unknown
scripts before allocating the timeout sweep address.

Use the per-deposit parameters when constructing the FSM, sign
descriptor, and unilateral expiry sweep so derived-address recovery uses
its owning script and key.
Associate fractional loop-ins with their operation-specific static
change address so recovery restores the descriptor needed to reconstruct
signed transactions. Backfill legacy fractional swaps to the original
address.
Create a fresh static address for fractional loop-in change and send its
descriptor to the server. Reconstruct signed HTLCs with the persisted
parameters and verify cooperative batch change by output script.
Multi-address loop-ins sign and construct transactions from the
parameters attached to each deposit and their dedicated change address.
The legacy root address fields therefore became write-only, but
populating them could still abort signing, sweep handling, or recovery
when the root lookup failed.

Remove those fields and lookups, select the FSM from the protocol
version persisted with the swap, and set that version before
constructing new state machines. Keep the root-parameter lookup used by
autoloop expiry calculation and add regression coverage for recovery and
unsupported persisted versions.
Create a fresh static address for partial-withdrawal change and identify
it in the confirmed transaction through its active change-family script,
without assuming output order or count. Record withdrawn and change
amounts by script identity.

Keep all withdrawal outputs in the PSBT without separate signing
metadata while preserving full-withdrawal behavior.
Let loop static deposit create and fund a fresh receive address through
lnd SendCoins. Validate funding arguments before address creation and
require explicit confirmation unless --force is set, including for
non-interactive and first-use deposits.

Allow NewStaticAddress RPC callers to fund a requested existing static
address by resolving it through the active script index. Expose the
nested request through the client RPC, require swap:execute permission,
and cover the CLI new-address and daemon existing-address funding paths.
Regenerate RPC and CLI documentation.
Include the owning static address in every deposit RPC response and CLI
listing. Users can distinguish deposits created by different receive and
change addresses without reconstructing scripts externally.

Calculate blocks until expiry from each deposit owner instead of the
legacy root address, and reject deposits whose owning parameters are
missing. Centralize deposit response conversion and update generated RPC
artifacts, regression coverage, and command replay fixtures.
The CLI previously recognized an uninitialized static-address seed by
searching arbitrary gRPC error text. Any wrapping or wording change
could suppress the L402 backup warning before a user funded a newly
derived address.

Map ErrNoStaticAddress to codes.NotFound at the RPC boundary and
classify that status in the CLI. Retain compatibility with older daemons
only for an exact Unknown-status message, avoiding the broad substring
match, and cover both sides with regression tests.
A static-address account can now receive deposits across multiple
derived addresses, so the singular summary field can no longer describe
the current receive address. Removing or repurposing field 1 would break
existing clients.

Keep the wire value as the legacy/root derivation address, formally
deprecate it, document the expiry as the shared CSV delay, and direct
CLI users to derive a fresh receive address. Rename the server locals to
make the compatibility behavior explicit and regenerate protobuf and
Swagger artifacts.
Cover per-deposit address ownership and operation-specific change
outputs across the shared SQL persistence boundary. Reconstruct the
deposit, loop-in, and withdrawal stores to verify ownership and change
metadata survive restart.
The sweep request handler only compared the number of prevouts the
server sent against the number of sweep inputs. A list of the right
length could still contain duplicate outpoints or reference outpoints
the sweep doesn't spend. The prevout fetcher then returns nil for an
input and NewTxSigHashes panics on the nil dereference, taking down
loopd on a malformed server request.

Reject duplicate prevouts while building the prevout map and require a
prevout for every sweep input before computing sighashes.
Rename the legacy key family and address-manager parameter alias, use
root terminology consistently, and consolidate wallet UTXO listing.
Keep key-family values and the underlying script parameters unchanged.

Document script map keys, issuance helpers, and test invariants. Include
the output script in missing-parameter errors, preallocate address store
results, and remove the constant-value test. Update CLI and RPC docs.
Keep NewStaticAddress address-only and move wallet funding into the new
FundStaticAddress RPC. Require wallet:fund alongside swap:execute and
loop:in so existing swap or address-URI macaroons cannot authorize
wallet funding. Explicit grants for the new RPC remain supported.

Preserve new-address and existing-address funding behavior, route the
static deposit CLI through the new RPC, and regenerate all bindings.
Reserve the removed experimental fields and document macaroon upgrades.

Exercise the production interceptor with signed and expired macaroons,
and cover address-only calls, funding validation, request preservation,
existing-address funding, and recovery after a wallet broadcast failure.
Require exactly one legacy static address when deposits exist before
migration 22 assigns their address IDs. Use a temporary CHECK constraint
shared by SQLite and PostgreSQL; databases without deposits remain
valid.

Cover valid backfills, missing and ambiguous ownership, unchanged schema
and data on rejection, and retry after repair and migration-version
reset. The existing framework still marks a rejected migration as dirty.
Encode listing addresses directly from their canonical P2TR scripts,
validating the script shape and using the configured network. This
avoids rebuilding Taproot trees and aggregating keys for every record
without introducing a cache.

Check encoding against full reconstruction on three Bitcoin networks
and reject malformed or non-Taproot scripts.
Quote selected deposits with one exact active-set lookup, retaining
presence, state, duplicate and expiry checks without rendering history.

Refresh deposits before taking the unspent response snapshot and filter
against active Deposited records. Historical database rows cannot revive
outputs removed during reconciliation or return stale confirmations.
Keep the lowest-ID active address alongside the script index so receive
and change issuance can find the root without scanning all addresses.
Publish the cached root only after successful activation and update it
under the same mutex as the index.

Cover restart recovery and repeated import failures so caching cannot
make an unavailable root appear ready.
Read wallet UTXOs outside the map lock, then match their scripts
directly against the active address index under a short lock. This
removes the full-index allocation and copy on each poll while
preserving confirmation bounds and allowing address activation
during the wallet RPC.

Test filtering across addresses and verify wallet RPCs do not hold
the map lock.
Load persisted addresses in bounded pages using an ascending ID cursor
during startup and root recovery. Read wallet watches once and publish
the complete runtime index only after every page and import succeeds.
The existing bulk-read API remains available for compatibility.

Test sparse IDs, page boundaries, failed-page recovery and restart
behavior, including the SQL query on SQLite and PostgreSQL.
Replace the issuance mutex with a context-aware gate so queued callers
can cancel without disturbing the current issuer. Keep root creation,
recovery and derived issuance serialized and retryable. A private mutex
now explicitly guards the active script index and root.

Test canceled root and derived waiters, subsequent progress and
concurrent first callers sharing one root.
Remove the blank line before the closing brace flagged by CI. This is
formatting only; test behavior is unchanged.
@hieblmi
hieblmi force-pushed the multi-address-core branch 2 times, most recently from 26983e9 to ad14247 Compare September 28, 2026 14:28
Compare lnd's next legacy/root, receive and change indices with the
highest persisted Loop keys before activating addresses. The legacy
family also holds every static loop-in HTLC key, so its target covers
the highest persisted HTLC key index as well. Advance only lagging
counters, verify the restored wallet against the family's persisted
address key first, and leave equal or ahead counters untouched without
creating any Loop addresses.

Allow startup reconciliation to outlive the fixed manager deadline while
preserving RPC timeouts, cancellation and error propagation. Add
recovery, restart and failure coverage plus documentation.
The deposit expiry sweep passed only the client pubkey to lnd. After a
wallet restore lnd has not yet re-derived the static-address key, so
its pubkey lookup misses and falls back to locator (0, 0), producing an
invalid timeout-path signature. Pass the deposit address' key locator
so lnd derives the signing key directly for root, receive, and change
deposits.
Document fresh receive-address derivation, lazy seed initialization,
funding-address lookup hardening, and the swap:execute permission
required by address creation. Regenerate the CLI, gRPC, Swagger, and
man-page documentation and add feature, breaking-change, and recovery
release notes.
//
// The server receives this proof material with swap and withdrawal requests and
// verifies it against the L402's server key and expiry before co-signing any
// input.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are root and multi addresses treated differently here? Can the Loop server distinguish which is which?

Could you add these details to the godoc, please?

var filteredUtxos []*lnwallet.Utxo
for _, utxo := range utxos {
if bytes.Equal(utxo.PkScript, staticAddress.PkScript) {
if _, ok := m.activeStaticAddresses[string(utxo.PkScript)]; ok {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested many scenarios of multi-address and identified an interesting property. This is not pre-existing.

Restoring an older Loop DB (with lnd's wallet unchanged) hides deposits at receive/change addresses issued after the backup: this filter only knows the restored DB's address rows, and counter reconciliation doesn’t rebuild missing ones. I confirmed empirically that some funds are still unspent past CSV maturity, with no automatic sweep. The single-address version could rediscover later deposits through its retained root.

Could we add recovery of the missing 42061/42062 addresses, or explicitly agree on the weaker backup guarantees before release?

if hasChange {
changeAmount := f.loopIn.ExpectedChangeAmount()
f.loopIn.ChangeAddressParams, err =
f.cfg.AddressManager.NewChangeAddress(ctx)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the Loop server does not support multi-address yet, we have a problem if this code actually runs.

Every partial loop-in now requests derived change, but the loop server rejects any ChangeOutput, so even root-only partial loop-ins stop working. I confirmed empirically that root-only partial withdrawals succeed but leave derived change the server refuses to co-sign (CSV recovery still works).

Could we gate these operations on server support, or explicitly require multi-address enabled before rolling out this client? If someone runs this as-is from master branch, we can get this problem.

Comment on lines +130 to +134
changeAmount += txOut.Value
continue
}

withdrawnAmount += txOut.Value

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's rewrite it in more readable way using else instead of continue

Comment thread cmd/loop/staticaddr.go

var depositStaticAddressCommand = &cli.Command{
Name: "deposit",
Usage: "Create and fund a new static loop in address.",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this command? Could loop static new just print a command for lncli to fund this new address? IMHO embedding this flow here is an extra complexity and extra braveness of the loop command itself.

Comment on lines +700 to +705
params := addressManager.GetParameters(txOut.PkScript)
if params == nil || int32(params.KeyLocator.Family) !=
swap.StaticAddressChangeKeyFamily {

continue
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A pending partial withdrawal created before upgrading has change at the legacy root (family 42060). The new change detector only recognizes family 42062, so it returns no change script. The new store loop (SqlStore.UpdateWithdrawal) then counts both outputs as withdrawn, with zero change. The old two-output logic in SqlStore.UpdateWithdrawal classified the second output as change correctly.

For example, a confirmed transaction paying 100,000 sat to the destination and 49,000 sat back to the root gets recorded as 149,000 withdrawn, zero change. This affects history/accounting, not ownership of the funds.

I'd either retain the existing one/two-output handling, or fix legacy change recognition before keeping the new loop.

" \"id\": \"68262a104c9ec325de6bec37b8e31bd875bbd2f5f0b9ce2da20cf0bd636fc448\",\n",
" \"outpoint\": \"edcdab8f0b1138d853a453b8b7a5ac3c694bd53ad38b7ccf062e45f99440e6e6:0\",\n",
" \"state\": \"WITHDRAWING\",\n",
" \"static_address\": \"\",\n",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it empty in all the fixtures? Maybe we need to re-generate them? I guess AI can do it easily with minimum blast radius.

Comment on lines +203 to +208
if err := validateExpirySpend(
confirmedTx, f.deposit.OutPoint,
f.deposit.TimeOutSweepPkScript,
); err != nil {
return f.HandleError(err)
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmt

Comment on lines +176 to +181
confChan, confErrChan, err :=
f.cfg.ChainNotifier.RegisterConfirmationsNtfn(
ctx, &spendingTxID,
f.deposit.TimeOutSweepPkScript,
DefaultConfTarget, heightHint,
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we handle spend, reorg and confirmation events in one loop (with WithReOrgChan on the spend subscription)? If sweep A confirms once, gets reorged out, and replacement B confirms, we currently keep waiting for A’s txid. On a spend reorg, we should cancel the old confirmation watch and follow the deposit’s next spender, then wait for its three confirmations. Otherwise B can confirm while the deposit stays stuck until loopd restarts.

Comment on lines +582 to +583
func (m *Manager) GetTaprootAddressFromScript(pkScript []byte) (
*btcutil.AddressTaproot, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to make this a free function, not a method. Currently it looks like the manager has something to do with the address (e.g. makes sure the address belongs to it).

ChainParams can be passed explicitly as an argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants