Lightning Labs Blog - The Official Blog of Lightning Labs

Welcome to 2020, Lightning community! We’re working to do our part to kick off the new decade right with the release of lnd v0.9-beta! This latest release includes two new payment types, Multi-Path Payments and Keysend, as well as custom TLV infrastructure that can be used to create even more payment and application types. We’ve also added more sophisticated access control for lnd nodes thanks to the new Macaroon Bakery (it’s not exactly what it sounds like, but still exciting!). For node operators, there are a series of improvements in v0.9-beta including additional data about routing peers, more tools for pathfinding, and more automated channel rebalancing. As always, there are a lot of smaller improvements as well, many of which are summarized below and more fully described in the release notes.

Multi-Path Payments

Those who have been following Lightning development have likely heard of “Atomic Multi-Path Payments,” which allow for payments to be split into multiple parts by a payment sender and then joined back together by the payment receiver. This brings many potential benefits, including simplified UX, since funds from many channels can be combined and used as if they were a single balance. The ability to split payments may also improve routing, particularly for large payments, since split payments may be easier to route around liquidity bottlenecks.

In lnd v0.9-beta, we’re making a significant step in this direction with the ability to receive “multi-path payments” (MPPs) (the non-spontaneous variant). Getting to this milestone required a number of underlying components, particularly the addition of custom Type-Length-Value (TLV) data payloads (described below), two TLV fields added to support MPPs, handling for invoices that support MPP, and updated RPC calls.

In addition to the UX and routing advantages provided by MPP, there are security advantages as well. MPP invoices include a new payment_secret value which is included in the onion-encrypted data structure sent along with each payment. Because the payment_secret is encrypted end-to-end, intermediary routing nodes aren’t able to probe for the final destination and potentially steal funds. With this small tweak, MPP also provides additional safety in cases of invoice overpayment which can be an issue for donation use cases. More specifics about these topics can be found in the release notes. Because of these safety and security improvements, the underlying MPP mechanism is now used as the default for what we call “single-shot” lnd v0.9-beta payments. “Single-shot” MPP payments are essentially an updated, safer form of “normal,” unsplit Lightning payments.

To recap, the logic for receiving MPPs is included in lnd v0.9-beta and the sending of “single-shot” MPP payments is now the default. However, the logic for splitting payments into multiple parts during the sending process isn’t yet enabled. MPP splitting is an area of active development that we’re working on for lnd v0.10.

“Spontaneous” payments

One of the most frequently requested features for Lightning has been the ability to send “spontaneous” payments, in which the receiver of a payment doesn’t need to be involved in the payment ahead of time and doesn’t have to generate an invoice before the payment is sent. These “spontaneous” or “push” payments are more like what Bitcoin users have grown accustomed to, as opposed to invoice-based systems like credit cards. In lnd v0.9-beta, we’re introducing an experimental feature called Keysend that allows for spontaneous, invoice-less payments. Like Multi-Path Payments, Keysend uses custom TLV onion payloads to transmit the payment preimage so that a payment receiver can settle payments without an invoice. A future release of lnd will also support AMP, which is essentially a sharded Keysend payment that supports additional features such as recurring payments.

Extensibility and Flexibility

As mentioned above, both the Multi-Path Payments feature and the Keysend feature rely on TLV onion payloads to transmit encrypted data between payment senders and receivers. In lnd v0.9-beta, the TLV feature was made more useful by the addition of support for custom_records, which enables developers of new Lightning features to specify their own custom data that can be sent along with users’ payments. MPP and Keysend are only the first features to use this functionality, and we hope the Lightning community will find this feature useful for many new applications. Whatsat, an experimental application of chat over Lightning, is another early example.

Another improvement that makes lnd v0.9-beta more flexible is a change to how channels are initially funded. Many Lightning users know that channels are funded via a 2-of-2 multisig bitcoin transaction, but in previous lnd versions, the way funding transactions were negotiated and created was largely managed behind the scenes by lnd’s internal wallet. In v0.9-beta, the steps in the funding process have been modularized and exposed so as to give developers more flexibility. In particular, future channels will be able to be funded by transactions created outside of lnd, multiple funding transactions could be combined together into a single batched opening transaction, and with future support for Partially Signed Bitcoin Transactions (PSBT), hardware wallets will be able to be used in the channel funding process. This change also provides some of the groundwork for even more advanced future functionality such as channel factories.

Security

lnd developers and node operators are generally familiar with the use of macaroons, which are required when issuing RPC commands to lnd nodes. However, in prior versions of lnd, the full potential for macaroons wasn’t fully realized. In lnd v0.9-beta, we’ve added a Macaroon “Bakery” which enables the creation of macaroons that can define permissions for customized combinations of RPC calls. This more fine-grained control can be used to support more complex permissions levels required by larger organizations. Custom macaroons can also be used to facilitate partnerships in which external parties are able to view limited sets of data or execute particular actions on nodes operated by other entities.

Another security-related change is the addition of support for option_upfront_shutdown which allows a user to specify “up front” where funds will be sent when a channel is cooperatively closed. Normally, this address can be changed in real-time when channels are closed, but for cases in which a node operator would like to provide greater protection and deterrence against node compromise and funds theft, option_upfront_shutdown provides an additional layer of security.

Liquidity management

The ability for a user to balance channels by sending funds from one channel to another has been talked about for a while in Lightning circles, but until recently, there was no built-in support for doing so. This has changed in lnd v0.9-beta with the addition of “circular payments” and supporting infrastructure. This should be particularly useful for node operators who have multiple channels and more sophisticated liquidity management requirements. With this new set of extensions to the RPC interface, explicit rebalancing can be executed in as little as two commands to lnd.

Routing

As discussed in the Routing Node Guide blog post series, we expect that over time, routing nodes will become more selective about the quality of nodes they connect to. In lnd v0.9-beta, we’ve taken an initial step toward supporting that vision with the addition of a “channel fitness” subsystem to track the uptime of channel peers. This information may be particularly useful for reinforcing or pruning routing connections.

In lnd v0.9-beta, the Mission Control subsystem also gets a series of upgrades, including a new factor that evaluates routing success probability for nodes, not just at the channel level as was previously calculated. This can optimize pathfinding in situations where a node has a large number of channels of varying quality. This factor will hopefully also serve to encourage routing nodes to manage channels more carefully so as to improve their node-level scores. lnd v0.9-beta also includes improvements in probability estimation and historical payment success and failure tracking. A new RPC call QueryProbabiity was also added to take advantage of the fact that Mission Control now takes payment size into account when calculating payment success probabilities.

lnd v0.9-beta also adds a couple smaller pathfinding-related improvements, and for node operators, control over min_htlc settings was simplified, as was disconnecting from peers.

Mobile

To make lnd builds simpler for mobile wallet developers, lnd v0.9-beta now includes a new mobile SDK called falafel. There are also new features designed to make it simpler to unlock mobile wallets as well as make mobile API security more flexible.

Developer Features

In lnd v0.9-beta, we’ve continued to add features to make lnd more developer-friendly. Based on feedback from the developer community, the mix of different units for different RPC calls was somewhat confusing, so in lnd v0.9-beta, all RPC calls now support millisatoshi or mSAT units. Other improvements include better handling for insufficient balance errors, uniform hex encoding (particularly helpful for REST/JSON API users), and more control over the QueryRoutes RPC call used for pathfinding.

Additional features for developers include more control over message signing, a new subscription RPC that can be used to more efficiently track peer activity, and the ability to specify a custom address to send closed channel funds to (if option_upfront_shutdown mentioned above wasn’t used).

Often, node operators and developers use Docker to get lnd nodes up-and-running more quickly, and lnd v0.9-beta includes a number of changes that improve the lnd Docker workflow. These include making it possible to restart lnd Docker containers without losing data, improving container rebuild speed, supporting custom builds from alternate Git repos, and more easily spinning up btcd and lnd together.

To make contributing to lnd more streamlined for developers, the continuous integration test suite was made more modular.

Compatibility

There were a number of compatibility changes made in lnd v0.9-beta that will primarily be useful to developers of other Lightning implementations. These include handling for invalid onion payloads, a single feature bit namespace, ordering of validation of short channel IDs, support for the payment_secret field in BOLT 11 payment requests, and fixes for gossip query responses.

Bug fixes

In lnd v0.9-beta, the bug fixes continued, starting with a fix made to address a recently discovered vulnerability in the Sphinx protocol that allowed the receiver of a payment to ascertain the number of hops in the payment path. Additional smaller changes include isolation of database migration code, handling of commitment signatures, a fixed failure message, canceling of expired invoices, and fixed notifications when addresses are reused.

Conclusion

The 2010s took Bitcoin from humble beginnings and a few early developers to becoming widely known, featured regularly in mainstream media, and with millions of enthusiastic users. Still, Bitcoin and Lightning are just getting started and we’re thankful to all of our community members who are braving these early waters with us. The 2020s promise even greater strides forward for lnd, Lightning, and Bitcoin, and the people we’ve met in person, on Slack and even (most of) those on Twitter have provided us with incredibly useful feedback, feature requests, bug reports, code, and words of encouragement, all of which are greatly appreciated and which contributed to the next step in Lightning’s development and today’s release of lnd v0.9-beta. Thanks to you all, and happy new decade!