The Node Operator's Guide to the Lightning Galaxy, Part 1

in Posts

by Bryan Vu

As the Lightning Network has grown and we’ve worked with many community members who have been running Lightning routing nodes, we’ve been wanting to provide more information about our ideas and experiences running routing nodes. We hope that this Routing Node Guide blog post series will help those who are interested in running routing nodes do so more easily and effectively, with less trial and error. Hopefully, having this information available in a more consumable form will also help the community evolve the state of the art in Lightning Network routing. The first part of this series will give a high level overview of routing requirements as well as a walkthrough of some of the basic lnd options and parameters and how they apply to routing node operators. Subsequent posts will discuss some of the underlying principles of Lightning routing, creating and managing channels, securing nodes, and monitoring nodes. We’ll also cover some common troubleshooting issues and future directions in routing.

Who is this guide for?


This guide is intended for advanced Lightning users who are interested in contributing to the early routing network and potentially earning small amounts of bitcoin from routing. Note that the Lightning Network is still in its nascent stages, and operating a routing node involves a good deal of technical skill, experimentation and trial and error. In this guide, we’ll describe some of the best practices that we’ve developed with the caveat that these practices are a work in progress and we’re actively collaborating with the community to evolve this set of practices as the network grows and scales.

What is a routing node?


A Lightning Network routing node is a node that is configured and capitalized (with bitcoin) so as to facilitate the routing of Lightning Network payments. Some routing nodes have only small amounts of bitcoin allocated and route a small number of payments, while other routing nodes have hundreds of channels and many bitcoins routing many payments. Regardless of size, however, routing nodes should maintain high levels of uptime and availability. This requires appropriate hardware as well as time, skill, and capital.


We recommend the following minimum hardware for operating a routing node:

Reliable storage - Whether hard disk or SSD, a Lightning routing node updates its database with every payment, and the integrity of this database is paramount. In cases of data corruption, channels must be closed so as to prevent loss of funds (see static channel backups and automation). Because of this, highly reliable storage options are recommended. We do not recommend using SD or flash storage for lnd’s channel database.

In addition, we recommend that routing node operators run full bitcoin nodes as well, whether bitcoind or btcd. This requires approximately 300 GB of storage as of this writing (July 2019), but we recommend 512 GB to leave room for growth.

Reliable network access - Particularly high-volume routing nodes may need high-bandwidth connections, but for most nodes, any stable, reliable broadband network connection should suffice. Wired connections are generally preferable to wireless.

Reliable power - Since routing nodes are intended to be online at all times, battery-powered or mobile devices (e.g. phones or tablets) aren’t good choices for routing. However, uninterruptible power supplies can be useful in case of power outages.


In addition to the hardware requirements listed above, routing node operators will need time and sufficient technical skills to manage a reliable online service. Eventually, we hope to provide more tools to automate most basic routing node operations, and our recently released lndmon will hopefully make monitoring routing nodes (and other lnd nodes) much easier. The broader Lightning community has begun developing tools as well, including Ride the Lightning, lndash and lndmanage. As most of the tools are in early phases of development, prospective routing node operators will still need to handle node management tasks directly and should be familiar with creating channels, sending and receiving Lightning payments, working with command-line interfaces, shell scripting, and data analysis. Additionally, because routing nodes control bitcoin, proficiency with general bitcoin security and private key management is required. This guide will hopefully provide some of the required information in these areas, but will not be exhaustive.


In addition to hardware, time and skills, the final requirement to run an effective Lightning Network routing node is capital (bitcoin). A routing node should maintain a balance between “outbound” capital added to the network in channels created by the routing node operator and “inbound” channels, which are initiated by end-users or other routing node operators. More details about channel creation and channel acceptance will be discussed in future sections, but nodes with more capital in more channels and/or larger channels will generally have more routing potential.

The final point to make in this section is that the requirements above are necessary, but not sufficient to operate an effective routing node. The final piece of the puzzle is good channel placement and good channel management. The majority of this guide will cover these topics. Also note that at this point, even effective routing nodes that are routing many payments per day may or may not be particularly profitable. Over time and as the network grows, this may change somewhat, but today’s routing node operators should not be expecting to make money fast.

Basic node setup


lnd and bitcoin node installation


The first step to running a routing node is to install lnd, using either the btcd or bitcoind backends. For node operators who want the fastest initial sync time, bitcoind is recommended. For those who would like to contribute to light clients running the Neutrino protocol, btcd is the best option.

Useful startup options


The following options can be specified either at the command line when starting lnd, or via lnd.conf. Below are listed some situations when you may need to use non-default values.

–externalip - Routing nodes should run with the --externalip command-line flag set. For nodes that are behind firewalls, more information on this can also be found in the “Accessing lnd from the Internet” section of this guide. The key point is that port 9735 (the Lightning Network default P2P port) must be externally accessible.

Currently, Tor support is also an option, though a routing node should support regular IP addresses or Tor, but not both. Tor nodes provide greater privacy, but at the cost of added payment latency, since each payment channel hop requires three Tor hops. Those interested in operating a Tor routing node can follow these configuration instructions.

–minchansize - the --minchansize flag specifies the minimum size that will be accepted when other nodes attempt to open channels to your node. As nodes get more popular and well-connected, operators may increase this value so as to only allow connections from other nodes that have made larger capital commitments. At the beginning, we recommend experimenting with a value around the BTC equivalent of about $20. This should be enough to route useful payments, while not being so restrictive as to filter out end-user wallets or small routing nodes.

–lnddir - specifies the lnd base directory, which will determine where lnd will store databases, logs, macaroons and other data associated with your node.

–tlsextradomain, –tlsextraip - the default TLS certificate generated by lnd will allow RPC and REST connections from localhost. If you would like to allow connections from other machines, you can use these flags to specify additional domains or IP addresses that will be allowed to connect to your lnd node.

–stagger-initial-reconnect - this should generally be set to true, particularly for routing nodes that have many channels. When true, this will reduce the memory consumed at lnd startup, since channel peer connections will be established over time rather than immediately all once.

–profile - enables profiling, which can be useful for identifying performance bottlenecks in the case that a node isn’t performing normally. The argument should be the port at which to server profile metrics.

–debuglevel - can be used to add or subtract detail from logs, which can be useful for debugging. debuglevel can be specified at the individual subsystem level, providing additional control for logging. See ‘lncli debuglevel’ below for adjusting debug message granularity for a running lnd node.

–nat - if a node is running on a network with Network Address Translation, the --nat flag can be enabled so that lnd will attempt to advertise and route to the correct external IP address. If this fails, forwarding will need to be manually configured on the NAT-enabled router or routers.

–rpclisten, –restlisten - to listen for RPC or REST connections on non-default ports, these options can be specified.

–maxlogfiles, –maxlogfilesize - by default, lnd will maintain three log files of 10MB each. For larger logfiles (that may be easier to search), or for longer log history, these values can be adjusted.

–unsafe-disconnect - by default, lnd won’t disconnect from channel peers. However, there are some cases for which disconnecting and reconnecting can resolve connectivity issues. Enabling --unsafe-disconnect will allow this.

Basic node monitoring and management


Once a routing node is up-and-running, the node operator will need to ensure that the node is operating as expected. The first aspect of this is to ensure that channel-related operations are working, and the second is to ensure that hardware and networking are working properly as well. The following sets of lncli commands can be used to monitor and manage a running node. Below is a short discussion of some of the relevant commands for routing node operators. More detail can be found using lncli -h or lncli [command] -h (e.g. lncli debuglevel -h).

updatechanpolicy - In order to experiment with fees and/or timelocks, updatechanpolicy can be used. The time_lock_delta parameter determines how long a node has to get an HTLC into the blockchain in uncooperative cases (more detail on time locks will be included in a subsequent post in this series). base_fee_msat is the fixed fee charged for every transaction regardless of size, and fee_rate determines what percentage rate is applied to calculate the part of the fee that’s based on transaction size.

Note that updatechanpolicy should not be heavily used, as propagating each change requires network bandwidth from the rest of the network and can also result in routing failures for clients who aren’t fully up-to-date. Those nodes that update too frequently will be more likely to have their scores downgraded (node scoring will be further discussed in a later post), or have their updates filtered out. No more than one update per day is likely to be reasonable, but the more stable a channel’s parameters are, the more likely it is to be chosen for routing.

describegraph - shows a list of all channels in the network. This graph can be analyzed and used in order to find potential nodes to connect to and also to identify potential gaps in the network that a routing node could fill.

listchannels - can be used to check the state and the balance of a node’s channels. If a particular channel is heavily unbalanced in one way or another, measures can be taken to rebalance the channel, either by routing payments to or from that channel or by using tools like Lightning Loop to add or subtract funds from the channel.

pendingchannels - lists the set of channels that are in a transition state, whether being opened or closed. This can be useful for tracking the state of channel transitions and identifying when funds will be available either for Lightning transactions or for on-chain transactions (in the case of channel closes).

getinfo - provides information about a node, is useful for determining whether a node is operating, is connected to the Lightning Network, synchronized with the blockchain, etc. The node ID or identity_pubkey is also useful when querying external tools for information about the node or when providing the information necessary for other nodes to create new channels to the node.

fwdinghistory - shows the list of transactions forwarded by the node, allows a routing node operator to view overall transaction forwarding volume as well as to determine which channels are being most heavily utilized.

feereport - shows the current fee schedule for each channel as well as the sum of fees earned over the previous day, week and month.

getchaninfo - returns information about any public channel in the Lightning Network or private channels for which the node is one of the peers. This information can be used to ascertain the liveness and capacity for a channel as well as fee rates, etc. This information may be useful node scoring.

getnodeinfo - contains information necessary for creating a channel to a given node (pub_key, addr) and also information that can be used to ascertain connectivity (num_channels, total_capacity) and liveness (last_update).

listpeers - shows the peers that the node has connected to, or that have connected to the node. Note that these are network peers, which are not necessarily channel peers.

connect/disconnect - connecting directly to selected peers may be useful in order to diversify sources for channel graph data, but is uncommon. Disconnect can be used in cases where a peer is causing problems due to connectivity bugs, message spamming, channel inactivity, etc.

closedchannels - can be useful when reviewing the history of a node for accounting or debugging purposes. The type of close (cooperative or force) can also be useful when determining when funds will become available on-chain.

debuglevel - similar to the startup option listed above. An example would be lncli debuglevel ATPL=trace if you were wanting to debug and get more detailed information about the functioning of the Autopilot.

Upcoming chapters


In the next post in this series, we’ll discuss some broader concepts underlying Lightning routing, including long-lived channels, channel selection, node scoring, channel acceptance, inbound and outbound liquidity and more. For those who are running routing nodes and have questions you’d like to have answered in this series, please feel free to send suggestions to @bvu on Twitter. For other questions, please join the conversation in our Slack community.