| Internet-Draft | Impact DLTs | August 2022 |
| Trossen, et al. | Expires 3 March 2023 | [Page] |
This document discusses the impact of distributed ledger technologies being realized over IP-based provider networks. The focus here lies on the impact that the DLT communication patterns have on efficiency of resource usage in the underlying networks. We provide initial insights into experimental results to quantify this impact in terms of inefficient and wasted communication, aligned along challenges that the DLT realization over IP networks faces.¶
This document intends to outline this impact but also opportunities for network innovations to improve on the identified impact as well as the overall service quality. While this document does not promote specific solutions that capture those opportunities, it invites the wider community working on DLT and network solutions alike to contribute to the insights in this document to aid future research and development into possible solution concepts and technologies.¶
The findings presented here have first been reported within the similarly titled whitepaper released by the Industry IoT Consortium (IIC) [IIC_whitepaper].¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 3 March 2023.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The current routing system was initially designed for a single purpose, namely reachability between end nodes. This capability is utilized in many higher layer technologies in the form of overlays. Distributed Ledger Technologies (DLT) are one such form of overlay with the aim to facilitate communication patterns that allow a distributed consensus among distributed, and generally unknown, participants in the DLT overlay.¶
The realization of a DLT overlay follows that of other well-known examples for distributed computing tasks, such as Torrents, distributed file storage, among others. That is, DLTs form their own overlay through contributing 'peers' that partake in the DLT. For this, reachability information (in the form of IP addresses) of other DLT peers is centrally maintained (in so-called 'bootstrap nodes') to establish peer-specific pools of peers, within which each peer in turn communicates for the specific purpose of the DLT. DLTs secure the transactions using transport-level methods. As an overlay, DLTs are little concerned with the underlying network(s) itself, simply utilizing the provided IP reachability service for their purpose.¶
Continuing on the insights first reported in [IIC_whitepaper], this document sheds light onto the realization of specific DLT overlay mechanisms from the perspective of the resulting impact on the utilized provider networks in the form of the actual communication taking place.¶
For this, we outline the communication patterns upon which certain forms of DLTs rely (Section 4.2) in order to implement the key DLT concepts (Section 3). Based on our insights of those communication patterns, we then identify a number of key challenges (Section 5) through initial experimental results (Section 6) within an example DLT platform (here, Ethereum [REF]).¶
Here, we explicitly recognize that those insights are highly dependent on the specific DLT mechanisms under investigation and are therefore not generally transferable to other DLT platforms and their realization. For instance, DLT platforms relying on proof-of-work for transaction verification tend to differ in their communication from those relying on proof-of-stake. However, this document does attempt to develop a wider methodology over time that may allow for quantifying the impact on underlying networks across those different types of DLTs.¶
While the quantification of DLT impact serves as an interesting benchmark into the possible costs for operating DLTs, the identified challenges give also rise to possible opportunities for network-level innovations to improve on the situation observed in our experiments, thereby reducing the identified impact on provider network. Section 7 outlines a possible realization of those opportunities through a constraint-based selection of communication relations, utilizing semantic information beyond IP reachability.¶
With this in mind, we position an improved DLT performance as a possible applicability for semantic routing, introduced in more detail in [I-D.farrel-irtf-introduction-to-semantic-routing], while also soliciting other possible realizations of an improved DLT performance through network-level innovations. Moreover, we draw connections with ongoing IETF/IRTF efforts (Section 8), where our insights may provide useful input.¶
Note: This document does neither discuss the particular rationale for selecting DLTs in order to realize the intended application purpose nor the specific DLT mechanisms eventually used. It therefore does not pass comment on the advisability or practicality of using DLTs and their solutions, nor does it define any specific technical solutions for reducing the observed provider impact.¶
The following terminology is used throughout the remainder of this draft:¶
There has been ample work, such as [DLT_intro] [DLT_intro2], among others, including in other SDOs such as the IEEE but also within the IRTF/IETF [DINRGref], on defining main DLT concepts; we refer the reader to those references for more details. We focus our brief introduction here on those concepts most important to understand from a communication perspective.¶
The core abstraction used in a DLT is that of a 'transaction', i.e., a cryptographically signed (set of) instruction(s) to modify a state machine, which in turn represents the distributed consensus the DLT is trying to maintain. These transactions are executed within the higher-level concept of a 'smart contract', which implements the specific DLT application, such as for cryptocurrency, storage management, decentralized governance, among others.¶
Valid transactions are maintained in a distributed 'ledger' in the form of hashed information referred to as 'blocks'. Consensus is represented through the longest available chain of blocks that can be obtained from another DLT peer.¶
The validation of transactions, and therefore the inclusion into the (distributed) ledger, is realized through the consensus layer, realizing computational operations, such as Proof-of-Work, Proof-of-Stake, and others. There has been much discussion on the implications of those computational aspects, e.g., on energy consumption, which is not the focus of this draft.¶
Figure 1 provides an overview of a typical layering within a DLT architecture. The focus of this draft is on the layers below the session, i.e. the communication that needs to be upheld in order to facilitate transactions and block exchange within the DLT system.¶
+------------++---------------------------------------------------------+ | Application|| User | DLT | DLT | DLT |Decentralized| | Layer || Interface | Wallet | Explorer | Analytics | Finance | +------------++---------------------------------------------------------+ |App Protocol|| Identity | Token | Storage | DLT |Decentralized| | Layer || Mgmt | Mgmt | Mgmt | Oracle | Governance | +------------++-----------------------------+---------------------------+ | Contract || Transaction | Smart | | Layer || Engine | Contract | +------------++-----------------------------+---------------------------+ | Consensus || PoW/PoS/DPoS/PBFT/Raft/etc. | | Layer || | +------------++-------------------+------------------+------------------+ | Session || Transaction | Block | Account | | Layer || | | | +------------++-------------------+------------------+------------------+ | Transport || TCP | QUIC | UDP | | Layer || (+TLS) | | | +------------++-------------------+------------------+------------------+ | Network || (DNS + ) IP | Service | Pub/sub | | Layer || | Routing | overlay | +------------++-------------------+------------------+------------------+ | Resource || CPU | Storage | Transport | | Layer || | | Network | +------------++-------------------+------------------+------------------+
With our focus on the communication impact of DLTs, we now tease apart the communication as it usually takes place in a DLT in order to realize the transactions within a distributed ledger and the maintenance of the latter. We first outline the interactions at a higher level before delving into the communication patterns that result from those.¶
As stated in the introduction, these insights are currently limited to those obtained from Ethereum, a proof-of-work based DLT platform. Future draft revisions will enrich this section with any differing insights from other DLT realizations and platforms.¶
We can distinguish three core interactions in a DLT:¶
We can see from those interactions above that communication in a DLT is multipoint in nature, i.e., transactions or information (such as blocks) are sent to a set of DLT peers, not just a single one.¶
Important here is that the set of DLT peers is a randomized sample from a larger pool of available DLT peers; this is to achieve diffusion among many DLT peers, avoiding repeated communication with a fixed set of DLT peers and thereby reducing the threat of collusion of information through a malicious set of DLT peers.¶
The consequence of that varying random nature of the multipoint diffusion, however, is that repeated unicast replication is utilized instead of efficient network-level multicast; this constitutes a first recognizable impact on provider networks.¶
In the following subsection, we now focus on the communication patterns that are utilized to achieve the aforementioned interaction. Special attention is here given to the establishment of the pool of DLT peers that is used in the multipoint operations that are executed for each interaction, be it a transaction or the commitment of a newfound (ledger) block.¶
As mentioned before, it is key for any DLT peer, be it a client or a miner, to establish and maintain a 'pool of peers' from which it can select a set of DLT peers for each intended interaction. Figure 2 outlines those steps, detailed in the following. Our insights on realization were obtained from an Ethereum based experiment, using the go-ethereum release V1.10.2-stable on a Linux-based machine, operating out of Munich, Germany.¶
The final size of the pool is a matter of local configuration (in our case about 28k DLT peers, significantly less than the size of the overall DLT network, which was about 500k at the time of the experiment).¶
Also, a DLT client may commence with transactions (to the DLT overlay) already while the pool creation is still ongoing, thereby progressing to the last step in Figure 2 once a suitable set of DLT peers can be obtained from the overall (and possibly still growing) local pool of peers.¶
+-------------------+ if DLT peer connection failed
| Obtain list |<--------------------------------------+
| of DLT peers |<--+ |
+-------------------+ | if pool size +--------------+---
| Node | | smaller than max | Maintain peer |
| discovery | | | connectivity |
+-------------------+ | +-----------------+
| Transport | |
| security | |
+-------------------+ |
| Capability +---+
| exchange |
+-------------------+
|
| add discovered peers to pool of DLT peers
\|/
+--------------------------------+
| Obtain set of DLT peers |
| from pool of DLT peers |
+--------------------------------+
| Transactions |
+--------------------------------+
Considering the observed communication patterns in the previous section, we can identify a number of challenges that need addressing:¶
To shed some more light onto the possible impact on provider networks, stemming from some of the challenges in Section 5, we conducted experiments, using the same setup described in Section 4.2. More details (and suitable graphical representations of our initial results can be found in [IIC_whitepaper]).¶
Here, the goal was to undergo the steps needed to build up the needed pool of DLT peers, after which we sought to synchronize to determine the longest blockchain available in the discovered pool. The resulting geographic spread of the discovered DLT peers included all continents albeit with an expected clustering of nodes North America, Europe, Asia, and Australia, with only few discovered in South America and Africa.¶
Our first target was to differentiate types of DLT peers that stem from the communication patterns in Figure 2. Specifically, we came to differentiate the following types of DLT peers:¶
In our experiments, we determined at about 18% of peers are of the last type, i.e. successfully contribute to DLT purposes, while about 2% are of the third category, about 12% are non routable peers and about 68% are signalling peers. In other words, almost 80% of all attempted discoveries fails either because of the lack of reachability or mismatching capabilities.¶
Looking at the bandwidth usage across the different peer types allows for shedding some light on the communication that needs to be carried through the participating provider networks.¶
Given the amount of data for each blockchain synchronization, it is not surprising that, despite forming a mere 18% of peers, the 'data peers' account for about 58% of traffic in the overall system. This is followed by the 'dropped data peers' with about 31.5% (since still much data is sent albeit unsuccessfully). Both non routable and signalling peers account for a total of slightly under 10% of data used.¶
Although the amount of data that is wasted here accounts for (significant) total of about 42%, the data-heavy operation of synchronization large amounts of (blockchain) data is mainly to blame for this; however, the synchronization has to happen for any DLT peer to start operating as a possible DLT miner, so is not avoidable.¶
The challenges outlined in Section 5 lead us to outline possible opportunities for network innovations that may address those challenges and reduce the observed impact on provider networks. We stress here that none of the suggested approaches constitute solutions for those opportunities but merely possible starting points beyond which further study is required:¶
Addressing model: With the DLT overlay being realized over an IP network, each DLT peer is being addressed via its IP(v4/v6) address. With the discovery step selecting a dedicated DLT peer (through its IP address), the discovery steps (see Figure 2) include dedicated steps to ensure the reachability of the specific DLT peer under discovery. Until reachability can be ensured, traffic (in the form of PING/PONG messages) and latency (through sending those messages, while needing to wait for a timeout in case the DLT peer is not routable) need to occur, despite the DLT peer not being eventually used for communication.¶
Constraint-based peer selection: Following on the aspect of relying purely on reachability information in the form of IP addresses, the discovery steps in Figure 2 further include a number of capability-dependent selection criteria to finally include a DLT peer in its pool of peers. Specifically, the security and capability exchange may lead to a disconnect from a successfully contacted DLT because of such exchange leading to mismatching capabilities. Furthermore, even after an initial capability exchange being successful, the actual transaction itself may be constrained by capabilities such as available resources (e.g., bandwidth or CPU), leading to unsuccessful communication, which in turn will need to be compensated with including another DLT peer into the diffusion request.¶
Diffusion multicast: The multipoint replication of the transaction request to a set of DLT peers, chosen from the larger DLT pool maintained at the initiating DLT peer, increases the overall system but, in particular, individual client bandwidth usage, which in turn impacts the provider network by needing to provide the necessary resources for the replicated sending.¶
Both, DLTs as well as routing innovations, are subject to investigation in a number of related IETF and IRTF efforts. For instance, the Decentralized Internet Infrastructure RG [DINRGref] has been studying various aspects of DLTs and blockchains. Our findings in this draft may provide additional input into the work of this RG, while we would solicit feedback from this group of experts into the specific insights we have derived so far.¶
There is no standard way of providing interoperability between DLT networks. This results in difficulty of transferring or exchanging virtual assets from one DLT network to another. An interoperability architecture is being proposed in the IETF [I-D.hardjono-blockchain-interop-arch] to permit two gateways, belonging to distinct DLT networks, to conduct a virtual asset transfer between them while ensuring the asset does not exist simultaneously on both networks. The Open Digital Asset Protocol (ODAP) [I-D.hargreaves-odap] is a gateway-to-gateway protocol to perform a unidirectional transfer of a virtual asset.¶
Blockchain technologies, and thereby DLTs, have also been proposed for use in network functions itself. For instance, the work in [I-D.mcbride-rtgwg-bgp-blockchain] proposes to position BGP as DLT-managed transactions and thus, to utilize the power of a permissionless (DLT-based) management infrastructure to improve on resilience and trust into the operations performed within BGP, such as origin announcements and BGP updates. Such proposition, however, opens the question on the exact nature of such infrastructure but also its impact in terms of incurred traffic, particularly when operating at scale.¶
Furthermore, routing innovations under the label of 'semantic routing' have been the topic of recent work, see [I-D.farrel-irtf-introduction-to-semantic-routing] for an overview. With the examples of service routing as possible approaches to realize the opportunities outlined in the previous subsection, a stronger linkage to this activity should be considered.¶
While the DLT standardization efforts in IEEE SA mainly focus on the upper layers of the DLT architecture, the decentralized identity related standards (e.g., P2958 [P2958] and P3210 [P3210]) that are currently under development might be relevant for addressing specific challenges in the DLT network layer.¶
The work initially presented in [IIC_whitepaper] focused on the specific impact that DLT operations may have on provider networks, thereby turning the attention not to the specific applications of DLT but what their realization may mean to the underlying network operators.¶
Although attempting from the onset to base our insights on actual experiments we conducted, we recognize that those insights are only the start to a possibly wider understanding beyond this initial work.¶
We therefore solicit not only feedback on the specific findings presented in the previous sections, but also to specific questions that our work has led to:¶
As for the next steps for this draft, the authors seek to deepen the current insights through further conducted experiments, providing more insights on the disconnects experienced by the system and the costs for maintaining the pool of DLT peers.¶
Furthermore, the authors will more directly link to relevant network innovations, particularly in the service routing and instantaneous multicast domain, with the goal of providing estimates of improving on the operational costs of DLTs through such new network innovations.¶
This draft is a living document, originating from an initial study in the impact of DLTs on provider networks [IIC_whitepaper].¶
As such, the authors solicit feedback from the wider DLT and network community to improve on the insights, transfer them onto more DLT systems, and shed light onto how possible network innovations could improve on the identified issues.¶
This document does not introduce or modify any security mechanisms. The nature of DLTs is to provide a high level of transactional security through immutability of the data in blocks. But 51% attacks are possible amongst miners particularly on smaller, private blockchains where legitimate miners could be prevented from completing blocks and new blocks could be created by illegitimate miners. Smart contracts could become vulnerable if a function calls the wrong contract either intentionally or through human error. Transactional data meant to be private might be exposed. DLT attacks most often involve accounts being hacked outside of the DLT domain.¶
Since the IP addresses of DLT peers are exposed in the DLT system, the DLT network layer might be subject to privacy leakage. This document does not introduce any mechanisms for protecting IP address privacy and the methods described in [I-D.ip-address-privacy-considerations] could be employed to enhance the privacy of DLT peers.¶
This draft does not request any IANA action.¶
This draft acknowledges the work done in the IIC Industrial Digital Ledger focus group, leading to the whitepaper in [IIC_whitepaper]. We would like to thank the co-authors of this whitepaper for their work, specifically David Guzman (Huawei Technologies), Abhijeet Kelkar (GEOOWN Consulting), Xinxin Fan (IoTex), Mike McBride (Futurewei Technologies), Lei Zhang (iExec), Ulrich Graf (Huawei Technologies) and Dirk Trossen (Huawei Technologies) but also Stephen Mellor (IIC staff) who oversaw the process of organizing the contributions.¶