Internet-Draft Problems and Requirements of Addressing April 2022
Li, et al. Expires 29 October 2022 [Page]
Internet Engineering Task Force
Intended Status:
Y. Li
Tsinghua University
H. Li
Tsinghua University
J. Liu
Tsinghua University
L. Liu
Tsinghua University
Q. Wu
Tsinghua University

Problems and Requirements of Addressing in Integrated Space-Terrestrial Network


This document presents a detailed analysis of the problems and requirements of network addressing in "Internet in space" for terrestrial users. It introduces the basics of satellite mega-constellations, terrestrial terminals/ground stations, and their inter-networking. Then it explicitly analyzes how space-terrestrial mobility yeilds challenges for the logical topology, addressing, and their impact on routing. The requirements of addressing in the space-terrestrial network are discussed in detail, including uniqueness, stability, locality, scalability, efficiency and backward compatibility with terrestrial Internet. The problems and requirements of network addressing in space-terrestrial networks are finally outlined.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 29 October 2022.

Table of Contents

1. Introduction

The future Internet is up in the sky. We have seen a rocket-fast deployment of mega-constellations with 100s-10,000s of low-earth-orbit (LEO) satellites, such as Starlink [STARLINK], Kuiper [KUIPER] and OneWeb [ONEWEB]. These constellations promise competitive low latency and high capacity to terrestrial networks. They expand global high-speed Internet to remote areas that were not reachable by terrestrial networks, resulting in a tens-of-billions-of-dollar market with 3.7 billion users in rural areas[ITU-Measure], developing countries, aircraft, or oceans.

A salient feature for LEO mega-constellations is their high relative motions to the rotating earth. Unlike geosynchronous satellite or terrestrial networks, each LEO satellite moves fast (e.g., 28,080 km/h for Starlink), causing short-lived coverage for terrestrial users (less than 3 minutes). This yields diverse challenges for the traditional network designs.

This memo outlines the problems and requirements of addressing in integrated space-terrestrial network. It starts with the basics of satellite mega-constellations, terrestrial ground stations/terminals, and their inter-networking. It analyzes how high space-terrestrial relative motions yields challenges for logical topology, addressing and their impacts on routing. Then it discusses the requirements of network addressing in space-terrestrial network for uniqueness, stability, locality, scalability, efficiency and backward compatibility with terrestrial Internet.

2. Terminology

GSO: Geosynchronous orbit (at the altitude of 35,786 km).

NGSO: Non-geosynchronous orbit.

LEO: Low Earth Orbit (at the altitude of 180-2,000 km).

MEO: Medium Earth Orbit (at the altitude of 2,000-35,786 km).

ISL: Inter Satellite Link.

NAT: Network Address Translation.

GS: Ground Station, a device on ground connecting the satellite.

FIB: Forwarding Information Base.

3. Basics of Space-Terrestrial Network

As shown in Figure 1, a space-terrestrial network for terrestrial users consists of the satellite or constellations, terrestrial terminals, and ground stations.

3.1. Satellites and Mega-Constellations

Satellites can be classified based on their relative motions to the earth. A satellite can operate at the geosynchronous orbit (GSO, at about 35,786 km altitude) or non-geosynchronous orbits, such as low earth orbits (LEO, <=2,000km) and medium earth orbits (MEO, between 2,000 km and 35,786 km). Satellites at higher altitudes offer broader coverage, while satellites at lower altitudes move faster.

Historically, communications in space were dominated by GSO satellites. As shown in Table 1, GSO offers excellent coverage at high altitudes, but at the cost of long space-terrestrial RTT (>=200ms) and low bandwidth (<=10Mbps, due to bit errors in long distance transmission). Instead, recent efforts seek to adopt satellites at lower non-geosynchronous orbits, with a special interest in low-earth orbits. Together with Ku (12-18 GHz) and Ka (26.5-40GHz) bands, these satellites promise competitive bandwidth and latency to terrestrial networks( [LOWLATENCY-ROUTING-SPACE], [SPACE-RACE], [NETWORK-TOPO-DESIGN]). Due to low coverage for each LEO satellite, a mega-constellation is necessary to retain global coverage. Table 2 exemplifies popular LEO mega-costellations in operation. They are enabled by recent advances in satellite miniaturization and rocket reusability.

         +-----------+                 +-----------+
         | Satellite |_______ISL_______| Satellite | (Space Segment)
      _ _|___________|              _ _|___________|
         /|           \                /|
        / |            \              / |
       /   (Bent pipe)  \ |          /
      /               _ _\|         /
+---------+               +------------+
|  Mobile |               |   Ground   |
| Station |               |  Station   |            (Ground Segment)
|_________|               |____________|

Figure 1: A simplified space-terrestrial network architecture
Table 1: Differences between GSO and NGSO.
Orbit Altitude (km) RTT (ms)
GSO GEO 35,786 240
NGSO MEO 2,000-35,786 12-240
LEO 500-2,000 2-12
Table 2: Low-earth-orbit (LEO) satellite mega-constellations in operation.
Constellation Num. satellites Num. orbits Altitude (km)
Starlink 1584 72 540/550
720 36 570
348 6 560
172 4 560
Kuiper 1156 34 630
1296 36 610
784 28 590
Telesat 351 27 1015
1320 40 1325
Iridium 66 6 780

3.2. Evolution of Space-Terrestrial Network

Terrestrial users access satellite networks via terminals (e.g., satellite phones, onboard dishes, IoT endpoints) or ground stations. Ground stations can serve as network gateways (e.g., carrier-grade NAT in Starlink [STARLINK-CGNAT] and Kuiper [KUIPER-CGNAT]) and remote satellite controllers (e.g., telemetry, tracking, orbital update commands, or centralized routing control).

3.2.1. "One-to-one" satellite communication

Early satellite communications favor the simple "bent-pipe-only" model (Figure 1), i.e., satellites only relay terrestrial users' radio signals to the fixed ground stations without ISLs or routing. This model has been popular in GSO satellites with broad coverage (2G GMR [GEO-MOBILE-RADIO-INTERFACE] [ETSI-TS-101], 3G BGAN [BGAN] [ETSI-TS-102], and DVB-S [SATELLITE-COMMUNICATIONS]), and recently adopted by LEO satellites in OneWeb (4G) [ONEWEB] and 5G NTN [STUDY-NR-SUPPORT] [SOLUTION-NR-NTN]. However, this model suffers from low LEO satellite coverage. To access the network, both terrestrial users and ground stations must reside inside the satellite's coverage. Due to each LEO satellite's low coverage, most users in remote areas with sparse or no ground stations cannot be served. As shown in Table 3, under current Starlink (no ISLs so far) and ground station deployments ([STARLINK-GS-FOUND], [AZURE-GS], [AWS-GS], [GOOGLE-DATA-CENTER], [AZURE-CLOUD-STARLINK], [STARLINK-GS-MAP]), 27%-52% global populations cannot be served by the "one-to-one" model (depending on how many satellites each ground station can simultaneously associate to). Most under-served users are from remote areas (e.g., Africa), thus causing revenue loss for operators.

The ground station aggregates traffic from all satellite users and becomes the single-point bottleneck. In reality, Starlink's LEO satellites generate 5 TB telemetry data per day for the ground stations to process [REDDIT]. With limited space-terrestrial radio link capacity, its ground stations have limited the LEO network's total capacity [COMPARISON-THREE-CONSTELLATION]. Similarly, each OneWeb's ground station must process 10,000 terminal handovers per second [ONEWEB-GS]. Deploying dense ground stations in these remote areas could mitigate above two problems. However, it is expensive and lowers commercial competitive advantages to terrestrial networks.

3.2.2. "One-to-many" networked ground stations

To this end, networked space-terrestrial infrastructure is crucial for global coverage and single-point bottleneck elimination. To date, inter-satellite links (ISLs) are under early adoption([BEIDOU-TEST], [TheVerge-STARLINK-SPEED]). The recent "burn on re-entry" regulations from FCC also slows down the adoption of ISLs[SPACEX-CLAIM]. As a near-term remedy, routing with distributed ground station networks is adopted. There are two variants. The ground station-as-gateway is adopted by Starlink and Kuiper. Each ground station is a carrier-grade NAT that offers private IP[RFC0791] for terrestrial users. The ground station-as-relay [USE-GROUND-RELAY] mitigates ISLs with ground station-assisted routing, but is vulnerable to intermittent space-terrestrial links in Ku/Ka-bands. Fundamentally, the "one-to-many" model heavily relies on global deployments of ground station networks, thus offsetting LEO satellites' advantages and competitive edges to terrestrial networks.

3.2.3. "Many-to-many" networked satellites

To unleash LEO mega-constellations' potentials and long-term success, the networked LEO satellites are under rapid deployments[KUIPER][STARLINK]. Today's networked LEO satellites typically have a microwave space-terrestrial radio interface and at least 4 laser/microwave inter-satellite links (ISLs) [LOWLATENCY-ROUTING-SPACE][USE-GROUND-RELAY] (2 for intra-orbit satellites, 2 for inter-orbit satellites). With this capability, recent work has explored topology design [NETWORK-TOPO-DESIGN], low-latency routing [LOWLATENCY-ROUTING-SPACE][SPACE-RACE], inter-domain routing [Giuliari20Internet], orbital computing [ORBITAL-EDGE-COM][IN-ORBIT-COM], and security [ICARUS] in LEO networks. We take a forward-looking view to simplifying LEO networks in the first place and helps these efforts fulfill their merits in space.

Table 3: Global population that could access Starlink in its current "bent-pipe-only" model.
Global Africa Oceania South America Asia European North America
1-SAT association 48.71% 19.52% 42.85% 49.63% 43.49% 91.00% 87.50%
2-SAT association 57.30% 24.37% 56.58% 53.90% 55.91% 94.33% 91.23%
4-SAT association 67.04% 26.13% 60.31% 63.16% 71.34% 95.46% 95.04%
8-SAT association 73.04% 29.17% 60.68% 65.65% 80.28% 96.91% 98.86%

4. Problems in Space-Terrestrial Network Addressing

In terrestrial and GEO satellite networks, the logical network topology, addresses, and routes are mostly stationary due to fixed infrastructure. Instead, LEO mega-constellations hardly enjoy this luxury, whose satellites move at high speeds (about 28,080 km/h). The earth's rotation further complicates the relative motions between space and ground. In this section, we will analyze how high relative motion between space and ground challenges addressing due to topology instability, and its impact on routing[INTERNET-IN-SPACE].

4.1. Unstable Space-Terrestrial Topology

High physical mobility incurs frequent link churns between space and terrestrial nodes, thus causing frequent logical network topology changes. For all mega-constellations in Table 2, the topology changes every 10s of seconds[INTERNET-IN-SPACE]. The link churn populates with more satellites and ground stations.

In terrestrial mobile networks (e.g., 4G/5G), such physical link churn can be masked by handoffs without incurring logical topology changes. This method works based on two premises. First, all link churns occur at the last-hop radio due to user mobility, without affecting the infrastructure topology. Second, all cellular infrastructure nodes are fixed, resulting in a stable logical topology as "anchors".

However, neither premise holds in non-geosynchronous constellations. Instead, infrastructure mobility between satellites and ground stations becomes a norm rather than an exception. This voids cellular handoffs' merits to avoid propagation of physical link churns to logical network topology: They are designed for user mobility only, and heavily rely on the fixed infrastructure as "anchors." Therefore, 5G NTN lists satellite handoffs as an unsolved problem ([STUDY-NR-SUPPORT], [SOLUTION-NR-NTN]), and the latest 3GPP 5G release 17 defers its mobility support for satellites [TEC-SPECI-GROUP-MEETING] due to significant architectural changes. While Starlink uses handoffs to migrate physical links between satellites and ground stations (every 15s [STARLINK-CGNAT]), its logical topology and routing are still be repeatedly updated at high costs.

4.2. Inconsistent "Locations" for Space/Terrestrial Nodes

Each space/terrestrial node has two notions of "locations": The logical location in its topological address, and the physical location in reality. With repetitive topology changes, a static network address can hardly ensure its logical location in the topology is consistent with the fast-moving node's physical location in reality. Then to correctly forward data, a network should choose one of the following designs:

  1. Dynamic address updates

    A node can repetitively re-bind its physical location to its logical network address, thus incurring frequent address updates or re-binding. Under high mobility, this could severely disrupt user experiences or incur heavy signaling overhead. Table 4 and Table 5 project the address update frequency when using legacy IP addresses[RFC0791] for logical interfaces. In this scheme, the terrestrial users' logical IP[RFC0791] address changes if it re-associates to a new satellite (thus new interfaces and subnets) to retain its Internet access. Due to high LEO satellite mobility, each user is forced to change its logical IP address[RFC0791] every 133-510s. Every second, we observe 2,082-7,961 global users per second should change their IP addresses.

    Table 4: Frequency of each user's logical IP address update.
    Starlink Telesat Kuiper Iridium
    Every 133s Every 510s Every 510s Every 458s
    Table 5: Number of terrestrial users that change logical IP address per second.
    Starlink Telesat Kuiper Iridium
    7961 2082 5673 2379
  2. Static address binding to a fixed gateway

    This is adopted by the cellular networks and Starlink [STARLINK-CGNAT] and Kuiper's[KUIPER-CGNAT] initial rollouts. Each user gets a static address from the remote ground station (via carrier-grade NAT), which masks the external address changes and redirects users' traffic. This mitigates user address updates, but cannot avoid gateway's external address updates when changing satellite interfaces (detailed below). It also incurs detours and long routing latencies for remote users from ground stations (e.g., 18,000 km detours and 370 ms extra delays in [lai2021icnp]).

4.3. Impact on Routing: Frequent Routing Updates

The inconsistent locations in addressing further impact the network routing. As space and terrestrial infrastructure nodes physically move fast, the logical routing in cyberspace expires frequently. It must be updated frequently, thus threatening various routing schemes:

  1. Distributed routing: Repetitive re-convergence.

    In distributed routing, network nodes distribute topology information to others, locally compute forwarding tables, and eventually reach a global consensus on routing paths (i.e., convergence). Before global routing convergence, there is no guaranteed network reachability. With high mobility, each LEO satellite can only offer very short-lived access for a ground station(<=3 minutes in Starlink). Frequent topology updates cause repetitive routing re-convergence and thus low network usability. For intra-domain routing (e.g., OSPF[RFC2328], IS-IS[RFC1142]), most mega-constellations suffer from low network usability. Even the the size of constellation is samll, the network needs more than fifty seconds to converge after each handoff while using OSPF[TIMESLOT-DIVISION]. For inter-domain routing (e.g., BGP[RFC4271]), [Giuliari20Internet] and [NETWORK-IN-HEAVEN] show frequent logical topology changes cause BGP[RFC4271] re-peering, thus sharpening the instability of global Internet routing.

  2. Centralized routing: Repetitive global updates.

    In the centralized routing, a ground station predicts the temporal evolution of topology based on satellites' orbital patterns, divides it into a series of semi-static topology snapshots, schedules the forthcoming global routing tables for each snapshot, and remotely updates the routing tables to all satellites (e.g., via SDN[RFC7426], MPLS[RFC3031], or SRv6[RFC8754]). LEO mega-constellations pose stresses on centralized routing's scalability, stability, and complexity. Due to frequent link churns, the topology snapshots and FIBs explode with more satellites, ground stations, and faster mobility. Moreover, every satellite should locally load these new FIBs upon snapshot changes, which is vulnerable to transient global routing inconsistencies and thus black holes or loops.

5. Requirements of Addressing in Space-Terrestrial Network

Except from the basic properties like clusterability, network addressing in space-terrestrial network should also meet the following requirements:

5.1. Uniqueness

In integrated space-terrestrial networks, each user's address should be globally unique. This property calls for address allocation and duplicate address detection mechanisms.

5.2. Stability

This property ensures that the addressing of each node does not change with the movement of users or satellites. With this property, location-based routing will be more stable, avoiding routing convergence caused by the high dynamics of integrated space-terrestrial network. It also reduces the frequency of network addresses updates and reduces the impact on users' network services.

5.3. Locality

For any two users or satellites, this property ensures that if their addresses are closer, their actual physical distances should be closer. It not only guarantees the unified logical and physical locations, but also simplifies the design and implementation of location-based routing.

5.4. Scalability

The addressing of integrated space-terrestrial network should be designed to accommodate as many satellites and terrestrial nodes as possible. What's more, it should scale to more satellites and terrestrial users if needed.

5.5. Efficiency

In order to avoid frequent addresses updates, the design of addressing in space-terrestrial network should not require static address binding to remote gateways (e.g., carrier-grade NAT). The addressing method should ensure consistent cyber-physical locations, thus easing physically shortest paths without detours.

5.6. Backward Compatibility with Terrestrial Internet

To ensure seamless expansion of the global Internet ecosystem, the addressing in space-terrestrial network should be backward compatible with terrestrial Internet. It should be compatible the standard IPv6 addressing formats, and facilitates inter-networking to external networks without modifying terrestrial infrastructure. For backward compatibility with IPv4, we recommend adopting 4over6 transition for integrated space-terrestrial networks.

6. IANA Considerations

This memo includes no request to IANA.

7. Security Considerations

The present memo does not introduce any new technology and/or mechanism and as such does not introduce any security threat to the TCP/IP protocol suite.

8. References

8.1. Normative References

Postel, J., "Internet Protocol", STD 5, RFC 791, DOI 10.17487/RFC0791, , <>.
Oran, D., "OSI IS-IS Intra-domain Routing Protocol", RFC 1142, DOI 10.17487/RFC1142, , <>.
Moy, J., "OSPF Version 2", STD 54, RFC 2328, DOI 10.17487/RFC2328, , <>.
Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, DOI 10.17487/RFC3031, , <>.
Rekhter, Y., Li, T., and S. Hares, "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, , <>.
Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S., Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-Defined Networking (SDN): Layers and Architecture Terminology", RFC 7426, DOI 10.17487/RFC7426, , <>.
Filsfils, C., Ed., Dukes, D., Ed., Previdi, S., Leddy, J., Matsushima, S., and D. Voyer, "IPv6 Segment Routing Header (SRH)", RFC 8754, DOI 10.17487/RFC8754, , <>.

8.2. Informative References

"Amazon AWS Ground station: Easily control satellites and ingest data with fully managed Ground Station as a Service, 2021", <>.
"CNBC. Microsoft partners with SpaceX to connect Azure cloud to Musk’s Starlink satellite Internet, 2020", <>.
"Microsoft Azure. New Azure Orbital, ground station as a service, now in preview, 2020", <>.
"China tests inter-satellite links of BeiDou navigation system", <>.
"Broadband Global Area Network (BGAN)", <>.
Portillo, I.D., Cameron, B.G., and E.F. Crawley, "A technical comparison of three low earth orbit satellite constellation systems to provide global broadband", Acta Astronautica, 2019, <>.
"ETSI. TS 101 376-1-3: GEO-Mobile Radio Interface Specifications; Part 1: General specifications; Sub-part 3: General System Description", <>.
"ETSI. TS 102 744-3-6: Satellite Earth Stations and Systems (SES); Part 3: Control Plane and User Plane Specifications; Sub-part 6: Adaptation Layer Operation", <>.
"GEO-Mobile Radio Interface", <>.
Giuliari, G., Klenze, T., Legner, M., Basin, D., Perrig, A., and A. Singla, "Internet backbones in space", ACM SIGCOMM Computer Communication Review, 2020, 50(1): 25-37, <>.
"ZDNet. SpaceX to put Starlink ground stations in Google data centres, 2021", <>.
Giuliari, G., Ciussani, T., Perrig, A., and A. Singla, "{ICARUS}: Attacking low Earth orbit satellite networks", 2021 USENIX Annual Technical Conference (USENIX ATC 21), 2021, <>.
Bhattacherjee, D., Kassing, S., and M. Licciardello, "In-orbit computing: An outlandish thought experiment?", Proceedings of the 19th ACM Workshop on Hot Topics in Networks, 2020, <>.
Li, Y., Li, H., Liu, L., Liu, W., Liu, J., Wu, J., Wu, Q., Liu, J., and Z. Lai, ""Internet in Space" for Terrestrial Users via Cyber-Physical Convergence", Proceedings of the 20th ACM Workshop on Hot Topics in Networks. 2021, <>.
"ITU. Measuring digital development: Facts and figures 2020, 2020".
"Amazon receives FCC approval for project Kuiper satellite constellation.", <>.
"Amazon Kuiper", <>.
Lai, Z., Li, H., Zhang, Q., Wu, Q., and J. Wu, "Cooperatively constructing cost-effective content distribution networks upon emerging low earth orbit satellites and clouds", 2021 IEEE 29th International Conference on Network Protocols (ICNP). IEEE, 2021.
Handley, M., "Delay is Not an Option: Low Latency Routing in Space", Proceedings of the 17th ACM Workshop on Hot Topics in Networks. 2018, <>.
Klenze, T., Giuliari, G., Pappas, C., Perrig, A., and D. Basin, "Networking in heaven as on earth", Proceedings of the 17th ACM Workshop on Hot Topics in Networks. 2018: 22-28, <>.
Bhattacherjee, D. and A. Singla, "Network Topology Design at 27,000 km/hour", Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies. 2019, <>.
"OneWeb constellation", <>.
"OneWeb Gateways Require Complex Hughes Engineering, 2021", <>.
Bradley, D. and B. Lucia, "Orbital edge computing: Nanosatellite constellations as a new class of computer system", Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, 2020, <>.
"Online discussion with Starlink Engineers about satellites’ capability, 2021", <>.
Roddy, D., "Satellite communications. McGraw-Hill Education", <>.
"Solutions for NR to support non-terrestrial networks (NTN)", <>.
Bhattacherjee, D., Aqeel, W., Bozkurt, I.N., Aguirre, A., Chandrasekaran, B., Godfrey, P.B., Laughlin, G., Maggs, B., and A. Singla, "Gearing up for the 21st century space race", Proceedings of the 17th ACM Workshop on Hot Topics in Networks. 2018, <>.
"Spacex claims to have redesigned its starlink satellites to eliminate casualty risks", <>.
"SpaceX Starlink", <>.
"Petition of Starlink Services, LLC for Designation as an Eligible Telecommunication Carrier", <>.
"Tesmanian. SpaceX Starlink Gateway Stations Found In The United States and Abroad, 2021", <>.
"SpaceX Starlink Ground Station Map, 2021", <>.
"Study on New Radio (NR) to support nonterrestrial networks", <>.
"Technical Specification Group Meeting #91E", <>.
"With latest Starlink launch, SpaceX touts 100 Mbps download speeds and "space lasers" (though the system still has a ways to go)", <>.
Li, J., Li, H., Liu, J., Lai, Z., Wu, Q., and X. Wang, "A Timeslot Division Strategy for Availability in Integrated Satellite and Terrestrial Network", 2021 IEEE Wireless Communications and Networking Conference (WCNC). IEEE, 2021: 1-7, <>.
Handley, M., "Using ground relays for low-latency wide-area routing in megaconstellations", Proceedings of the 18th ACM Workshop on Hot Topics in Networks. 2019: 125-132, <>.

Authors' Addresses

Yuanjie Li
Tsinghua University
Hewu Li
Tsinghua University
Jiayi Liu
Tsinghua University
Lixin Liu
Tsinghua University
Qian Wu
Tsinghua University