Why didn’t COVID-19 break the internet?

Just a few months into its fifty-first year, the internet has proven its flexibility and survivability.

In the face of a rapid world-wide traffic explosion from private, public and government entities requiring employees to work from home to help curb the spread of the coronavirus, some experts were concerned the bandwidth onslaught might bring the internet to its knees.

All indications are that while there have been hot spots, the internet infrastructure has held its own so far – a silver lining of sorts in dreadful situation.

How is the internet handling this situation?

First, what does the internet look like? It consists of access links that move traffic from individual connected devices to high-bandwidth routers that move traffic from its source over the best available path toward its destination using TCP/IP. The core that it travels through is made up of individual high-speed fibre-optic networks that peer with each other to create the internet backbone.

The individual core networks are privately owned by tier-1 internet service providers (ISP), giant carriers whose networks are tied together. These providers include AT&T, CenturyLink, Cogent Communications, Deutsche Telekom, Global Telecom and Technology, NTT Communications, Sprint, Tata Communications, Telecom Italia Sparkle, Telia Carrier, and Verizon.

These backbone ISPs connect their networks at peering points, neutrally owned facilities with high-speed switches and routers that move traffic among the peers. These are often owned by third parties, sometimes non-profits, that help unifying the backbone.

The backbone infrastructure relies on the fastest routers, which can deliver 100Gbps trunk speeds. Internet equipment is made by variety of vendors including Cisco, Extreme Networks, Huawei, Juniper Networks and Nokia.

Cisco said it has been analysing traffic statistics with major carriers across Asia, Europe, and the Americas, and its data shows that typically, the most congested point in the network occurs at inter-provider peering points, Jonathan Davidson, senior vice president and general manager of Cisco’s Mass-Scale Infrastructure Group, wrote in a blog on March 26.

“Our analysis at these locations shows an increase in traffic of 10 per cent to [41 per cent] over normal levels,”  Davidson stated.

“In every country [with peering points in Hong Kong, Italy and France and Russia seeing the biggest traffic jumps], traffic spiked with the decision to shut down non-essential businesses and keep people at home. Since then, traffic has remained stable or has experienced a slight uptick over the days that followed.”

While overall the story has been positive, the situation hasn’t been perfect. There have been a variety of outages according to traffic watchers at ThousandEyes, which writes weekly reports on outages among ISPs, cloud providers and conferencing services.

Globally, the number of outages to ISPs hit a record high of 250 during the week of April 20-26, 124 of the in the U.S. The number of outages is the most since the end of March, but two issues – fibre cuts in CenturyLink’s network and a broad Tata Communications outage – helped push that number up. Typically though,  these problems have not been caused by networks being overwhelmed with traffic.

Resilient by design

Network planning, traffic engineering and cutting-edge equipment can take most of the credit for the internet’s ability to adjust in times of need.

“IP was built to last through any sort of disaster, and the core was built to live through almost anything,” Davidson said. “Over the years there has been a tremendous amount if infrastructure and CAPEX spending to build out this massive network. We are no longer in the days of the wild west of years ago; the internet is a critical resource and the expectations are much higher.”

Indeed, the principle of over-building capacity is one of the key reasons the internet has performed so well.

“Network capacity is critical. Our network team and engineers have been able to keep the same amount of capacity or headroom on our networks during this crisis,” said Verizon’s Irlando. “We continue to augment capacity and connectivity.”

“There was some anxiety as traffic began to ramp up at the start. We’ve seen a 35 per cent increase in internet traffic – but ultimately the networks have handled it quite well,” said Andrew Dugan, chief technology officer at CenturyLink.

Internet planning actually took into account the demands a pandemic would place on the network, Dugan said. “CenturyLink and other providers began developing pandemic plans more than a decade ago, and we knew that part of the response would rely significantly on our infrastructure,” he said.

People who build large IP networks engineer them for unexpected congestion, he said.  Dugan pointed to three factors that are helping the internet successfully support the increased volume of traffic:

• Networks are built with redundancy to handle fibre cuts and equipment failures. This means creating capacity headroom to support sudden disasters
• Network monitoring helps operators anticipate where congestion is occurring, allowing them to move traffic to less congested paths
• ISPs have been building out networks for years to account for increasing demand, and planning specifications help prevent networks from reaching capacity

When building fibre backbones, ISPs often bury the cabling to protect it from storms and accidents that can take down above-ground power grids.

Since much of the cost of deploying the cable is in the labour to dig the trenches, while they’re at it, most ISPs install more fibre strands than they have a current use for, according to ISP OTELCO. This so-called dark fibre can take the form of additional cables or cables with unused fibre strands within them that optical switches can light up quickly as the need arises.

“We had some infrastructure segments that ran hot,” Dugan said about the Covid-19 spike in traffic on CenturyLink’s network, “but we are fibre-based so we quickly were able to add capacity.”

And ISPs are adding more fibre all the time, which “is key to ensuring networks can meet growing demands and provide support in times of crisis, like these,” Dugan said.

The shifting last mile

Fibre may be commonplace in the largest internet backbones, but it is much less so in the last-mile connections that reach homes. While fibre is the fastest home internet option by far, availability is still scattered in the US, according to Broadbandnow.

Due to the high cost of installing fibre service directly to homes, ISP connections are still predominantly served by coax cable TV services even major cities. Chicago, for example, only has 21 per cent fibre availability as of 2020. Dallas has about 61 per cent, and that’s actually high compared to other major metros in the US, the company stated.

When work-at-home orders came down in March, the source of internet traffic shifted dramatically. Rather than coming from business sites connected by high-bandwidth links, suddenly significant amounts of traffic was coming from private homes, dumping more traffic onto the access networks during what would otherwise be off-peak hours.

It was a significant enough issue that AT&T CEO Randall Stephenson noted it during the company’s first-quarter financial call with analysts.

“What we are seeing is the volumes of network usage moving out of urban and into suburban areas… and we are seeing heavy, heavy volume on the networks out of homes,” Stephenson said. Work-at-home employees, students doing online classwork and online shopping added to the load.

But as CenturyLink’s Dugan noted, the work from home activity is generally happening during the day while peak internet usage continues to occur in the evening when people generally consume video and gaming. This has helped balance out the additional internet use.

In addition, traffic engineering may be able to find less congested routes if the traffic load gets too great. When that’s not possible, providers have to look elsewhere. For example, U.S. Cellular boosted its mobile broadband capacity in six states by borrowing wireless spectrum for 60 days from other carriers who owned the licenses for those spectrum bands.

AI and automation help dodge issues

Other attributes have helped the internet’s performance as well. For example, AT&T said its artificial intelligence is helping remotely troubleshoot problems with customer equipment and identify issues before they become problems.

“We’ve expedited deployments of new AI capabilities in certain markets that will allow us to balance the traffic load within a sector and across sectors to help avoid overloading specific cells and improve the experience,” AT&T stated.

Increased use of automation has also had an impact by enabling network engineers to quickly manage traffic, Dugan said. “Service providers who invested in software-defined networking prior to the coronavirus crisis may have been more responsive to changing traffic patterns than ones that are still using legacy or hybrid networks,” Dugan said.

Going forward Verizon’s Irlando said he doesn’t think current internet traffic levels are the new normal. “No one knows the future, but we will not have 90 per cent of America working from home,” he said.

One indication of remote-worker impact comes from a Gartner survey of 317 CFOs and finance leaders in March that said 74 per cent of businesses will move at least five per cent of their previously on-site workforce to permanently remote positions post-Covid-19.

Cox’s Hart says the situation underscores the need to continue investing in the backbone. He says his company will spend $10 billion over the next five years to build out network capacity, improve access, drive higher speeds and improve latency and security.

Internet access isn’t universal

There is one overarching problem the Covid-19 crisis is shining a light on: the digital divide. For an estimated 3.7 billion people worldwide, internet access is either unavailable or too expensive, and that is palpable when connectivity to the outside world becomes essential.

“Internet access has become increasingly vital to our health, safety, and economic and societal survival. As cities and countries across the globe ask their citizens to stay at home, billions of us are fortunate enough to be able to heavily rely on the internet to fill the gaps in our work and life,” wrote Tae Yoo, senior vice president of Cisco Corporate Affairs in a blog about the digital divide.

“There is no silver bullet on how to solve this problem,” Dan Rabinovitsj, vice president of connectivity with FaceBook. “It’s going to take a lot in investment and innovation from network operators to drive costs out of the ecosystem so that they can pour more money back into the network. Infrastructure is having its moment right now, everyone is depending on it.”

“The internet is moving from huge to absolutely massive. It’s moving from being critical to being essential to economies, businesses and governments,” Cisco’s Davidson said. “As a result of Covid-19, we’re getting a glimpse of what the internet of the future is today.”

―ARN