Consolidation

Is Internet traffic consolidating, i.e., moving towards a larger fraction of traffic involving a small set of large content providers, social networks, and content delivery platforms? It certainly appears so, though more research on this topic would be welcome. But what might this mean for the Internet? And are our technology and architecture choices affecting some of these trends? Would different technology change the trends in some fashion?

Of course there are many different areas where consolidation might occur. For example, provision of content and services is observed to concentrate in the hands of a few entities; likewise, ISP services often achieve better efficiency at scale. Even operating systems, Web browsers and other tools experience this kind of tension, with too many or too few choices creating different kinds of problems. We all probably recognise at least some forms of these trends. In general, an efficient market such as the Internet tends to enable winners to take large market shares.

The IAB's remit is to focus on technology, but of course we also want to invite input on the implications and externalities of those arrangements. Technology affects economics and vice versa. The Internet technology community continues to make decisions that have ramifications on Internet systems, just as we are subject to forces that affect them: the consolidation trend also raises relevant and interesting technical and architectural questions. 

For instance, a year ago we talked about large-scale denial-of-service attacks, and how various entities can deal with them. While the largest attacks affect all players (see, for instance, the Dyn attacks in October 2016), it is also true that large cloud- and content delivery providers can better deal with such attacks due to their scale. This is one reason among many that drives many network services to such providers. Another reason is the drive towards lower latency services, which can be best provided through globally distributed data centers.

As technologists, one question we have is whether there are changes in technology that would help reduce technically-driven large-player advantages. This would lead to lower barriers to entry, more entities able to provide any particular service on the Internet, less dependence on a few players, and more choices and greater resilience for the network as a whole. 

Of course, it may well be that technology improvements are hard to come by. For instance:

• Providing distributed, low-delay services seems like something that naturally fits larger entities better than small.

• Some technical issues may appear more suited for improvements, but have historically not been easy to solve, such as spam. Today, it is becoming increasingly difficult to run your own mail services, essentially forcing many organisations and individuals to employ larger providers. The issues relate directly to size of entities; no one can afford to disconnect from the largest providers. But as a small entity, there is little leverage to convince peer entities or various supporting white/blacklist entities to deal with you properly.

Nevertheless, recognising the risks of consolidation in both current and proposed future technologies is the first step in proactively avoiding those risks where possible.

One avenue where further IETF work would be beneficial is better support for federation, which would enable entities (be they large or small) to work together to provide global services.

Another big question is whether there are assumptions about the Internet architecture that no longer hold in a world where larger, more centralised entities provide big parts of the Internet service? If the world changes, the Internet and its technology/architecture may have to match those changes.

One example of this is that more and more content is becoming available locally, from a content delivery or provider function directly on your own ISP's network. This trend seems strong, and eventually most of the content will be delivered this way, reducing the role that global IP connections across the Internet play. By some metrics this has already happened; what practical - positive or negative - impacts might this have on the Internet technology?

Changes in business landscape may also affect standards work and processes. Small sets of significant players can drive standards work faster, but at the same time the larger entities are not as dependent on standardised solutions as smaller ones would be. 

What do you think about this topic? What research on this topic should be driven forward? What IETF topics that should be pursued to address some of these questions? If you are interested on this or other architecture-related topics, please subscribe to the IAB architecture-discuss mailing list as one forum for discussion.

By Jari Arkko, Mark Nottingham, Christian Huitema, Martin Thomson, and Brian Trammell for the Internet Architecture Board (IAB)