Unpredictable Patterns #24: Centralization, networks and the curse of connection
On models of network evolution, human nature, technologies of deliberation and product / society fit
Dear reader,
A good week, I think, and one in which I had the opportunity to think a bit about networks and technology. The theme for this week’s note is the idea of centralization, how it relates to networks and what it means for democracy. I hope you find it interesting!
Is the Internet centralized?
Geoff Huston wrote an interesting and thoughtful essay this week, called ”Centrality and the Internet”. Huston, who is the chief scientist at APNIC, examined the hypothesis that the Internet is slowly becoming as centralized as the telco-network and that we are losing something important in the process:
”It certainly seems that the Internet is now the realm of a small number of enterprises that dominate this space. This is no longer a diverse, vibrant environment where new entrants compete on equal terms with incumbents, where the pace of innovation and change is relentless, and users benefit from having affordable access to an incredibly rich environment of goods and services that is continually evolving. Instead, today’s Internet appears to be re-living the telco nightmare where a small clique of massive incumbent operators imposes overarching control on the entire service domain, repressing any form of competition, repressing innovation, and extracting large profits from their central role. The only difference between today and the world of the 1970s is that in the telco era, these industry giants had a national footprint. In contrast, these days, their dominance is expressed globally.”
His fear of centralization as a pathology of the Internet is not unique - there are more and more voices now worried about the Internet losing something fundamental as it grows increasingly centralized, and a new dimension is emerging as a key tension in Internet policy - the tension between centralized and decentralized. Where we used to worry mainly about open and closed systems we now need to also care about systems that are centralized and decentralized. A rough, simplified picture of this could look something like this:
The open - closed axis is about the amount of control that is imposed on the service in question, how easy it is to manipulate or change it. We can debate how we would distribute the different services in the image, and I think there is a lot to be said for placing Blockchain technologies all over the place, but one reason for putting it in the closed / decentralized is because of one of the core advantages in this technology - decentralized but yet authoritative information that cannot be easily manipulated. Search can be manipulated, and is designed to discover new things so ends up being open, but centralized in a single search engine.
Huston’s argument, then, could be cast in the following terms:
(i) The Internet is evolving towards more centralization.
Huston understands that if he is to make this argument, he needs to also show a mechanism that leads to this centralization - what is driving the Internet up the central axis? Huston’s candidate is advertising:
”It appears that there are very real dynamics of scale in advertising. Advertisers want their message to be seen by the greatest number of potential consumers, and advertising platforms want to provide a service to the greatest number of potential advertisers. The larger the platform, the greater the potential of the platform to meet both of these requirements. The result is intense pressure to consolidate in this market. ”
Huston’s argument here then becomes a bit complicated - he argues two different things, it seems - one is that the Internet is becoming more /centralized/ and the other that the advertising market is becoming more and more /consolidated/. In addition to this he raises the question of market /concentration/ as another component in the challenges he sees.
It may seem as nitpicking to point out that centralization, consolidation and concentration are different things, but they are - and if the question really is if the Internet is becoming more centralized we have to also add an assumption to Huston’s model: that the Internet’s topology is responding to the structure of markets and the concentration of capital.
We can then develop Huston’s criticism further:
(ii) The Internet is becoming more centralized, due to the dominance of advertising business models that inherently derive great benefit from scale, and hence consolidate markets to great concentration of capital.
If this is indeed the case we should, as Huston is, ask if this is a development that we want, and if we should not do something to stop it. But first we need to examine the argument more closely, since there is another model that also fits with the facts and is interesting to explore.
Centralization vs preferential attachment
The story of the Internet evolving from a decentralized state to a centralized state is complicated by the fact that the Internet has gone through a phase of massive growth.
Whenever we compare the Internet at time t with the Internet today we are comparing its state at that time with its state today, and so we need to ask if any changes we see are just a part of that growth, or if there are other growth paths that the Internet could have taken.
This in turn requires that we propose a rule under which the Internet has evolved, and then examine the consequences of it growing under that particular rule.
As best we know the rule that we think governs Internet growth is some version of the preferential attachment rule. This rule is simple, and suggests that the Internet will evolve according to the ”rich-get-richer”-scheme that we see in many other networks as well.
The way to imagine this is to imagine that we have a network with a few websites, and then we add a few new ones. The new ones link to each other and to the already existing ones. How do they choose what to link to?
If we assume that this process is completely random, the structure of the resulting Internet should be something like the normal distribution - a lot of websites with a lot of links in the middle and the tails seeing fewer and fewer links and so the resulting network would also look like that.
If we instead assume that there is some increased probability that a new website links to an old one, and that tenure over time creates a kind of linking gravity, then we get a very different structure. We end up instead with what is sometimes called a power law distribution in which few websites have a lot of the links.
This will look a lot like centralization, but it is in fact a function of the way the network grows.
The challenge here is that we look at the way the network looks, and we forget the network’s history. In most cases the history of phenomenon is more interesting than its current state.
An instructive example of this is Schellings segregation model. Thomas Schelling, a brilliant game theorist and economist, suggested that we could study the segregation in large cities by devising a very simple model: we assume the city is a grid and that every person lives in one of the boxes on the grid.
We then posit a very simple rule: every person in this grid has 8 neighbours (the surrounding boxes) and we can set a threshold of when they will move because they feel that their neighbors are not enough like them in their current neighborhood.
Essentially, Schelling suggested, we are asking at what threshold we will start seeing segregation. If people were striving for all 8 to be like then we would suspect that we have a problem of xenophobia and racism built in to the system, but the fascinating truth turned out to be different.
This is what a simulation of Schelling’s experiment looks like in the brilliant simulation tool NetLogo at the threshold of 30% (that is, people in this model want roughly a third of people to look like they themselves do - they do not even want to be in majority white /black neighborhood).
It starts out like this (the image on the right are different people randomly distributed across a city grid):
And ends up in a stable state that looks like this:
As you can see this is an entirely segregated city. But the driving force of this is not deep-seated racism that requires all white communities, but instead just a more modest desire to have 1/3 of people in a neighborhood be like us.
What Schelling shows us is that we need to be careful when we speak of the drivers behind how complex systems end up with structural qualities. Huston’s model, where advertising is the key driver, builds on the observation that the large actors today are - to some degree - advertising funded, but does not recohnize that this may be because the way the network evolved may have made that the most reasonable business strategy. Now, I am not arguing that they have nothing in common or that there is no correlation or causation - just noting that cause and effect here are not trivial.
If we just assume preferential attachment, and no other causes - and so that the network structure is dependent on a small bias towards the already existing nodes, we can use NetLogo to observe the following evolution. We start with two nodes:
And now we evolve the network with a slight bias towards already existing nodes:
This network is, to some degree, just like the Internet, centralized — but not because of any extra assumptions about advertising as a scaling business model. Occam’s razor suggests that we do not need to blame advertising for this particular structure.
But we still are left with a really important question: what is it that is driving preferential attachment? Why do we see this not only in the Internet’s structure, but also in other media markets?
What drives preferential attachment?
In 2006 a group of researchers published an intriguing piece of research in which they had constructed a fake music market download site (yes, kids, we downloaded music back then).
They partitioned the download site into several different versions and so were able to essentially run, in parallel, an experiment to examine two things: if people liked all songs equally or if there was some inequality in their choices across the songs on the site, and if the /same/ songs where chosen in each version of the site.
The experiment was a way to find out how power law distributions come about, and - interestingly - if the ”quality” of the content mattered.
Their findings were simple, yet important: when users knew about other users’ preferences - the researchers simply displayed number of downloads to create this transparency - the result was a pronounced, power law, inequality between the songs - a few songs had the majority of the downloads.
But with small differences in the initial conditions the songs that came up top in the results varied! The really good songs always won - so quality does have an impact - and the bad songs were downloaded less, but the list beyond the outliers depend more on chance and initial conditions.
When people did not know about each other’s choices these effects were much less pronounced and inequality was much lower.
This seems to suggest that in any system where we have transparency into how others choose, we will see rich-get-richer effects, and it is interesting to ponder why. At this point I would like to propose a simple conjecture:
(iii) Human social cognition is often shared and so when we can we put a value on consuming the same information, news and participate in the same cultural events. This leads to power law like distributions in how we engage with content networks.
Underlying this are further results about how human networks evolve. In the literature on networks there is a lot written about this, and the conclusions seem to suggest something interesting.
Human social networks have a tendency to become tighter and tighter over time.
One driving mechanism here is something called triadic closure.
If I have two friends who are not friends, they will soon have the opportunity, the incentive and a bit of transitive trust to close that link in our shared network - and as a result they will also become friends, closing the network and creating new links.
This, paired with the value of shared social cognition, let’s us formulate a new hypothesis:
(iv) Social networks have an inherent social gravity that tends to make them tighter and tighter over time.
But what do we mean when we say tighter? One way to answer that question is to talk of degrees of separation. This is a term that you have probably heard in the phrase ”six degrees of separation” and the popular saying is that everyone on earth is connected to everyone else by merely six jumps across friends of friends of friends…
The original experiments in sociology here suggested that in open societies (like the whole of the US) the degree of separation was roughly six, and this has since been replicated in a lot of different studies.
In more closed social networks, like professional networks or villages, the degrees of separation quickly sink to much lower. In the once popular Oracle of Kevin Bacon, where you can figure out any actor’s Bacon number - their level of connection with Kevin Bacon as calculated by if they have been in a film with him (1) or with someone who has been in a film with him (2) and so on - and the average degree of separation there is below 3.
(iv) can then be rephrased as
(v) Social networks tend to decrease their degree of separation over time.
If this is the case we need to ask why we see 6 turning up all the time? The real answer to that, it turns out, is that we did not have the technology to support further reduction of the degrees of separation.
But now we do.
The curse of connection
The degrees of separation is sinking, due to the powers of connection afforded us by new technology. What we call communication technology is really more of connection technology, and it enables the natural social gravity to draw us even tighter.
Researchers from Facebook, looking at the giant network component the services has started, suggest that we are now at 2.9-4.2 degrees of separation.
In other words: we have rebuilt the village social network on the Internet, and so are shifting from the more distanced and anonymous ”society” of the city to the ”community” of the village, to speak with German sociologist Ferdinand Tönnies.
At the same time we are increasing the amount of information and the speed with which that information flows massively. The tighter network allows for more people to get information quicker than ever before.
Our powers of discovery of information have grown orders of magnitude. But what about our powers of deliberation?
First, let’s look at what happens when we deliberate in groups, when we do deliberate. In smaller groups, as Cass Sunstein has shown, deliberation has the unfortunate effect of polarizing views. As the Internet has recreated the dynamics of the smaller group, we should expect to see such polarizing effects as mere artifact of the fact that the public sphere has turned into the private network where we discuss with other likeminded people.
Second, let’s also look at how we deal with information overload. When there is too much information flowing in our networks we need heuristics to guide us, so we start thinking tribally to manage the information load.
Set political positions and polarization provide us with filters that allow us to make sense of the world at a reasonable cognitive cost - something that is hardly possible otherwise.
So, as a mere function of tighter networks and increased amounts of information we seem to get quite worrying effects, that go beyond what Huston has been suggesting to the basic functioning of democracy.
This curse of connection is somewhat surprising given that ”connection” has been a singularly positive word, associated only with positive externalities.
Can we fix it?
Networks of deliberation
Before we address the question of if we can fix this, we have to examine another aspect of the problem, and that is what network topologies work best for problem solving. If we agree that the Internet is evolving into some kind of scale free network based on a power law structure, we can ask what kinds of political regimes we should expect this to generate? We can then also ask the question of what network topology is best compatible with deliberative democracy - and explore different fixes.
The scale free networks we have explored seem to contain the social structure that human beings will tend to adopt if left to their own devices. If I believed that there was a state of nature, with philosophers like Hobbes, I would argue that whatever political regime sits on top of a scale free network most naturally, is our state of nature.
Now, I believe we are in the unique position to change our nature, and so do not subscribe to necessary natural states - but let’s play with the idea for a while! The natural state would be one in which a few of the nodes have a lot of the connects - that suggests some kind of elite society, perhaps an aristocracy as these networks stabilize over time. (It is not surprising that the great theorist of the elites - Pareto - also showed great interest in power laws.)
The democratic revolutions in the Western world could even be viewed as the imposition of a different kind of network - I would suggest a modular, spatial one - on the scale free networks to ensure new forms of influence and deliberation. Cities, regions, nations, international institutions - these modular networks represented a great invention and innovation in how we organize our societies and were encapsulated in law and institutions.
It would amount to the reintroduction of space and distance on our networks, and slowing us down.
Democracy exists in the tension between network topologies.
If we want to explore this idea further, one interesting way to do it is to look at how problem solving in networks can be rated for efficiency and accuracy. My guess here is that scale free networks are efficient, but rarely accurate or very novel seeking how they solve problems (but this is just a guess) - and if I had two groups of equivalent talent and knowledge, but one organized as a modular network and the other in a scale free network I would choose the former for better expected outcomes.
So, we started with the idea that the Internet is becoming centralized because of advertising and we are now arguing that the Internet evolved into a scale free network that is reducing the degrees of separation between us and that this might be affecting our polities.
I think this shows that the centralization / decentralization discussion opens up a really interesting set of questions for us, and the solution space is also intriguing.
10% less connection 10% more distance
There are essentially two different ways of addressing what we have suggested is happening here.
The first is to try to attack the information flows on the networks and reintroduce friction and try to check the information. Here we find the approach of Twitter, trying to make us read what we retweet and the legislative push from the EU on misinformation and fact checking.
The assumption must be that if we can just fix the information flow and the quality of the information we will be alright, and we will be able to deliberate successfully in scale free networks.
Then there is the second approach, which is to try to impose a different network topology. Here we find initiatives like Tim Berners Lee’s work on decentralized Internet solutions, the idea that we should cap social network connections at much lower values (and reduce the so-called passive networks that are externalities in our social networks). Some hope that other ideas around blockchain can lead to different topologies as well.
The assumption here seems to be that the network topology is produced by network effects that are inherent in the protocols or architecture we employ, rather than in our human nature.
This is a tremendously interesting area of innovation and I really believe that there is a great future in designing disconnection and distancing technologies that help deliberation.
Different disconnection models - where you, for example, isolate groups or just create modularity and reintroduce spatial elements - may lead to entirely new ways in which we can deliberate across great groups, and create even more interesting political models and deepen our democracy.
There is no doubt in my mind that we can do this, and do it well — if we agree that there is something about the way our networks interact that has impact on our political organization as well.
So what?
As a strong believer in technology and an optimist, I think this has a few obvious implications.
First, I think we need to think actively about discovery and deliberation, and recognize that technologies of deliberation are at least as important as technologies of discovery. This, in itself, is not small task - but it is important. And to preempt the ”technical fix”-criticism I can see coming, I do not think that there is a piece of gadgetry or software that fixes this - it has to be organizational, institutional and social innovation as well.
Second, I think there is a real question here about not so much the celebrated product/market fit we often discuss but a broader more interesting product/society fit. This means that technology companies had better start rethinking their internal hierarchies where the engineer reigns supreme. Don’t get me wrong - I have an enormous respect for engineerings and think that engineers are creating enormously important things. But when we talk about ”building” in today’s society we are talking about engaging in social change and that requires a humanist view, a deep understanding of the human condition and of how we organize legally, politically and economically as well. Building should be our ideal, but again - with a product / society fit as the key heuristic.
I am also curiously, and perhaps naively, very optimistic about our ability to do this, and to harness the Internet for a rejuvenation of democracy - and I agree with Huston in his concluding remark:
“The result appears to be that this Internet that we've built looks like a mixed blessing that can be both incredibly personally empowering and menacingly threatening at the same time!”
As always, thanks for reading, and let me know any thoughts or ideas you have! I am looking forward to your feedback.
Nicklas