Unpredictable Patterns #134: Organizing science in the age of AI
On organization, methodologies, schools of science and pattern matching - with Anastasia Bektimirova
Dear readers,
This week’s post is a collaboration between myself and the brilliant Anastasia Bektimirova. For those of you who do not know Anastasia, she focuses on science and technology policy and strategy, and heads this agenda at The Entrepreneurs Network, a London-based think tank. You can read more about her here on TEN's website. Anastasia writes in a personal capacity, so nothing in this piece should be seen as representing the views of her current or past employers, of course. She also co-authored this nice piece with Alex Chalmers.
Organzing is hard
Here is a prediction: science will be more important in the coming 3 decades than it has been in the past 3 decades, and we stand to unlock the equivalent of many decades of today’s incremental progress in a single generation.1 The main driver of this acceleration will be artificial intelligence, and related technologies - but it is not a given that we get to enjoy this progress; we have to organize for it.
Organization is hard - for multiple reasons. Re-organization re-allocates power, resources, legitimacy and decision rights inside an already existing equilibrium established under different conditions, and the inertia associated with such equilibria is significant.2 Yet, organization is the key to the use of technology, and it is only in use that technology has socio-economic impacts at all. Without embedding in institutions, tools remain latent potential. This is as true for manufacturing technology as it is for AI. Organization limits or enables diffusion of technology as well - who can apply it, at what pace, to which problems - shaping everything from productivity growth to cultural change.
Organizing science might be among the hardest of organization problems because “science” is a loosely connected ecosystem of tightly connected institutions. Research funders and coordinating bodies, universities and their departments, labs and different institutes are all tightly connected through a loose network of collaboration, publishing, career ladders and funding. Any change is resisted both by the equilibrium of the organization in question and the overall ecosystem, including the shared interests of incumbents invested in the status quo.
This inertia can have beneficial effects: it helped preserve the UK’s participation in the European Union science programs - since these were too interwoven with UK capabilities to be dissolved in any reasonable way. This was a net benefit for both the UK and the EU. But this same resilience pairs with slowness: the system of science adapts grudgingly to other kinds of shocks.
Artificial intelligence is an interesting example of such a shock, and there has been plenty written about how to best deploy AI in science – arguably providing ready roadmaps that now only require political will in science policy and the willingness of funders to align incentives. Most of the proposals are wisely incremental, looking for first steps that can be taken without challenging any stakeholder or incumbent power structure. Useful as that is, they tend to leave deeper architectural assumptions intact.
We believe that it could also be useful to set aside incumbent constraints and approach the question of how to organize science as a system designed for AI-mediated discovery from scratch. Let’s ask how science as a whole could be organized if we started today, with the AI-tools available to us, and look at what we can learn from this thought experiment.
Designing science from first principles - the importance of method
Organizing science from scratch requires starting with how scientific inquiry is done and how institutions are built around it. Disciplines became the enduring “home bases” for scientists - each one a community with sub-communities, with their own knowledge base and favored methods. Disciplines are organizational forms that supply the durable architecture - departments, journals, conferences, career ladders for science, and they are often organized around methods. Methods work as the engines that drive change and collaboration within and across those organised communities. Methods identify groups by activity and behaviour, and groups also build sub-ecosystems of journals, conferences and appointments to reinforce their networks, which in turn defends and entrenches methods.
A lot of what we see in terms of scientific organization flows from how methods are embedded in and across disciplinary homes.
Indeed, there is an argument that scientific progress is often driven less by facts challenging paradigms and forcing a rethink of existing science, than by new methods being introduced and used to build relative advantage between groups. Science is not a disembodied activity - but takes place in specific communities and contexts.3 Where and with whom it takes place matters - and the old notion of scientific schools may be out of fashion, but still permeates a lot of scientific institutions even today - in practice, as combinations of problems, methods and training.
So what should we ask in the age of AI? We need to start our thought experiment by asking which roles and method combinations to elevate, how they connect across disciplines, which shared platforms they rely on, and how we need to resource and govern them. This, obviously, is a hard task - but one that will prove interesting in our rethink of organization overall.
Let’s sketch out a first, rough model of how to think about science - and let’s try to be almost simplistic about it.
First, science is about patterns. All science - humanities, physics, linguistics - study patterns in different ways. Patterns, in turn, are expressed in data. Any organization of science, then, would start with a shared data layer for all of science, essentially building the foundation of scientific inquiry through the shared storage of data across different institutions. This would require eliminating any silo - such as commercial lock-in in journals4, sequestered data in different data repositories or proprietary data kept by private collaborators in public research.
This shared pattern layer would be the basis of any scientific inquiry, and these patterns would include novels, ethnographic notes, measurements from telescopes, legislation, case studies - anything you can imagine we can capture. This doesn’t collapse meaning or erase evidence standards, but simply makes it easier to connect methods to materials. The old qualitative and quantitative split would lose organizing power - all science is pattern-centric, and we respect that there are different kinds of patterns, but in the interest of discovering new insights we will not discriminate against any kind of pattern. A short story or poem is as much a pattern as a super-collider data set - and can be represented and queried in ways that allow tools and teams to move more easily across fields. The possibilities then become endless - if all scientists share a foundational layer of knowledge to build upon, we maximise the chances of serendipitous connections and interdisciplinary breakthroughs.
Second, we need to agree that patterns can be approached in different ways. David Krakauer, head of the Santa Fe Institute, has suggested that science might be thought of as divided into two different domains: machine learning for fine-grained predictive tasks, where we build on the correlation of data, and complexity science where we provide coarse-grained schemata to understand systems and causes.5
Krakauer’s distinction is closely related to the distinction made in the theory of science between explanation and understanding, but replaces explanation with prediction. This, again, is not a new move - but it is one that is salient as we see the emergence of a stronger predictive aspect of science.
Science then should be organized around prediction and understanding - and understanding is as important as prediction. Crucially, when we look at patterns anything can be predicted - poems or proteins - to varying effect and with various uses. There is no need for distinguishing between natural sciences, social sciences and the humanities - since they are all composed of patterns - at the level of representation and reasoning. All become just different facets of a single continuum of pattern-seeking and knowledge-building, united by shared tools for prediction and understanding. At the level of experimentation and intervention, however, domain-specific craft, such as wet-lab practice, instrument calibration, biosafety, standards of evidence still governs how claims are validated and used.
Now, what we have so far is a unified science based on patterns organized around predictions and understanding, or machine learning and complexity science, building on a shared pattern layer incorporating all knowledge.
To most this will read either like disrespectful computer imperialism for science or dangerous relativism, suggesting, for the first category, that we do not respect the fine nuance needed in hermeneutics for the humanities or, for the second category, that we are willing to accept any pattern as equal to any other, rejecting notions of quality, statistical value and veracity. We mean neither. We claim that such a shared knowledge substrate would make cross-field, multi-disciplinary work easier.
We accept that something like this is unlikely to happen, but let’s talk about what we can learn from this thought experiment for more realistic scenarios.
Initially, the focus on patterns - a focus on data, really - is key. We firmly believe that when our civilization looks back in a 100 years at our time, one of the things that they will be horrified by is the way that we slowed down scientific progress by creating commercial, organizational and national silos for data of various kinds. The emergence of a shared data layer - a shared layer for exploration and experiment - should be inevitable, and the liberation and reform of scientific publishing cannot happen soon enough. The advent of AI raises the opportunity cost of fragmentation: AI allows us to collate the enormous amounts of published results, data and findings in ways that can accelerate progress by an order of magnitude - leaving that on the table because of the special interests of a few actors is not morally defensible.
Second, the focus on prediction and understanding - in correlation - allows for the emergence of a new kind of social science, as well as a new form of the humanities. We have seen that already in computational social science and digital humanities. Instead of reducing them to “just quant”, the computational turn has allowed many researchers to become bilingual: to use computational pattern-finding at scale and to deepen causal, contextual and interpretive work. Social systems and humanities study some of the most complex and important systems we are aware of. We need to explore them as such, and learn more about everything from legal systems to history and art. By treating these as patterns, where both predictive questions and questions about understanding can be asked, we ensure that we do not reduce them to mathematics. Introducing AI does not absolve us from the second prong of Krakauer’s model: the coarse-grained understanding of a complex, interconnected world.
Ultimately our goal here should be to overcome Snow’s two cultures, and show them for what they truly are, two aspects of the study of complex patterns.
But…policy?
So far so good, but how do we translate this into policy today? A surprising, and we admit disconcerting, conclusion is that we need to elevate the discussion about the future of the scientific method. The theory of science as a subject has become technical, specialized and lost contact with discussions about shared progress as a civilization. Let us try a controversial opinion here: that the scientific method should advance society as a whole, and serve human progress in ways that are clear, understandable and measurable. Methods that do not clear this mark should be scrutinised and, over time, deprioritised. This is not a call to abandon, for example, interpretive methods, but a call to design a portfolio in a way that makes progress legible and shareable.
This is not the same as saying that we should only engage in immediately useful or applied science - we should still engage in foundational science and exploration - but in ways that ensure that the results can be used to build further insights. The scientific method needs to produce results that compound into bettering our situation.
Francis Bacon, often regarded as a pioneer of the scientific method, emphasized exactly this: the practical utility of science as a means to enhance human power over nature, improve societal conditions, and alleviate human suffering. In his view, true knowledge should not be sought merely for intellectual satisfaction, curiosity, or debate, but for its operational and beneficial applications—essentially, to "command nature" through obedience to its laws and to restore humanity's dominion over the world.6 This perspective is evident across his major works, such as The Advancement of Learning (1605), Valerius Terminus (ca. 1603), and Novum Organum (1620), where he critiques speculative philosophy and advocates for empirical inquiry leading to inventions and discoveries. This does not exclude either social science or the humanities, by the way, assuming that the methods employed to pursue either are the right ones.
This is, we realize, an argument for a certain intolerance - and a narrowing of science in a way that some will perceive as exclusionary, but the value of science in society broadly directly hinges on the quality of the scientific method employed in conducting science overall. Re-inventing and re-visiting the scientific method, 400 years after Bacon, is not just prudent, but necessary to understand where we stand today.
Science needs to become a widely shared, engaging project again - and AI provides an opportunity to accomplish this - but such a re-invention requires that it is re-organized from first principles.
Thank you for reading,
Anastasia and Nicklas
How would you measure this, you may ask? Perhaps as investment, public and private, but also as talent streams and public attention — the hypothesis here is that a basket of such measures will show that science will be moving from a special interest to a key shared project in our societies.
Especially power is tricky here. As noted in Knights, D. and Roberts, J., 1982. The power of organization or the organization of power?. Organization Studies, 3(1), pp.47-63 a lot of the challenges in organization flow from people mistakenly thinking power is a thing we possess rather than a relationship we create.
This of course is a point made by many theorists of science - such as Bruno Latour - but it still is curiously absent from the policy discussion of science.
This remains a different and distinct issue, but research suggests that copyright in science restricts progress with unclear benefits. See eg Biasi, B. and Moser, P., 2021. Effects of copyrights on science: Evidence from the wwii book republication program. American Economic Journal: Microeconomics, 13(4), pp.218-260.
The whole discussion is worth listening to:
Bacon was not narrowly technocratic. The Advancement of Learning contains long discussions of history, poetry, ethics, and civil knowledge. He argued that all forms of learning could be improved by his method, not just natural philosophy.
 | A guest post by
|