Unpredictable Patterns #147: Death and intelligence
On masochistic research programs, mortal AGI, phatic communication and love
Dear reader,
This week’s note reflects something I have been thinking about peripherically for some time now, and it sort of starts in the recognition that the way we think about artificial intelligence is both anthropomorphic and weirdly mechanomorphic. The perhaps most articulated mechanomorphic element in the current discourse about AI is the absence of death in understanding intelligence - this piece re-introduces death as a key component in what we are and how intelligence evolves.
Mortality and intelligence
We have abstracted away death from our lives. Today, in the West, it is rare to have seen a dead human being up close. We have seen photos, perhaps, or videos of accidents where we know that the people in the footage did not survive - but really have seen a dead person, even one who died of old age, is increasingly a rare experience. It should not be surprising, then, that we also minimize the role of death in understanding ourselves - yes, we are mortal, but we would not like to talk about it.
But what if death is defining for us in ways that we may be ignoring? What if death, to take a topical example, is a necessary component in intelligence? Can the fact that we are finite be the very thing that also led to the evolution of intelligence, and can that finitude be a key part of what intelligence really is?
Today, that is very much not the view. Silicon Valley-ideology believes death is a deplorable design mistake, and something to be overcome. The idea that we can, and should, live for ever is alive and well in the Californian sunshine - from the focus on health, supplements and diet to cryogenic facilities preserving whatever you can afford to preserve for future resurrection (head only slightly cheaper than full body preservation).
Intelligence, on the other side, is viewed almost like a natural force or a disembodied capability that can be likened to software. In this view, intelligence is treated as if it was independent of time, a potentially infinite phenomenon. Intelligence is substrate-independent, some argue, and that image easily drifts into intelligence also being independent of time.
Let’s examine the counter-view.
As if death meant something
The only examples we have of intelligence in our world are mortal beings. There are no, at least no empirically evidenced, immortal beings and we know of no immortal intelligences. This also means that all intelligence is oriented toward its own finite nature - we discount value over time, for example, because we know, with Keynes, that in the long run we are all dead. Death is a great clarifying and prioritizing force in our lives - and we even say, repeatedly, things like “life is not long enough to bother with that”. Death inspires satisficing, across the board.
This insight is recognized in various technological models. Reinforcement learning algorithms routinely employ discount factors that weight immediate rewards more heavily than future ones, a mathematical acknowledgment that the horizon matters. Time-bounded search in game-playing systems accepts “good enough” moves rather than pursuing exhaustive analysis, trading theoretical optimality for practical decision-making within real constraints. Even large language models, despite their apparent timelessness, operate under context windows and computational budgets that force a kind of artificial mortality onto their reasoning processes.
These are all examples of viewing death as a time limit imposed on us, and adjusting systems to reflect the fact that we know that intelligence does not have access to infinite time. Intelligence requires prioritization, because time is scarce.
But what if other aspects of intelligence also flow from our finitude? Take agency: we desire things, we want - and we want to do things before we die. The idea of the “bucket list” may seem silly, but it is right there: evidence that our desire, too, is structured by the finite nature of our existence. What does the bucket list of an immortal entity look like? Desire also plays out against a finite horizon.
Prioritization and agency are both related to mortality, then. On top of this we have our long term care for coming generations, knowing that we will not be here - and knowing that we have but a little time to try to improve the world they will inherent. So we learn to plan, and we learn to learn. These two are also connected to our mortality.
And mortality is at the heart of another function that I think is undervalued: forgetting. Death devours memory, clearing the path for adaptive behaviours on a population level and minimizing the risk for loops of suboptimality. With death society forgets us, and as it does so it evolves.
We can continue, but once we start to look closely it does seem at least likely that death, mortality, is a pre-requisite for intelligence. Intelligence isn’t a capacity for solving problems, after all, it’s a capacity for choosing which problems to solve. And that capacity only makes sense against finitude.
Models of intelligence
The reason this matters is not just because death is so undervalued in understanding ourselves, and intelligence, but because it suggests that maybe we have been trapped in trying to understand intelligence inside-out. We have our own experience of intelligence, as something in our heads, and we think we can recreate that phenomenon and that this phenomenon is independent of time. But if death - an external fact of life - structures intelligence intimately across a number of different dimensions, then we might need to shift our perspective and look at intelligence as an effect of various external factors. Intelligence is not software in the head, it is the effect of the execution of software outside of the head.
The reason this, in turn, is interesting is that it allows us to invert a lot of different questions about intelligence. Ilya Sutskever recently asked how evolution installs social emotions in our heads (I simplify, and obviously he is much smarter than I am so bear with me) - but maybe that is the wrong question. What if social emotions are just error correction we experience when social connection with others is disturbed, lost or modified in some way?
Let me explain: if we believe that evolution had to install fairness in our heads, we need a plausible mechanism for how that happens. But if we assume that social connection is primary, rather than the result of interacting individual intelligences - that is, we assume that the “we” come before the “I” - then we can say that when we feel that something is unfair we just pick up on an error in the social connection that comes before our sense of individuality, and we experience that disturbance in the social field.
Emotions, overall, then are not experiences produced by the brain - but absences or disturbances felt as the social connection field is shifted or distorted in some way. Emotions are reflections of social connection and primary to individual experience, and intelligence. We are social first, individual second in this model and evolution never worried about installing social emotions in our heads, as it created fields of connection across our eusocial species that we then react to.
Take another example: think about the child, crying out in the night to make sure someone is there - this is a ping, an attempt to connect with the social field, the “we” and then it calms down as the mother responds. This is something that we call, after linguist Roman Jakobsson, phatic communication and it is one of the most underrated functions of language.
Roman Jakobson suggested there are six functions of language, each corresponding to a different element of the communication situation. Language is referential when it orients towards subject matter, emotive when it expresses states of mind, conative when it asks questions or commands, metalinguistic for when we discuss language itself and poetic when the form takes the foreground.
The phatic function, however, concerns the channel itself, language used to establish, maintain, or verify contact (”Can you hear me?” or small talk that keeps conversation flowing). It is not impossible that phatic communication may be the primary function for language, and that all that comes after that is what allows for the evolution of intelligence and emotions on an individual level. And at its core, phatic communication is focused on survival, and so conditioned on and produced by our mortality.
Building mortal machines
What would it mean to try to build artificial intelligence as if death mattered? How would we approach AGI if we set ourselves the task to build from the community to the individual, and not start by modelling or building individual intelligence? If we took seriously the possibility that individual intelligence is secondary to community intelligence?
Wittgenstein notes somewhere that one of the most common roots of philosophical quandaries is that we are held hostage by an image. What does the image look like that holds us hostage in the case of the current paradigm in artificial intelligence? Here are what I think are the main elements:
First, it is an image of intelligence as individual. And I don’t think I mean that we should try for hive minds or collective intelligence, I think it may be more foundational than that: that our intelligence is in all real aspects reflections of social relationsships of various kinds. Yes, there are innate biological constraints on what you can reflect back, and history matters as it always does in evolution, but if we entertain the idea that intelligence is a mirror rather than a set of algorithmic steps, we may start to ask ourselves what it is that we mirror and how mirrors can be built better.
Second, I think it is an image of intelligence as immortal. I struggle here, because immortality is not quite right, and I do not want to say that we see intelligence as atemporal, or infinite, but it is somewhere in between those two: we underestimate the importance of finite horizons for the evolution and functioning of intelligence.
Third, the image is one of a scalar. We seem to believe that it makes sense to say that A is more intelligent than B on a scale. This reduces intelligence to a set of measurable capabilities and this is practical, and we need metrics for capabilities, there is no doubt about it, but I think it obscures that intelligence may be something like a carrier wave - and that capabilities are shared across the reflections of intelligence in a community in ways that are both complex and hard to decompose.
If we want to break away from this image, we could start to think seriously about how to build mortal machines, and not just time-limited, but mortal - structuring them so that the ever-present experience of mortality influences everything about their evolution, behaviour and capabilities. Designing such mortality into artificial intelligence - systemic mortality - is also a way to think about long term architectures for safety and security, but that the idea here is broader, it is that death may be one of the missing components on the way to AGI.
Practically, building “mortal” AI could involve things like exploring finite lifespans deeper, building social-first, phatic agent architectures or building evaluations and benchmarks for adaptation for these ecosystems — there is a wealth of possible ideas here. Building mortal machines is also very likely to teach us something about ourselves, and our own intelligence.
Masochistic research programs
Taking death seriously is hard, and the notion that emotions are just absences is almost depressing - because both challenge things we want to believe about ourselves: we want to aspire to immortality and we believe that emotions are the most valuable about us, now that intelligence, and even creativity, seems to be up for grabs with ever-better models.
But this in itself may be a reason to explore these ideas more. There is an argument for adopting a masochistic heuristic in exploring artificial intelligence: assume that what we cherish most about ourselves is very different from what it is and perhaps easier to understand than we would want.
In many ways that masochistic research program has brought us to where we are today, and now we balk at moving further down that road - we want to be special, and so we try to safeguard the things that we believe make us unique like emotion, and we shy away from the idea that death might be instrumental and essential for intelligence - because it extinguishes the hope of immortality that technology kindles.
And all is not bleak. There is another interpretation of the theory that emotions are error corrections on encountering absences - and that is that we all are aware of a state in which social connection is pure and complete, and that we have the ability to experience it and engage in it. This original state of our emotions and the root of our intelligence is a state in which we recognize ourselves in the mirrors of others, and share the wonder of the world - the state of equilibrium all humans innately seek under the horizon of our own mortality.
Something we call love.1
But, of course, love may simply be the brief silence of a system whose error-correction has nothing left to correct.
Thanks for reading,
Nicklas
Remember Rilke: “And this more human love (which will fulfill itself with infinite consideration and gentleness, and kindness and clarity in binding and releasing) will resemble what we are now preparing painfully and with great struggle, the love that consists in this: that two solitudes protect and touch and greet each other.”