Unpredictable Patterns #131: Human judgment and artificial intelligence

Evolutionarily shaped randomness, decisions and solomonic AI

Dear reader,

August is unfolding quietly, with time for reading and thinking - and it is wonderful. I hope that you have time to also exercise your judgment skills, as these will become more and more important! Here a short piece on why judgment preserving architectures may be key to using AI in the future.

Human judgment

Human judgment is an interesting phenomenon - a simple and naive version of what it is could go something like this: given an ambiguous set of circumstances, facts, guesses, intuitions and, yes, biases, we have the ability to make a judgment call, a snap judgment and decide one way or another.

What exactly is it that is happening in our minds when we do this? Theories disagree: some argue that we maximize some parameter and so make a decision to ensure such maximization. Others say that we choose randomly, essentially throwing mental dice and picking one decisions over another as the dice fall. And then there are those who argue that human judgment ultimately is connected to unknown and largely unexplained phenomena on quantum-levels in the brain.

In science fiction author Iain M Bank’s Culture novels, humans are called upon by the gigantic artificial intelligences known as Minds for some kind of quality that looks a lot like judgment: minds all converge when looking at a problem fitness landscape, and this means that they need help to get out of their own maxima - especially if they are warring with other minds, say. Humans are brought in because they essentially inject some kind of randomness into the decision making and force exploration, rather than the AIs exploitation.

This is an intriguing perspective: the idea that there is value in randomness in decision making is not new at all, but if that is the case we could imagine an AI just consulting any sufficiently good randomness generator or source — there are after all plenty of such randomness sources - randomness as a service is a viable idea!¹

But Minds go to humans. Why? It is unlikely that Banks spent a lot of time thinking about this problem - but one never knows - and if he had he might just have said that Minds like keeping humans around and discuss with them, but there are other possibilities here as well. One such possibility is that there is something unique about human-generated randomness.

Now, psychologists and con-artists will tell you that human beings are notoriously bad at randomness. If you ask someone to think of a random number and write it down you will get number sequences that are far from random.² In fact, they may - if you make them long enough - be unique to you and be used to identify you! So, why would such really bad randomness be attractive to an AI?

It is at times like this that it is good to remember Orgel’s second rule: evolution is cleverer than you are. What if - and admittedly we now enter speculative territory - human randomness is shaped by evolution in such a way as to be on average more helpful to decision making than pure randomness? What if judgment is randomness evolutionarily shaped by our ancestral environments?

If so, judgment has a unique quality that we should want to preserve as much as possible in any future decision making architecture - because it will help calibrate AI-systems through the injection of a specific kind of evolutionarily shaped randomness.

Preserving judgment

Judgment, of course, is worth preserving any way - because it is connected tightly to autonomy and agency. We believe that decisions that are made by human beings are special, and this bias - I think that is what it is - is strong. If asked if you wanted to be sentenced by an AI or a human being you might reflexively say that you would prefer a human being, since you would expect that they could exercise Aristotelian equity and look at the particulars in your case, and see you and your uniqueness.

However, as pointed out by legal scholar Orly Lobel, you might want to think about that twice if you belong to a minority, or if the judicial district you are in is an anomaly. Lobel has argued that anyone appearing before a court should have the right to an algorithmic process, if they so choose.³ Sometimes removing human randomness from the process and returning to the mechanical execution of a process might be preferable - just as nature may well have designed our randomness to be adapted to the world in such a way as to be superior to pure randomness in some cases, that same process may have designed a unique brand of human error and injustice.

For Minds this should have been a cause for concern - but perhaps the questions were carefully designed and doctored to avoid that, and this leads us to another interesting question: how do we design decision architectures in such a way as to maximize the role and usefulness of human judgment?

If we look at education, to take an example that is increasingly a focus of policy makers, we would probably want to figure out how we can build tutor-systems that maximize the training and exercise of judgment in complex issues. The current model of “socratic” AI where the AI asks you until you get it, is good for teaching factual subjects — but what about ethical, political or other humanities subjects? A socratic AI would have to end up in classical aporias - the elevated confusion that ends all early platonic dialogues - and this may not be helpful to students who actually have to make decisions, just as Socrates was attacked by some of the sophists for never getting to the decision at hand.

What we could imagine instead is Solomonic AI - an AI that forces the student into the role of a judge, adjudicating an issue in a simulated discussion, forcing a decision rather than just getting advice from an AI. Such architectures would preserve and develop students’ judgment, possibly strengthening their autonomy and agency as well.

Futures

In Bank’s future AI relies on human judgment in rare, limited cases - the majority of decisions in his fictional universe surely being made by the Minds themselves. This provides an interesting thought experiment, where we can ask which kind of future we want to live in if we look at how decisions are made, and by whom.

Let’s say that we have X number of decisions in a society, and that out of these X decisions Y are made by humans. This would, to some extent, be a good measure of human autonomy. If Y/X trends to zero we are not making any decisions at all and our autonomy is lost. If X=Y all decisions are made by humans and we largely ignore the benefits of automation, better decision making etc. We could also speak of the number of decisions made with the help of machines, Z, and try to look for good decision compositions across possible societies.

But this way of approaching futures is flawed - not least because we make many more decisions today than our ancestors did (at least I think so, but this is a hypothesis - I could be wrong here if we show that the amount of decisions made by an average individual is constant over time, but that the subject of those decisions shifts). So we need to look for which decisions matter - and how to qualify what constitutes a decision.

A machine choosing one caching pattern rather than another is in some respects making a decision, but this decisions would never have been made by a human so we have this class of decisions that seem to be machine-centric.

Maybe one way around this would be to say that what we care about are those decisions where humans have exercised their judgment - and that we want to preserve the sum total human judgment exercised today as a measure of our autonomy and our freedom. If so, we need to build architectures that preserve judgment — and ditch ideas like “human-in-the-loop” because such architectures are explicitly not judgment-preserving at all, they merely create some moment in a complex decision process where a human is exposed to the decision being made. The negative right to inject a judgment where someone feels it is needed should not be our ideal.

A proper taxonomy for decisions built from observation and study of human societies, and focused on which decisions should be made how in the future, is badly needed.

In practical terms this will matter a lot for how AI is used and implemented across different sectors: law, health, education — all of these sectors will need to build solomonic AI architectures and figure out how to not just preserve, but perhaps even augment and deepen our specific kind of randomness.

Thanks for reading!

Nicklas

1. See eg https://www.random.org/

2. See eg Schulz M-A, Schmalbach B, Brugger P, Witt K (2012) Analysing Humanly Generated Random Number Sequences: A Pattern-Based Approach. PLoS ONE 7(7): e41531. https://doi.org/10.1371/journal.pone.0041531 - Participants produced two 300‑digit sequences (1–9). Using pattern‑based analysis (Damerau‑Levenshtein), researchers predicted next digits with up to 27% accuracy (vs. 11% by chance); they even identified which sequences came from the same individual with ~88% accuracy. LLMs expand on these tendencies, see eg Van Koevering, K. and Kleinberg, J., 2024. How Random is Random? Evaluating the Randomness and Humaness of LLMs' Coin Flips. arXiv preprint arXiv:2406.00092. See also arguments for why human randomness may be rational: Williams, J.J. and Griffiths, T.L., 2008. Why are people bad at detecting randomness? because it is hard. In Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 1158-1163). Austin, TX: Cognitive Science Society and statistically explainable: Warren, P.A., Gostoli, U., Farmer, G.D., El-Deredy, W. and Hahn, U., 2018. A re-examination of “bias” in human randomness perception. Journal of Experimental Psychology: Human Perception and Performance, 44(5), p.663.

3. See Lobel, O., 2023. The law of AI for good. Fla. L. Rev., 75, p.1073.