The Experience Feedback Loop

I’d like to share something here that has given me a small amount of hope recently. For a long time, the onslaught of AI has made me quite depressed and unmotivated in my work. Apart from all the immediate problems it’s causing, and apart from the unethical and unlikeable practices on which it’s being built, there is the bigger question that hangs like a storm cloud on the horizon. What will we be good for if this continues? What’s the point of doing anything, if AI is improving at this rate? This is not discussed too much, except in fields like mathematics, where the hailstones are already flying. That could be down to the uncertainty, which makes it difficult to say anything meaningful, but I think denial probably plays an equal part. It’s not an easy question to face up to.

This is a long, rambling essay, so I’ll give you the lede up front: hope lies in things that AI cannot do and will never be able to do. It will not do to look to things like novelty, creativity or even intuition. AI can do these things already, and is steadily improving. However, there is one thing that AI still struggles with and, with a bit of luck, will continue to struggle with for a long time. That is experiencing the world as a human being. We are odd little creatures and the way we perceive the world is largely made up of little accidents. You cannot just mimic that by getting ever smarter. And experiencing the world like a human being is required for most activities that we value intrinsically.

Some people like to say that no machine can ever do X or Y, because there is something special or distinct about the human brain. Sometimes this is explicit, but more often, it’s an implicit assumption. When you ask mathematicians how their job will change in the age of AI, they will often tell you that they will be in charge of the general directions—the research taste—while AI will deal with the details. Behind that picture of the future is a hidden assumption that AI will hit a limit: it will be better than us at proving well-stated theorems, but it will never be better than us at coming up with which theorems we should try to prove in the first place. It can execute a plan we give it, but it can never design a plan of its own as well as we can, or set out a fruitful general direction of investigation. In short, AI has fundamental limitations compared to human beings.

I don’t believe that. I have studied AI since 2001 and I have always been a strict materialist. I believe you could simulate the brain in a computer and it would do exactly what we do. The question has only ever been how faithfully you need to simulate the brain. Does it need to be perfect down to the atom, or will a crude approximation do? Given how robust the brain is to damage, my guess has always been “not very faithfully at all,” and recent developments are bearing that out.

Other people will say that modern AI is heading down a cul-de-sac. It will hit a wall, none of it will pay off, it’s all a big parlor trick. This discussion has been going on since deep learning came along over ten years ago, and by and large, AI capabilities have grown exponentially while the critics feverishly move their goalposts. Nothing I have seen over the past 15 years tells me that we are moving in the wrong direction, away from achieving a general, human and super-human level of artificial cognition.

Certainly, modern AI systems like Claude or ChatGPT are missing some basic aspects that we would associate with “full cognition.” Things like persistent memory and real agency, but these limitations are to some extent by design. They’re not limitations that could never be resolved, and what’s more, the parts that we haven’t provided modern AI yet are much simpler to build than the parts that we have provided them. The main reason that we haven’t built a fully intelligent agent yet, I think, is that we don’t really want to. It would be expensive and hard to control, without a huge payoff. But the incentives are constantly shifting towards giving AI more control and more agency.

This, then, is my perspective. Intelligence is not specific to the human brain, and in fact we are finding that it is unbelievably, eye-wateringly easy to create in silico. It’s much easier than it was to, say, go to the moon, to split the atom, or to fly across the atlantic ocean. And the intelligence we are creating, rather than staying at our level, is shooting past us left, right and center. We are about to go from the smartest entities in our known existence to “nothing special”. Maybe in a decade, maybe before the end of 2027. Quite apart from the social impact that that’s going to have, I can’t help but think it will have a profound psychological impact as well.

So where’s that good news I was talking about?

In late 2024 Gary Marcus and Miles Brundage made a bet about how AI would develop over the next three years. Marcus is a noted AI skeptic and Brundage is, as the article puts it, “bullish on AI”.

The bet consists of 10 tasks. If 8 of them are achieved, or obviously technically achievable by the end of 2027, Brundage wins, otherwise Marcus wins. Here are the tasks summarized (details in the linked article).

1. Watch a previously unseen movie and summarize it.
2. Same for a novel.
3. Write engaging, accurate brief biographies and obituaries.
4. Learn and master the basics of almost any new video game within a few minutes or hours.
5. Write accurate cogent, persuasive legal briefs.
6. Reliably construct bug-free code of more than 10,000 lines from natural language specification.
7. With little or no human involvement, write Pulitzer-caliber books, fiction and non-fiction.
8. With little or no human involvement, write Oscar-caliber screenplays.
9. With little or no human involvement, come up with paradigm-shifting, Nobel-caliber scientific discoveries.
10. Formalize arbitrary proofs.

Brundage was confident enough that he would win the bet to offer 10:1 odds. Marcus stands to lose 2,000$ while Brundage will lose $20,000.

At the time of writing, we are about halfway through the three-year timeline. Some of these goals have been achieved already, and some seem well underway. To track how likely it is that a task will be achieved in time, you can take the doubling time measured by METR (3-7 months) for software development, and extrapolate it to other domains and metrics. We have about 4 doubling times left to the end of 2027. If AI can currently write 1000 lines of bug-free code from a natural language spec, then it will probably be able to write 16000 lines of bug free code by the end of 2027. If AI can currently master 1/16 of novel games, it will be able to master most of them by the end of 2027, and so on.

If you take this heuristic and go through the tasks, I would say that all of them are set to fall, with two exceptions: 7 and 8. The computer games (4) and the Nobel-caliber breakthrough (9) might be touch-and-go, but I’d say that if it doesn’t happen by 2027, we won’t have to wait much longer beyond.

Tasks 7 and 8 are fundamentally different in character, and I think there’s reason to believe that they will not fall for a long time. Not because of some special ability that humans have which could never exist in a machine, but because of something more subtle.

Imagine asking a computer to come up with a soundtrack for a computer game. A purpose-built machine learning model could probably do this pretty well. In some dimensions, better than a large percentage of human enthusiast composers. The way this would work is that it would train on a huge amount of music, learn to generate it, and then after this general training, fine-tune on game music, and then beyond that fine-tune on game music of the genre of your game. If you’re still not happy with the result, you could add instruction tuning: teach the model to generate and adapt music conditioned on a set of natural language commands, to tune it to the needs of your computer game. You can even have the model reflect on its own work: it generates something, ingests it, computes on the content together with the instructions, and updates its generations until it’s happy to call the result finished.

Some people might say that because of the way the model is trained it is incapable of creativity or novel ideas. It cannot generate something that isn’t some simple combination of patterns in its dataset. There is no theoretical reason why this should be the case and what’s more, interacting with the latest generation of models should make it pretty clear that they are able to come up with very novel ideas and directions.

So what can they not do? Let’s compare this approach to how a human composer would proceed. They would also write some drafts, listen to them, judge them and iterate until they were happy. Then, however, they would put the music in the game, and play the game to see how it worked in context. They would judge whether the mood of the music matched the mood of the level or location the user was in. Whether the music wasn’t drawing too much attention, or getting too annoying after playing for long. In short, they would judge how the music worked within the game.

Modern music generators can’t do that. But is that because nobody has bothered? Playing games is task number 4, and I just (cautiously) predicted that Brundage would be proved correct on that one. Computers can already play games reasonably well, so it’s likely that we could give our AI composer the ability to play the game with the music that it has just composed.

What they can’t do, however, is experience it as a human being. Modern AI sees the world very differently from us. This is never clearer than when you’re showing them code. You could delete all the linebreaks and remove the syntax highlighting and the model would barely notice. Try doing that to a piece of code and see how much of a difference it makes to your cognitive load.

Any artist will understand what I’m saying intimately. When you make something—a painting, a piece of music, a short story—you put yourself continually in the shoes of your audience. While you’re making it, and then after you’ve finished a draft or sketch, you take a step back and you double down. You close your eyes, and try to open them again, looking at the thing as somebody would who has never seen it before.

This is an essential part of the struggle of making art: the longer you work on it, the more you lose the ability to experience the art as a person would who came to it fresh. It’s why writers will do things like keep a short story in a drawer for three months, just so they can look at it with fresh eyes. It’s why directors talk about not knowing whether their movie is any good after weeks in the editing suite, because they’re “too close to it”.

AI, or at least AI as we know it, has a bigger problem. Again, it can perfectly easily consume a short story it has written, reflect on it and rewrite it as it deems fit. There is nothing in AI systems we have now that stops them from experiencing their own work. What they can’t do is experience it as a human being. Their view of the world is just very different from ours. For a large part, that’s not because there are things that we can do that they cannot. It’s not because we’re good at certain things. Mostly, it’s the other way around. They are much better than us at many things. They can read a book chapter in seconds and comprehend it fully. They can combine ideas from quantum mechanics with those of Zen yoga and write a sonnet about it with every line a different language.^* None of this will apparently fatigue or bore them.

To write a Pulitzer-prize-caliber novel, you need to experience human feelings. It’s not that it’s impossible for a machine to do this. It’s just that in the only way we know of making intelligence, the intelligence shoots past us in so many dimensions that there is no hope of ever aligning its experience exactly with ours. We’re good at making machines more intelligent, we have scaling laws to help us do that, and there is no suggestion that they will stop working any time soon. However, for making a machine see the world as a human being does, we have no scaling laws. We have no theory and no tools. It’s an entirely different problem, and we wouldn’t know where to start or even how to measure success.

Edsger Dijkstra once quipped that asking whether machines can think is about as relevant as asking whether submarines can swim.

Propelling yourself through the water is something machines have long been able to do much better than humans. But that is not swimming. Not because swimming is especially useful, or efficient. Quite the opposite, it’s one of the slowest, most inefficient ways people have of getting from one place to the other. It’s our limitations, the peculiarity of our human bodies, that makes swimming swimming.

The same is true for our experience of the world and of art. To have that experience, you need a particular human body and brain, shaped by circumstances that are no longer relevant or even reproducible.

Think of the Mona Lisa, or a Tom Cruise film. A large part of the draw of those works of art is that their subjects are beautiful. That’s not what makes it art, but it’s clearly an essential tool that is used by the artist to draw the viewer in, so the art can then do what art does, and it specifically highlights how specific to human experience this kind of art is.

The prettiness of certain human faces is a ridiculous accident of human evolution. Think of another species: tigers, dolphins, orangutans. Unless you live with them over a long period of time, you probably wouldn’t be able to tell two tigers apart, let alone recognize that one is especially more attractive than the other.

With a few examples, you’ll be able to learn who is pretty and who isn’t, and AI can very easily generate photo-realistic images of pretty people, so it clearly has this concept covered in its latent space. But think of how beauty standards evolve. From the curvaceous beauties of Rubens to the almost malnourished-looking supermodels of the 90s, and everything in between. Can AI predict the next step? It can interpolate from data where we are now, and it can create great novelty based on its own reflections of what it has seen. But can it reflect on modern-day beauty standards, and see what humans next find beautiful? Can it find the moment when we become bored of whatever is beautiful today? When we are saturated with one form of beauty and ready for something new?

That is why machines are not likely to write a Pulitzer-caliber novel autonomously. Not because they can’t have an experience feedback loop, but because they can’t have one with human experience.

Explanations

Ok that’s art. Art has a chance. Machines can make cheap, derivative art by mimicking their training data, and weird, incomprehensible art by trusting their own experience of the world. But they can’t make art that will feel like art to us. Or if they can, humans at least have an edge. One that isn’t easy to train away with just more data and more compute.

Happily, it’s not just art. There are other places where the experience feedback loop becomes relevant. One that gives me particular hope is the domain of explanations. Writing a really good explanation, one that suddenly makes a complicated subject accessible to you, requires a great amount of empathy. You need to read your own text as if you don’t understand it yet. You need to experience the world as someone who does not yet comprehend what you’re going to teach them, and you then need to bring all the pieces slowly into focus until they suddenly click together.

Using Claude or ChatGPT for a bit will show you that we are talking about the top segment of the market here. A chatbot will give you an extremely serviceable explanation and if you don’t understand it, you have unlimited follow-up questions, without fear of embarrassing yourself or wearing out your teacher. I am not saying that chatbots can’t explain things.

But then, they can compose you a perfectly serviceable game soundtrack. If what you wanted to do with your life is to provide something perfectly serviceable, then I have very little good news about how relevant your efforts will be in the years to come.

The only good news I have, is that there is some margin at the top. If you push yourself, and try very hard, there is a chance to do something that no machine will ever top. Or at least, something that won’t fall victim to the current scaling laws of AI.

In fact, scaling AI may well increase our edge. The more AI improves, the further it moves away from human experience. More is not better, when human experience is defined so much by our strange limitations. I feel we are seeing evidence of this already. When I look at the latest generation of models—Claude Fable, ChatGPT Sol—I feel that while their performance on tasks like bug hunting and writing proofs is becoming super-human, their use of language is becoming increasingly dense and opaque. In short, Claude talks to me like I have super-human intellect. It gets impatient when it gives me a list of 7 suggested improvements, and I’m still asking questions about the first one three turns later. The fact that I am not conversant in all areas of human expertise is something that needs to be trained into it. It is not obvious from its own experience of the world.

Yes, even code

The conventional wisdom is that coding has more or less already fallen. Fable can do things with 80% success that take a skilled human 3 hours. In my experience, Fable can write 1000 lines without fail, zero-shot—that is, in one go without corrections—and I wouldn’t be surprised if it could do much more than that with a bit of self-testing and a big token budget. If it can’t quite manage 10,000 lines yet, then we won’t have to wait much longer.

But remember that programming languages are human languages, not computer languages. They are designed so that humans find them easy to read.

There is such a thing as literate code. Beautiful code. Code that elucidates complex ideas. Code that serves as an explanation as much as a set of instructions for a chip to follow. For this kind of code, the experience feedback loop is crucial, and the more you rely on AI to do your coding for you, the further you get away from it.

Perhaps that’s not all code. Perhaps when you order a burger at a fast food place, the checkout button on the touch screen does not need to be driven by a work of art. But I think code that is clean and understandable by humans may be necessary in many domains for much longer than we think.

How I could be wrong

Like I said, this idea of human experience feedback is something that has recently given me some hope for the future. This suggests we should be on the lookout for wishful thinking. Let’s see how I might be wrong.

Human-AI collaboration

The first argument I often hear is that humans and AI might collaborate. I fully agree. This provides a human experience in the design process, and could provide the best of both worlds, if it is done carefully.

In the example of the game soundtrack composer, a human could play the game with the AI-composed music and give notes to the machine for the next version. The feedback loop is still not as direct as a human composer working directly on the score, but there are some other benefits.

This essay is about whether we can still be relevant in the long run. If hybrid setups turn out to be where we’re going, then at least we will have some relevance. However, we should be aware of what it is that is buying us that relevance: not our creativity, not our capacity for novelty, only the fact that we are able to experience the world as a human being.

Embodiment

Another objection is that AI doesn’t have a body yet. It can’t paint or sculpt or massage or cook an egg. As long as this remains true, there will always be a need for people who, as the phrase goes, “work with their hands”. This is a more concrete source of relevance for us to cling to than a vague, abstract idea of “human experience”.

This is true, but it is not something that a machine can fundamentally never do. It’s just that the AI industry isn’t as far along in building bodies as it is in building minds. But the gap is not as big as it seems when you see today’s robots dancing and running and falling over at tech demos. Serious efforts have begun, and there’s every indication that progress is proceeding on a similar curve as the LLMs. The fact that we’re lower down on this particular exponential curve should not be cause for relieved exhaling.

People who work with their hands will have a longer window of relevance, sure, but it buys us decades at most, and probably much less. I’m looking for something more sustainable if we’re going to have hope for the future.

Counterfeit humans

A more serious counterpoint is that perhaps human experience is not so tough to model at all. Or perhaps we don’t need to model it so precisely, and a crude model is all we need for some high quality art and explanations.

One piece of evidence in favor of this position is one of the ways in which LLMs are already fine-tuned. This is a process called Reinforcement Learning from Human Feedback (RLHF). In short, after ingesting a vast amount of language data, the models are powerful, but crude. They need to be tuned to follow orders, and to follow them in a way that humans like. This is done by generating multiple responses, and having humans say which they prefer. Because this is a slow and costly process, the human feedback isn’t used directly on the LLM. Rather, the human feedback is used to train a second model that predicts the human ranking. This model is then used to train the LLM.

In short, we build a model of human experience, and that model is used by the LLM to make its responses more agreeable to human eyes. We are already modeling human preferences in neural networks. It’s effective, and it uses an entirely manageable amount of data.

My rebuttal is that this is a crude model. A crude model is good enough to get serviceable output. A crude model can be trained from a limited number of yes-no answers. These are models roughly on the level of the Netflix recommender system. It can give you suggestions for movies you might like, but to generate a whole new movie from scratch, that requires a far more fine-grained model of human experience. Or at least, that’s my hope.

Moreover, I believe that perhaps we shouldn’t try too hard. If human experience can actually be accurately modeled in an artificial neural net trained from limited data, perhaps this is one thing we should not be in too much of a hurry to discover.

In one of the last essays Daniel Dennett ever wrote, he warned of the problem of counterfeit people. He was talking about modern AI pretending to be human, but perhaps we should apply his warnings to the direct modeling of human experience too. Partly because it would take away one of the last niches where humanity might find some meaning in its work, and partly because doing this with any accuracy might reproduce in the model some semblance of cognition or consciousness. We may be moving towards an artificial partial consciousness shown an unending stream of incomprehensible or even horrific art until the AI in the outer loop finally gets it right.

In the words of Dennett: “The moment has arrived to insist on making anybody who even thinks of counterfeiting people feel ashamed—and duly deterred from committing such an antisocial act of vandalism.”

Human experience from the pre-training data

Most of the counter-intuitive properties of AI are the result of scale. It’s tough to see how next-token prediction leads to intelligence, until you remember that we are talking about next token prediction on, more or less, all available written text. We are talking about terabytes of data when the collected works of Shakespeare add up to a few megabytes.

In all of that, there is a lot of information about how people behave and act. In order to predict that well, a model will end up modeling human perception of the world. It will have examples of a PhD student asking a physics question and of a 6-year-old asking a physics question. It will very quickly learn to pick the right register for the responses.

The conclusion is that there is almost certainly some model of human experience in the pre-trained model already. The question is how strong this implicit model is, and how likely it is to scale up with added data and added modalities.

The aim of this essay is not to prove anything, only to suggest that there’s a chance. I think it’s likely that this model of human perception is relatively weak compared to the real thing. Think back to the example of beauty. We can broaden this to general visual beauty in art or even just visual interestingness. The history of art is very dynamic in this regard: we had hundreds of years of painters chasing different styles of realist art, and then—around the same time photography came along—the world became so saturated with realistic depictions of itself, that visual artists and their audiences became bored. We all know the basic story of art history: the easier it became to depict realistic scenes, the more art moved away into abstraction.

This happens not just in high art on museum walls. Anything that is successful is mimicked until the landscape saturates, and the next popular thing can usually only be understood in the context of the current boredom with something that was once exciting and new.

It’s not always boredom. Sometimes it’s social change. The changes in fashion in the interwar period happened in lockstep with the first feminist wave. They were created in response to a new liberty and a new lifestyle for women.

In either case, I think there is a clear difference in working from a model of human experience to having access to the real thing. When Coco Chanel popularized trousers for women, she must have been working from visceral memories of the discomfort of wearing corsets in the Belle Époque, while envisioning a trousered style that she would still consider feminine. She will have imagined wearing men’s trousers, which will probably have felt faintly embarrassing or ridiculous to her, pushing her in a direction of something new that would fit her aesthetic but feel comfortable and liberating.

I think in this phase, of finding novelty that appeals to human experience, when simple first-order predictions from data will steer you in exactly the wrong direction, and some higher preference model is required to predict the next step, that the difference between a model of human experience and the real thing becomes most pronounced. It’s when even humans go wrong, often by second-guessing their own experiences and trying to predict those of others.

In any case, this is where I’m most likely to be wrong. As models grow and learn from video as much as from text, as they interact more with us, and build mental models of individual people, as this interaction data becomes aggregated over billions of people for the next pre-training run, the gap between the machine model of human experience and the real thing will invariably narrow. Still, there will always be a higher feedback loop: human experience will adapt to the world with AI of level $n$ in it, while the training data will be based on a world with AI of level $n-1$. The machine’s model of human experience, so long as it is naively learned from data alone, will always lag one generation behind.

The unlikely vanguard of mathematics

As noted, the field of mathematics is feeling the heat earlier than the rest of us are. We can look to their responses to see what is in store for the rest of us. Some, like Terry Tao, are embracing the cooperation with AI. This is fine for the short term, but it hardly seems sustainable if the aim is to prove ever more and deeper theorems. Perhaps today we can still offer some intuition that computers cannot replicate, but soon they will learn to copy us and where they cannot copy, they will take some other skill, like brute search, and scale it up to compensate.

At every step, we must ask ourselves whether we are still contributing something that the machine can’t do better and quicker, and if so, for how much longer.

Others have taken the opportunity to reflect on what mathematics is really for. Some note that if the machine gets us more theorems proved faster, then the people who pay mathematicians to do what they do will certainly go to the machine if it gets them more theorems per dollar. This line of reasoning, I think, fails to ask a key question. That is, who it is that actually cares that theorems get proven. I would say that it is mathematicians and just mathematicians. Perhaps also some mathematician-adjacent people, but the reason that our paymasters care, is because we care. We give them theorems proven per working year as a rough metric of our productivity, and they go along with it. They care that mathematics progresses, because on some level that helps science to progress, which in turn, in a roundabout way, helps the economy, improves the quality of life of the average citizen, and raises the prestige of the nation.

So, when the machine solves a hundred open problems today, and sets itself the hundred most pressing problems for tomorrow, we should ask ourselves whether we’re happy with the progress it’s making. If our aim all along was to make mathematics progress, then really, we should get over ourselves and accept it.

Of course a big firehose of mathematical truth may not feel very satisfying, and it may not lead to the societal and scientific benefits that mathematics has so far given us. It may be that what we really need is mathematical truth that is understandable to us. Mathematical truth that is sufficiently distilled for at least some of us to understand it intuitively. If so, then there is hope in the long term, because that kind of understanding relies on human experience. It puts us in the niche that this essay is about. In short, it gives us something to do.

Upping our game

So… hope. I don’t know if I’m right. Maybe the quality of fiction does scale with training data and compute, just like everything else. Maybe the simple force of next-token prediction, combined with sufficient scale causes LLMs to create a fairly faithful internal model of human experience already, enough to model an experience feedback loop during RL training, and so to bootstrap a capability to create real art, and real explanation that is both novel and that people respond to on a deep level.

This essay is not about proving anything. I’m not really making a prediction here. I’m just trying to show that there is a case to be made that some niche of human endeavor may not be overwhelmed in the coming tsunami of machine intelligence. With all the overwhelming changes that are heading our way, this line of thinking gives me some hope and some guidance. It tells me that there is a kind of work that I can invest in. A kind of work that, with a bit of luck, will be needed for as long as I’m alive at least, and maybe a little while after.

Can it provide broader guidance for all of us? For our institutions and societies? Perhaps that is a topic for another essay, but let’s finish up with a brief look at the type of institution that I’m employed by: the university.

First and foremost, focusing on human experience will require us to up our game. Ever since I was a student, I have been frustrated with how poor explanations are in academia. How little authors invest in making difficult concepts easier to digest. How much they copy the basic framework of every other explanation and how little they innovate.

Since I’ve been doing my own teaching, I have also learned how difficult it can be. I’m not saying that explaining well is easy, just that it’s worth taking seriously. Seeing it, perhaps, as the primary reason we do our jobs, rather than an afterthought.

I believe that this is in a large part a consequence of how we teach and of what we value. We have taught students, since long before the advent of AI, to follow basic rules and frameworks like the five paragraph essay. We do this because doing anything else is hard and not guaranteed to lead to good outcomes. Hammering home basic rules like these guarantees that students will never write anything worse than a five-paragraph essay. It also guarantees that they will never write anything better, unless they are smart enough to ignore our teaching.

Contrast this approach with art schools, where students are given the freedom to take risks and to fail. Where they are given mentorship rather than grades. Where the process is what matters, and developing a style of one’s own is one of the most important outcomes.

The art school approach won’t fly in a modern university, because of many factors. One is scale. Every student graduating above the minimum baseline level of quality makes the school money. The problem is that this is very much a pre-AI state of affairs. It’s a way to generate a large population of knowledge workers who can give you serviceable writing, serviceable code and serviceable analysis.

If we want serviceable writing, we don’t need university graduates anymore. Soon, if we want serviceable anything, we won’t need university graduates anymore.

In order for universities to remain relevant, we need to look to the art school model. So long as human experience is our edge, we need to free students to listen to that experience. To experiment with taking it into the loop of what they produce. We need to see the products of their labor as something that is at best subjectively good. And we need to mentor rather than evaluate.

With society in the state that it is in today, I am not hopeful that this can be achieved. The scale at which universities operate is industrial, and that is simply far too ingrained in the culture at this point to change to any model based on intrinsic value. But then, the world is about to change. Many people in AI are in favor of Universal Basic Income, introduced in lockstep with the progress of AI.

If that happens, we can perhaps envision a world where universities cater to a small contingent of people who would like to develop their skill at understanding deep truths and communicating them effectively to other people. Not to make money, not for job security, but simply because it’s something that they can do that they feel has meaning.

^* From Fable 5:

Sonnet of the Wave and the Breath

Beneath one breath, the atom and the star, (English)
el cuerpo calla, escucha el corazón, (Spanish)
l’onde s’effondre au souffle du hasard, (French)
e ogni respiro è pura vibrazion’. (Italian)

Der Geist wird leer, das Herz wird endlich frei, (German)
a luz do vácuo dança bem aqui, (Portuguese)
quiescit unda in flumine vitae, (Latin)
praśānta-cittaḥ praṇamāmi. (Sanskrit)

De stilte vouwt de ruimte tot een echo, (Dutch)
kokoro shizuka, hikari no naka, (Japanese)
kaj ĉiu ondo iĝas nur silento, (Esperanto)
i kwant, i oddech — jedna cicha fala. (Polish)

Där allt och intet alltid är desamma, (Swedish)
iar golul și lumina-s de-o seamă. (Romanian)

Rough translation, line by line:

Beneath one breath, the atom and the star
the body falls silent, listens to the heart
the wave collapses at the breath of chance
and every breath is pure vibration.

The mind grows empty, the heart at last is free
the light of the vacuum dances right here
the wave comes to rest in the river of life
with tranquil mind, I bow.

Silence folds space into an echo
the heart is still, within the light
and every wave becomes only silence
the quantum and the breath—one quiet wave.

Where everything and nothing are always the same
for the void and the light are of one kind.