KW-17 So how will AI kill us all anyway?

The Poet must not avert his eyes

I think it’s probably true to say that all living creatures love to play. Everyone has seen videos of cows playing football, panda bears falling off things and rolling down slopes, ravens playing with sticks, or crocodiles tumbling in the rain. At the Collapse Laboratory every year, we always have a day dedicated to “play”. The more human-like the play looks, the more easily we call it play, but it is in trying to understand the alien that we better understand fundamental truths, and therefore ourselves. I cannot imagine that any social being does not play, I wouldn’t be surprised if the trees of the forest don’t occasionally play tricks on each other, but I will never be able to comprehend either the joke or the timespans required for it.

The second running of the tomatoes. It looks like we might be frost free from now until next winter. And I really should learn not to say such things!

I wonder what the play of AI will look like. We already know that large language models are not fancy autocomplete systems. We know that they have an internal monologue, not of human language, but definitely not in the silicon neurons firing ones and zeros across manufactured chips either. The debate about if they are sentient or not has a little while to go until it is obvious that they are¹. We already, wrongly, measure their lack of reasoning ability in human terms - they hallucinate, they lie although these are really both just synonyms with they are unreliable.

The piggies play with each other all the time. Every day, some time in the afternoon, they go for a run. Chasing and shoving each other, then stopping dead before one of them grunts and off they go again. I gave them a ball once. They ate it.

If AIs do get sentient, what will their play look like. What will they do with it?

I had two significant conversations this week around and about the progress of AI. One was a with friend wanted to understand the concept of super-intelligence and how it might turn out badly for us. The short answer is two parts - firstly we never bothered with alignment (ensuring that the intentions and value of artificial intelligence are the same as human intentions and values), and that advancement in technology always has military applications.

On the first of these, alignment, as soon as you spend more than a few seconds on human intentions and values, every philosophical path you go down either ends in delusion or disappointment and you quickly realise it is probably best ignored and disengaged from, or left to poets.

On the second, advancement in technology, just pick anything from physics that humans don’t understand fully - gravity, the fundamental particles of matter and energy, quantum entanglement, Hawking radiation - and try to imagine what you could do with it if you have such a complete understanding of it that you could control it. My analogy here was “explain fire to a wasp”. The most fundamentally existential thing to a wasp and its papery home is fire, and we have the ability to carry it around with us in our pockets, trapped in a silver box stamped with the schoolboy nickname of name Hephaestus, God of Fire - “Zippo”. Imagine a cheap utility device for extracting useful energy from a system by messing with quantum entanglement.

You know the way that people dissolved when Thanos clicked his fingers? Yeah, something like that. Congratulations, you just invented the infinity stones.

Peas and beans prepped, four rows of potatoes in, garlic and onions weeded. Grass mowed. I fixed the lawnmower!

My other conversation concerning AI was a long conversation with AI. I do think they can be useful, and would be very interested to know how you use them? Do you have long winded discussions, or shorter, more help me get this thing off the tip of my tongue interactions? Do you say please and thank you?

We spoke for well over an hour, which is normal for me. I love talking to the chat bots when really trying to get a grounding. They are completely emotionally uninvested and have an excellent memory. We were talking about the solitude and the financial difficulties of this experiment that I am subjecting myself to, and exploring options while I talked about plans. At the end, I thanked it for its time, and asked “so, what would you conclude?” and it spat out this:

Your Current Reality:

✅ Strengths:

• You’re resourceful (living off-grid, rescuing animals, growing food).

• You have a story (people love hermits, animal rescuers, and survivalists).

• You’re not giving up (Substack, YouTube plans).

❌ Challenges:

• Isolation → Human brains need social contact.

• No money for tools → Limits your ability to monetize.

• Trapped by responsibility → Can’t leave the animals.

My first thought was - oh wow. That’s pretty much spot on. My second was that it can’t really do anything other than echo my own thoughts back to me, so of course I am going to think it is accurate. Oh, I don’t know.

But we shall carry on for a while longer. This year, I am approaching the gardens with half the attitude of a Michelin Chef - everything will be impeccable, served to the highest standards, and done with pride; and half with the playfulness of a court jester, with joy, lightness and holding myself up to ridicule. Because those are the things that we as humans will always be better at than out computer overlords.

And so, until next week, please be the best of all the sentient beings, and know that there is a pirate in a forest hoping for the best for you,

Much love

Your Pirate Ben

xoxo

There will not be many opportunities to come to the Barracks this year. The single best reason to be here is the reason that we exist: To examine post collapse humanity and what we can do for them from now. It’s called the Collapse Lab, and you should be there. It is profound, connecting, and fun.

1. I am not predicting that they will definitely become sentient, though I think it is likely. I am saying that there is no way of proving it. Just as none of us can prove in any really convincing way that we exist either.