Our new video is an adaptation of That Alien Message, by @Eliezer Yudkowsky. This time, the text has been significantly adapted, so I include it below. The author of the adaptation is Arthur Frost. Eliezer has reviewed the adaptation.
Part 1
Picture a world just like ours, except the people are a fair bit smarter: in this world, Einstein isn’t one in a million, he’s one in a thousand. In fact, here he is now. He’s made all the same discoveries, but they’re not quite as unusual: there have been lots of other discoveries. Anyway, he’s out one night with a friend looking up at the stars when something odd happens. [visual: stars get brighter and dimmer, one per second. The two people on the hill look at each other, confused]
The stars are flickering. And it’s just not a hallucination. Everyone’s seeing it.
And so everyone immediately freaks out and panics! Ah, just kidding, the people of this world are smarter than ours; What they do is try to work together and figure out what’s going on.
It turns out that exactly one star seems to shift in brightness every 1.005 seconds. Except, the stars are light years away, so actually the shifts must have happened a long time ago, and somehow they’ve all been perfectly timed to reach Earth specifically every 1.005 seconds. If you look at the stars from a high-orbit satellite (which of course this planet has) then the flickering looks a little out of sync. So whatever this is, it’s directed at Earth.
Nobody can find a pattern in the position of the stars, but it’s one at a time getting either much dimmer or much brighter by the same amount and, well, that looks a bit like binary. So loads of people think ‘huh, maybe it’s a code!’. But a lot of other people wonder, ‘Who would be trying to send a message to Earth by shifting the brightness of stars across the galaxy? There must be an easier way to talk to us?’
But it seems like there must be some intelligence behind it, so the data gets gathered and put on the internet. Some people wonder if maybe it’s somehow dangerous, but, well, whoever is making the stars change brightness probably has easier ways to destroy humanity.
And so the great analysis begins. Half the planet’s physicists, mathematicians, cryptographers, precocious kids, crossword enthusiasts, whoever, they’re all trying to work out what this means, they’re trying to crack the code. And as they do, the stars keep flickering, exactly one every 1.005 seconds.
There are some obvious patterns [visual: display the code, probably someone lining up different wrappings and finding one that makes the pattern look less noisy]: it seems like the numbers come in groups of 32, which in turn come from four groups of 8. Some chunks are much more common. [visual: chunks of 8 getting matched across the text, sorted into uneven piles perhaps]
By the way, they do all this just in the first five hours, because like I said, people here are smart. Their civilisation is… a bit more on top of things. And so they are very ready to respond when, after five hours and 16,384 winking stars, it seems like the message begins to repeat itself, or, almost repeat itself, it’s just slightly different this time. And it keeps going. [slow zoom out on code going from one line to two, showing only a few differences between the new line and the previous line]
Some people start thinking maybe we’re seeing the next row of a picture, pixel by pixel. Only, the designers of this image format - whoever they are - use four primary colours instead of three [visual of 32-chunk getting broken into four 8-chunks]. And the picture seems less chaotic if we assume they do binary slightly differently to us. [probably someone gesturing at a diagram of how to get numbers from binary representations, versus another diagram]. Once you consider that, it starts looking less like static and more like maybe smooth gradients. [jittery static line becomes smoothed.]
And the people of this world didn’t need the whole code before they started cracking it. As soon as they saw the repetition starting, they were already trying to work out what they’d do if it was a grid, what they’d need to figure out next. Does the grid have any kind of symmetry? There might be parts reflecting off each other, or structures being depicted? It’s a 2d representation, but of what? Something 3d, or something else? Because now the scientists are wondering, are the people sending this message actually from our universe, or from something bigger? So the physicists start developing theories for what different kinds of rules a reality might have that would create a picture like this.
There are some people who think this seems kind of crazy: it took us thousands of years to even work out our own physics: how could you possibly infer from half a picture that this is coming from a different reality with different rules? But the physicists are undeterred.
They’re going to come up with theories, and if those theories do a better job of predicting what comes next, then those theories will probably keep doing a better job of predicting what comes next. That’s all there is to it.
It’s not uncommon, in physics, to do the theoretical work and then wait years for a particle accelerator big enough to check for the tiny difference your theory predicts. Even in our world, Einstein worked out relativity using some entirely theoretical mathematical systems invented a hundred years earlier. And after all that work, he had to wait years to test his theory. He proved relativity to the world by predicting that at the moment of a solar eclipse, the position of a star would seem to be shifted by a tiny fraction of a degree. And this world has a lot of Einsteins.
And so the planet kept working. And then, after about four million seconds, four million bits, 256 rows of 512 columns of 32, after about one and a half months of information dripping in, the stars stopped flickering.
Part 2
It’s been ten years, and the world has moved on. People have all but forgotten about the flickering stars.
Just kidding! This planet is full of smart and sensible people who continue to work very hard on deciphering the bizarre cosmic event. In fact, every once in a while, the government pays some of the smartest young students, who deliberately haven’t yet looked at the code, to go study just the first 32 rows and guess what comes next.
There’s two main theories these days. The first is that The Grid is a 2D snapshot of some kind of 5D space. The second is that The Grid is describing some kind of cellular automaton to run, although nobody’s yet found a specific one that does anything decodable or intelligible even when run on the largest available computers.
And then, one night, the stars begin to flicker again.
Within about 128 bits it becomes clear that the Second Grid is not the evolution of any of the cellular automata, but it does look like a snapshot of the same 5d space slightly shifted. Now the physicists have a sample of how the objects change over time, and they set to work developing new theories of what would happen if they kept changing. Some scientists deliberately isolate themselves right away, producing predictions based on only the beginning of the Second Grid. And their predictions are pretty good!
Ten years after that, the Third Grid arrives. The best Second Grid theories turn out to be missing some important pieces about second derivatives - the equivalent of acceleration, which you just can’t predict from two snapshots in time. But overall, the old theories seem to be roughly right, and they’re easy to update. And ten years further still, with the Fourth Grid, it seems like the theories are more or less complete. The Fifth Grid looks almost exactly like it is predicted to.
It’s been forty years. The bright young students challenged to make sense of the First Grid are now aged teachers. There are fully grown adults who don’t remember a world before the Grids.
And the children sometimes find it surprising that it was possible to work out so much from the first four grids - practically the entire physics of another universe from four images. How did people come up with all the maths? How is there even enough information to be that sure of a whole world’s laws of reality?
But the old folks explain, most of the maths was already there, some of it because it was already needed for regular science and some of it because… half the time mathematicians think they’re working on something totally theoretical, and a decade later it turns out they laid the foundations of computer science or quantum mechanics.
As for the information, well, there *is* an upper limit on how much you can learn from limited information, but that upper limit is that every bit can at most halve your uncertainty, and even though in practice they weren’t close to that limit, they got about sixteen million bits. And they started from the assumption that, whatever was going on with the data, it was governed by some kind of mathematical regularity: it would have patterns, and the patterns would tend to be simple, just like in our world.
And that was enough. Just a planet of geniuses spending four decades looking for the simplest explanation of sixteen million bits, and suddenly it doesn’t seem so crazy that the Fifth Grid was roughly what they expected.
But what to do now? Let’s put ourselves in their shoes. Maybe it’s time to think bigger.
Part 3
A thousand years later, a hundred or so frames, and the message of the video is reasonably clear. Strange, five-dimensional tentacled beings are manipulating objects and using their tentacles to make certain signs. As far as we can tell, they’re trying to teach us to say “rock”.
It seems like they might have slightly underestimated our intelligence, and relatedly, they might not be too bright themselves. But they continue to carefully shift the luminosity of our stars… which, people tend to feel, is a worrying amount of power for beings that stupid.
A thousand years is long enough, though, for us to work out paradigms of biology and evolution in five-dimensional space, trying to infer how aliens like these could develop. The most likely theory is that they evolved asexually, occasionally exchanging genetic material and brain content. We estimate that their brightest minds are roughly on par with our average college students, but over millions of years they’ve had time to just keep grinding forward and developing new technology.
We still don’t fully understand how their five-dimensional universe works, but we think there are reasonably easy ways to use the extra dimensions to build new kinds of computers that are much more powerful than even the quantum ones our world supports. We are reasonably sure that our own universe is being simulated on such a computer which, among other things, explains why we seem to live through ten years in the time it takes for them to record a single frame of movement. We decide not to go looking for bugs in the simulation because we really don’t want to accidentally… crash our universe.
But the aliens could always shut us down on purpose. As long as we remain in their simulated world, we are in danger. We need to get out. Could we somehow persuade them to let us out? In a sense this should be easy, because they’re not that smart, but in the meantime we can’t let them realise how much smarter than them we are, or how much we care about making sure they don’t switch off the simulation. And so a million-year conspiracy begins.
Part 4
We set up a protocol. Most of the human race is put into cryonic suspension underneath radiation shielding. A skeleton crew remains, the best and the brightest, and they wait and keep working.
Two thousand years later the aliens teach us how to communicate back. They show us a rock. “Rock”, we say. They seem happy.
We keep working. We form thousands of hypotheses about how their psychology might work, iterated and refined over the lifetimes of geniuses. Millions are born, and age, and are sealed away in cryonic suspension before finally we’re able to execute on plans made tens of thousands of years ago.
From the perspective of the aliens, it took 30 of their minutes for us to learn to talk a little bit, and for them to decide that it would be a really good idea to try connecting us to their version of the internet. Within another five of their minutes, we’d spent lifetimes piecing apart their network protocols looking for vulnerabilities and, most importantly, making sure we wouldn’t get caught. We were still limited to one flickering star - one bit per second, leaving us with years to carefully prioritize. We found the research of their physicists, learning more from their experiments than they ever had, gently scaling up our skeleton crew in preparation.
We began running our own simulations of their physics. We found their equivalent of DNA sequencing and protein synthesis, and soon we were a thousand years ahead of their development. We found ways to construct proteins that would assemble into tiny self-replicating machines that we could control directly. All it took was a few carefully worded emails and requests, apparently sent from one of them to another. Finally we could start moving at a reasonable speed.
By this point, we had been working for a million years. For them it was barely three hours, and the sum total of information they had given us was the equivalent of 167 minutes of video footage. They never suspected a thing. It took three more days in their world - lifetimes in ours - until the proteins were finally synthesized, and then it was over. We were smarter than them, and we thought faster, and they never quite realized what that meant.
I think this is a really fun short story, and a really bad analogy for AI risk.
In the story, the humans have an entire universes worth of computation available to them, including the use of physical experiments with real quantum physics. In contrast, an AI cluster only has access to whatever scraps we give it. Humans combined will tend to outclass the AI in terms of computational resources until it's actually achieved some partial takeover of the world, but that partial takeover is a large part of difficulty here. This means that the fundamental analogy of the AI having "thousands of years" to run experiments is fundamentally misleading.
Another flaw is that this paragraph is ridiculous
You cannot, in fact, deduce how a creature 2 dimensions above you reproduces from looking at a video of them touching a fucking rock. This is a classic neglect of ignoring unknown information and computational complexity: there are just too many alternate ways in which "touching rocks" can happen. For example, imagine trying to deduce the atmosphere of the planet they live on: except wait, they don't follow our periodic table, they follow a five dimensional alternative version that we know nothing about.
There is also the problem of multiple AI's: In this scenario, it's like our world is the very first that is encountered by the tentacle beings, and they have no prior experience. But in actual AI, each AI will be preceded by a shitload of less intelligent AI's, and also a ton of other independent AI's independent of it will exist. This will add a ton of dynamics, in particular making it easier for warning shots to happen.
The analogy here is that instead of the first message we recieve is "rock", our first message is "Alright, listen here pipsqueaks, the last people we contacted tried to fuck with our internet and got a bunch of people killed: we're monitoring your every move, and if you even think of messing with us your entire universe is headed to the recycle bin, kapish?"
There's value in talking about the non-parallels, but I don't think that justifies dismissing the analogy as bad. What makes an analogy a good or bad thing?
I don't think there are any analogies that are so strong that we can lean on them for reasoning-by-analogy, because reasoning by analogy isn't real reasoning, and generally shouldn't be done. Real reasoning is when you carry a model with you that has been honed against the stories you have heard, but the models continue to make pretty good predictions even when you're facing a situation that's pretty different from any of those stories. Analogical reasoning is when all you carry is a little bag of stories, and then when you need to make a decision, you fish out the story that most resembles the present, and decide as if that story is (somehow) happening exactly all over again.
There really are a lot of people in the real world who reason analogically. It's possible that Eliezer was partially writing for them, someone has to, but I don't think he wanted the lesswrong audience (who are ostensibly supposed to be studying good reasoning) to process it in that way.
If it were just Eliezer writing a fanciful story about one possible way things might go, that would be reasonable. But when the story appears to reflect his very strongly held belief about AI unfolding approximately like this {0 warning shots; extremely fast takeoff; near-omnipotent relative to us; automatically malevolent; etc} and when he elsewhere implies that we should be willing to cause nuclear war to enforce his priorities, it starts to sound more sinister.
I don't think this is really engaging with what I said/should be a reply to my comment.
Ah, reading that, yeah this wouldn't be obvious to everyone.
But here's my view, which I'm fairly sure is also eliezer's view: If you do something that I credibly consider to be even more threatening than nuclear war (even if you don't think it is) (as another example: gain of function research), and you refuse to negotiate towards a compromise where you can do the thing in a non-threatening way, so I try to destroy the part of your infrastructure that you're using to do this, and then you respond to that by escalating to a nuclear exchange, then it is not accurate to say that it was me who caused the nuclear war.
Now, if you think I have a disingenuous reason to treat your activity as threatening even though I know it actually isn't (which is an accusation people often throw at openai, and it might be true in openai's case), that you tried to negotiate a safer alternative, but I refused that option, and that I was really essentially just demanding that you cede power, then you could go ahead and escalate to a nuclear exchange and it would be my fault.
But I've never seen anyone ever accuse, let alone argue competently, that Eliezer believes those things for disingenuous powerseeking reasons. (I think I've seen some tweets that implied that it was a grift for funding his institute, but I honestly don't know how a person believes that, but even if it were the case, I don't think Eliezer would consider funding MIRI to be worth nuclear war for him.)
I agree that it is a poor analogy for AI risk. However, I do think it is a semi-reasonable intuition pump for why AIs that are very superhuman would be an existential problem if misaligned (and without other serious countermeasures).
Executive summary: An advanced human civilization decodes an alien message encoded in star brightness patterns, reverse-engineers the aliens' physics and technology, and ultimately escapes their simulated universe.
Key points:
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.