This is a nice writeup and summary.
I personally think that this is yet more evidence that formal control is a path which is more promising than others. If you can formally prove that your code, when properly executed, has certain properties then that gives you some hope that those properties will be durable during and after a hard left turn.
Things like, if you had a magic wand, formally proving that any AI designed by a formally controlled AI will also be formally controlled. That way even if it whooshes and completely redesigns itself there is still some hope.
I would love to see the amount of resources going into formal methods be multiplied by 10x or 100x, I think if we built a really solid field, where all of modern mathematics and computer science is formalised and people write formally verified code by default because it's safer and there are good libraries to do that, then in that environment the control problem becomes easier, if still extremely hard.
I've been thinking quite a lot about the problems of dating apps and had some ideas and would be interested in talking about it.
I'd suggest a dating site that improves these things in the following ways.
I feel like this could lead to an experience which is quite heavily managed and structured which is great for people who maybe aren't so confident and aren't sure how to go forward. I think it would be great if one of the main goals was teaching people the skills of building great relationships rather than just assuming that if you connect people that's enough for it to all work out on it's own.
Yeah I think that's a good point. I mean I could see how you could send a civilisation the blueprints for atomic weapons and hope that they wipe themselves out or something, that would be very feasible.
I guess I'm a bit more skeptical when it comes to AI. I mean it's hard to get code to run and it has to be tailored to the hardware. And if you were going to teach them enough information to build advanced AIs I think there'd be a lot of uncertainty about what they'd end up making, I mean there'd be bugs in the code for sure.
It's an interesting argument though and I can really see your perspective on it.
Nice writeup.
"The primary theory is that alien civilizations could continuously broadcast a highly optimized message intended to hijack or destroy any other civilizations unlucky enough to tune in."
One question I have is whether this is possible and how difficult it is?
I mean if I took a professional programmer and told them to write a message which would hijack another computer when sent to it isn't that extremely hard to do? I mean even if you already know about human programming you have no idea what system that machine is running or how it is structured.
Isn't then that problem like 10,000x harder when dealing with a totally different species and their computer equipment? The sender of the message would have no idea what computational structures they were using and absolutely no idea of the syntax or coding conventions? I mean even using binary rather than trinary or decimal is a choice?
Yeah I think there is some similarity there. I think I was assuming that the matches are made algorithmically, it's just that they are made at a sustainable pace rather than in a flood at the beginning.
Thanks for the comment :)