This is a linkpost for https://www.lesswrong.com/s/5Eg2urmQjA4ZNcezy/p/rzqACeBGycZtqCfaX
OR: Big Timelines Crux Operationalized
What fun things could one build with +12 orders of magnitude of compute? By ‘fun’ I mean ‘powerful.’ This hypothetical is highly relevant to AI timelines, for reasons I’ll explain later.
Cool angle and thought experiment, makes this all a bit more concete. My timelines for transformative AI already were close to what you're handwaving at, but I'm really happy to see more thought in this direction and hope that this inspires more people to take soon-ish tansformative AI even more seriously.
From my inside view it feels pretty unsettling to think about the changes I expect to happen in the coming decade or two. I wonder if you think the EA community is too slow to update their strategies here. It feels like what is coming is easily among the most difficult things humanity ever has to get right and we could be doing much more if we all took current TAI forecasts more into account.
You guessed it -- I believe that most of EA's best and brightest will end up having approximately zero impact (compared to what they could have had) because they are planning for business-as-usual. The twenties are going to take a lot of people by surprise, I think. Hopefully EAs working their way up the academic hierarchy will at least be able to redirect prestige/status towards those who have been building up expertise in AI safety and AI governance, when the time comes.