What Do the Frontier Labs Actually Believe?

I listened to a recent episode of Cal Newport's "Deep Questions" podcast, where he interviews Emile P. Torres. Cal says that if you want to understand what is happening in AI, you have to understand the futurist ideologies that shape the people running the frontier labs. Torres is a researcher who studies eschatological and utopian movements. They spent a decade inside this movement as a true believer, published in it, worked as a research assistant to Ray Kurzweil, did a stint at Cambridge's Centre for the Study of Existential Risk, and then defected in 2019. TESCREAL Torres coined " TESCREAL " with the computer scientist Timnit Gebru around late 2022. They were trying to name the seven ideological drivers behind the race to build artificial superintelligence : Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism. This is also roughly the chronological order these ideologies emerged over the last 40 years. Transhumanism is the backbone and the rest are mostly variants with different emphases. It advocates using technology to push humans past our biological limits: longer lives, better cognition. Torres's claim is that transhumanism is the water Silicon Valley swims in: most people there don't call themselves transhumanists for the same reason Torres has never publicly declared themselves a round-Earther. Cosmism widens the scope: why stop at re-engineering the human organism when you could re-engineer entire galaxies, maybe intervene on the fabric of spacetime itself. Rationalism was founded by Eliezer Yudkowsky, a transhumanist, extropian, and singularitarian, on the thesis that to build these technologies without being destroyed by them, we first need to optimize our rationality. Effective altruism emerged from the rationalist blogs (Overcoming Bias, LessWrong) and swapped the target: optimize our morality rather than our rationality. One branch of this is "earning to give": take the highest-paying job you can so you can donate the most. Sam Bankman-Fried is the cautionary example. He ran FTX as an earning-to-give project: if the math says the expected good of the donations outweighs the harm of the means, the means stop mattering. Longtermism is cosmism with an explicit ethical foundation, namely total utilitarianism: we are obligated to maximize the total amount of value in the universe across cosmic space and time. That last one is important. Cosmism says a cosmic post-human future would be …