Takeaways from the podcast: Terence Tao – How the world’s top mathematician uses AI
I just came across this new interview of Terence Tao on the Dwarkesh Podcast, and it’s well worth sharing. Much to my relief, the host is not annoying. (I’ve recently seen quite a few hosts who struggle to ask good questions or guide interviews toward more insightful responses...)
Back on track, since I also use AI to facilitate my research (exploring new ideas then exploit and examine them myself, as well as handling secondary tasks like running experiments and plotting), I can resonate with Terence’s observations and thoughts.
In fact, I’ve been wondering whether the human thought-generating process is inherently stochastic. If so, do there exist some (perhaps transient) random “seeds” that influence how our thoughts are generated? Perhaps learning—whether human or machine—can reshape these seeds, gradually calibrating our intuition to better reflect the underlying principles (if exist) of the real world.
Here are some of my personal takeaways:
- Analogy of AI doing research
"These AI tools, they’re like jumping machines that can jump two meters in the air, higher than any human. Sometimes they jump in the wrong direction, and sometimes they crash, but sometimes they can reach the tops of the lowest walls that we couldn’t reach before.
(...)
But it’s a different style of doing mathematics. Normally we would hill climb, make little markers, and try to identify partial things. These tools either succeed or they fail. They’ve been really bad at creating partial progress or identifying intermediate stages that you should focus on first." - Elegance
"Often progress has to be made not by adding more theories, but by deleting some assumptions that you have in your mind." - Sandbox (like AlphaGo Zero)
"If we had access to a million alien civilizations, each with a different development of history and science in different orders, then maybe we’d actually have a decent shot at understanding how we measure what progress is and what is a good strategy. We could maybe start formalizing it and actually having a framework. Maybe what we need to do is start creating lots of mini-universes or simulations of AI solving very basic problems in arithmetic or whatever, but coming up with their own strategies for doing these things and having these little laboratories to test. There are people who investigate what’s the smallest neural network that can do 10-digit multiplication and things like that. I think we could learn a lot just from evolving small AIs on simple problems." - Serendipity
"When I was a grad student, I would go to the library to look for a journal article. You had to physically check out the journal and read the article. You could browse through and sometimes the next article was also interesting. Sometimes it wasn’t, but you could accidentally find interesting things. That has basically been lost now. If you want to access an article, you just type it into a search engine or an AI, and you get exactly what you want instantly. But you don’t get the accidental things you might have found if you’d done it more inefficiently.
(...)
You actually do need a certain level of distraction in your life. It adds enough randomness and high temperature. I don’t know the optimal way to schedule my life. It just seems to work."
Here's the link to this podcast: https://www.youtube.com/watch?v=Q8Fkpi18QXU