Alex Kontorovich: I’m constantly talking to AI systems, and I’m learning a lot of math a lot faster than I’d be able to. Because the way math is written, you know, I sometimes give a paper to a grad student to read, and they come back a week later, and I say, “So how far did you get?” “Well, I didn’t really get past the first page, to be honest.” “Oh, really? Tell me.” “Uh, well, actually, I didn’t get past the first paragraph.” I say, “Okay, fine. Why not?” “Well, the first sentence is by the perturbation theory of non-local systems and I don’t know what any of that means, so that’s kinda where I’m stuck.” I’m like, “Okay. You don’t know how to read a math paper. You don’t– You skip the first couple of paragraphs. Go straight to, you know, the Lemma 3.” “And that’s where you can start working and work your way backwards to what the thing is doing.”
But in the future, there is no “I don’t know what this means,” ‘cause you just upload it to an LLM, and you start asking it questions, and it will zero in on the zone of proximal development, the place where here’s everything that you know, and here’s the next thing you need to hear so that you start learning. And if people use that effectively, they can really supercharge their education. I don’t just mean kids. I mean old fogies like me- … can learn new fields. You know, to learn a field, like this is what our graduate students are going through. They have to completely rewire their brains. Like, learning math is still hard.
Up until past the master’s level into the PhD level, you are not expected to produce new mathematical theories. Everything that they can do is something that is in a textbook somewhere. And yet it’s still extraordinarily, extraordinarily difficult for people to- … Learn to solve problems that are solvable by lots of other people, never mind problems that aren’t solved. So learning math is hard. Learning new math is even harder, discovering math, whatever, working out solutions to math problems. But I think it’s going to get easier, and this tool is gonna be an amazing resource for people to be able to get to that zone of proximal development much more rapidly and develop their understanding much more rapidly.
Lex Fridman: Yeah, I’ve been having so much fun on topics I’m not even… And I would love to go back to grad school with the tools we have now. But one of the ways my brain likes to learn is constructing visual examples, toy examples, the simplest possible example that still contains the insight there, and especially if it’s interactive. Like, if I can, like, move stuff, it’s like, oh, you immediately, like, get it. All these words that the community agreed mean something, you’re supposed to know all the notation, all that kind of stuff. If I can just know that this little symbol means this, oh, okay, cool, all right. And just everything kind of falls into place.
Also, you fall in love with the subject, and I feel like I can take on any problem that’s completely outside of my field. Like, I can understand, like, protein folding. I can, like, play with the parameters and play. I can take a DeepMind paper and actually understand by taking any aspects of it and playing with it, and what’s a simple example? Ask crazy aforementioned pothead questions. So yeah, this is super fun. The question is whether people will be motivated to learn. This is the problem, is the problem of human motivation. That’s the real question.
Steven Brunton: Well, the key ingredients there are curiosity, asking good questions, and being able to call bullshit. Like the- … critical advisor skills of, like, getting information where you’re not the expert, and knowing if you need to dig deeper or push back. And I think, you know, there will always be people who are curious and not curious, and who ask good questions and don’t. We need to be really promoting that more than we ever have. It’s a little scary to me that we’re expecting our undergraduates, our young students, to interact with these tools when they’re learning, when their brains aren’t fully formed. We’re asking them to have critical, not just critical reasoning skills themselves, but, like, advising skills. How do you take in information and tell if it’s true or not, or how to question it? And that’s a huge task for us in education, is how do we- … train people to have good advisor skills when they’re kids?
Lex Fridman: Yeah, I think the hardest problem isn’t undergrad. The hardest problem is, like, middle school ‘cause, like, that, I… What is the point at which homework starts getting hard? That, at that point, why do the homework?
Steven Brunton: There’s the temptation, right?
Lex Fridman: That thing will just… At first, I’m saying, see, I’ve come to appreciate the value of doing difficult things because of all the suffering, but I think because I was forced to. But if you’re not forced to do it because LLM can, AI can just help you, I don’t know what that, what kind of society that creates.
Alex Kontorovich: But math has the longest head start on that question because calculators came out, what, 70 years ago? And we still ask kids to learn their times tables. At least some of us do, and those who said, “No, kids don’t need to learn times tables,” you know, to the extent that teachers listened to that and kids listened to that and didn’t learn- … they really suffered. There’s a great, great amount of human suffering resulting from people saying you don’t need to know times tables ‘cause calculators can do it. And my hope is that we can learn from that and say just because a calculator can do it, you’re gonna be much better off if you can also do it yourself. Doesn’t mean if I have to do some calculation, obviously I’m reaching for a calculator.
But I could in, in principle do it myself, and that’s what allows every time you give yourself that skill and master a skill. Kids love mastering skills. How happy are they when they learn to tie their shoelaces for the first time? You would think it’s like, “It’s just shoelaces. Who cares?” No, it’s the greatest thing ever. You wanna celebrate every little bit of independence and you want to be able to think. You wanna be independent from the machines. Just ‘cause the machine can do something doesn’t mean you don’t wanna have the capacity to do it yourself.
From Lex Fridman Podcast #503 (video, 3:11:34–3:17:45); text from the official transcript.
This passage is very meta and very illuminating. The metacognitive way of learning they describe is itself worth learning, and what they say about the changes the AI era brings is also valuable.