The fact that few students are self motivated enough to use it makes sense....but you are telling me that, in 4 years, _so_ few were motivated to use it that you can't report on whether or not it makes a difference for the minority that do?
I was among those who, when Khanmigo was first announced, were pretty excited about it's potential. I then waited for data on the results....and kept waiting.....and kept waiting. And now four years later this is apparently what we are going to get. I think that this is enough for me to decide that Khanmigo, regardless of whether or not a student actually engages with it, doesn't make much learning difference. At some point, the absence of (reported) data becomes data in itself.
I still believe, in principle, that AI tutors could be massively helpful for learning. But apparently we haven't yet figured out how to take that principle and turn it into reality.
I think the AI can not understand the situation of the student so it does not know what the student does not know. Therefore it can not guide the student through the main hurdles of learning and understanding a topic. Whereas a human tutor was once a human-student, AI never was.
I mainly use AI for learning things these days. The biggest bottleneck is always providing the machine with your context in sufficient detail that it can understand how to help you. When learning a topic it isn't always clear cut on how to do that, as you're likely missing much of the vocabulary necessary to get the AI to give you the answer you want.
AI is perfectly capable of teaching you quantum mechanics if you understand music theory. However, unless you have a full understanding of music theory, you'll need to explain to the machine what you know, and that takes trial and error that most students won't bother with.
Until he shows a comparative study, and also any disaster information like mental health effects, then his entire claims three years in should be treated as bullshit.
Honest question: how many of you tech bros have used this platform with your own children? If you won’t dog food it, quit claiming it’ll help the disadvantaged. Please.
On one hand I will grow old knowing I'll always have a job because a lot of kids never will have researched anything in their lives and won't know how to deal with anything an LLM can't solve. On the other hand between this and most kids having had a 2 year covid gap in their learning, who the heck is going to pay my retirement and be my doctor when I'm old?
When I was young 20-30 years ago older people were saying that the Internet would make us dumb. Why learn anything when information is always readily available one search request away? Videogames were supposed to make me a blood-thirsty maniac, and don't even get me started on readily available porn. "Kids these days are lazy and don't want to learn" is one of the oldest memes in human history, with documented use going almost as far back as writing itself.
The flipside of that take is that if you listened to technologists, then educational TV/CD-ROMs/laptops/the internet/tablets/educational games/digital blackboards/MOOCs/etc. were going to completely revolutionize education - but looking at the evidence, it doesn't seem like students have gained much at all from any of it.
I remember an educator ranting to me a long time ago that the only data-proven ways to meaningfully improve educational outcomes was to reduce classroom size and make sure kids got enough sleep + fed well enough, everything else was just a waste of time.
From all my years of schooling one of the biggest factors is the combination of level of interest from the student, with parent involvement following that, once you cross the basic threshold of stable home with regular sleep and food. Some kids don't care and even perfect parents won't matter, but disinterested parents also drag a bunch of kids down.
There has been a measurable and noticeable drop in attainment starting with smartphones entering the classroom, supercharged by COVID chaos, and finally with AI cheating being just the latest assault on learning.
Ask teachers that have been teaching for 10 years. Ask the professors how today's kids are different than the ones of yesteryear.
The move to de-tech the classroom will eventually help out I expect, but keeping kids (and adults!!!) from using cognitive shortcuts so they can develop their own sense of what's reasonable instead of taking information from a bought-and-paid-for oracle is going to remain a problem.
I don't even think Khan Academy's original teaching revolution quite panned out.
I still remember when Khan Academy first came out, there was talk that teachers would go obsolete because teaching would become centralized and delivered over video.
Khan Academy to me is still just a YouTube channel trying very hard to be something more.
Well it wasn't really a teaching revolution. It was a marketing job around a YouTube channel that purported to be a teaching revolution.
The thing is people want more than material. They want the material to be accredited and examined. Otherwise there is no demonstrable credibility from doing it.
And there's a whole world out there of higher quality material with has that accreditation and examination structure around it. And it existed, sometimes for decades in the case of The Open University, before Khan Academy appeared. But it costs money.
Indeed, reverse classroom, everybody getting access to high quality content, video then interaction, learning paths, etc.
Well, in practice it's still about the amount of time a pupil does train with the right oversight and that is precisely the bottleneck that hasn't been alleviated.
Many of the ed-tech products that are introduced in the classroom are solutions looking for a problem. Not many have much understanding of the context of teaching and learning processes. And yet, as history of ed-tech tells us, gung-ho business optimism will sell these to managers/administrators as THE solution. The most important stakeholders in the process the teachers an students, who are end-users of the said product/technology, are left out in this decision chain.
amazing in theory with the perfect user in the perfect use case,
misused in practice with terrible consequences for society at large.
Sure the one student who already excels, is motivated, understand what the concept to learn is, that actually completing exercises helps them to learn might, possibly, thrive. All other students, the vast majority, will try to "game" the (terrible) evaluation system to get good grades by cheating WHILE avoiding the very challenge that make the learning possible.
I think it's particularly telling that the teacher dropped the product and the superintendent is saying it's going well. Maybe I'm biased from going to New Orleans public schools, but I have my doubts about how tapped in the superintendent is of the overall strategy.
That said I do think it's particularly hilarious that KA's strategy to students not wanting to use the product is to make the product more integral to the experience.
Weird. No mention of the technical aspects, essentially just blaming average students for not being engaged enough with their simplistic ChatGPT clone. No wonder they have not dared yet to give out actual usage metrics. If I was given the choice between an inferior product that probably lags significantly behind on all features and one of the standard offerings from OpenAI, Google or Anthropic, I'd question why I should use this thing too. According to their website, they position Khanmigo like this:
>Unlike other AI tools such as ChatGPT, Khanmigo doesn’t just give answers. Instead, with limitless patience, it guides learners to find the answer themselves. In addition, Khanmigo is the only AI tool that is incorporated with Khan Academy’s world-class content library that covers math, humanities, coding, social studies, and more.
The first differentiation is literally just prompting (if at all). Nowadays you can tell any chatbot to behave that way. The second one may have been an edge before tool use was widely common, but with all chatbots now having access to the internet and code execution, it seems like this has also become a dud. This product was a nice idea on paper, but the fast technical evolution of the field has largely left it in the dust.
I mean, isn't the whole khan academy approach "we know better how to teach everything"? It's not surprising that they'd think they have more enlightened prompts than anyone else.
They had really cool math videos and got given too much money, that's about the story.
The wish for an _AI revolution_ in learning seems to have been granted by a monkey's paw. Articles like this, or [0], or browsing r/teachers [1], or even talking close-ones in college, give a rather grim view of AI use.
A para from from [0] makes it seem that students understand that LLM use doesn't lead to learning, but still do so. Do they not see effort put into learning worthwhile?
A few months ago, I overheard some college students talking about their classes.
One was complaining about an assignment they needed to do that night, and
another incredulously asked why they wouldn’t just have ChatGPT do it. The first
replied, “This is my major, I actually need to learn stuff in this class. I use
AI for my other classes.”
I myself use LLMs for learning (using ChatGPT's study mode for instance r.i.p)
and can see that there's a right way to use it—you reach for it when you hit a wall, not to avoid the friction of developing an understanding.
From what I understand tho, most of LLM use for learning is just LLM used as a tool for cheating. Even tfa mentions something of the sort:
few of Musall’s most advanced students have taken advantage of AI to learn new
topics. But, as far as she can tell, more students are using it to just find
answers
The article attributes _skill issue_ as part of the problem, but how much of that
is a motivation or awareness issue. How do you make student realize that learning is worth it?
I don't even think it is a monkey's paw. That implies that it looked like it would solve a big problem in our initial estimation.
But "students will use the cheating machine to cheat" was obvious from the release of ChatGPT3. There was never some period of time where AI looked like it was a net positive for students only to be revealed to have an unexpected harm.
Even from the folks who claim to use LLMs to learn rather than cheat or avoid work, I've seen so many people admit that they are actually using it to harm themselves. "Oh, I only ask ChatGPT for the answer for really hard problems." Yeah man, doing the hard problems is how you learn.
You can be very AI-skeptic in various ways and still think that this is a fair take.
I teach and supervise students as master's level courses, and about 15% of them have intrinsic motivation to learn. These students have set up their own AI tutors with prompts and know way more than me in certain areas of the field, they are extremely ahead of their class.
The issue in my country is that you equate education with getting a safe job. 20 years ago, you needed a high-school degree in social science to get a government job. 10 years ago you needed a bachelor in social sciences to get the same job. 5 years ago you needed a bachelor in economy/engineering to get the same job. Now, because of recessions this is stretching to masters degrees.
You can't expect people who just want a job and a comfortable life and NEED to go to uni for this to want to be curious and want to learn.
And on other side education attainment has become metric for governments. More degrees and higher the degrees are better it will be for the economy somehow. Where there is likely quite a lot of jobs that don't actually need the degree.
> about 15% of them have intrinsic motivation to learn [...] they are extremely ahead of their class.
Feels like whatever tool they'd be given, they'd be ahead anyway. What's more worrying IMHO is, are the remaining 85% faring even worst than they would have before because they are learning even less, not just slower than the 15% learning faster. Namely is the gain for the few a loss for the majority?
You are right on both counts. I do think that it's however different on the first concept. Before, they would be ahead but still capped by their university. If you come from a uni in a 100k person city, you probably would not have the material nor the best teachers. Now you can have literal Stanford quality education (by accessing Stanford's open source lectures) as well as the collective aggregated knowledge of humanity in the chat interface. The curiosity/intrinsic motivation is the only limit except for perhaps compute.
As for the other question, its mixed. I think about 20% of students understand that they are fucked if they just delegate it all to LLMs, they still go through the ropes and show up to class but do the minimum.
However most are down the deep end in various degrees. I have seen students with 5 different 3000-line files for 5 questions for the same lab where each file has 3 lines of code different. This never happened even when the students cheated by accessing old labs online or plagiarizing before.
I believe that what will happen (because universities move really slow on policy and education on LLM use), is that pre-LLM, the university had a normal distribution of skills upon graduation. A company could trust that someone with a degree knew X and Y. With this however, you have more of a bimodal distribution, some know nothing and some know it all, so then the trust in universities deteriorates. I think we will see much more IQ-test/practical tests in hiring processes as the trust falters for that a degree equals something.
>Kristen DiCerbo, the organization’s chief learning officer, said AI can only respond to students based on what they ask. And it turns out, she said, “Students aren’t great at asking questions well.”
Ignoring whether or not this is a good idea in the first place, what about inverting the loop? Have the robot drive the interaction.
Yeah, that's weird. I've actually been working on an AI tutor for my kids that I'm thinking about open sourcing, but it drives the conversation in new directions using a concept graph that it can poll via tool use, and find the knowledge frontier for that learner.
It's been fascinating to watch - my kids are really into Slay the Spire, and it had a discussion about a decision tree they use when fighting one of the enemies, and then it used that to bridge to writing some python code and walking them through it. Another time, with dinosaurs, it went with them through the k-pg extinction event, and what really killed the dinosaurs - the kids thought the explosion - it walked them towards the sun dimming, and why food getting more scarce filtered for small mammals, our ancestors, and smaller dinosaurs.
on one hand I wish I would be smart enough to build something like that.
on the other hand, I was playing a lot Slay the Spire few years back and I would love to talk about with my kids while they play. Going from that it is not job of the parent to explain why dinosaurs are extinct?
Ha I love talking with them about it, too, but my 7 yo’s stamina for talking about it is a lot higher than mine. But yeah, not trying to abdicate my parenting role here, just looking to get a little supplemental help, and cover things I forget to sometimes. Also a nice launching point for me to delve deeper into things with them.
How about completing the loop? Pose subject matter questions to them throughout the day, maybe via something like mobile push, collect their answers, immediately grade their results, and then actively reward them for performance.
All of the things brick and mortar schools are uniquely bad at.
> "It’s an early indication of the limits of AI to drive massive learning gains..."
I thought Sal's revolution was the idea of flipping the script on primary school learning: in-class homework & at-home video lessons.
I'm not surprised. Students are not rewarded when they ask _curious_ questions--rather, they're admonished for not paying sufficient attention.
Personally, my first use of ChatGPT was to ask tangential questions on JavaScript while taking a LinkedIn learning course on VueJS. I found ChatAI an excellent substitute for Reddit and StackOverflow, which is how I would have followed these inquiries before. Of course, I'm not a primary-school-age learner. I had to learn _How To Learn_ from experience.
I have some two million KA points, and I'm currently using it to guide me around learning basic physics - I've never used Khanmigo not even once. If I need to ask AI type questions I go off site to Claude or GPT or deepseek.
As other people have noted, asking a.k.a <i>typing</i> questions, especially math-type is fatiguing, and there's no substitute for pen and paper and thinking hard.
KA would be better off using AI on the supply side (but heavily curated) to have more assignments, or better assignments in some sections.
But it's important to recognize KA for what it is, and it's an excellent way to have some sort of a basic curriculum, especially when self-studying, and all of the instructors have great teaching personalities, as far as I can deduce from the approach in the videos.
This honestly seems like a flawed approach. Kids don't show up in the first grade, or the sixth, 9th, or really 12th, as the initiators of their educational journey.
An AI-based education system should have embedded in it "I am here to teach this person Geometry. Here is a list of the topics to cover, with a breakdown of steps for each including an intro section, a study section, a test section, and the meta material to go along with it.
Best comment in this thread. An AI-based education system as you described, would deploy the _technique_ the KA bot is built around, sparingly and at narrow times, to deliver a lesson to great effect.
The poor engagement of the KA bot becomes clear--a teaching technique in not an education system.
I agree. I just can’t understand how people who work in education seem to be incapable of learning anything about education! They couldn’t have predicted that just having the most amazing tutor in the world available all the time just wouldn’t make any difference?? Students don’t go to school because they’re eager to learn. Don’t they know this? I think an effective tutor must be human or at least a human who can use the AI tutor on behalf of the child, which means the parent. Because your job as a tutor is not just spill knowledge, it’s to keep the student engaged ( just awake may be a challenge ) and make sure they are doing what they are supposed to.
The educators keep making blunders like in this case. Hundreds of years of teaching and sorry but looks like your field is still trying to figure out the very basics of your discipline.
Might be harsh, but more I think about education more I think that the main job for many if not most kids or students in general is making them to learn. By various incentives be it punishments or rewards. Only really at highest levels is there any self-direction. And even there is plenty of external factors like not being paid anymore if you do not publish something.
Eh, working on rebuilding a Montessori school for a couple years showed me that kids are very intrinsically motivated to learn, just not always what the person who’s there to teach them wants them to learn that day. But if you enable that self drive, and gently steer by exposing them to new things at the right level, they can learn a tremendous amount.
Our kids elementary school (Germany) recently voiced concerns that pupils - especially boys - do not fulfill the self defined learning tasks in the expected time..
There may be no upper boundary to how much out of touch educators can be.
I think ai has revealed one of the biggest gaps in our education system: the majority of students don’t really care about knowledge-even when tested on it.
To give an example, I have a friend who learned system design through Claude in order to get a job interview (and he got really good at system design)while I have another friend who copies and paste ChatGPT responses in order to get a B on a reflection assignment.
This highlights that there is legit use case for personalized learning and growth via AI-but these are the people who seek knowledge with or without AI.
Whereas the majority of students actively tries to do the least as possible on assignments even if they get 0 value out of it
Right. Most people are deeply uninterested in learning anything, not just in K-12, but at pretty good colleges like mine.
Modern AI has made me a more productive teacher—-I produce higher quality material and have more time for research.
But the impact on most students is negative. It is another thing to engage with, which they won’t unless forced. The only way to learn is to do the work yourself. An AI tutor can get you unstuck faster, but that’s typically bad. Learning to be productively stuck on something for days without making visible progress is an important skill that most people never learn.
There is an LLM button on every other website now. "Chat with your lesson", "chat with your food", "chat with your photos". People are not clicking them because they are just visual noise at this point.
It was honestly a very cool gimmick few years ago, when it was new. Chatting with your photos, imagine that! ... then after few tries you're like "well... I just want the last 10 photos because I want to share a photo of my son playing piano to his grand parents" and the UX, because it's actually well craft based on feedback from users, not only has a button for just that... but even default to this.
So... yeah, it got old quick. Genuinely cool for a bit but now "we" as users just want good UX. Now give me the FAQ that I can search through then an email if it's not in there.
PS: FWIW I do believe in a long-tail fashion, for few users who are not into scripting, might not be developers (or believe they could become) it could help find very few very niche use cases with solutions.
Derek Muller of Veritasium has a presentation on why the promises of a revolution in education are never fulfilled. [0]
His hottest take is we're already close to the optimal process for learning, so technology isn't going to improve it. Learning takes work, and no technology can do the work for you.
I think the teacher's job is not just to teach but more importantly to ingnite curiosity in students on new fields / subjects. If a genuine interest / curiosity in subject is missing then no amount or medium of tutoring can help.
AI is great for the curious. But its not yet there where it can proactively engage with students to generate interest.
>Khan Academy recently announced an overhaul of its product that provides students additional academic practice. Now Khanmigo is incorporated directly as a way students can get advice as they’re working through specific problems. A spokesperson said the organization made this change because “students were not seeking out Khanmigo’s help as much as we had hoped.”
Dear Lord, how is this any different from Microsoft sticking Copilot or Google sticking Gemini in every single offering? They're literally saying that people aren't using the chat bot enough so they're going to force it on people inside the product.
Well, there's a good way and a bad way to do this. Embedded AI (vs standalone AI) can do a lot of interesting things. For example, a good teacher will watch students and identify when someone is struggling (or thriving), and carefully offer to help, vs just waiting for students to approach them and aka questions.
I'll probably open source and Show HN the AI tutor I've been working on for my kids at some point, but working on it has given me a little insight into the problem.
The biggest thing is motivation. First off, if Khanmigo requires them to type and read everything, that's going to get tiring fast for most kids. But I don't know how you could do voice in a school setting - mine uses STT/TTS, but with 20 kids in a room, it'd be chaos - STT accuracy and diarization with 2 is already really challenging.
Motivation is helped a bit by following their interest, but it seems like KA is having trouble guiding the kids when they prompt it that way. That was a pretty big issue with mine early on - the kids would talk to it for an hour about whatever topic they were interested in at the time, but it would never branch into something new.
The tutor I'm working on solves it by having a concept graph that covers a lot of learning, from the basics like math, dinosaurs, etc to other developmental topics like 6 year old boundary-pushing humor, and two LLM threads - one that handles the conversational turns, and another one in the background that strategizes and steers the conversational thread by looking at the concept graph connections and considering how ready they are for each, and then injecting steering notes into the conversational thread. Basically system 1 and system 2 thinking. And after sessions, it'll make a basic plan of where to start next time, and what might be interesting to offer up.
I mentioned this in another comment, but I've been really pleasantly surprised at the quality of the tutoring, especially when it bridges into new topics - one of my sons is really into slay the spire, and at different times it’s used that as a launching-off point into probabilities, decision trees, python code of the algorithms he thinks about as he's facing different enemies, and general strategies on different facets, and my other son was really into sharks, which it has bridged into extinct sharks like megalodon, how scientists derive how it looks given cartilage's lower propensity to fossilize, bridging to dinosaurs and their fossils, the K-PG extinction event, how food scarcity filtered for smaller animals like the ancestors of birds, and our small mammalian ancestors. And a whole bunch of other topics.
It's been pretty great in that way, but my biggest open question at the moment is how to get them to engage with it on their own on a more regular basis - they go to it occasionally for random questions, but to get good coverage of that huge knowledge graph would take much more. And fundamentally, I think that human engagement still just has a number of important aspects to it that it's lacking, and I'm not sure if it's possible to replace those well enough.
Which explains the poor engagement you observed. To me it seems like a _technique_ I'd expect a skilled educator to deploy, sparingly in narrow use-cases, when it's nescessary to probe a students interest.
Ha yeah, maybe so. But the main thing is actually Socratic more than just going down rabbit holes, I was summarizing a few extended sessions there. But being Socratic, it also demands thinking, and they’re not always up for that.
Hi Eric, would like to understand how you approach that steering. It's a problem statement I've been working on as well, would like to compare notes. Couldn't find your contact - mine is in my profile.
what is the point of teaching anyway when fundational knowledge are becoming obsolete?
i think what should be taught is the metacognative ability - like how to retrieve knowledge, how to ask the right questions towards a certain goal. knowledge itself are easily accessible with ai. now the difficult part is the ability to discern actual knowledge from llm halucination bs, the ability to retrieve the required knowledge given a scenario.
this still requires some foundational grounding — you can't detect bullshit with zero context. but the balance shifts from memorization to retrieval, iteration, verification. honestly i think it is more about critical thinking and philosophy.
> what is the point of teaching anyway when fundational knowledge are becoming obsolete?
1. It isn't
2. As you acknowledge, you need some 'foundational grounding', but the amount needed is quite a lot
3. The best way to teach metacognitive (and all other) skills is within a context
> the balance shifts from memorization to retrieval, iteration, verification
This has been trumpeted with every poorly-thought-out educational change, and it's a marker of unfamiliarity with the space. Memorisation hasn't been the focus ever; it's always about the other skills, and (some) memorisation is useful as part of that.
> “Students aren’t great at asking questions well.”
In my interactions with my kids public school and their teachers, they're goal is ram content down their throat and test for retention, not foster an environment open to questions
Had a teacher claim straight up they don't believe the system works and are just in teaching for benefits and summer vacation
IMO Sal Khan's revolution hasn't happened because the adults in charge right now are ignorant and inept but incredibly vain nonetheless
Humans have 3 inch chimp brains. The expectations on those poor brains are too high. Its like expecting dogs to one day figure out how to run a mcdonalds. We do what we can. Theory of Bounded Rationality applies.
The vanity part has to do with Status Signaling. And status signaling is one of few know hacks that work at population scale in keeping all the different chimps together. Granted the status signalers some times get carried away and forget what its purpose is. See Theory of the Leisure Class.
>In my interactions with my kids public school and their teachers, they're goal is ram content down their throat and test for retention, not foster an environment open to questions
Is that actually true though? Average American students (especially those in the public school system) are not excellent test takers, and they're even worse at rote memorization. If this is actually the goal they're not achieving that either.
I think it’s not right to blame teachers, because you cannot control that right? Agree it’s far too hard to find aspirational teachers, but they are naturally in demand and will be snatched up by well paying positions, like private schools. The tool is supposed to help given the current suboptimal teaching practices.
The fact that few students are self motivated enough to use it makes sense....but you are telling me that, in 4 years, _so_ few were motivated to use it that you can't report on whether or not it makes a difference for the minority that do?
I was among those who, when Khanmigo was first announced, were pretty excited about it's potential. I then waited for data on the results....and kept waiting.....and kept waiting. And now four years later this is apparently what we are going to get. I think that this is enough for me to decide that Khanmigo, regardless of whether or not a student actually engages with it, doesn't make much learning difference. At some point, the absence of (reported) data becomes data in itself.
I still believe, in principle, that AI tutors could be massively helpful for learning. But apparently we haven't yet figured out how to take that principle and turn it into reality.
I think the AI can not understand the situation of the student so it does not know what the student does not know. Therefore it can not guide the student through the main hurdles of learning and understanding a topic. Whereas a human tutor was once a human-student, AI never was.
I mainly use AI for learning things these days. The biggest bottleneck is always providing the machine with your context in sufficient detail that it can understand how to help you. When learning a topic it isn't always clear cut on how to do that, as you're likely missing much of the vocabulary necessary to get the AI to give you the answer you want.
AI is perfectly capable of teaching you quantum mechanics if you understand music theory. However, unless you have a full understanding of music theory, you'll need to explain to the machine what you know, and that takes trial and error that most students won't bother with.
Family exceptions and support as well as the students self drive are the main indicators of a students academical improvement in my experience.
Until he shows a comparative study, and also any disaster information like mental health effects, then his entire claims three years in should be treated as bullshit.
Honest question: how many of you tech bros have used this platform with your own children? If you won’t dog food it, quit claiming it’ll help the disadvantaged. Please.
It’s human nature really, and we see this in our jobs, instead of using LLMs to learn to do our jobs better, we replace work with automation.
It’s going to be quite hard to motivate students to learn now that they know answering can be automated.
90% of teaching kids is actually managing energy and motivation. Of course an obsequious facile robot can’t actually help there.
I never thought about it that way, but that matches my experience with my own kid.
On one hand I will grow old knowing I'll always have a job because a lot of kids never will have researched anything in their lives and won't know how to deal with anything an LLM can't solve. On the other hand between this and most kids having had a 2 year covid gap in their learning, who the heck is going to pay my retirement and be my doctor when I'm old?
When I was young 20-30 years ago older people were saying that the Internet would make us dumb. Why learn anything when information is always readily available one search request away? Videogames were supposed to make me a blood-thirsty maniac, and don't even get me started on readily available porn. "Kids these days are lazy and don't want to learn" is one of the oldest memes in human history, with documented use going almost as far back as writing itself.
The flipside of that take is that if you listened to technologists, then educational TV/CD-ROMs/laptops/the internet/tablets/educational games/digital blackboards/MOOCs/etc. were going to completely revolutionize education - but looking at the evidence, it doesn't seem like students have gained much at all from any of it.
I remember an educator ranting to me a long time ago that the only data-proven ways to meaningfully improve educational outcomes was to reduce classroom size and make sure kids got enough sleep + fed well enough, everything else was just a waste of time.
From all my years of schooling one of the biggest factors is the combination of level of interest from the student, with parent involvement following that, once you cross the basic threshold of stable home with regular sleep and food. Some kids don't care and even perfect parents won't matter, but disinterested parents also drag a bunch of kids down.
There has been a measurable and noticeable drop in attainment starting with smartphones entering the classroom, supercharged by COVID chaos, and finally with AI cheating being just the latest assault on learning.
Ask teachers that have been teaching for 10 years. Ask the professors how today's kids are different than the ones of yesteryear.
The move to de-tech the classroom will eventually help out I expect, but keeping kids (and adults!!!) from using cognitive shortcuts so they can develop their own sense of what's reasonable instead of taking information from a bought-and-paid-for oracle is going to remain a problem.
I don't even think Khan Academy's original teaching revolution quite panned out.
I still remember when Khan Academy first came out, there was talk that teachers would go obsolete because teaching would become centralized and delivered over video.
Khan Academy to me is still just a YouTube channel trying very hard to be something more.
Well it wasn't really a teaching revolution. It was a marketing job around a YouTube channel that purported to be a teaching revolution.
The thing is people want more than material. They want the material to be accredited and examined. Otherwise there is no demonstrable credibility from doing it.
And there's a whole world out there of higher quality material with has that accreditation and examination structure around it. And it existed, sometimes for decades in the case of The Open University, before Khan Academy appeared. But it costs money.
Indeed, reverse classroom, everybody getting access to high quality content, video then interaction, learning paths, etc.
Well, in practice it's still about the amount of time a pupil does train with the right oversight and that is precisely the bottleneck that hasn't been alleviated.
Many of the ed-tech products that are introduced in the classroom are solutions looking for a problem. Not many have much understanding of the context of teaching and learning processes. And yet, as history of ed-tech tells us, gung-ho business optimism will sell these to managers/administrators as THE solution. The most important stakeholders in the process the teachers an students, who are end-users of the said product/technology, are left out in this decision chain.
Classic technological innovation trope :
Sure the one student who already excels, is motivated, understand what the concept to learn is, that actually completing exercises helps them to learn might, possibly, thrive. All other students, the vast majority, will try to "game" the (terrible) evaluation system to get good grades by cheating WHILE avoiding the very challenge that make the learning possible.Who could have guessed.
I think it's particularly telling that the teacher dropped the product and the superintendent is saying it's going well. Maybe I'm biased from going to New Orleans public schools, but I have my doubts about how tapped in the superintendent is of the overall strategy.
That said I do think it's particularly hilarious that KA's strategy to students not wanting to use the product is to make the product more integral to the experience.
> She says there’s been more enthusiasm for the product among administrators than teachers in her school.
That is a warning sign if ever there was one.
Weird. No mention of the technical aspects, essentially just blaming average students for not being engaged enough with their simplistic ChatGPT clone. No wonder they have not dared yet to give out actual usage metrics. If I was given the choice between an inferior product that probably lags significantly behind on all features and one of the standard offerings from OpenAI, Google or Anthropic, I'd question why I should use this thing too. According to their website, they position Khanmigo like this:
>Unlike other AI tools such as ChatGPT, Khanmigo doesn’t just give answers. Instead, with limitless patience, it guides learners to find the answer themselves. In addition, Khanmigo is the only AI tool that is incorporated with Khan Academy’s world-class content library that covers math, humanities, coding, social studies, and more.
The first differentiation is literally just prompting (if at all). Nowadays you can tell any chatbot to behave that way. The second one may have been an edge before tool use was widely common, but with all chatbots now having access to the internet and code execution, it seems like this has also become a dud. This product was a nice idea on paper, but the fast technical evolution of the field has largely left it in the dust.
I mean, isn't the whole khan academy approach "we know better how to teach everything"? It's not surprising that they'd think they have more enlightened prompts than anyone else.
They had really cool math videos and got given too much money, that's about the story.
The wish for an _AI revolution_ in learning seems to have been granted by a monkey's paw. Articles like this, or [0], or browsing r/teachers [1], or even talking close-ones in college, give a rather grim view of AI use.
A para from from [0] makes it seem that students understand that LLM use doesn't lead to learning, but still do so. Do they not see effort put into learning worthwhile?
I myself use LLMs for learning (using ChatGPT's study mode for instance r.i.p) and can see that there's a right way to use it—you reach for it when you hit a wall, not to avoid the friction of developing an understanding.From what I understand tho, most of LLM use for learning is just LLM used as a tool for cheating. Even tfa mentions something of the sort:
The article attributes _skill issue_ as part of the problem, but how much of that is a motivation or awareness issue. How do you make student realize that learning is worth it?[0] https://arstechnica.com/science/2026/04/to-teach-in-the-time...
[1] https://www.reddit.com/r/Teachers/
You never realize the beauty of just learning cool stuff in college and exploring around until youre like 26 and graduated for 4 years
I don't even think it is a monkey's paw. That implies that it looked like it would solve a big problem in our initial estimation.
But "students will use the cheating machine to cheat" was obvious from the release of ChatGPT3. There was never some period of time where AI looked like it was a net positive for students only to be revealed to have an unexpected harm.
Even from the folks who claim to use LLMs to learn rather than cheat or avoid work, I've seen so many people admit that they are actually using it to harm themselves. "Oh, I only ask ChatGPT for the answer for really hard problems." Yeah man, doing the hard problems is how you learn.
You can be very AI-skeptic in various ways and still think that this is a fair take. I teach and supervise students as master's level courses, and about 15% of them have intrinsic motivation to learn. These students have set up their own AI tutors with prompts and know way more than me in certain areas of the field, they are extremely ahead of their class.
The issue in my country is that you equate education with getting a safe job. 20 years ago, you needed a high-school degree in social science to get a government job. 10 years ago you needed a bachelor in social sciences to get the same job. 5 years ago you needed a bachelor in economy/engineering to get the same job. Now, because of recessions this is stretching to masters degrees.
You can't expect people who just want a job and a comfortable life and NEED to go to uni for this to want to be curious and want to learn.
And on other side education attainment has become metric for governments. More degrees and higher the degrees are better it will be for the economy somehow. Where there is likely quite a lot of jobs that don't actually need the degree.
> about 15% of them have intrinsic motivation to learn [...] they are extremely ahead of their class.
Feels like whatever tool they'd be given, they'd be ahead anyway. What's more worrying IMHO is, are the remaining 85% faring even worst than they would have before because they are learning even less, not just slower than the 15% learning faster. Namely is the gain for the few a loss for the majority?
You are right on both counts. I do think that it's however different on the first concept. Before, they would be ahead but still capped by their university. If you come from a uni in a 100k person city, you probably would not have the material nor the best teachers. Now you can have literal Stanford quality education (by accessing Stanford's open source lectures) as well as the collective aggregated knowledge of humanity in the chat interface. The curiosity/intrinsic motivation is the only limit except for perhaps compute.
As for the other question, its mixed. I think about 20% of students understand that they are fucked if they just delegate it all to LLMs, they still go through the ropes and show up to class but do the minimum. However most are down the deep end in various degrees. I have seen students with 5 different 3000-line files for 5 questions for the same lab where each file has 3 lines of code different. This never happened even when the students cheated by accessing old labs online or plagiarizing before.
I believe that what will happen (because universities move really slow on policy and education on LLM use), is that pre-LLM, the university had a normal distribution of skills upon graduation. A company could trust that someone with a degree knew X and Y. With this however, you have more of a bimodal distribution, some know nothing and some know it all, so then the trust in universities deteriorates. I think we will see much more IQ-test/practical tests in hiring processes as the trust falters for that a degree equals something.
>Kristen DiCerbo, the organization’s chief learning officer, said AI can only respond to students based on what they ask. And it turns out, she said, “Students aren’t great at asking questions well.”
Ignoring whether or not this is a good idea in the first place, what about inverting the loop? Have the robot drive the interaction.
Yeah, that's weird. I've actually been working on an AI tutor for my kids that I'm thinking about open sourcing, but it drives the conversation in new directions using a concept graph that it can poll via tool use, and find the knowledge frontier for that learner.
It's been fascinating to watch - my kids are really into Slay the Spire, and it had a discussion about a decision tree they use when fighting one of the enemies, and then it used that to bridge to writing some python code and walking them through it. Another time, with dinosaurs, it went with them through the k-pg extinction event, and what really killed the dinosaurs - the kids thought the explosion - it walked them towards the sun dimming, and why food getting more scarce filtered for small mammals, our ancestors, and smaller dinosaurs.
on one hand I wish I would be smart enough to build something like that.
on the other hand, I was playing a lot Slay the Spire few years back and I would love to talk about with my kids while they play. Going from that it is not job of the parent to explain why dinosaurs are extinct?
Ha I love talking with them about it, too, but my 7 yo’s stamina for talking about it is a lot higher than mine. But yeah, not trying to abdicate my parenting role here, just looking to get a little supplemental help, and cover things I forget to sometimes. Also a nice launching point for me to delve deeper into things with them.
Doesn’t change the fact that the students have to ask for what they need.
If you can’t articulate what you want it becomes a guessing game
You could prompt them through the material. But you're right, that's only gonna take you so far if you don't know where they're having problems.
Then why do you need a language model for this?
How about completing the loop? Pose subject matter questions to them throughout the day, maybe via something like mobile push, collect their answers, immediately grade their results, and then actively reward them for performance.
All of the things brick and mortar schools are uniquely bad at.
> "It’s an early indication of the limits of AI to drive massive learning gains..."
I thought Sal's revolution was the idea of flipping the script on primary school learning: in-class homework & at-home video lessons.
I'm not surprised. Students are not rewarded when they ask _curious_ questions--rather, they're admonished for not paying sufficient attention.
Personally, my first use of ChatGPT was to ask tangential questions on JavaScript while taking a LinkedIn learning course on VueJS. I found ChatAI an excellent substitute for Reddit and StackOverflow, which is how I would have followed these inquiries before. Of course, I'm not a primary-school-age learner. I had to learn _How To Learn_ from experience.
I have some two million KA points, and I'm currently using it to guide me around learning basic physics - I've never used Khanmigo not even once. If I need to ask AI type questions I go off site to Claude or GPT or deepseek.
As other people have noted, asking a.k.a <i>typing</i> questions, especially math-type is fatiguing, and there's no substitute for pen and paper and thinking hard.
KA would be better off using AI on the supply side (but heavily curated) to have more assignments, or better assignments in some sections.
But it's important to recognize KA for what it is, and it's an excellent way to have some sort of a basic curriculum, especially when self-studying, and all of the instructors have great teaching personalities, as far as I can deduce from the approach in the videos.
This honestly seems like a flawed approach. Kids don't show up in the first grade, or the sixth, 9th, or really 12th, as the initiators of their educational journey.
An AI-based education system should have embedded in it "I am here to teach this person Geometry. Here is a list of the topics to cover, with a breakdown of steps for each including an intro section, a study section, a test section, and the meta material to go along with it.
That would work.
Best comment in this thread. An AI-based education system as you described, would deploy the _technique_ the KA bot is built around, sparingly and at narrow times, to deliver a lesson to great effect.
The poor engagement of the KA bot becomes clear--a teaching technique in not an education system.
I agree. I just can’t understand how people who work in education seem to be incapable of learning anything about education! They couldn’t have predicted that just having the most amazing tutor in the world available all the time just wouldn’t make any difference?? Students don’t go to school because they’re eager to learn. Don’t they know this? I think an effective tutor must be human or at least a human who can use the AI tutor on behalf of the child, which means the parent. Because your job as a tutor is not just spill knowledge, it’s to keep the student engaged ( just awake may be a challenge ) and make sure they are doing what they are supposed to.
> how people who work in education seem to be incapable of learning anything about education
The people who work in education don't have this issue; the people who work in tech and assume that gives them expertise in education do.
The educators keep making blunders like in this case. Hundreds of years of teaching and sorry but looks like your field is still trying to figure out the very basics of your discipline.
Educators aren't making these blunders though; again, this is non-educators trying to force tools that educators don't want.
Most teachers I know would be delighted if tech companies and management stopped trying to push tools on them that aren't fit for purpose.
Might be harsh, but more I think about education more I think that the main job for many if not most kids or students in general is making them to learn. By various incentives be it punishments or rewards. Only really at highest levels is there any self-direction. And even there is plenty of external factors like not being paid anymore if you do not publish something.
Eh, working on rebuilding a Montessori school for a couple years showed me that kids are very intrinsically motivated to learn, just not always what the person who’s there to teach them wants them to learn that day. But if you enable that self drive, and gently steer by exposing them to new things at the right level, they can learn a tremendous amount.
Our kids elementary school (Germany) recently voiced concerns that pupils - especially boys - do not fulfill the self defined learning tasks in the expected time.. There may be no upper boundary to how much out of touch educators can be.
I think ai has revealed one of the biggest gaps in our education system: the majority of students don’t really care about knowledge-even when tested on it.
To give an example, I have a friend who learned system design through Claude in order to get a job interview (and he got really good at system design)while I have another friend who copies and paste ChatGPT responses in order to get a B on a reflection assignment.
This highlights that there is legit use case for personalized learning and growth via AI-but these are the people who seek knowledge with or without AI. Whereas the majority of students actively tries to do the least as possible on assignments even if they get 0 value out of it
Right. Most people are deeply uninterested in learning anything, not just in K-12, but at pretty good colleges like mine.
Modern AI has made me a more productive teacher—-I produce higher quality material and have more time for research.
But the impact on most students is negative. It is another thing to engage with, which they won’t unless forced. The only way to learn is to do the work yourself. An AI tutor can get you unstuck faster, but that’s typically bad. Learning to be productively stuck on something for days without making visible progress is an important skill that most people never learn.
> An AI tutor can get you unstuck faster, but that’s typically bad.
I like struggling with interesting problems. But spinning your wheels is not progress. So, IMO, getting you unstuck is a generally good thing.
There is an LLM button on every other website now. "Chat with your lesson", "chat with your food", "chat with your photos". People are not clicking them because they are just visual noise at this point.
have you noticed the obnoxious coloured swirly around the "chat with x" buttons too? just in case you didnt notice the button...
It was honestly a very cool gimmick few years ago, when it was new. Chatting with your photos, imagine that! ... then after few tries you're like "well... I just want the last 10 photos because I want to share a photo of my son playing piano to his grand parents" and the UX, because it's actually well craft based on feedback from users, not only has a button for just that... but even default to this.
So... yeah, it got old quick. Genuinely cool for a bit but now "we" as users just want good UX. Now give me the FAQ that I can search through then an email if it's not in there.
PS: FWIW I do believe in a long-tail fashion, for few users who are not into scripting, might not be developers (or believe they could become) it could help find very few very niche use cases with solutions.
Derek Muller of Veritasium has a presentation on why the promises of a revolution in education are never fulfilled. [0]
His hottest take is we're already close to the optimal process for learning, so technology isn't going to improve it. Learning takes work, and no technology can do the work for you.
[0] https://www.youtube.com/watch?v=0xS68sl2D70
I think the teacher's job is not just to teach but more importantly to ingnite curiosity in students on new fields / subjects. If a genuine interest / curiosity in subject is missing then no amount or medium of tutoring can help.
AI is great for the curious. But its not yet there where it can proactively engage with students to generate interest.
I wonder what is the current state of coursera? Do people still study there or is it dying like stackoverflow.
>Khan Academy recently announced an overhaul of its product that provides students additional academic practice. Now Khanmigo is incorporated directly as a way students can get advice as they’re working through specific problems. A spokesperson said the organization made this change because “students were not seeking out Khanmigo’s help as much as we had hoped.”
Dear Lord, how is this any different from Microsoft sticking Copilot or Google sticking Gemini in every single offering? They're literally saying that people aren't using the chat bot enough so they're going to force it on people inside the product.
Well, there's a good way and a bad way to do this. Embedded AI (vs standalone AI) can do a lot of interesting things. For example, a good teacher will watch students and identify when someone is struggling (or thriving), and carefully offer to help, vs just waiting for students to approach them and aka questions.
I'll probably open source and Show HN the AI tutor I've been working on for my kids at some point, but working on it has given me a little insight into the problem.
The biggest thing is motivation. First off, if Khanmigo requires them to type and read everything, that's going to get tiring fast for most kids. But I don't know how you could do voice in a school setting - mine uses STT/TTS, but with 20 kids in a room, it'd be chaos - STT accuracy and diarization with 2 is already really challenging.
Motivation is helped a bit by following their interest, but it seems like KA is having trouble guiding the kids when they prompt it that way. That was a pretty big issue with mine early on - the kids would talk to it for an hour about whatever topic they were interested in at the time, but it would never branch into something new.
The tutor I'm working on solves it by having a concept graph that covers a lot of learning, from the basics like math, dinosaurs, etc to other developmental topics like 6 year old boundary-pushing humor, and two LLM threads - one that handles the conversational turns, and another one in the background that strategizes and steers the conversational thread by looking at the concept graph connections and considering how ready they are for each, and then injecting steering notes into the conversational thread. Basically system 1 and system 2 thinking. And after sessions, it'll make a basic plan of where to start next time, and what might be interesting to offer up.
I mentioned this in another comment, but I've been really pleasantly surprised at the quality of the tutoring, especially when it bridges into new topics - one of my sons is really into slay the spire, and at different times it’s used that as a launching-off point into probabilities, decision trees, python code of the algorithms he thinks about as he's facing different enemies, and general strategies on different facets, and my other son was really into sharks, which it has bridged into extinct sharks like megalodon, how scientists derive how it looks given cartilage's lower propensity to fossilize, bridging to dinosaurs and their fossils, the K-PG extinction event, how food scarcity filtered for smaller animals like the ancestors of birds, and our small mammalian ancestors. And a whole bunch of other topics.
It's been pretty great in that way, but my biggest open question at the moment is how to get them to engage with it on their own on a more regular basis - they go to it occasionally for random questions, but to get good coverage of that huge knowledge graph would take much more. And fundamentally, I think that human engagement still just has a number of important aspects to it that it's lacking, and I'm not sure if it's possible to replace those well enough.
You built a wikipedia-rabbit-hole chat bot.
Which explains the poor engagement you observed. To me it seems like a _technique_ I'd expect a skilled educator to deploy, sparingly in narrow use-cases, when it's nescessary to probe a students interest.
Ha yeah, maybe so. But the main thing is actually Socratic more than just going down rabbit holes, I was summarizing a few extended sessions there. But being Socratic, it also demands thinking, and they’re not always up for that.
Open to suggestions for how to improve it!
Hi Eric, would like to understand how you approach that steering. It's a problem statement I've been working on as well, would like to compare notes. Couldn't find your contact - mine is in my profile.
what is the point of teaching anyway when fundational knowledge are becoming obsolete?
i think what should be taught is the metacognative ability - like how to retrieve knowledge, how to ask the right questions towards a certain goal. knowledge itself are easily accessible with ai. now the difficult part is the ability to discern actual knowledge from llm halucination bs, the ability to retrieve the required knowledge given a scenario.
this still requires some foundational grounding — you can't detect bullshit with zero context. but the balance shifts from memorization to retrieval, iteration, verification. honestly i think it is more about critical thinking and philosophy.
> what is the point of teaching anyway when fundational knowledge are becoming obsolete?
1. It isn't
2. As you acknowledge, you need some 'foundational grounding', but the amount needed is quite a lot
3. The best way to teach metacognitive (and all other) skills is within a context
> the balance shifts from memorization to retrieval, iteration, verification
This has been trumpeted with every poorly-thought-out educational change, and it's a marker of unfamiliarity with the space. Memorisation hasn't been the focus ever; it's always about the other skills, and (some) memorisation is useful as part of that.
> It doesn’t necessarily make students motivated to learn or fill in gaps in knowledge needed to ask questions.
Who would have thought?
Kristen DiCerbo quote from the article
> “Students aren’t great at asking questions well.”
In my interactions with my kids public school and their teachers, they're goal is ram content down their throat and test for retention, not foster an environment open to questions
Had a teacher claim straight up they don't believe the system works and are just in teaching for benefits and summer vacation
IMO Sal Khan's revolution hasn't happened because the adults in charge right now are ignorant and inept but incredibly vain nonetheless
Humans have 3 inch chimp brains. The expectations on those poor brains are too high. Its like expecting dogs to one day figure out how to run a mcdonalds. We do what we can. Theory of Bounded Rationality applies. The vanity part has to do with Status Signaling. And status signaling is one of few know hacks that work at population scale in keeping all the different chimps together. Granted the status signalers some times get carried away and forget what its purpose is. See Theory of the Leisure Class.
>In my interactions with my kids public school and their teachers, they're goal is ram content down their throat and test for retention, not foster an environment open to questions
Is that actually true though? Average American students (especially those in the public school system) are not excellent test takers, and they're even worse at rote memorization. If this is actually the goal they're not achieving that either.
"It's all focused on rote memorisation" is a really popular dismissal of the education system that betrays a lack of familiarity with it.
I didn't claim they were successful just paraphrased what a teacher explained as their mandate
I think it’s not right to blame teachers, because you cannot control that right? Agree it’s far too hard to find aspirational teachers, but they are naturally in demand and will be snatched up by well paying positions, like private schools. The tool is supposed to help given the current suboptimal teaching practices.
Teachers claiming to be self aware such they don't care and are just there for the benefits can absolutely take some blame
They could quit and free up the slot for someone who does care
> Musall no longer uses Khanmigo in her class. She says there’s been more enthusiasm for the product among administrators than teachers in her school
…sounds a lot like Investors versus those who actually perform “work” as it’s defined in research literature.
But I’m sure a shoe company pivoting to AI isn’t a sign of a bubble about to burst, nope.