source: google deepmind: measuring the impact of learning with ai in sierra leone and beyond
level: research
google deepmind and fab ai ran an eight-week randomized controlled trial with 1,763 junior secondary students across 12 schools in sierra leone. the study tested how guided learning in gemini affected math progress. students using the ai tool gained 0.258 standard deviations in math scores compared to a control group, equal to 1.2 to 1.7 years of typical learning progress. gains were even larger when teachers used gemini in about half their lessons, reaching 1.8 to 2.5 years of progress.
the ai was designed to avoid giving direct answers. analysis of over 113,000 interactions showed that 91.4% of student conversations focused on building conceptual understanding. gemini responded with scaffolding questions 76% of the time and gave direct solutions only 2% of the time. over the trial, skill-building queries rose from 68% to 90%, while solution-seeking questions dropped from 25% to 10%. teachers reported shifting from lecturers to facilitators, and 69% of students met or exceeded usage targets, far above the typical 5% for voluntary edtech.
the trial highlighted an achievement gap: students with stronger initial math skills benefited more. future work will expand trials to other countries and explore areas like metacognition. the team released a teacher training guide and a playbook for running similar randomized controlled trials. the results suggest that carefully designed ai can support teachers and improve learning when it prioritizes understanding over answers.
why it matters: the study provides rigorous evidence that ai tutoring can significantly accelerate learning in low-resource settings when designed to promote critical thinking, not just answer delivery.
source: google deepmind: measuring the impact of learning with ai in sierra leone and beyond