Khan Academy’s mission is to provide free, world-class education to everyone, everywhere. This blog post explores Khanmigo, our AI tutor and teacher assistant, and how it helps us achieve this mission at scale. We’ll delve into the problems we’re tackling, our solutions, technical challenges, and future plans.

The Challenge: Bridging the Widening Learning Gap

The pandemic exacerbated existing learning gaps. Data reveals that while many third-graders perform at grade level, a significant number fall behind by eighth grade. Khan Academy believes AI can help close these gaps by providing personalized tutoring at scale. Khanmigo, our AI tutor for learners and assistant for teachers, represents a significant step towards this goal.

Khanmigo: An AI Tutor Unlike Any Other

Khanmigo isn’t just another chatbot. We designed it to emulate the Socratic method, guiding students to answers through carefully crafted questions. Unlike some general AI models, Khanmigo prioritizes accuracy, especially in math, and integrates seamlessly with Khan Academy’s existing content.

Here’s how Khanmigo differentiates itself:

• Socratic Approach: Instead of giving away answers, Khanmigo asks leading questions, encouraging students to think critically and arrive at solutions independently.
• Math Accuracy: Khanmigo emphasizes accuracy in mathematical calculations, identifying errors and offering specific guidance.
• Deep Content Integration: Khanmigo accesses and utilizes the context of Khan Academy lessons, providing tailored support based on the student’s current learning activity.
• Personalized Learning: Khanmigo offers personalized hints, feedback, and support based on individual student progress.

Beyond Chat: Reimagining Essay Writing and Coding with AI

Khanmigo goes beyond the traditional chat interface to transform how students approach essay writing and coding. Our writing coach guides students through outlining, drafting, and revising, providing feedback on evidence, structure, style, and tone. This iterative process, facilitated by AI, fosters deeper learning and encourages continuous improvement. A similar approach is planned for code writing, further enhancing the learning experience.

The teacher dashboard provides insights into student interactions with Khanmigo, including conversation history and flags for potentially inappropriate content. This feature promotes transparency and allows educators to monitor student progress effectively. Khanmigo’s AI-driven features are designed to improve the learning experience and empower both students and educators.

Building Khanmigo: A Journey of Rapid Prototyping and Iteration

Khanmigo’s development wasn’t a meticulously planned project with rigid timelines. Instead, we prioritized rapid prototyping, iteration, and close collaboration between builders and domain experts. This agile approach allowed us to quickly test ideas, learn from failures, and refine our approach based on real-world feedback.

Key takeaways from our development process:

• Resist Perfectionism: We embraced a “ship it and iterate” mentality, focusing on delivering value quickly rather than striving for perfection from the outset.
• Trust Intuition: We combined data-driven decision-making with the expertise of learning scientists and educators.
• Iterate Rapidly: We used demo-driven development to test and refine features, ensuring they met the needs of our users.
• Embrace Change: We adapted our core values and product development process to support this new way of working.

Transforming Khan Academy into an AI-First Organization

Integrating AI into Khan Academy’s DNA required a shift in organizational structure and processes. We established an AI Platform team to bridge the gap between infrastructure and product development. This team owns the AI developer experience, manages the AI router, and provides expertise to other product teams.

Key changes included:

• Formation of an AI Platform team: This team focuses on the AI infrastructure and ensures seamless integration of AI capabilities across Khan Academy.
• Refined Product Development Process: We adopted a flexible process that allows for different levels of code quality based on feature confidence.
• Updated Core Values: We modified our core values to emphasize intuition, rapid iteration, and a focus on delivering “wow” experiences.

Overcoming Technical Hurdles

Building an AI tutor at scale presented numerous technical challenges. We tackled these head-on, developing innovative solutions to ensure accuracy, performance, and scalability.

Some of our key technical solutions:

• Math Agent: Allows the AI to defer to a specialized math agent for complex calculations.
• Chain of Thought Prompting: Improves reasoning and accuracy by guiding the AI through a structured thought process.
• Contextual Information: Provides the AI with relevant information from Khan Academy lessons and step-by-step solutions.
• Tutoring Accuracy Dashboard: Monitors the quality of tutoring over time and identifies areas for improvement.

Looking Ahead: The Future of Khanmigo

We’re continually improving Khanmigo, focusing on enhanced accuracy, expanded language support, and improved scalability. Key initiatives include:

• Math Accuracy Benchmark Dataset: We’re developing a benchmark dataset to evaluate the accuracy of AI tutors, providing a standard for the field.
• Integration with Wolfram: We’re exploring integration with Wolfram Alpha to handle complex mathematical computations.
• Model Fine-tuning: We’re continually fine-tuning our models to improve performance and address specific learning needs.
• Multi-Model Approach: We’re leveraging multiple models for different workflows and exploring more efficient models like GPT-4 and smaller language models.
• Dynamic Scaling: We’re working towards dynamic scaling of our infrastructure to optimize resource utilization and cost-effectiveness.

Khanmigo represents a significant step forward in our mission to provide a free, world-class education for anyone, anywhere. We believe AI has the power to transform education, and we’re committed to harnessing its potential to empower learners and educators around the world. We invite you to explore Khanmigo and experience the future of learning.