Main Point
Evaluate AI-supported education through the learner's developing capability. A useful explanation becomes part of learning when the learner attempts, checks, revises and later applies the idea under appropriate conditions.
Key Points
- Define what the learner should independently explain or do.
- Combine worked examples with attempts before revealing answers.
- Use feedback to address a visible misunderstanding, then collect a revised response.
- Revisit the relationship over time and test it in a changed situation.
- Provide usable support and examine the teacher work required to maintain it.
1. Nia's Learning Objective
Nia is a fictional learner studying after work. An assistant offers an explanation when her teacher is unavailable. Suitable language, accessible presentation, working connectivity and a correction route affect whether this help is usable.
Her objective is to distinguish two mechanisms:
| Situation | What changes | What the learner should explain |
|---|---|---|
| An application retrieves a current policy document for an answer | Information supplied to the task | The source context changes while the model’s learned internal settings, called parameters, remain unchanged |
| A learning process updates parameters using examples | The model's parameters | A training process changes the learned model |
The teacher wants Nia to apply the distinction in a different case later. This criterion determines what evidence to collect.
2. Make the Thinking Visible
A complete assistant-generated answer can help Nia finish a page while leaving her reasoning uncertain. Begin instead with a worked example, then close it and ask her to explain the relationship from memory. Active retrieval is that attempt before checking.
Suppose her answer says that retrieving the new policy trained the model. Save this attempt. It reveals the misconception that feedback needs to address.
Useful feedback might ask which parameters changed and direct her to the relevant step in the example. Let her revise, then check the revision against a reliable explanation. The teacher can use this record to decide whether another example or discussion is needed. Generated feedback also needs accuracy checks.
3. Collect Later Application
Change the surface details while preserving the relationship. Present a different retrieved document and a separate parameter-updating process. Ask for an explanation before showing the answer.
Revisit a related task on a later occasion. Spaced practice provides encounters over time; the later response reveals what is available then. Choose timing and support around the task. A reminder schedules an opportunity; the learner's attempt supplies the learning evidence.
The IES September 2007 practice guide recommends spacing learning, combining worked solutions with problem-solving exercises and retrieval-based review. These principles concern learning and memory. Applying them with AI requires checking whether the activity genuinely elicits those actions and whether the material is accurate.
4. Evaluate the Implementation
An October 2026 IZA discussion paper reports a randomized University of Maryland study of a course-integrated AI tutor. Within sections of the same course, access was associated with lower final grades. Participation in the existing course learning-management platform declined in that comparison and the broader sample; academic estimates in the broader sample were less clear and not statistically significant. The research is preliminary and specific to its implementation, with limited uptake and other AI available. Its course outcomes do not directly settle Nia's particular delayed transfer performance.
For a local class, record the objective, permitted assistance, actual use, assessment conditions and outcomes. Investigate differences between learners without assuming that access to the same tool creates the same opportunity or benefit.
5. Teacher and Institutional Choices
Teachers can diagnose recorded errors, check feedback and choose the next task. They need time and authority to do that work. Institutions choose the support, assistance rules and assessment conditions.
Nia may need narration or a saved restart point. Another learner may need an offline route or interface adjustment. Test these supports with the relevant people. An independent task should assess the intended reasoning while providing appropriate access; document its conditions so outcomes are interpretable.
The useful sequence is objective → example → independent attempt → feedback and revision → later changed task. It gives learners and teachers an inspectable account of what is improving and what still needs attention.
Learning Checks
M31-L11-Q01: Why should an easy explanation not be equated with learning? Describe the difference between familiarity during reading and independent recall or application later.
M31-L11-Q02: Compare the two fictional institutional support arrangements in the worksheet. Explain effects on teacher and learner work, possible exclusion and the evidence needed to judge learning.
Record your responses before consulting the separate key.
Practice
Use M31-L11-P01 to design a provenance-study cycle with a worked example, independent attempt, specific feedback and later transfer question. Keep answers concealed until attempts are recorded.
Use M31-L11-P02 to compare institutional arrangements for access, teaching workload, permitted assistance and assessment. Identify a responsible choice and an outcome measure with its limitation. The separate key supplies illustrative reasoning; no actual learner success is given.
Sources
- Harold Pashler and colleagues, Institute of Education Sciences / What Works Clearinghouse. *Organizing Instruction and Study to Improve Student Learning*, September 2007. Practice guide. Recommendations on spacing, worked examples, retrieval and explanatory questions; a learning-and-memory foundation rather than an evaluation of a current AI product. Retrieved 26 October 2026.
- Jing Liu and colleagues. *The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement: A Randomized University Trial*. IZA Discussion Paper 18989, October 2026. Paper. Preliminary University of Maryland implementation evidence; distinguish same-course comparisons from the broader sample and retain uptake/measurement limitations. Retrieved 26 October 2026.
Nia and the worksheet cases are fictional teaching examples. They illustrate assessment design without supplying measured learning gains.