Focusing on “Evidence of Learning” in the Age of AI
The Obstacle is the Way author Ryan Holiday likes to say, "Belief in yourself is overrated. Generate evidence."
In learning, evidence is how we see a demonstration of understanding and acquisition of skills.
We’ve always had an evidence problem in most education settings, but now with Generative AI able to create “evidence” in a matter of seconds, we are at a crossroads in what acceptable evidence of learning actually looks like.
Looking to have students generate evidence inside your LMS by answering discussion or quiz questions? Coursology.io is an AI-powered Chrome extension that will solve and answer all of these in a matter of seconds.
Looking to have students write a research paper, properly citing sources etc? Jenni.ai is an AI-powered tool that 10x the speed and accuracy on these types of assignments.
Math problems for HW? Khanmigo and Photomath.
Science lab reports? ChatGPT, boom, done.
We could go on, but I think you get the point.
The world has changed, and here we are, once again, as the ones who will have to be adaptable and flexible enough to evolve our practices for 2023 and beyond.
The Silver Lining
The silver lining in all of this is that good education practices are still good education practices.
Learning hasn’t changed. However, how we plan learning experiences and gather evidence of learning is going to be different moving forward.
This post is set to focus on one of the biggest reasons we look for evidence of learning: Transfer.
Transfer is one of the most important topics in education, that we rarely talk about. We know it matters, we design entire curricula with the goal of transfer, but in the daily grind of teaching and learning, it can often be months without a conversation about transfer.
What is transfer? Here is how authors Julie Stern, Krista Ferraro, Kayla Duncan, and Trevor Aleo define it in their book, Learning That Transfers: Designing Curriculum for a Changing World.
Learning Transfer: Using our previous learning to understand or unlock a completely new situation.
This is generally the goal of education: Not to "prepare" our students for something that we can foresee, but instead, help them to prepare themselves for situations neither of us could predict.
Can We Really Design Learning That Transfers?
The answer is a resounding yes. However, even with that emphatic YES, this work can only be done intentionally. The authors argue that not only can we teach for transfer, but we can also do so while teaching "less":
What if we selected the most powerful, transferable, organizing ideas from our curricular documents, and anchored everything we explored in those concepts? Could this help educators turn off the conveyor belt of "covering" an endless list of objectives while also ensuring students are prepared to tackle topics they encounter without a teacher's guidance? Yes, it can. We can both teach less and prepare our students to tackle more.
To do this, we can use a simple framework for teaching concepts and their connections. Enter the ACT: The Learning Transfer Mental Model.
ACT is a powerful way to reframe the learning process (all images from Learning That Transfers).
Step 1: Design learning experiences that help students acquire knowledge of single concepts.
Step 2: Connect those concepts in a relationship.
Step 3: Transfer those concepts and connections to new situations.
What Strategies Can We Use to Promote Transfer?
The authors do a fantastic job of breaking down some questions we can ask to develop transfer strategies in their book:
1. Recognize the concepts that apply: Which concepts are at play in this situation?
2. Engage prior understanding of the conceptual relationship: What do I already know to be true about the relationship among these concepts? What specific examples support my understanding?
3. Determine the extent to which prior understanding applies: What makes this new situation different from the situations I’ve seen in other learning Does what I understand about the relationship between these concepts apply to this new situation? Which parts of my prior understanding transfer and which don’t?
4. Modify and refine understanding based on the new situation: How has transferring to this situation refined or reshaped my thinking? (Stern et al., 2017)
Along with these questions, we can focus on seven shifts that also help develop transfer throughout the learning process:
Evidence of Learning in an AI World
Author Jay McTighe has long-been one of the leading advocates for authentic performance-based assessments. In books and articles from 1998, 2008, 2013 and even as recent as 2017 McTighe offered a look into the real reason we should generate a variety of evidence:
Traditional types of assessments offer sufficient ways of measuring students' knowledge and basic skills. For example, we can use multiple choice or fill-in-the-blank test items to gauge students' knowledge of historical or scientific facts. However, to properly assess conceptual understanding, long-term transfer, and other complex skills, we need greater use of authentic, performance-based measures in which students are asked to: 1) apply their learning to a new situation, and 2) explain their thinking, show their reasoning, or justify their conclusion.
Authentic tasks are like the game in athletics. While the players have to possess knowledge (the rules) and specific skills (dribbling), playing the game also involves conceptual understanding (game strategies) and transfer (using skills and strategies to advantage in particular game situations). Assessing what matters must include assessing performance in a "game" in addition to tests of requisite knowledge and skills.
That bolded line above may not be true in today’s world as those types of low-level assessments and assignments are just the type of task that Artificial Intelligence loves to answer.
It’s why “playing the game” is so important.
If I coach a player in basketball, I can see if they can do dribbling drills, shooting drills, passing and rebounding drills in practice. Yet, I truly understand their level of skill and understanding when they are playing the game doing all of these at once in a real environment.
Similarly, that basketball player can take what they learned in those drills and games and apply them when playing other sports such as lacrosse, soccer, or hockey—which would be transfer. McTighe takes a step further on what we are looking for in these types of authentic assessments that enhance learning:
This list explains why I (McTighe) am an advocate for expanding the use of authentic tasks and projects in schools. Such assessments offer more than just another way to measure student achievement. Like the game in sports or the play in theater, authentic performances are motivating to learners. They give relevance and purpose to learning, and they underscore the need for practice.
Authentic tasks also influence teaching. Coaches recognize that their job is not to simply "cover" the playbook play-by-play and teach individual skills. They understand that knowledge and skills are in service of larger ends, and that their role is to prepare players for authentic transfer performance in the game. Performance-oriented teachers in all subjects understand this role as well.
Next Steps
A few important lessons I’ve been reminded of since diving back into Stern and McTighe’s work.
We already know what quality evidence of learning looks like. Everything that worked 10 years ago still works today. What doesn’t work is the rote, low-level type of assignments and assessments that never promoted transfer.
Training around AI should not be focused solely on using AI tools, or how to handle students using AI to cheat. I’m not saying it isn’t important, however, when we focus on authentic tasks and assessments that enhance learning, we’ll worry less about AI’s role in doing student’s work for them.
Evidence may look different in 2023 and beyond. It doesn’t always have to be big EPIC authentic learning. It often is small, simple changes.
Instead of 20 math questions, just answer three, but screencast yourself going through the problems and talking about your thinking in each step. Instead of writing a 5-page paper at home, write an 11-minute essay in class, use AI to revise it for HW, then share what changes you made and why in a short presentation.
Maybe I should have titled this post, “Focusing on Transfer in An AI World” — but to be honest, it takes a lot of time to plan around transfer as the ultimate goal for learning. We all want to get there, but along the way, focusing on authentic ways to showcase evidence of learning may be the best way to lay the building blocks for transfer, again, and again throughout the year.
