Peer and AI Review of Student Writing

Peer review is a signature pedagogy of writing instruction. What happens when you take that established structure and add in a layer of AI-generated feedback on student writing? You get PAIRR: Peer and AI Review and Reflection, an approach to integrating AI into writing instruction developed by a team of faculty at California public institutions.

On the podcast this week, I talk with two members of that team, Marit MacArthur and Anna Mills. They share how important that second R in PAIRR is. That's the step where students consider the feedback provided both by a peer and by AI and decide what feedback to incorporate in their next draft. That step builds student agency and AI literacy. We also talk about the importance of prompt testing for a project like this, linguistic justice, and much more.

Writing instructors have been on the front lines of generative AI in education since ChatGPT was released in late 2022. The ability of students to use AI as a ghost writer has led not only to hard questions about academic integrity but deep discussions in the field about what it means to teach writing. The pressure to respond to generative AI has led to a lot of innovation in writing instruction, including the PAIRR approach.

As Justin Reich recently argued in The Conversation, it will likely be several years before we have robust research on what adaptations to generative AI work best for student learning. That's why I'm glad the PAIRR Project is collecting data across dozens of courses and hundreds of students. It's perhaps the best example of embracing-AI-but-critically I've seen in writing instruction, and I'm looking forward to learning more about what they find.

For now, however, you can learn about the PAIRR approach through my conversation with Marit MacArthur and Anna Mills. You can listen here, or search for "Intentional Teaching" in your podcast app.

Reading Strategies (with and without AI)

Last month in the newsletter, I wrote a piece about the use of AI-generated summaries of reading assignments. I argued that we point students towards a variety of scaffolds for their reading, including social annotation, pre-reading lectures, and post-reading class discussions. We can think about AI-generated summaries as part of this collection of supports, and we can also push ourselves to view these activities as ways to teach students how to read. I would like to see students who grow to be savvy about when to use an AI summary to help them with a hard text, when to seek out useful background information on a reading, and when to engage in conversations with others to deepen their understanding.

I posted that piece on LinkedIn and learned about a couple of related resources from a couple of colleagues. I wanted to share those resources as a follow-up here in the newsletter.

• Kevin Kelly recommends the Metacognitive Awareness of Reading Strategies Inventory (MARSI) to help students understand their own current set of reading strategies--and which ones they might develop. Kevin writes, "I like to engage students in active exploration to find out what helps them make sense and make meaning of course content." After his students complete the inventory, "we discuss whether or not they agree with the results, and they commit to exploring at least one concrete strategy they intend to try as a result of this self-assessment." After two weeks of testing the strategy, they report back. I really love this very intentional approach to building students' reading skills!
  ​
• Marc Watkins points to Learn Your Way, kind of a focused version of NotebookLM where you can upload a PDF and use NotebookLM tools to read and make sense of it. Unlike regular NotebookLM, you can only upload a single PDF, but you still have access to the NotebookLM audio and video overviews, mind maps, and quizzes. There's currently a wait list to get access to the new tool.

Five Teaching Applications of Custom AI Chatbots

Finally this week, I wanted to point you to a recent blog post of mine that builds on a lot of work I've been doing this fall at the University of Virginia and elsewhere helping faculty figure out what they might do with custom AI chatbots.

There are lots of ways (good and bad) that a commercial chatbot like ChatGPT or Gemini or Claude can be used in teaching and learning, but the default behaviors of these chatbots aren’t always ones we want for a particular pedagogical application. There are, however, a variety of tools for designing AI chatbots for particular purposes, from ChatGPT’s GPT tool to Boodlebox’s education features to the Cogniti platform developed at the University of Sydney, and I’ve been tracking how instructors are choosing to use these tools.

As I like to do, I've been drafting a set of categories to make sense of these pedagogical applications of custom AI chatbots, and I share my current list of categories in "Not Your Default Chatbot: Five Teaching Applications of Custom AI Bots." I define and point to examples of (1) course assistants, (2) assignment coaches, (3) tutor bots, (4) feedback bots, and (5) conversation simulators. I'd love reader feedback on this typology. If you're experimenting with custom agents, do your agents fit one or more of these categories? Or are you use them for other ends?

Fall Break Birdwatching

You might have noticed I didn't send out a newsletter last week. I was away for fall break on a cruise with my family. It was the first cruise for most of us, and my first cruise with kids. We had a blast, with highlights including the most beautiful beach I've ever seen in Cozumel, Mexico, and getting to see some 2500-year-old Mayan ruins on the Yucatán Peninsula.

I also added seven new birds to my life list! It's not easy birdwatching with three kids, but it's also hard to avoid seeing brand new birds when you travel that far from home! And I got a few decent photos along the way, which I'm happy to share here.

Thanks for reading!

If you found this newsletter useful, please forward it to a colleague who might like it! That's one of the best ways you can support the work I'm doing here at Intentional Teaching.

Or consider subscribing to the Intentional Teaching podcast. For just $3 US per month, you can help defray production costs for the podcast and newsletter and you get access to subscriber-only podcast bonus episodes.