– I am here with Derek Bruff, and I’ll let him introduce himself with his formal titles. But Derek is one of my favorite thinkers about education and technology. And so we’re gonna talk today about artificial intelligence. And Derek is the author of one of my favorite books about technology and education, “Intentional Tech.” And so I’m gonna let him introduce himself really quickly, and then we’ll get into the conversation. So where are you at right now, Derek?

– Yeah, I’m happy to be chatting with you, Jim. I am a visiting associate director at the Center for Excellence in Teaching and Learning at the University of Mississippi where I get to work with folks like our friend Josh Eyler, who’s the director there. So I’ve been enjoying getting to know Mississippi, its campus, its culture, and figuring out how I can help foster good teaching there.

– Excellent. All right, so let’s get into the conversation. Let me just sort of step back and invite you to think about or talk a little bit about what’s your journey been with artificial intelligence, generative artificial intelligence from when ChatGPT sort of burst onto the scene and then where you are now in terms of your thinking.

– Yeah, so ChatGPT was launched in November of 2022, and I happened to not be teaching right then, so I actually had a little time to kind of play around with it and read up on it. And so I think a lot of folks were taken by surprise. It was kind of a terrible time for a new transformative technology to hit the scene, late in the fall semester where faculty are trying to get through the grading and the semester and don’t have a lot of time before classes start in the spring. And so it has been a bit of a journey. I do feel like, somewhere in there I realized like this is gonna be a tool that is going to have an impact on any discipline. At the time, it was any discipline that deals with words, which is most of them. Since the technology’s progressed, I think that’s even expanded to some more quantitative disciplines as well. But, you know, part of the challenge was that ChatGPT, when it hit the scene, it could do these things. It could output text that was very much like what we often ask our students to do in pretty basic assessments, right? Here’s a question, give me three or four paragraphs in response. And that kind of, it’s not the most innovative or interesting assessment, but it’s a very commonly used assessment across higher education. And ChatGPT seemed to be really good at doing that one thing, right? And so that was I think a lot of the disruption is because it aligns so well with a very commonly used form of assessment in higher education. You know, there was a lot of discussion about the hallucinations and the falsehoods and the made up stuff. That’s still an issue with ChatGPT and the other tools but, you know, they’re getting better at that kind of thing. A lot of the tools will now integrate with search in different ways. And so it helps to do a little bit of interior error checking on what they come out with. And so I think though, if I had to describe where I’m now, I feel like there are now a robust set of generative AI tools for text and image generation. They are increasingly used in a variety of professional contexts, although a lot of professional fields are still figuring out what to do with them too, right? What are they good for? What are they not good for? What are the ethics and legalities of using them? But as I think about preparing our students to finish college or university and go into these professional fields, whatever they may be doing, I think it’s imperative that we help our students prepare to use these types of technologies well. I think many of our students will need to know how to use them. And honestly, you know, I don’t know that all of employers will have a lot of time and opportunity to train their employees in how to use them well and ethically. And so I think some of that falls on us to do.

– That’s a great point, right? So like if the training that we’re offering to students or the uses that we put ChatGPT or AI to in our courses can help prepare them to do a good job with it when they’re off campus. So tell me a little bit about some of the ways in which you observe people, teachers or students using AI, generative AI in positive ways, productive ways. And so maybe a little inspiration for folks who are watching. How they could start thinking about the positive role it could play in their courses.

– Well, and I think about some of the teaching principles that I talk about in my book that I claim are true regardless of technology, right? But they inform our use of technology, right? So our students need practice with the skills and the knowledge, the ideas, the concepts that we want them to develop. They need practice, they need feedback on that practice. And so that’s true regardless of kind of what or where or how you teach. That’s just kind of how learning works. And so I’ve seen a number of faculty use tools like ChatGPT to generate opportunities for practice for students. So because these tools are good at kind of doing certain things, like you can ask them, you can ask ChatGPT or Bing Chat or Google Bard to write something in this particular genre or in this particular style, or even to try to solve a math problem in some fashion, it’ll come out with an output that is in the ballpark, and sometimes it’s a really good output, but often, it’s not a really good output, it’s just kind of sufficient, right? It has some inaccuracies or it gets some of the tasks, but not all of the task. And so what’s great about this is that if you want your students, you know, where you might in the past have had students draft something and then do some peer review with a colleague to give each other feedback on their draft as a way to practice a particular set of writing skills or critical thinking skills or analytic skills, you can now have ChatGPT generate passable drafts of things for your students to then critique and analyze. And I’m seeing this in a lot of different disciplines, and I think it’s really nice because it takes advantage of these tools’ ability to kind of generate original stuff all the time. So every student gets a different sample something to kind of critique and analyze and respond to. It also has the benefit of teaching your students some, or at least giving them the opportunity to think about the limitations of these tools, right? What are they good for? What are they not good for in your particular discipline? And so that’s one of my favorite is to have the tool and each student generate kind of sample stuff to critique and respond to. And you know, often students are more willing to critique what the robot outputs than what their peers.

– Ah, good point.

– Right? And so if the goal is to help them sharpen their critique skills, then sometimes it’s nice to have essentially a scarecrow for them to respond to, right?

– Yeah, that’s a good point. Actually, I was thinking about this. The other thing maybe what we have to consider is like sometimes these tools are effective in pushing us forward in some way, but there might be something a little that it’s kinda like two steps forward and one step back. And I was thinking about this in relationship to two things I’ve tried: having ChatGPT create quiz questions for me, and actually, it’ll do like lots of those, like if I’m teaching a novel and, you know, I wanna just sort of make sure that students are sort of staying on top of the reading, there can be 200 questions from a large novel. And that would be hard for me to come up with all those questions, right? And that’s fine. Like, I think that’s a great use for it. But in terms of like students creating flashcards, that’s another use that you can use for it. At the same time… And so I think that’s positive, right? If you’re creating lots of flashcards and you’re using them, that’s great. But the process of creating the flashcards is actually part of… It can be a learning tool to do as well. And so you’re getting some of that benefit, but maybe not quite as much benefit as you would if you actually created them yourself too. So I think we’ll probably have to keep weighing these things, right?

– Yeah because I think, you know, what I’ve seen is that an expert in a particular area can use a tool like ChatGPT to do expert level work but a novice in that area cannot, right? It outputs something and you need to kind of evaluate it and weigh it and see is this a good quiz question? Is it accurate? Is it useful? And so when you’re doing that yourself to generate quiz questions, you’re bringing your expertise in your domain to the evaluation of that. And so, yeah, maybe ChatGPT can save you some time, but you’re still leveraging your expertise. The tricky part is when you’re having students use these tools, how can you use them to help develop that expertise when it’s not there yet? And that’s where I think you need more kind of careful scaffolded activities for students.

– Yeah. Intentional tech, right?

– Yeah. Yeah.

– Yeah, I mean that’s a really good distinction, right? The expert versus the novice because I’ve read the novel 20 times, like I don’t need, right? I know what the novel says and I don’t need to sort of… The process of creating the quiz questions doesn’t necessarily help me push forward my thinking, but the students would benefit, the novice actually, it would benefit them to do the flashcards, right? Yeah, so it’s a good distinction here. So we’ve talked a little bit, actually, we’re sort of getting into the next question here, which is to think a little bit about, so going forward, try to be intentional about our use of it. What do we need to watch out for going forward? Like what are the areas of concern that we wanna keep our eye on as we are embracing it and using it productively? But yeah, where are your concerns?

– Yeah, I do think this kind of central issue of how can we, you know, in each particular domain or discipline, how do we figure out where is it helpful to students? Where is it impeding or short-cutting their learning, right? And so I’ve been using this metaphor, I have a 20-month-old child and so he is not quite yet to learning how to ride a bike, but I will have a decision to make soon. Do I put him on a bike with training wheels, right? So with training wheels, those are the little wheels on the sides that helps balance the bike so the kid can learn how to pedal and steer, but he doesn’t have to worry about balance. Eventually though, you want to take those training wheels off so that he can learn to balance, right? But there’s also the balance bike, which is a bike with no pedals. And so he actually has to learn to balance and steer first and then learn to pedal later, right? And so the question is I’ve heard from a lot of parents who are ahead of me in this process that the balance bike is a better way to learn how to ride a bike. That getting the balance skill down first is actually more critical than learning to pedal first. And so it makes me think, in my discipline and the things that I’m teaching, what sequence would be helpful? Like where do my students need training wheels? Where do they need a balance bike? Where is the AI gonna be helpful to take up some part of the load so they can focus on some other part of learning and then later you can have them come back to the part that the AI was helping with and when is it going to be a crutch that’s gonna kind of slow them down overall, right?

– [James] Yeah, yeah, yeah.

– So like if I want my students to be thinking visually, to put together a presentation or a digital story or something where they’re using words and pictures to try to convey ideas together, a hard part of that task can be finding useful visuals. So it could be that if I have my students use an image generator that can quickly spit out kind of versions of whatever you ask it to, they don’t have to focus so much on finding copyright-free visuals that do just the right kind of metaphorical thing for them. And so that may be an area where having an AI tool can help them with kind of part of that task while they think more conceptually, what are the images I want to include in this application that help me convey what I’m trying to say? The construction or the finding of the images may be a skill that we can defer. They can develop that later if they really need to. For now, we’ll let the AI kind of do that piece for them because it’s not as central to the more conceptual thinking that I want them to develop.

– Yeah. That’s great. And let me just finish with one last question here, which is when first, actually, maybe within the last few months people have been, last few months, first, people were talking about prompt engineering. This is a really important skill that we need to teach in the future. And recently, I’ve been seeing people say, “It’s actually not that difficult to prompt ChatGPT to get you what you want essentially.” And so I think there’s one person, Ethan Mollik, who has a very useful, I think it’s called “One Useful Thing.” He has a–

– Newsletter, yeah.

– And I think I saw him make this argument and then I actually saw someone like post an image of like a set of prompts by like a novice user and it got what they needed. So like, I mean is is this a skill that we really need to focus on or what do you think?

– I do think that’s a changing target, right? So I think in 2023, yes, that was a useful skill because the tools were still kind of hard to use and hard to get the right thing out of it. I think you’ll see a few things that are changing in the area. One is that the tools will probably get better at inferring what you might want from a kind of simple prompt. Part of that is that I think we’re gonna see more tools that are tuned to particular tasks or particular domains or content domains. And so if you’re working with, so this is a example I heard. You know, like if you’ve got a big business and you’ve got a knowledge base, lots of articles with information about all of your business processes and how you support customers and all that kind of stuff, and you’ve trained GPT, an AI tool on that knowledge base, it’s gonna be really good at answering a certain set of questions about how your business works. And so your prompts won’t have to be as good to get the right thing out of them. And so that’s one thing I think we’re going to see more – ’cause ChatGPT is a general tool, right? You can ask about anything and it will give you something. We’re gonna see more specific tools that I think will be kind of easier to use for certain purposes. The other thing is that you can find recipes for prompts essentially. Someone else has figured out this is the kind of thing, you know, this is the six prompts I want to give my ChatGPT to do this kind of output and you can just follow someone else’s recipe pretty easily. Well, I think there needs to be some kind of literacy about what prompts are and how they work. I don’t know that you need to do a deep dive in being a prompt engineer. Ask me again in 12 months. We’ll see.

– Right. I think that’s probably true for everything we talk about today and everything writes about artificial intelligence, right? All right, great. Thanks so much for this conversation. Fascinating as always. And good to see you again.

– Yeah, thanks for having me, Jim. This was fun.