– Hi everyone, I’m Niya the faculty developer here at OneHE, and I’m excited to be with you today with Vince Granito, who’s going to be talking to us about integrating AI into different types of assessments, welcome, Vin!

– Oh, thank you, glad to be here.

– I was hoping you could start just by telling our community a little bit about yourself and why, and how you’re interested in this topic?

– Okay, so my main function is I am a psychology professor here at Lorain County Community College. So we are a two year school in northeastern Ohio, outside of Cleveland, so that’s my main role, but then I am also serving this year through our Teaching and Learning Center as the AI Faculty Fellow for AI and Assessment. And so, I jumped on the whole AI thing shortly after ChatGPT came out almost two years ago now and have been sort of researching and investigating and doing a lot of stuff with it ever since, so…

– Well that’s awesome. We have a lot of AI resources, as you know, it’s a very timely topic, so we’re thrilled to have your expertise here.

– Yeah, I think it’s an exciting area and it’s also an area that is sort of causing, as you can imagine, issues between faculty who are really excited about it and then faculty who might be a little bit hesitant about the new technology.

– Yeah, I appreciate that. We have encountered a spectrum too of anxiety to excitement, sometimes both simultaneously, at least for myself. So yeah, I appreciate you acknowledging that. Well, let’s jump right in, talk to us about different kinds of assessments, integrating AI and what you’re learning and practicing.

– Okay, so I think for me, one of the big discussion or one of the big areas, especially around the debate around AI and talking a little bit about it, is so many people are putting AI sort of in the center of this whole conversation and I like to think of it, I like to reframe it for people and say that maybe what we need to have is pedagogy and sort of the science of teaching and learning as sort of the centerpiece for which AI is just one of the many orbits around that particular area and so, for me, I think a lot of, if you’re gonna use AI in your classroom, if you’re gonna use AI as a faculty member, think about it, how, okay, how is this sound pedagogy that was gonna be able to help my students be successful in being able to look at those types of things?

So for me, I think centering it in pedagogy is paramount to any of the debates that we have when it comes to AI. And so taking that, I will typically start with my classes and say, “Okay, let’s talk about then the different levels of assessment that we have out there.” So typically in higher education, we’ll talk about three levels of assessment, we’ll talk about diagnostic, we’ll talk about formative, and we’ll talk about summative. So usually it follows along where you’re at within the semester, so diagnostic tends to be more beginning of the semester, these are the ways in which we are sort of assessing students, where are they at? What are some of the preconceived notions that they might be coming into the class with? And those types of things.

Formative assessments, these are usually your typical every week quizzes or every other week tests that you might have or the various homework assignments that you might have throughout the semester. And then, the summative assessment that is sort of the wrap up at the end. So these would be your final exams, your final presentations, your final paper, a final portfolio that you might be turning in as part of that. And that type of assessment then is checking to see did we meet the outcomes? Did we meet the outcomes that we had for the class over the course of the semester?

And so for me then, the question becomes, okay, well, how can we now incorporate some AI things into then, each of those three levels? And keep in mind that probably the traditional ways in which we’ve thought about how we do things at each of those three levels has to change now in the age of AI, because obviously this is gonna… The new technology that is out there is gonna completely alter the way in which we go about doing those types of things.

– Yeah, I really appreciate you reminding us that, especially with this technology, it’s kind of still a humans first approach and the point is that as with any tool, it’s pedagogy first and then you think about how the tool is gonna fit in, which may mean like you completely overhaul an assessment or an assignment, but it’s that intentionality piece and that planning that’s really still important.

– Yeah and I think that’s been the part that’s been most exciting about this for me, so I’m 25 years into a teaching career and it’s been one of those things that has sort of re-energised me, I know a lot of my colleagues are saying, “Geez, you have a spark that hasn’t been there for the last couple of years!” And I think part of it is I have really sort of forced myself to go back and take a look at all my assessment techniques and things like that. I remember doing a virtual conference, I think two years ago, where they were talking a little bit about AI use and they had a student panel there where the students talked a little bit about the way in which they’re using AI and there was a faculty member who asked a question and said, “Can you see this from our point of view though that we’re fearful that this potentially could be a source for students being able to cheat?” And there was a student on the panel who actually said, “Can I answer that question?” Or, “Can I make a comment on that?” And she said, “We’re not cheating on your assignments because there’s this new technology out there that allows us to be able to do that.

Most of the time, the reason we cheat on assignments is because either the assignments are pretty boring or there’s no relevancy to what it is that we’re gonna be doing after we graduate.” And that sort of hit home to me and sort of forced me to have to go back and look at all of my assessment techniques and figure out, okay, what do I need to redesign? In some cases I felt like there was relevancy to what it was that they were gonna be doing after graduation, but what I realised was I wasn’t communicating that correctly to my students, I wasn’t connecting the dots for them. And so, in many cases it forced me to figure out, “Okay, well how do I frame this assignment then so that it makes it intentional on the part of what it is that I’m doing for the students to see how this is gonna be important to what you’re gonna be doing after you graduate from college.

– That’s such a powerful example and one of the things that does excite me about GAI, or this technology, it is that idea of re-energisation that you talked about, like part of teaching should be self-reflection and evolution and flexibility and adaptability and this is really causing us to look at our intention and actually meet it and revise if we’re not doing that.

– Right, right, yeah and I think it’s very easy, especially people who’ve been in a teaching career for a number of years to sort of get stuck or continue doing the feel good assessments that we’ve always done, but then, is it really helping then the students, moving forward? And so, for me, like I said, that’s been sort of the part for me is that ability of being able to take a look at these assignments and saying, “Okay, how can I redo these now?” And maybe incorporate some AI use into it to allow students, because I know then if part of that goal then is the relevancy of what we’re gonna be doing after graduation, we know employers are saying that we are looking for students who are gonna have a lot of these AI literacy skills and so that means having to incorporate some AI use into some of these different assignments and some of these different areas.

– Yeah and I’m glad you you went there because that was gonna be my next question actually. You talked about the diagnostic, formative, and summative assessments, which, across different disciplines can look like different things, but I was gonna ask you, so are you using AI to kind of rebuild those or are you integrating AI for students to work with, or both?

– Yeah, so I think it’s mainly just integrating it into some of the things that I was already doing. So let’s take, for example, for diagnostics, and really this could be adapted for any discipline, but in my case, in the term, in the case of psychology, in psychology, a lot of times what’ll happen is students will come in with a lot of preconceived notions as to what the field is about, what the nature of human nature is and those types of issues and so what we’ll do is we’ll typically start at the beginning of class, one of the first classes, and talk a little bit about, well, what are some of their ideas? What are some of the myths that they might have that are out there?

This semester, for example, it just happened to coincide, our first day happened to be during the full moon. And so one of the things that came up was, we’ve always heard that people tend to act crazy around the full moon and we’ve got a lot of nursing students, we’ve got a lot of criminal justice students, where they’re saying, “We’re hearing people who work in criminal justice,” or, “We’re hearing people who work in nursing saying, ‘Yes, people definitely act differently around full moon.”

And so what I do is once we go through a lot of those myths, what I’ll do is I’ll have students now then go home, put those myths into an AI tool, ChatGPT, CLAW, Gemini, whatever, it’s completely up to them as to which tool that they would use and then bring those back to the next class and what we’ll use that, is we’ll use that as a discussion starting point. So we’ll say, “Well, what did the AI say? What did it come up with?” And then once we start going through that, that’s now where I can now connect the dots to factual sources in the field of psychology. So the way I’ll follow that up with, with them is I’ll say, “Okay, how do we know that the output that we got from AI is accurate?” And that’s where we then get into talking about how do we look up literature in psychology? What are peer reviewed research studies that are done?

And then that’s that connection, so the assessment or the assignment that I have as part of that is to provide the output that they get from AI, but then in connection with that, the sources that they were able to find where the actual research shows, no, that’s just more of an illusionary correlation that we just tend to take notice of crazy behavior more around full moons, that there are typically crazy behaviors at all times of the month, not just when there’s a full moon. And they’re able then to follow that, follow that up with the research. And so that’s an example of how I’m allowing them to use AI, but then connecting it back to what it is that we want them to get from understanding that psychology is a science.

– I love that and I love how it’s like a double layered success skills event, they are learning about disciplines specific concepts, but they’re also, like you said earlier, learning about those AI literacies, but also just broader higher education skills like creative thinking, like research. I imagine you must have to talk about ethics, maybe, with input and output as you’re talking about research too.

– Yeah, so that’s always, when I do workshops for faculty here, I always tell them that if you’re gonna incorporate AI, there always should be that sort of dual purpose of number one, hitting the content areas that you want them to be able to hit, but then also talking a little bit about the AI literacy, the accuracy, the biases that come out sometimes on some of the outputs that you get, the ethics of the types of things. So I always have, whenever we do any type of AI, I usually have some reflection questions, what worked with the AI tool, what didn’t work? Ethically, how do you think, are there appropriate times that you think we could use AI in the field of psychology and are there times where you think it might be inappropriate?

And students are usually very creative at being able to come up and say, “Yeah, I can see if I’m a therapist, I’m not sure whether I would necessarily…” Or, “If I’m seeing a therapist, I’m not sure whether I would necessarily want that therapist putting my information into an AI tool to be able to figure out what it is that’s going on.” So they’re able to come up with a lot of the different ethical, appropriate, inappropriate uses of the AI tools.

– Nice, now I know your example’s gonna inspire educators in our community to go out and maybe try this, so I’m wondering if you can talk a little bit more just about the practical elements of kind of the take home AI assignment. Do you give your learners a specific prompt or how do you guide them to input into the AI?

– Yeah, so early on I find, and again, this is part of that AI literacy, is early on, I find that providing them with specific prompts is a little bit easier, but then as the semester goes on, one of the things that I try to do, and when I provide them a prompt, I might provide, see what I’m doing here with this particular prompt so that they’re able to see that so that by the end of the semester they might be using it a little bit more themselves. And I always try to get them to think about it in more sort of specific ways that might be individual to themselves. So for example, one of the assignments that I have for the formative assessment might be, let’s say they’ve got a concept that they’re having a difficult time being able to understand, so one assignment that I have, I teach classical conditioning and the learning perspective, and one of the problems that students always have is applying the unconditioned stimulus, conditioned stimulus, unconditioned response, conditioned response.

So an assignment that I have them do is to go to an AI tool, and the prompt that I give them is “Create a fictional case study that will emphasise or help me be able to understand classical conditioning.” But then the additional piece that I have them add to it is, “Create this case study around,” and then have them fill in some interest area of theirs. So if I’ve got a student who’s into video games, they might say, “Create the case study around a student who’s interested in video games,” or, “Create the case,” if I’ve got a student athlete, “Create a case study around soccer,” or “Create a case study,” if somebody’s into gothic novels, “Create a case study related to gothic novels,” and now what happens is when they turn those, so they turn those assignments into me, but then we also use those as discussion, small group discussion pieces, because then that now will allow the other students in class to get to know one another and understand a little bit about their background, so obviously not each person’s output is gonna be exactly the same, because everybody’s got a different interest area that they actually include as part of that. So that would be an example for something that I would do for that particular assignment and then sort of help them along with that particular prompt to be able to include then, their own information. So I sort of give them the structure, but then sort of include your own information as part of it.

– I love that and I love the scaffolding that you’ve built into it and the way that you… It seems like you’re very conscientious about returning to their use of the tool and like having them critically reflect on it, which I think is so important.

– Yeah, yeah and again, that’s a suggestion that I always do whenever we have workshops here, is there always should be sort of a reflection of the process and talking a little bit about what worked, what didn’t work, just by changing this word, how did that change? Or it could even be the multiple types of output that you get based on the different types of tools that you use. So, so many of my students now are starting to use a lot of the AI tools that are built into some of the social media platforms, so for example, you’ve got Meta in Instagram, so a lot of my students, as opposed to using ChatGPT or Gemini or Copilot or something like that, they might be using Meta in Instagram.

So how does that output compare to something that you might get in ChatGPT? Which again, I don’t really do this as an assignment, but this could be a possible assignment where you’re doing a comparison and contrasting between the different types of outputs that you get based on the different types of tools or the different types of outputs that you get based on the different types of prompts that you put in. So now, again, that could be something that could be helping learn the content material, but then at the same time helping with that AI literacy of seeing how the different tools have strengths and weaknesses.

– That’s an awesome idea and I feel like you have been a wealth, both of philosophical and practical information during this interview and I think people are gonna be really inspired, to try this out with their pedagogy and their teaching philosophy.

– Yeah, like I said, it’s an area that I’m really excited about, it’s an area that not only have I been doing with faculty, but now we’ve been doing a lot of community outreach. So my mother who’s 82 years old, keeps bugging me about, “What is all this AI stuff?” We actually had a workshop here a couple of weeks ago that we did how do use AI for seniors? And we ended up getting over 45 seniors come in that wanted to find out a little bit about just practical uses on how they could use AI, around planning a trip or planning a meal or planning an exercise program or something like that.

– That’s awesome, well, we usually always leave the last word to our experts, so no pressure, but also pressure to share one last message with the community. Do you have anything you want everyone to know about AI?

– Yeah, so I would just say, again, I think just centering pedagogy, first and foremost. What are the outcomes for the classes that you have? Either a class outcome or a program outcome and thinking about that first and foremost before thinking about, okay, how do we incorporate the AI and stuff into it? And I think what that will do is that will help sort of shrink that gap between sort of the AI enthusiasts and the faculty that might be a little bit more hesitant, because I think that’s a common agreement that we can all come to, that pedagogy really should be front and center of everything that we do.

– Agreed and thank you so much for your time. This has been such a great interview and I feel like I’ve learned so much, so really appreciate you taking the time to share with us.

– Okay, thanks!