- Hello, everyone. I am Eddie Watson. I'm the Associate Vice President for Curricular and Pedagogical Innovation at the American Association of Colleges and Universities. I also serve as the Executive Director of Open Educational Resources and Digital Innovation. And I'm here again today with my friend, Chris.

- Hi, Chris Hakala. I am the Director for the Center for Excellence and Teaching Learning and Scholarship at Springfield College in Springfield, Massachusetts and I'm also a professor of Psychology. And we're happy to be here today talking with Michael Palmer. Michael, if you can just take a moment and introduce yourself. We have some questions we'd like to ask you.

- Yeah, good morning and thank you for the invitation. My name is Michael Palmer. I'm the Director of the Center for Teaching Excellence at the University of Virginia. I've been here now in the center for 20 years, a fairly large center of 12 full-time faculty and staff. UVA, if you don't know, is a large public institution considered an R1, so that'll help situate some of my comments today.

- Right, thanks so much for that. So I'll just dive right in. I guess the first question is something that we hear a lot these days, is about the notion of a CTL and how to maintain relevancy. And so I guess a piece of that, you know, might be centered within a question like how do you and your center actively position yourselves as a valuable resource for faculty? And so in addition to sort of like that strategy question, we'd love to hear some specific examples of things that you do to, you know, ensure that positioning.

- Yeah, absolutely. So our center is almost exclusively focused on instructional development, although we do a lot of organisational development work as well. I'm gonna focus a lot on the... I think your question gets more at the instructional development work. I think at the end of the day, Centers for Teaching and Learning are really about affecting change in instructors teaching beliefs and practices. And one of the questions you then begin to ask yourself is, why would instructor wanna change those two things, right? And so there's maybe at least five ways you can think about that. One is exposure. Exposure to new literature through things they read, and things instructors read, or through workshops that a center might put on, or through their colleagues. They might also be a disruptive force, right? And we've seen many of these in the last few years from a pandemic to social, political, racial unrest around the country to a very disruptive force in Generative AI. Maybe also the instructor has identified some undesirable outcomes. Maybe students aren't learning the level they want, or they're not engaged, or there's educational inequities that they've identified. And which leads to the kind of the fourth way is that maybe their practice is not aligned with their values, right? So they recognise that there's something's not right and so they think about, "Well, how do I align? What I do in the classroom better to those values?" And then finally, I think, you know, they change because it's a natural consequence of scholarly teaching. And so as they get more immersed in what a center offers, they become more scholarly in their approach and then want to change both their beliefs and practices more. And so if you think about those five things, you know, and designing CTL interventions around those, then for us, the primary way that we do that is through what we call our Course Design Institute. Many of these exist around the country, they come in all sorts of shapes, and sizes, and flavors. Ours has been around since 2008, and gone through continuous revision cycles, had over these days, over a thousand instructors here at UVA go through that. It sets up everything else we do. It's everything we do is either feeding into that Course Design Institute, or it's supporting on the back end of that institute. So some of the things on the front-end are very typical of CTLs, workshops, learning communities, things like that. And then you come out of the back-end and you have really immersive experiences that are on a range of topics like alternative grading, or equitable collaborative learning and things like that. So, for us, the key linchpin to everything we do is a Course Design Institute.

- All right, thanks so much, very interesting. Chris?

- Yeah, so thanks Michael for that. That's actually really interesting the way you use that as sort of your hub to do all of your activities. And so it leads us to a question of, how do you assess the effectiveness of these engagement activities? The whole notion of the Course Design Institute. What are you doing to assess, to make sure that it's having the impact that you wanted to have with both the faculty and the students?

- Yeah, of course, this is a challenge in our work. It's, yeah, impact where, right. There's a lot of impact points. There's impact on the instructor and then it's downstream, there's impact on students, and further downstream there's impact on institutional culture of teaching and learning. And as you move downstream those questions get harder and harder. And, you know, centers often default to measuring instructor's perceptions and satisfaction. Not that that's where they stop, but that's a common thread through all of this. One of the unique things about the center here at UVA is that most of the educational developers in the center are faculty, and as faculty they have scholarly expectations. And so we've taken a scholarly approach to a lot of our work, namely that as part of our own research, we're studying our interventions in a way that allow us to both understand the impact of those interventions and then also provide a mechanism of feedback mechanism to allow us to improve those mechanism, those interventions. If we take the Course Design Institute as an example, and I think many of your listeners will be aware of some of the work we've done in this space, our Centers, our CDI has been in place since 2008 and we've done pretty rigorous studies from the perception of studies, but then also measuring instructor confidence around a whole range of pedagogical practices to examining syllabi, to see how those material artifacts change after going through this, to observing courses to see if their beliefs aligned with the practices. And then really going further downstream, and this was some really cool work by Lindsay Wheeler in my center, and Dorothy Bach, that actually looked at student outcomes and they measured changes in DWF rates. You know, and part of that work has also helped us think about how has the institution changed in terms of are we more learning centered institution? We don't have great answers on that, but we have some inferences we can make on that. You know, and one other hidden thing, and I think this came out in Mary Wright's recent book on Teaching and Learning Centers is the impact of your work on other centers. And so our CDI model has been replicated at a number of institutions and pieces have been borrowed. And so it's not only just impact for our instructors, but it's impact broadly across the educational development community.

- Yeah, that's great too. I love that point about it having the impact across educational institutions because that means then the impact might again, flow through to the student learning, which is what we're trying to do. And I wanna come back to something you said before, before I give Eddie the next question. I like your notion of leaning into instructional effectiveness as what your center's doing. Not a lot of centers want to do everything for everybody, and I like the focus. So I think that's really helpful for what you're doing with looking at the assessment of the effectiveness because you're not trying to measure three million different things. You've just gotta focus. So, that's good.

- Yeah, let me just add one piece. I think it's important. We don't assess for the sake of assessing our programs. We're often not trying to... We don't set out to prove a goal for someone, prove something to an administrator. What we set out to do is answer questions that we're interested in, that we think will give us information to help do what we do better. And so that's because of the way we're set up. The types of faculty that I've hired, it's allowed us to do some really rigorous assessment work that other centers aren't able to do just because of the constraints of their context.

- Thank you, thank you. Eddie?

- Yeah, you know that is a really rigorous set of models. I mean, thinking about some of the other conversations that we've had, I think this may be the most rigorous set of sort of evaluations of effectiveness and outcomes and the like. I guess I'm wondering how you use that data to further advocate for your center. I mean, can you share examples of how you've effectively communicated with senior leadership regarding the impact your center is having on campus?

- Yeah, that's a good question and maybe we'll put effective in quotes, because it is sometimes hard to know whether it's effective or not. The fact that nobody's bugging us for other information suggests that maybe it is effective, right. And my center's pretty well funded and staffed. So, I think there's some evidence of effectiveness. You know, it's a multi-pronged effort, right. From, we produce an annual report like most centers do. We actually produce two annual reports, and my marketing communication person hates me for it 'cause it's a ton of work. One of those reports is very text heavy. And that goes to our direct report in the Provost office. The other report is very visual. It's meant for external consumption. And that consumption is, you know, folks like the university's president and Provost and Deans of the schools, and directors of adjacent center to ours that have interest in teaching learning. But then it also goes to folks that have friends of the university who've contributed through our endowment and things like that so there's that. We have a monthly newsletter, that's another way for us to get the word out. And even though it's opt-in for most people, we just sign the Deans and the Provost up. And so they get that, they get that message. And then anytime we publish any of these studies, right. I'll send this on to a group of people in particular, our direct report, and the Provost office and just say, "Here's research we've done. It's in this peer review journal. This is what we found out from that research. And this is how it's improving what we're doing." So it's really kind of active in informing of what we do. And then, you know, we've also been fortunate enough to win a few national awards through some of our research and scholarship. And so those are always an opportunity to celebrate my team members and let upper administration know. So again, so far, the folks that we report to are not asking us about numbers. We're actually, we're setting the narrative for them. We're telling them what's important? And how we think is best to gauge that impact. And so far they've taken that on good faith, I think.

- It's excellent. Thanks so much for sharing those strategies. Chris?

- So we're curious about this, and we've been asking this of a number of different directors because it's one of the biggest concerns that people have about teaching centers. And that is, how do you get people to become engaged in the center who are not the people who typically go? And it's interesting because in a place like mine where I know everybody, 'cause we're only 200 faculty, there are faculty who should be coming who don't. Who I have to work to get at events. So we're curious what kind of strategies you use to get people who might not have attended in the past to attend?

- Yeah, so I think anywhere you go, there's a continuum, right. There's faculty who... Instructors who run out to instructors who will never take advantage of your services no matter what you do. And then there are instructors who will take advantage of everything you do. And then there's the continuum in between and it's really thinking about how do I make sure that that person who is kind of on the cusp, actually tips over and dips their toe into your services. And so, you know what I laid out on that first question, the five different ways that instructor that would promote, they would encourage an instructor to change their practice and beliefs, I think can guide programming of a center, right. And so even though we know one-off workshops aren't effective at changing instructor beliefs or practices, we do know they get them excited to do other things, right? And so that's part of the strategy. We also know that... Or, I should say, we don't know all of the things that are going to create excitement through exposure, right? So you think about how can I vary this? Even if it's the same thing, even if it's a Course Design Institute, how do we let a faculty member instructor know that this institute is for them, right? So how do you couple things like designing courses with Gen AI? Or designing courses for equity, designing courses for transparency, designing foreign language courses, designing, you know, you name it. How do you open up windows so an instructor will say, "Oh, okay, I didn't realise this was for me." It is though. And so it's really thinking about all of those different kind of onboarding places where you can capture their imagination. I would say, you know, one thing that's really paid off and I'm sorry I didn't figure this out sooner. And if someone were to ask me what's the number one thing I should do when I'm starting a center? Hire a marketing and communication person. That person is invaluable. They are worth their weight in gold. Because part of the problem is consistent messaging. And if you don't have someone sending out a regular newsletter that's attractive, that is well-written, if you're not doing an annual report that people wanna read, if you're not framing workshop descriptions and titles in just the right way, you're missing a whole group of people. So, you know, I think that's probably helped us more than... All the other things I just said is just having a good marketing communication person.

- It's interesting what you say about the one-off workshops. It gets people excited. I often liken most of the workshops that we run except for like course design things, as almost like a rock show to get people to buy the album.

- Absolutely.

- You get in there and then they're excited and then they wanna talk to you. And that's where we make most of our traction is, is one-on-one meetings with faculty. So, so thank you, Michael.

- And I think it's being very intentional about that, right? So you bring a particular person in, think about what are those ancillary activities that you can have up and running right away and it might be actually before. So when we bring in a speaker, we're often set up book groups. So there's a bunch of people reading the book before they get there. The instructor comes and then there's a course design opportunity out of it or some other thing. And so you're building on that momentum right then and there. You wait a semester, it's gone, right? So you really need to capitalise in the moment.

- Thanks, Michael. Really appreciate it.

- Well, I love the notion. And I guess I hadn't really thought about it in this term because I know when I was a CTL Director, I kind of did all of the marketing myself and that's definitely not my background. So I mean, the thought that the number one priority is to hire a marketing communications person is just brilliant. And I don't think that a lot of people's minds would necessarily go there first. So thanks for sharing that idea.

- Yeah, let me give you one example. I was in a group with... It was a conversation with STEM faculty, Science, Technology, Engineering, Mathematics faculty who were talking about ways that they might improve teaching. And I happened to be on research leave that semester so I just snuck in and I listened. And there were department chairs, and senior administrators that are all sitting there and at one point someone says, "I think it would be really helpful if someone could come in and observe our classes and give us feedback." And everybody's like, "Yeah, that would be so awesome." And of course, this is a service we've done for over 20 years. Why do they not know about it? Well, they don't know about it, it's 'cause we're not consistently messaging that we do these things. And it was at that moment when I said, "Okay, we've gotta do much better at this."

- Yeah, it's a great example. And it's interesting because we do send out a consistent newsletter every week that has... It's attractive and it has all those links and it is a pain to do because we don't have a marketing person. And so we all have to pass eyes over it and it actually does eat up time. Now it does what you're saying Michael, but it's really a lot of work. Having someone, that would be their... I mean that's a great piece of advice for us to take outta this. I really appreciate that piece of advice.

- And for budgeting. Think about how do you mesh that with a, say like a project manager, which is another important position, right? So how can you get two positions outta one? I think for smaller centers, that's the way to think about it.

- Okay, great. Thank you.

- Well, Michael, thanks so much for your time today for sharing your thoughts and your expertise. This has been really enlightening. We really appreciate it.

- Yeah, thank you for the opportunity.

- Thanks, Michael.

- Bye-bye.