– Hi, everyone. I’m Niya Bond, the faculty developer here at OneHE. and I’m so excited to be joined today by Juuso Nieminen, who’s going to be talking to us about inclusive assessments. Juuso, if you would, just tell the community a little bit about yourself, where you’re from, and how you got interested in this timely topic.

– Hi, I am Dr. Juuso Nieminen. I work as a senior research fellow at Deakin University in Melbourne, Australia at the Center for Research in Assessment and Digital Learning. And yeah. In my work, I do research on assessment often from the viewpoint of student identities, belonging, inclusion. I’m quite interested in the social effects of assessment on student lives, particularly these themes. And yeah, I just got quite interested in the matters of how assessment is related to the matters of inclusion. When I was doing my postdoc research on students disabilities in higher education, and I started to focus on those assessment encounters that the students would have in university studies, and it seemed like assessment indeed seemed to play quite a major role. Of course, there are many other factors. study was this massive topics of inclusion and belonging. But yeah, I simply want to zoom in a bit closer in what exactly is the role of assessment here? It’s significant, but it’s not all there. Maybe that’s the kind of a summary of the work that I’ve been doing.

– Yeah, that’s really interesting, and I think it’s so important because you mentioned social impacts and broader, like belonging impacts, and sometimes, there can be this narrow view that like assessment is something that just is like a one and done happens in a classroom of any kind or modality, but like it has ripple effects and impacts on, like you said, on identity and other things. So that seems really important to chat about today.

– Exactly. And I always find it fascinating to research as indeed as an academic because I feel like students already have the answers to this question. Students are the ones who live through these moments and live through these experiences. So a lot of my own work has focused on trying to understand the student experience of these matters. And I do think that’s quite an important direction for our research in this area.

– Yeah, definitely. Now you mentioned the student experience. What are the things that you found, you know, related to assessment and student experience?

– Well, maybe something to begin is the idea that assessment seems to be quite a universal barrier for students in many higher education settings. Recently did review study with my colleagues Anabel Morina and Gilda Biagiotti where we had a look at all the previous published studies on the assessment experience of students with disabilities, defined very proudly in the study, and really seemed that assessment was a kind of an antagonist in these stories of these students in these studies, which makes me a bit, maybe a bit shocked as an assessment researcher. I mean, it’s a bulk of my life that I’ve devoted to matters of assessment, design, and designing students and that assessments and making sure that assessment promotes student learning. But actually, when we have a look at research on how many diverse students experience assessment in higher education, it’s not always that rosy or positive as we assessment researchers sometimes make it seem. Exams are definitely something that jumps out of the literature. That does seem to be a bit of a global issue that’s maybe a bit unsurprising because in so many different higher education systems across the globe, we do have an institution of assessment adjustments or I think accommodations would be a more the common term in the various context of providing, say extra time for students of disabilities or personal testing spaces. We do have this kind of institution that enables that exams could be, as we often said, leveled up, or the playing field would be leveled up, so to say. So in that sense is unsurprising that exams jump out from the literature. And this is often because of accessibility barriers related to, say, time limitations or the formats. And in some of my own work, I’ve really tried to dig a bit deeper in having a look at how these forms of assessments are read quite often have in higher education exams would be a good example of that. How they are quite fundamentally designed to promote certain types of knowing being often the ways in which students can represent their abilities are and, indeed, disabilities or their successes, their failures, that it’s quite narrow and one dimensional. And there are many practical limitations on that, but I do think that there’s something quite drastic going on there.

– Yeah, so you mentioned, you know, ways of knowing and doing. Can you talk a little bit more about that in relation to assessment?

– Yeah, sure. So if you think about, for example, like quite a typical situation of a closed book examination, students arrive in an exam hall, leave their backs on the side of the hall, sit down in rows so they’re not too close to each other, wait for the exam to begin. This is quite a global situation. I work with people by education systems, and this kind of situation can be identified in all of them. We often think about the accessibility of this situation from viewpoint of how we could make sure that all students can demonstrate their skills in equal ways. But then you could also have a look at the ways of knowing and being that this particular practice embodies. Now we allow students to represent their skills in a written form. They might be able to draw some pictures, images, crafts, mathematical formula, all sorts of things. But if we focus on this, say, pen and paper example, that’s the sort of a representation that we allow students to do in that moment. And then the question for me as a social analyst is what other things might be hidden here? But there are many things, embodied cognition, communal forms of being, knowing. And that’s really something that I’ve been trying to unpack in my own work. As students take part in various sorts of assessment practices, you know, what parts of themselves they can represent. When our students graduate from higher education, they are expected to be unique knowledge workers, as we often say. So who are they as students and what kinds of kind of bits and pieces does allow, assessment allow them to represent of themselves as those future graduates.

– Yeah, and probably just as importantly as you touched upon earlier is what parts of their identities are excluded by certain types of assessment, right? Because you’re interested in belonging.

– Exactly. Exactly.

– Now, some of the things you’re talking about just for me as an educator, remind me of universal design for learning. Multiple means of representation, multiple means of expression. Is that kind of tied to where you’re going or outside of it?

– Oh, absolutely. And indeed, some of my own work really concerns that this idea of university design, the idea that students should be able to demonstrate their learning in multimodal, diverse ways, and that we should also engage students in various different sorts of ways. Absolutely. That’s a big part of the work that I’m doing. Now, at least one of the criticisms that I would have towards the UDL frame is something that I’ve done myself, so this self-criticism as well, but often with these sorts of models comes the idea that as educators, we design something for our students. So this idea of incorporating student voice in the matters of inclusive practices or assessment practices, more broadly, university design definitely included. That’s certainly something that I would like to emphasize. Not, there wouldn’t already be a lot of work in universal design that incorporates student voice in one way or another, but it’s not always in there. So that’s something to emphasize.

– No, and I really appreciate that, because I feel like too much that happens in higher ed is done for students and not with students, and I know you’re really passionate about amplifying student voices and helping them be heard. So what does that entail when it comes to assessment?

– Oh, that’s a big question. Can I give you an example of of a study that I did with my colleague, Henry . A few years ago, we zoom into this one university course in the STEM field that was university designed, and there was a team of researchers doing some work on that course, and we did some mixed method research. And in these large scale findings, in terms of these large scale findings that we identified, the results were quite nice. We were able to demonstrate that with various quantitative instruments students, demonstrated quite nice learning outcomes, and they use sustainable learning strategies, blah, blah, blah, blah, blah, wonderful findings. But then we did this follow up study where we focused on three students only, students with learning disabilities and their experiences, Now, that was actually something very, very meaningful for us. We did this only after the course has already been designed and implemented. All these other wonderful findings had already been found and published. We thought, as researchers at that point, that, oh, we only have three students, like, will anyone be interested in reading this research? Often we think that we need to have bit more participant in a proper educational study. But actually what we found based on this experience was absolutely crucial. There were various accessibility barriers still in that course design that we had implemented, and it was such a powerful reminder that these voices should have probably been incorporated much earlier in the process. And of course were able to utilize those later on in the future, publish some critical work on this issue. So hopefully that will inspire others later on. But yeah, it’s a very thorny dilemma of how to incorporate that voice already in a much, earlier phase. And how to do that in assessment? I think that’s a very tricky case because assessment is often the one piece that students can’t really influence. And there are many pieces with that. There are many institutional policies. Oftentimes, teachers need to have their course or unique plan way before that course actually starts. So how could we incorporate students’ opinions and assessment in that case? That often runs against the institutional policies. That’s sort of the context but that has sort been an issue for myself.

– Yeah, as you were describing your study, I was wondering about that, you know, what did you find that these students spoke about in their own stories, and how could that be put into like practical change?

– Yeah, so in this case, the students had multiple barriers to note about the course design itself. For example, issues with digital accessibility of the learning management system that we were using, or lacking support and scaffolding when it comes to things like self-regulation, or in our case, self-assessment design, the students, some of them wish for more support on how to really track their own learning process. And these are, of course, things that might be helpful for all students, not just for this particular students. But I think something quite crucial that we also found in that particular study was that the students still, they referred themselves through these quite strong deficits views or narratives. And some of the stories that they had were almost, it was almost like they were describing this wonderful course design that was not for them, and you designed something lovely for the normal students, and I’m not one of those, so it doesn’t work for me. And that really started a much longer thinking person. I’m still on that journey of focusing on this experience of students with disabilities or disabled students who take part in these educational institutions and learn to think about themselves in those ways. And then when we talk about things like pedagogical design, or in my case, assessment design, how could it shift this much stronger discourses and narratives that we have about ourselves and our students? That’s wicked problem if anything is, but I do wonder if there are ways of, for things like universal design for assessment to really try to do that at least on a local level, try to allow our students to know themselves in ways that are not always that strongly tied to those, that deficit thinking.

– So at OneHE, one of our big things is that we always try to help educators implement something somewhat immediately in a practical way. Now, a lot of your stuff you mentioned, you’re kind of still on the journey trying to figure out. But if someone’s interested in thinking about or building more inclusive assessments, where can they start? Like, what’s a practical first step?

– This is always a tricky question for social and cultural analyst because I do think that sometimes when we talk about inclusive practices, it’s not something that we can change just like that. Whatever I do in my classroom tomorrow, I won’t be able to, you know, get rid of ableism in our societies right away. So I do think there’s always this danger of drawing too heavily on solutionism. That said, I’m a teacher. My background is in teaching. So I very much understand the drive to do something now and change something now in our own context, in our own classrooms for our own students. So I do think it’s a very relevant question. Just a bit of a disclaimer. One tip, there will be a practical tip one it really relates to that idea of student partnership in assessment. Now, within those institutional boundaries that you have those boundaries related to, say, assessment and creating and feedback policies that might surround your work, is there a way to, in one way or another, make use of the student voice in your assessment design? This could be something quite traumatic. For example, I’ve taught courses by myself where I’ve co-designed all assessment methods with my students, Which is often in many disciplinary fields, a very meaningful practice because when students graduate, they need to be able to assess their own work and they often the work of their peers or coworkers. So they do need to have these tools of, in a way, designing assessment methods. But of course, that’s not always possible. So something that could be implemented in a much more, on a much more minor scale might be that when you start a new course, you may ask your students to check the syllabus, come up with any questions that they might have. I always post my syllables to my students in a way that they can leave anonymous comments, if there’s something unclear, and I always leave some room for at least minor co-design practices that I might tweak based on those student comments and feedback. That might be something that you could be able to do. And when it comes to the inclusivity of assessment, I do think that it’s always important to leave room for safe anonymous comments to make sure that you hear the voices of the diversity of students. My own work often focuses on disabilities or disabledness. That’s of course only one arm. Part of a much more, much broader more intersectional issue here. So making sure that you capture the voices that really matter. For example, if there are some accessibility barriers related to your own assessment design, that you hear from your students. But of course there are students might not always be able or willing to share those views if that’s not really safe and anonymous. So that’s one secret.

– Yeah, I really like that. So I have two follow up questions. The first is, how do you create places where students can leave anonymous comments? Can you give an example?

– Yep, I’ve done this in kind of a lo-fi way in just with a pen and paper in a classroom. But I’ve been able to use learning management systems. Not sure you or the listeners would use in their, in your own institutions, but there are digital tools that you can use for the very, very simple purpose of students leaving comments in a way that is not tracked. So just a comment.

– Okay, perfect. And I really liked that you gave the caveat that, you know, these are larger systemic issues that we’re not gonna overcome with like one small change. It’s gonna take time and lots of intention. But I also really liked that you mentioned small ways that we can kind of rebel against those systemic issues even in our own classrooms by listening to student stories and making space for them. Not just like a one and done, but throughout a learning journey. And as you mentioned even before at the very beginning through pedagogical partnerships.

– Exactly. And doing that in assessment, not just in teaching and learning or–

– Yeah, any kind of assessment.

– I think that’s a very powerful act because so often, students can’t really have their voices heard over the matters of assessment design, policy practice.

– Yeah. Well, I’ve really enjoyed this conversation. It’s been enlightening, and I’m definitely going to be reading more about assessment and its many impacts and ripple effects. We always like to leave the last word to our experts. So is there anything about assessment, inclusiveness, student voices that you’d like to share that we haven’t maybe covered?

– Something that I could say, and this might, hmm, this is a tricky tip because it might not be that helpful for individual teachers or educators, but on a system level, I do think that it’s very important that we take this idea of student partnership or co-design to the level of assessment adjustments or accommodations as well. Because those are often the practices that students can’t co-design in any kind of way. They are determined on an institutional level. Again, tricky maybe for individual educators to implement this idea. But particularly when we talk about students in higher education, they are often the experts in matters that consult themselves. They know their own case. So having their voices heard in our assessment accommodations, design, implement it, and evaluate later on whether they’re actually helpful, that I think is something that we really must be doing in the future.

– I love that, and I love that it’s advocating to bring student voices into policies and procedures that impact them directly and that they’re a part of. So of course they should have a voice in that.

– Exactly. And maybe something to add is that, I hope I haven’t given too optimistic or an unrealistic picture of something like student partnership or student voice. Those are always messy and complex things. And it is often the case that students are not experts in assessment designs as educators are. So there always some very tricky ethics and practicalities involved in hearing the student voice as if that would be just one monolithic voice, that’s various voices that an educator might need to synthesize. Some of those voices might be something that you need to disregard or suppress. Maybe they are not very meaningful for one reason or another. So at the very least, we need to be having these tricky dialogues with our students. But I hope that I didn’t give the impression that it will always be very easy.

– Oh, no, it’s clear that there’s complication, but that it’s worth digging into that complexity.

– Exactly.

– Well, thank you again so much for your time. This has been a wonderful conversation, and I hope we get to chat again one day.

– Thank you so much.