Podcast: Data Informed Teaching Practices

Published: Monday, May 9, 2022

A conversation on exploring the idea of data informed teaching practices.

On today's University of Arizona Futures in Digital Learning podcast, we bring you a conversation with instructional designers from the Digital Learning Team as they explore the idea of data informed teaching practices.

Speaker 1 (00:04): On today's University of Arizona Futures in Digital Learning Podcast. We bring you a conversation with instructional designers from the Digital Learning team, as they explore the idea of data informed teaching practices. On today's University of Arizona Futures in Digital Learning Podcast. We bring you a conversation with instructional designers from the Digital Learning team, as they explore the idea of data informed teaching practices.

Speaker 2 (00:17): Here's the literature in these three fields. Like those are three different fields. How can we pull them together as it relates to education and specifically online learning.

Speaker 1 (00:27): Join Adam Davi, Brian Hale, and Stephanie Tammen, as they discuss how to use data and research to help inform decisions and course design and instruction and explore ideas for implementing timely feedback and reducing student’s cognitive load. Welcome everybody to this month's Futures in Digital Learning podcast. I'm Adam Davi, a senior instructional designer with Digital Learning, here with my co-host

Speaker 3 (00:57): Brian Hale, an instructional designer with Digital Learning.

Speaker 1 (01:00): And this month we are joined by Stephanie Tammen, another senior instructional designer at Digital Learning. So, Stephanie, welcome.

Speaker 2 (01:08): Thank you.

Speaker 1 (01:09): Good to have you, and we are going to talk about data informed teaching practices. So what does that mean?

Speaker 2 (01:19): Well, you know, here's the thing I might be out on an island with how I interpret it, but what I like to think about when I think about data informed teaching practices, it's more than just the quantitative analytics data that we might default to thinking about, especially with online courses, when you could track everything. Um, I, you know, I like to incorporate some qualitative data as well. So if we look at the education research literature, what are these best practices for learning that we can incorporate into the online learning space? Um, so everything from like, what, what is, what are the measures of student success? Um, how do we get there? Um, how do we keep students engaged and, you know, wanting to come back for more of your content? Um, and there's, you know, lots of, lots of good stuff out there.

Speaker 1 (02:08): Yeah. I, well, and I think the word lots is, uh, is key there. I feel like you, when you look at, at all the research there that you can use um, to, to help, you know, inform your decisions. There's a lot to take in. So what, how do you narrow it down? Like, what do you look at specifically when you're looking at all of this, uh, to help and make your decisions?

Speaker 2 (02:34): Yeah. If you're going to attempt to read everything, that's just, it would be your full-time lifelong job, and you still want to get there. So you definitely need to narrow it down, just like any research field to a specific subsection that you are interested in or that appeals to you or applies to you. Um, and when I, you know, my background, I don't have a degree in education. My background is not in this field. And so it is overwhelming and daunting to start with, um, you know, jumping in it's like there's too much, but a few things that I found to be helpful are finding the professional organizations that either have their own journals, hold their own conferences, hold their own workshops and webinars. You don't need to start with the literature. Like you don't need to start with a Google scholar search or, you know, like that's the end.

Speaker 2 (03:24): So, start at the high level, watch some good, you know, YouTube videos. If they're out there, sign up for some free webinars. Um, and one of the best things that came out of this pandemic in terms of this is there's a lot of free virtual conferences now, and they are recorded and you could go watch them, um, after the fact. So, um, yeah. Do a little bit of that level of learning first, before you jump in and then specifically some like really good courses that I've taken. One is it's free. It's on Coursera. It's, um, Barbara, Oakley's learning how to learn. Have either of you watched that one?

Speaker 1 (04:00): I have not.

Speaker 2 (04:02): Okay. Well, at one point, I don't know if this is still true, but at one point it was the most highly taken online course. I don't know if that's the right way to phrase that, but it was the most popular book out there, massive open online force. Um, so I don't know if that's still true, but she is an engineer and, um, she's sort of transitioned into this world of teaching people, cognitive science as applies to learning. So how does our brain work? How do we learn? How does memory work, attention span? Um, and then how can you apply that it's really geared towards students or people learning. So it, how can you apply that to studying or like doing some really deep learning? Um, so yeah, I'd recommend that course. And then a benefit of signing up for that course is afterwards you're signed up for her Friday listserv every Friday. They send out an email with like new books in the field, or just new things to think about, or like new blogs. So that's a good way, like, and it's very approachable. And then another course I've taken, which was through Online Learning Consortium, the online learning consortium was neurocognitive and learning sciences or bringing neurocognitive and learning sciences to your teaching or something like that. And that was a, um, synchronous online course. So you sign up for it and you do have to pay for it, but that one was really good at like, here's the literature in these three fields. Like those are three different fields. Um, how can we pull them together as it relates to education and specifically online learning. So if you're really interested in this topic that I would recommend that one.

Speaker 1 (05:30): Awesome. That's some great advice, um, and, uh, a good way to help yourself not get too lost in, uh, the thousands and thousands of, uh, results you get on, on a Google scholar search. Yeah.

Speaker 2 (05:43): Yeah. Don't do that. You'll close your computer and walk away and never come

Speaker 1 (05:47): Back to it. Yeah, for sure. So, yeah. Okay. So, you know, well, let's, you know, let's kind of dive into something specific there, um, you know, what topics are you interested in? I know what I'm interested in. Um, but, but you're the guest, uh, we want to hear from you, what, what specific topics are you interested in as it pertains to, uh, you know, improving instruction

Speaker 2 (06:11): Well, as an instructional designer, you know, there are things that I can recommend and change as someone that helps faculty build a course. I'm also teaching right now. So then those are other things that I can do as the facilitator of the course. So I feel like those are two separate Stephanie's. Yeah. Um, but as an instructional designer, when we're thinking of course design, um, I like the information that tells me about how can we reduce students cognitive load. So how can we make these courses, you know, here at U of A they're on D2L, how can we make that D2L site easy to navigate? And like students know where they are in the course, they know what the expectations are of them both. Like, what do they need to do this week in the course, but also what are they expected to learn that week? Like what do they need to walk away knowing, um, or being able to do. Um, and then, you know, within a specific assignment, like, are they able to know the purpose of that assignment and like, what, what are they supposed to turn in and when and why, and how does that relate to everything else they do. I think like, as IDs, that's a very important aspect of our job.

Speaker 3 (07:20): Now, I would say also that, that what you just mentioned, having the student know when stuff is due, what is due, all of that is helpfully covered by some of the things we try and implement through quality matters.

Speaker 2 (07:33): Yes. And, you know, uh, universal design for learning, those are both very research backed organizations or what not organizations, whatever you would call it. Um, methods, procedures, rubrics. Yeah. They're not just like coming out of the blue. Those are all, you know, they are coming out of the data, those recommendations.

Speaker 1 (07:52): And I think with that too, it's, you know, important because cognitive load that's, that was something that stuck, you know, stood out to me as well. Uh, as something is how, you know, I, I have a quote that, um, for years has always been in my mind is, you know, how can you make things as simple as can be, but not simpler. Right. Um, so how can we,

Speaker 2 (08:10): I like that.

Speaker 1 (08:10): How can we make it simple for students? Um, but still maintain that, that level of rigor. Um, but when it comes to, to D2L yeah, like it's, it doesn't have to be complicated. It doesn't have to be it, the navigation shouldn't be rigorous, the content should be rigorous. Right, right.

Speaker 1 (08:27): Exactly. Um, you know, so how, how can we do that for students? And you know, it, it doesn't have to be, uh, you know, let's overhaul everything. Um, you know, but how, how can, how can we look at making one change this semester and then maybe adding another change next semester? uh, you know, so that it's not as overwhelming for an instructor either. Um, because as someone who, who came from teaching and, uh, you know, had, has had to overhaul things before, that's a lot.

Speaker 2 (09:04): Yeah. Now you're talking to the instructor, Stephanie, which is like, this is too much.

Speaker 1 (09:10): Exactly

Speaker 2 (09:10): Too much to do at once. So, you know, I'm teaching my course for the first time right now, I'm teaching it again in the summer. And I have so many things that I want to change. Because it's just so much to do the first build and then things don't run as you suspect, right. Students interact with the course differently than you intended them to. And, you know, even though I've been an instructor I've taught in person and I've designed a lot of courses teaching a course online is a whole different perspective. Um, and yeah, you absolutely need to do those incremental shifts and add, add these things as you go.

Speaker 3 (09:44): Yeah. So what are your tips for success as an instructor to keep a record of what you want to change?

Speaker 2 (09:52): Oh, I mean, this is slow budget, Brian. I have a Google doc and every week when I'm like, oh, that didn't work. I just add a note to my Google doc. And then I would say the other big tip is like my next course, the course isn't running until July. I am already working on it. Like it's April now I'm already trying to make those changes. So I don't feel rushed to make them by July. And then another thing is I, I would love to find a way to do this better, but I'm just trying to get as much student feedback as I can. So dropping in, um, Google forms, I just create surveys in Google forms and drop them into my announcements tool so that they're on the homepage of my course site. So when students open it and they don't have to open a new link, they're embedded there in the announcements tool. So they could just like click, gimme my feedback about like, Hey, what worked, what didn't work? Was this assignment super annoying? Or was it beneficial? Like, tell me, I don't know what you, you know, what you got from it. Um, you know, I see what you turned in, but I don't know what in your head, were you like super annoyed at this assignment or was it useful to you? So, um, yeah, just trying to listen to that student feedback, um, is another thing that I would definitely recommend instructors do, um, you know, U of a, has those course surveys, the, and of course student surveys, but you don't get that. Like, that's

Speaker 3 (11:08): The end of the course. You don't get that until the very end. Yeah. Right.

Speaker 2 (11:12): If you're not teaching it for a year,

Speaker 3 (11:14): Right. Yeah,

Speaker 2 (11:15): Yeah. Yeah.

Speaker 3 (11:17): So when you get the student feedback or you get your own feedback from your Google doc, do you take the student feedback and incorporate that into your Google doc? Or do you just kind of skim through it and, and maybe pick out, okay. Five people have said the same thing. So that's obviously an issue. Let me add that to my Google doc. And then, I mean, I, I love that because I'm a post-it guy, so, okay. The frame around my computer monitor is just covered in post-it notes, which can be very helpful because you have the information right there. I don't have to go look for that inform, but there are so many, sometimes it's like, okay, uh, which, which course does this post-it note refer to? And then, so you have all this stuff in your Google doc. How do you, how does that affect the scaffolding of your course? Do you look at it and go, uh, you know, I had this course scaffolded beautifully, but it's just not working out the way I want to, based on the data I'm getting back, And then do you go back to the drawing board and start fresh? Or do you just try to do an iterative, like Adam was saying, you know, as time goes by kinda thing.

Speaker 2 (12:24): Oh, for sure. Iterative and I can give you a specific example. Um, this course is 15 weeks. Students take a quiz at the end of each there's only eight modules. So, you know, each module covers more than one week or they spend more than one week on each module. And then at the end of the module, students will take a quiz. Um, and they have some other assignments that they do throughout the week. But the deadline for each module is the same, you know, it's whatever, in two weeks, Sunday at midnight, this module will close. Guess when students turn in everything

Speaker 3 (12:56): About 10:30 on Sunday night.

Speaker 2 (12:59): Yes. So if they don't understand the content, they maybe will reach out to me. It's very rare that they even do, but sometimes they'll reach out to me Sunday at 10:30 and say like, Hey, I'm not getting this. I'm already asleep by Sunday at 10:30. So, and then they have that quiz due. Um, so like I've realized this, that I need to set some deadlines earlier in the week to make sure that they're on the right track or, you know, maybe the week prior or something earlier. So I have time to give that student the feedback before they go in to take that quiz. I mean, it's still a low stakes quiz cause there's so many of them and they have a lot of opportunity, but, or to, you know, increase their grade or to learn, use the content again. But they take the low stakes quiz very seriously. So I'm not gonna say, oh, it's just a low six. Like, don't worry about it. If you get it wrong, like they are concerned about it if they get it wrong. So I should be concerned about it if they get it wrong. So yes, my next time I run the course, I'm planning on moving some deadlines up so I can see the work they've submitted before they go in to take the quiz so I can give them that feedback. Um, and let them know whether they're on the right track or not.

Speaker 3 (14:04): And I think that is so important, letting them know while they're learning, rather than expect them to learn all this stuff. And let's see at the end, if they actually have learned it, let's find out as they're going along so that we can prevent any stumbles, any roadblocks, um, any frustration, any cognitive overload on the point because think about it when you get frustrated and you're trying to, to learn material or find something out, you're gonna go to all of your resources and they may not be the right resources. They might not be telling you exactly what your instructor's looking for. So you're scrambling at the last minute to find all of this information that may or may not be what your instructor wants you to be learning. So giving them building in that feedback into your course design, bumping things up a little, um, rather than a Sunday at midnight is the due date. Well, maybe Friday night is when the big thing is due. Or, or the, the midterm check. Let's say for that module. That way you, as the instructor can give them feedback. Now, when we talk about data informed teaching practices, what does research say about providing feedback to students? You know, there's we have automatic rubrics in D2L that can, when a student submits information kind of grade their submission based on however, the instructor sets it up, maybe a three point scale, four point scale, et cetera. Um, but I, I think coming from a student, putting my student hat on, it's really important for me to know as I'm going, how I'm doing and getting feedback from the instructor. Not only does it tell me that yes, there's a live real person on the other end of my computer screen, helping me learn this stuff. Um, but it it's, it tells me as a student, yes, I'm moving down the right path. I am learning what I'm supposed to be learning. Maybe I need to go look over at this other thing over here for a little while, but I think it's really important that we have the research that shows us that providing feedback to students is important. And let's not just do it at the end of a course.

Speaker 2 (16:12): Yeah, you're absolutely right. So there's definitely a lot of research out there that shows providing timely. That's the keyword timely feedback to students, um, is valuable for their learning. So you want that feedback, you know, it doesn't have to be like personalized, customized feedback for every student in your course. That's just not doable unless you have a small course, then by all means do that. Cause that's wonderful TA or a TA. Yeah. I mean, I have 55 students, no TA, so I'm not gonna be in there giving everybody a paragraph of feedback every week. It's just this, this is not my full-time job. So, um, instead some things that you could do, like you mentioned Brian, the rubrics in D2L build that out, that sets up student expectations. And then when you grade, you basically it's like check the box that the student meets and the student can see that. So they'll know on that scale that you set up where they fell, but then even in D2L in every, you know, in assignments, in quizzes, in discussions, there's a text box that faculty can use, or instructors can use to input text. You can include a video note. So if you don't want to, if it's faster to speak than to type, you can, you know, record a little video or audio clip for your students, telling them, um, why they got the grade they got or where they need to go to figure something out that maybe they were a little bit lost on. Um, yeah. And providing that feedback, just like you said, Brian, unless the students know how they were in terms of like learning the content, but it also lets the students know like, oh, somebody's looking at this like, it's not just a robot. That's grading my work. Like there is a human over there. Um, that's going to see this and you know, is hopefully going to care and gimme feedback about it. Um, and then the other thing you said, Brian, that kind of sparked my like research backed mind is that if students don't know where they are in a class, they might feel like lost or stressed out. And there's lots of research that shows our emotional state will depict or will determine whether we are in a, in a place where we can learn or we can take a new content. So, um, if you are stressed, I mean, we're in a pandemic still two years later, like we're all stressed. So if you are stressed, you don't know where your next meal's gonna be. You don't know like how you're gonna pay rent this, this month. You're like not super excited about learning, you know, the mitochondrial dysfunction in the, its just not a priority. And so as an, in what you do is just like, be a human. That's what I always like to say, like be a human show, your students that you care that it's totally fine, that they have emotions. Like you are here to be in their support network. Like this relationship, isn't just transactional submit your content. And I give you a grade, like let's work together and do what we need to do to improve, like improve your learning wherever you started. Let's just make sure you go forward and that's the goal. Um, so yeah, be a human know, your students have emotions, that would be the like next part besides cognitive load. That's my second, most, um, biggest area of interest and research.

Speaker 1 (19:22): I'm, I'm glad you brought that up because I, I think that's been, um, that's been high on, on the list of things that I've been reading over the last year, year and a half is humanizing education. Um, and what that means and what are some little things, you know, that we can do to, you know, kind of humanize ourselves as instructors, um, and also, uh, you know, humanize the students and realize, like you said, it's not just a transactional, you know, exchange right. Where we're not, the people have lives outside of the little box that they're using to view your content. So, uh, just little things like that, and we've, we've kind of talked about some of those things in other podcasts, uh, you know, the, the evolution of, uh, late work policies in particular has come up and you know what that means and why, why are we penalizing students for, you know, for being late and, and what that looks like and how we can make some of those little changes. Um, and I think that goes back to something, you know, we brought up earlier, right? It doesn't have to be an overhaul, Um, but what are little changes, little things that we can do to, to humanize, uh, what we do in the, in the class.

Speaker 3 (20:29): So on that same track, um, you know, we want to have our instructors be humans and show their humanity, but how, what kind of data can we collect, which humanizes our students for the instructors. I know that some instructors have a introduction discussion post in their classes. Tell us about yourself. Um, like five things. What are your hobbies? Where do you live? Why are you in this class? And, you know, pick one other random fact you want to share with the class. Um, what other ways can we get data from our students? Um, we've mentioned, uh, putting surveys in your announcements and, and the end of, uh, class survey. Are there any other ways we can get immunizations?

Speaker 2 (21:16): Well, if you build assignments that are meaningful to students so that the student can relate the content to themselves, somehow that gives you information like, oh, this student decided to study this diet type, because like, that is important to them or culturally relevant to them, whatever it is. Like that's some data that you can get like about who your student is. Um, but also if you let students decide how they turn in their assignment, so you give them options, it doesn't have to be a five-page paper, maybe that student's really like a visual person and they want to turn in an infographic or a video, like giving that student options and how they want to interact with the content that you provide. That also gives you some insight and data into like, who, who are your students? Who is this, um, population that you are interacting with every week online?

Speaker 1 (22:10): Yeah. Student voice and choice. Um, Is, is huge. And I love that. Um, and that's something that I've been personally, I've been trying to push with instructors as well. It's like, yeah. What, how do we make this relevant for students? And how do we give them some choice in their learning.

Speaker 2 (22:26): And, and it's hard. It is, as I'm learning as the instructor, I'm like, I'll just take a quiz.

Speaker 1 (22:33): Well Yeah, you're right. It, it puts a little bit of extra stress on the instructor too. Right. If you're giving them a choice to say, you can turn in a paper, you can submit a video or you can create a webpage. Well, now it's like, well, how do I grade those three different things equally? Um, right. And that's, you know, back where rubrics come in, too. So everything comes full circle on this podcast in case you,

Speaker 2 (22:56): Yeah. I mean, as long as you've thought about it ahead of time, and you've planned for these different types of assignments, then it might be, it might seem like a bigger problem than it is.

Speaker 1 (23:06): Right. Exactly.

Speaker 2 (23:07): Like, think about it, let your students know, be explicit, be clear um, and students will be happy. They'll be fine. As long as it's not like, Hey, write a hundred page paper, or record a two minute video, like, right. Those obviously are the first page. Right.

Speaker 1 (23:23): Exactly

Speaker 3 (23:24): As, as both an instructional designer and a teacher, how often do you have to resist the urge at the very last minute before the next week of content comes out to go and change your content? Because like at three, am you woke up going, oh, I have a much better idea for assignment number four. Oh, let me go put that in right now.

Speaker 2 (23:42): It's constant. It's constant this morning in the shower. I thought, I already thought of like, oh, okay, next time it runs. I'm going to do this assignment instead. Yeah. It's constant. And I think it's a little bit of like the racehorse at the gate. Like as an instructional designer, you, you see so many courses and you get to spend so much time developing courses with instructors. So it's just, I have so much information in my head that I get to now use. So, um, yeah. That's like a benefit to this job is it directly applies to my, my night job.

Speaker 1 (24:15): You know, it's funny when I, when I was teaching middle school, we had, you know, seven periods a day and, you know, I, I would kind of split things up. So I wasn't teaching the same thing for all seven periods. But, um, but man, that first period class they were the Guinea pig class. you know, they, whatever happened there, uh, if it didn't work, it got changed by second period. you know, so when you ask Brian, like how, how do you resist the urge? Like sometimes it's like, oh yeah, no, this isn't going to work. so sorry, first period. But second, period's getting a whole different lesson created in five minutes, but we're doing it.

Speaker 2 (24:51): I mean, and that's an interesting point, like that is your own data. Right. Bringing it back to the topic of using data in your teaching practices. Yeah. Like that was your own little tiny study. Yeah. Um, so how do you weigh your own, your own data to the literature? And that's like something I haven't really spent time thinking about, but I'm going to assume that people are going to prioritize their own data because that's their own experience. Yeah.

Speaker 1 (25:16): That live student feedback is important too. Uh and you know, if you can build in ways to, to get that, uh, that feedback from students throughout your course, it's not going to, not only going to help your current course, but it'll help future courses as well. uh, you know, and I, I was recently, uh, in a discussion with Angela Gunder who used to work with us here at Digital Learning now works, um, with O L C and, you know, she was saying that she has her students annotate her syllabus at the beginning the semester. Um, and so that's, you know, again, some real time feedback, right. You create a syllabus and then you let the students come in and annotate it and you can see what, what changes should be made and what, uh, you know, what can be different based on, uh, the student feedback and the student perception. So, uh, it's an interesting concept. And I, I kind of, you know, like that.

Speaker 2 (26:11): You have to be like ready to actually enact those changes. Right.

Speaker 2 (26:16): Like, Hey, tell me what you think, and then I'm going to ignore it. Yeah.

Speaker 1 (26:20): So exactly. I mean, and you know, certain things, you know, obviously, uh, I'm sure can be taken with a grain of salt. Like, you know, why I'm sure every student would say, like, we don't need to do any readings in this course, right. You know, and that's just not, not completely, uh, plausible too much learning happens in this course.

Speaker 2 (26:41): Yeah. I went to a talk at the conference. I was just at last week and they were talking about like, you know, making sure, like, you want to make sure your students are like happy and engaged in your course, but like we, it's not just student satisfaction. Right. Right. Because then they would be like, give me an A, and I'm satisfied , there's got to be a little bit more going on in there.

Speaker 1 (27:01): Exactly. And so that's, uh, you know, but again, that comes back to, you know, what, what research is out there to show, like what, how we can find that happy medium right. And find that balance. And sometimes it's, uh, sometimes it's talking to the students themselves, sometimes it's just trial and error. Right. And see.

Speaker 2 (27:19): For sure. Yeah. And you can, you know, when things are bombed, because it's like crickets or, uh, the students, you know, I use Perusal the tool Perusal in my course where they do annotate the readings. And I have been super happy with it because their annotations are like very insightful um, that's actually one thing that when I, when the quiz grades are not so great, I could look at the annotations and be like, but they were really thinking about this. So that tells me that there was something wrong with my quiz. Right. So that's been, you know, another like personal data point that I'm using to redo that for next time. But yeah, Peru's a great tool where you can like gauge your student interest or their ability to grasp the concepts. Cause if it's all questions that are unanswered, if all their annotations are like, I have no idea what's going then, you know, as an instructor, like, okay, they have no idea what's going on, let's get in there. And it's a pretty low stakes way for students to make those, those annotations or those comments, at least in my class, they're going to get points for whatever. Like, they're, they're free to say, like, this is unclear to me. I'm not grading them on whether they understood the reading, I'm just grading them on. Did they think about the reading? Yeah. Um, and I found that students are more likely to like, in that roundabout way, tell me that they're confused then to email me directly right. And say, I did not understand that reading. So that's been a great tool in my course.

Speaker 1 (28:43): Well, and that's, uh, you know, a, a good way to set something like that up, uh, you know, as far as discussion and, and annotations go, you know, where you're not necessarily asking a question and, uh, and they think they need to find the right answer. Right. Uh, in which they, you know, sometimes we'll just repeat back the question and yeah. yeah.

Speaker 2 (29:04): There's no more minimum either.

Speaker 1 (29:06): Yeah. So, um, you know, but really it's just, you know, what, what are your thoughts, right. If you're confused, say you're confused, like, that's, that's the purpose of this as well. Or if you understand, if you have, if you have questions about it, bring it up. And so again, yeah, like you said, it's a good data point, but I think the research also points to that as being a valuable way to, to have students comment and respond to readings, to content, to, to each other even.

Speaker 2 (29:32): I was just gonna say, there's that whole field of research looking at metacognition So if you ask students to reflect on their learning, so, Hey, here's this thing, how do you, like, do you understand it? And then, um, how did you go about understanding it? Like, what were the steps you took to get from? I have no idea what's going on to, like, I have a little bit of idea now. Right? How do you study what works for you? What's your personal pathway to learning? That's been shown to be very beneficial, um, to, to student content intake, um, or even like putting a, you know, a pre quiz at the beginning of the module that they're not expected to know anything like, but it, it really will highlight, like these are going to be the important things to focus on. Like, Hey, look, you got 0% on this quiz. That's fine. But that just seems like you need to think about these things as we go through this module, that's also been shown to be beneficial.

Speaker 1 (30:22): That's great. We can get lost in all the research. Right. As we mentioned the beginning. Um, and, and, you know, so feel free for those of you listening to kind of take some of the things that we're talking about here today and pick and choose what will work best for you in your course, um, or in your course design as, uh, as you see fit. And there's lots of options out there to, to kind of think about what you need, um, uh, in there, you know, do you need something that is, is going to talk about how to use quizzes better? Do you need something that's going to talk about cognitive load or feedback, or, you know, any of the other wide array of topics that we've, we've already brought up and haven't even touched on, you know, an inkling of what's out there too. Um, as well. But, um, but I, I find this really interesting to, to think about like, how can we, how can we kind of narrow our focus and use that data and use that research that's out there to, to help us without getting overwhelmed?

Speaker 2 (31:23): I think, yeah. I think you just, as an instructor, you, um, maybe feel like something's not hitting right. Or they're not landing right. Where you wanted it to, or you just, you know, maybe it's going fine, but you're like bored. You've taught this class the same way and you like want to mix it up and you think, you know, what if I do away with my quizzes? Yeah. What can I do? You know, what are other ways to measure student success? And then yeah. Get into the literature on like problem based learning or on, uh, authentic assessments or, you know, what are the different ways that we can measure? You know, whether the students grasp the concept.

Speaker 3 (32:00): And, and one great way to do that is talk to your instructional designer. So many ideas. It's your instructional designer is like a cafeteria. They, I mean, you can pick and choose what you can, what ideas they have. And they're not going to be offended if you don't choose one of their ideas because they have so many of them, it's like, you know, one of the points of being an instructional designer is to help the instructor or the faculty come up with brilliant ways to get their material across to their students. Now I want to kind of throw a monkey wrench into using data here. what do you both think the difference is between using data to inform your course in an in-person course versus an online course?

Speaker 1 (32:55): I first just want to point out that we made it about 30 minutes before we had our first food analogy in this month's episode. So, um, Usually that happens within the first, you know, five to 10, but, so, yeah. Um, but great question, Brian, uh, you know, I, I think it's, you're, you're looking at different modalities are going to have different, I guess, for lack of a better word outcomes in the course, not necessarily the learning outcomes, but just the, the way the course is structured with the, the different modalities. So the research is going to say different things. You, you can engage students differently in a synchronous in person environment than yeah. Then yeah. You will in a, in an asynchronous online environment. Um, and so, you know, from, from that standpoint, I think the research is going to point in different directions, but it's still valuable to look at that research and, and see what I mean. I think we all know that standing up on a stage and talking at your students for an hour is not the most engaging thing. so, you know, and, and in that case, the same goes for online, having your students watch an hour long video of you talking is awesome.

Speaker 2 (34:05): That's what I was going to say. Yeah.

Speaker 1 (34:07): Yeah. Um, but, um, but how you present that content, you know, can still be different. Um, and, and how you have students interact with that content?

Speaker 2 (34:19): Yeah, I would, I think you're exactly right where, um, I don't, you know, the online versus face to face teaching, I think what a lot of the research is finding is like, it's way more similar than people had thought. Like they wanted to be able to parse it out and say, there are different outcomes with these different modalities and you need a whole different like faculty to teach each type. Cause they're so different, but they're not, it's still just teaching, but how you enact the recommendations are, is different. So, um, yeah, like student engagement, if you are talking at your students in a lecture hall for an hour, that's kind of like a lecture video for an hour, but it's even harder to pay attention to a video for an hour because like you're sitting in your house like, oh, the kitchen's right there. I'm just going to make a snack. I'm going to play with my dogs. Like, there's so many more distractions when you're at your house. So for sure there are different ways that you would try to keep your students engaged and then online versus, um, in person course. Um, and then things like that, that concept of being a human, like it's easier to be a human in person. It's just easier to be funny and be goofy. Like I find myself, prerecording my lecture videos for an online course. Like I default to this giving a business talk person and then I have to constantly remind, cause it's just me in a room, you know, looking at a camera, I have to constantly remind myself like, it's okay to be funny and make a stupid joke. And I'm sure it's not funny for the students. Like, I'm sure that, like, that was not a good joke, but at least I try. Yeah. And I'm the only one there to laugh at it because it's like just me and an empty room. So it's like way more awkward to be a human online.

Speaker 3 (36:05): When you are recording your videos, do you do your announcement videos differently than you do your lecture videos?

Speaker 2 (36:12): I don't make announcement videos. I know like shame, shame.

Speaker 3 (36:18): The one thing I tell my instructors is when you're going to do your announcement videos, do them on the couch with a cat in your lap and a cup of coffee because that shows your human, you know, it showed what your interests are. And plus it's a 32nd to a one minute video. It doesn't need to have Steven Spielberg quality

Speaker 2 (36:37): And it's gonna change every semester. So yeah, don't put a lot of energy into it.

Speaker 1 (36:41): You know, I, I have an instructor who, his online lecture videos, he uses memes and funny images and sound effects and things like that. And, uh, and he does the same thing in his in-person lectures too. And so he like that. And, and I know, you know, we, we recommend to keep them short, keep them, you know, right. To the point with, with those lecture videos. But he just kind of said, no, I want to use these things. They're funny to me. And even if the student don't laugh, like it's it, but it shows his personality as well. Yes. Um, and so it, you know, it's okay to have some of those, those type of things in there as well.

Speaker 2 (37:18): Yeah, I for sure need to get better at that. That's uh, that's on my list.

Speaker 1 (37:22): Yeah. It's hard though. It it's, you know, it it's, it's not an easy thing. Um, for everybody to just get on video and make it great. you know, one, one take and, and boom. You're done.

Speaker 2 (37:36): Yeah.

Speaker 1 (37:36): Yeah. But, you know, I wanted to bring up another story too, cuz it's uh, you know, we're we talk about student engagement and you know, we, as instructional designers have access to, or, or are aware of all, you know, these different tools and you know, things that we have, have licenses here on campus, like voice thread and play, pause, things like that. But there's all kinds of different tools that we can use out there as well for engagement. And I had an instructor asked me a, what can I use? I've been using voice thread and, and Stephanie, you mentioned this, sometimes things get boring. She was like, I'm kind of bored with it. I, I don't know if it's the right thing, what other tools are out there? And we started talking about different things and, and as we circle back, she's like, you know, I, I know the students like voice thread and they know how to use it. And, and I said, well, then keep it. like, I know like maybe you're, you're bored with it, but you're getting, you're getting this feedback from students that they like it, that they enjoy it. Um, you know, you don't have to do something different if it's working. Um, just because there's a flashy new tool out there doesn't mean it's the right flashy new tool for you to use. Um, and it's easy to get distracted. I mean, I, you know, I've had my car for 11 years now and I get distracted every time I see a new car drive by me on the street. Right. You know, it's been 11 years. I could use something different yeah. But it works. There's nothing wrong with it.

Speaker 2 (38:58): Yeah. Um, yeah, for sure.

Speaker 1 (39:00): Um, you know, so as, as you're looking through all of this, you know, really look to see, you know, really think about what's working already Um, and then, you know, and then you can focus on what's not, or maybe what needs to be tweaked.

Speaker 2 (39:18): Yeah. Another thing that's come out of, um, you know, conversations I've had recently is how you teach your course or, um, what your students want from the course is going to be different depending on your student population. Right? So, Brian, your previous question about differences between face to face and online, online students can be a different population. They're not always, but it can be a different population to face to face. Like typically if it's a purely online program, they're usually a little bit later, uh, in their career, they probably have had a career and maybe they're doing a change or they're in the military and like coming back into the job market, um, or they never had a chance to go to college. They had kids now they're coming back, going to college. So they're going to have different perspectives. Um, yeah, they're going to have different perspectives on the course, which is great, cause they have life experience, which will be really interesting for them to share out. Um, but they also probably want different things from the course. Um, I've been thinking of a lot about how do we build community in a online course and make sure like the students talk to each other and uh, you know, know that they're not alone. Like they are still part of the U of A student population and they have access to all the U of A everything else that the U of a students have access to. But somebody just asks this question to me a couple days ago, you know, do they want that? Or, or do they want to come in and have a checklist and cause they're busy. So they just say, tell me what's due. And when, and I don't care if I talk to another student, I just want to get it done cause I'm busy and that's been very difficult for me to cause I'm like, no, come be my friend. Let’s talk about that. This is so great. And they're like, no, I'm busy. Like my kids are my priority. Not you instructor so, um, just tell me, you know, go ahead, Ryan.

Speaker 3 (41:11): Make a really good point because a, a lot of the folks that do online learning have had a career, they have one job, maybe two jobs, a family to raise that's a third job. Um, and knowing that data as the instructor, you can tailor your course to that type of student but you should also ask that question. What is it you want from this course? Do you want to come in and all the boxes and not talk to another and therefore I'm not going to build in that interactivity because it's not important to you or is it more important to you to learn what you need to learn from this course? So you can move on to the next course, to the next course and then leave with a degree. You know, kind of data is really important when, when we're considering the level of our course, is it a 100, 200, 300? And then what is the population that we're serving? And luckily in what we do, we have folks, we have students from all over the country, all over the world, really taking our courses. And I think the, a big difference from in person to online is in person, you have that local geographic group of students and online really you've got the world is your oyster. And there's so many different things happening in so many different parts of the world that as an instructor and as an instructional designer, you need to be aware of all those things when you're putting your course together. And that's, that's another data point you've got to really look at when you're designing all this and then, oh my gosh, wait a second. No, this is too much data for me right now.

Speaker 1 (42:46): It's true Brian, you bring up a lot of things that I, I want to comment on. I'll try to keep this a little brief here, but, um, the one thing that you brought up and, and kind of touching on Stephanie, what you talked about with our different populations is the level of course, too, right. A hundred level course versus a 300 level course, a gen ed course versus a major requirement. Right. And that's going to kind of change as much as we don't, you know, we don't want to think it will, it's going to change the student's motivation in the course. Um, because they're, you know, I think research has shown that students are more motivated in courses that are more relevant to what they want to do and what they want to pursue. so, um, and I know that, you know, we're the, the Office of General Education here at, at U of A is, is trying to, to change that, to help, you know, make some of those gen EDS a little bit more relevant and maybe we can ask them to come be guests on a future podcast as well, to talk about that. That’s something to, to, to consider too, as you're designing your course, like what, who is your course for, is it for students in a major, is it for all students? Um, and how can, how can we implement student voice and choice into this course to make it more relevant to them and, and make it, you know, more engaging just from that standpoint to the students?

Speaker 2 (44:02): Yeah. I would challenge instructors to if, if that is the case, if they're, if they know that they have a very diverse student population, don't try to understand every single student you have and their culture and their background and their part of the, you're just not going to that. Don't try to do that instead, let them teach you. So have assignments that allow them to incorporate something about who they are or who they hope to be, what career they hope to do. Um, what's really interesting to them right now. Like let them build that into your assignment. Um, and then you get to learn, and then you also, like, aren't forced to know that at the get go, um, yeah, let them come in, let them teach you. Um, and just like you said, you know, major courses within their major research has shown that students are more motivated in those courses. You could trickle that down to assignments that are more, um, applicable to their real life. They're going to be more engaged in those assignments. So yeah. I love student centered.

Speaker 1 (45:05): I love student centered. I give them, yeah. Student centered, give them ownership over their own learning. Um, you're going to see a, a big change if you start doing those things and giving them that, that power in the course. Um, I really love that. So I think on that note, um, you know, we'll end on that high right there. uh, little, little podcast high for everybody, um, uh, to end with here in our, uh, this month's episode. And we will, uh, come back next month with, and as yet to be determined topic. Uh, but it'll be fun and exciting as you know, those of you who have listened to us regularly know, um, and hopefully lots more food analogies, but, um, but Stephanie, thank you for joining us today. I think this was a really enlightening conversation.

Speaker 2 (45:58): Yeah. Thanks for, um, enlightening me and, uh, putting up with me my banter.

Speaker 1 (46:04): We love it. You're welcome. Anytime. Yeah.

Speaker 3 (46:07): And you're also writing an article on research informed teaching for our next, uh, newsletter, right?

Speaker 2 (46:13): That's right. And hopefully there'll be some action items that you could take directly and, and use it in your next course that you're teaching or building.

Speaker 1 (46:23): Awesome. Perfect plug. Thank you. Um, just make sure you read the newsletter after listening to the podcast and, uh, tune in next month, uh, for another episode and make sure you check out our newsletters, uh, which come out every other month as well. So thank you all. And we'll see, next time,

Speaker 4 (46:44): The Futures in Digital Learning podcast is a production of the University of Arizona, Digital Learning. If you have any questions, comments, or ideas you'd like to share with our office, go to the contact us link on our website.

Authored By:

Adam Davi

Adam Davi
Senior Instructional Designer

Brian Hale

Brian Hale
Instructional Designer