Learning Outcomes and Learning Objectives are the core aspects of alignment in course design, but how do you tell them apart? There is some disagreement on how these two terms are used in the field of education- this month we're learning all about them and how you can learn how to tell the difference.
This episode features four members of our Instructional Design team as they guide you through the process of developing your outcomes and how to align them within your course. Listen and access the transcript below.
Speaker 1: In today's University of Arizona's Futures in Digital Learning Podcast. We're bringing you another conversation with DL instructional designers. The way we talk about online education has become increasingly sophisticated as more instructors step into the groin realm of digital learning. So educators and instructional designers need to work together to create the most effective approach in the online modality concepts can be blurry such as the difference between outcomes and learning objectives. So in today's conversation, we explore ways to help instructors and instructional designers create clear and well-structured courses for the online modality. Stay with us.
Adam: Welcome everybody to episode three of the Digital Learning Download podcast, Futures in Digital Learning. I am Adam Davy, Senior Instructional Designer for Digital Learning, and my co-host is here.
Brian: I'm Brian Hale an Instructional Designer over at Digital Learning.
Adam: And we have an exciting conversation today with the four instructional designers from our department, here to talk about learning outcomes and alignment. I know, I know, calm down.
Laura: Hi, I'm Laura Smith. I'm a senior instructional designer and I work predominantly with global micro campus courses.
Tiffany: I'm Tiffany McClelland. I am an instructional designer and also a quality matters coordinator. And I work primarily right now with social and vegan girls sites.
Reese: I'm Reese Davis. I work primarily with the college of law and global micro campus courses.
Nicole: Hey, I'm Nicole Schmidt and I coordinate the quality matters program across campus. So I am all over the place. Sorry, I can't do any better than that.
Adam: Laura Smith, Nicole Schmidt, Reese Davis, and Tiffany McClelland all here to talk to us today about this exciting topic. So welcome all of you to our podcast. We are excited to have you awesome. Let's get to the first question. So first of all, we need to clarify some terms before we get into this discussion in this conversation. So what is the difference between outcomes, objectives, and goals? And I know Nicole, you are one of the experts in our office about this. So I'm going to let you lead this off.
Nicole: Expert is such a strong word. I'm still learning, but yes, I do work with outcomes every day and think about alignment almost every day. The difference between outcomes and objectives is a little bit frustrating because the terms are used differently throughout the field of education. Sometimes they are used interchangeably and sometimes the meanings are switched from to the other, back and forth. So I'm going to sort of break down a little bit about how they're used here at the University of Arizona and how we use them here at the office of Digital Learning. So objectives basically you have one teacher-centered and one that is student-centered. Okay. So objectives at the University of Arizona are considered the teacher-centered description of the achievement of a learning goal, right? So, an objective is typically constructed in a longer form than an outcome. It may be measurable. It may not be measurable, but it basically outlines the teacher's intention of what they will cover and how that content or that instruction will lead to a specific kind of learning outcomes on the other hand, have to be specific and measurable. And they're usually more succinct than objective. And for the purposes of quality matters as all instructors that we work with to use measurable verbs from the Bloom's taxonomy, and they should be discrete items that are used in assessment, right? So an outcome is basically an action that students will perform to show that they have achieved the learning goal, and it should be measurable. That's why we ask it to be specific and discreet.
Adam: Excellent. Thank you. That's a great breakdown of those. And I want to open it up to the others on this podcast as well, to talk, you know, what are the sticking points with talking about outcomes versus objectives with faculty? Like, is there confusion on campus? Is there a challenge that you deal with and how do you kind of mitigate that challenge? And I'll let, whoever wants to jump in, jump in and, and start with this.
Brian: Oh, nobody wants to jump in. That's frightening.
Tiffany:I think sometimes instructors have issues with sort of the backwards approach to course design because they feel like it limits their autonomy to be creative and to be flexible throughout the course. So that is one issue that, that I run into a lot. And it is really kind of a tight rope walking to, to consider the balancing between like making sure that a certain level of quality is achieved that the courses understood that the learning expectations are understood and that the teachers have the flexibility that they need.
Laura: I think I'm in terms of definitions, once instructors understand the difference. I think most of them are on board and supportive of differentiating them and using them how we in our office would like to. To kind of talk about the positive end of that, that's good.
Tiffany: Yeah, sure. There's also a lot research showing that students really appreciate a well- structured course and really understanding what their teachers expect them to be able to do.
Adam: That's excellent. I think that's probably an undervalued piece of that is the student piece and what they need as far as the, the outcomes and objectives. When you do work with faculty, once you get past this first stage, where do you start with the outcome drafting process? You know, what is the first step? How do you approach this subject with faculty Reese? Any thoughts on this?
Reese: So when I start with faculty, the first question that I ask is really just, what do you want students to learn in this course? That's usually a good opener, and we can approach the conversation later about specific and measurable and all those wonderful things that we need. But yeah, just to get started. That's my first question. And then it's a lot of ebb and flow. We talk about what students need to learn, and then we talk about how we're going to measure that, and then have we accounted for those things? And so, yeah, it's a lot of back and forth, it's conversation. It's not just provide the outcome and that's the end of it.
Laura: I really love that Reese because you're talking about setting the learning goals first, which are ways they can be way broader and more conceptual than the outcomes and the objectives.
Tiffany: I like that it's student-focused, kind of what we've been talking a little bit already of, you know, what do those students need to know? What sort of questions would they have about this content or course. Where are they going and what do they need to get there?
Adam: I like to think of it as an inverted pyramid. When I start with that way too, you start big and then you kind of narrow it down and make it smaller and more specific.
Reese: It kind of comes from experiences where, when I've just asked instructors to, to draft those outcomes and you get just tons and tons of understand, you know, these very broad things. And I thought, well, maybe we need to start with that. But instead of them thinking, well, I did the outcomes to just start with, you know, tell me what they need to understand. And then, then we have something to build on.
Adam: Do you want to make a room full of IDs, cringe, start your learning outcomes with understand.
Tiffany: Step one is understanding, which are measurable and not. The idea is that, you know with an outcome, you have to be able to measure it. You know, I'll probably repeat that way too many times, but understanding it's really difficult to measure. Maybe some cognitive scientists can presume to do it or something like that, but it's generally things like what are some other ones, understand or appreciate, the big flashing red lights come on when we hear those, those verbs in an outcome.
Laura: Yeah, so it's helpful to bring in those bloom taxonomy charts and have some ideas of verbs to spark that and help them branch out from those.
Brian: It is really amazing how many verbs I have in the past used a two sheeter that's just full of measurable verbs. It's amazing how many verbs there are. And it's also equally amazing how many instructors just rely on the learn about whatever.
Tiffany: The teacher wants their students to learn. Right.
Reese: And kind of backing up to the measurable part. I'm working with having instructors actually write outcomes for it. They have multimedia assignments, which is kind of, it's an approach that I've been thinking a lot about what are we asking students to learn from doing a project like that? And it has to do with a lot of creative and digital literacies skills that they're developing. That can be a little bit difficult to measure if you don't spend some time with it, but that's that alignment is part of it, thinking about how do we make it measurable. And when you identify it, then thinking about what artifacts do you need to collect as part of that process? So it ties in really well with how we think about outcomes of really verbalizing what it is that you're having students learn from the content. And then how do we know that they learned it?
Brian: I think it's really important from the student perspective, why is the instructor asking me to do something in this particular way? Why do I have to learn this software? Why do I have to learn a format? You know? And when you can point back to an outcome or an objective or an end goal that says so that you will be able to, whatever, it just ties it all together so nicely. And I think it helps the student understand just a little bit better.
Reese: Yeah, that's exactly right. You know, we know that a lot of instructors assign it because they think it's, you know, it's a fun approach to doing an assignment, things like that. But some of the, the feedback from students is like, well, I put a lot of work into this and it doesn't really seem to matter in my grades. So why did I invest that time in it? But when the instructor, if you ask the instructor, why the instructor usually has a reason for that, but that's not always considered and articulated to students. So it is really helpful.
Tiffany: And that's why the outcomes are definitely student-centered, because it's the teacher's way of telling the students directly what they want them to do and giving them the why that they need to really feel like what they're doing is meaningful.
Adam: That's, I think an important buzzword there's meaningful, right? As you know, how do you, how do we make the learning meaningful for students and how do we make it impactful? Right. So that they, that they want to participate. They want to engage in the course. And a lot of that comes back to those outcomes. And then also, you know, seeing that, those outcomes aligned with the rest of the activities and the assessment. Right, and I think that's sometimes the tricky part is we can have, you know, a great outcome and then not measure it with the proper activity, right. Or vice versa, have a great activity. And then your outcome doesn't match with what the activity is actually accomplishing. So thinking about that with, you know, course design, how do you approach that? Like, what's it, you know, we look at the outcomes first and then look at the activities and then is there a back and forth what's a, what are some recommendations or what are some approaches that you all take with instructors with making sure that all of those things align, Tiffany, any thoughts on that with your work? Because you, do you see a lot of courses after maybe kind of they've run one or two times too and need some of that revision as well?
Tiffany: They do. And that alignment piece is it's something that is becoming more prevalent. People want to show how their materials or their assignments or they're interwoven, because you will have, there may be pieces to your course that you're your students, just, they can't see how everything is interconnected. And if they're going to invest the time, if they're going to invest their work, demonstrating this to your students is it's, it's incredibly valuable to both you and to them. One of the really brilliant ways I had seen in different courses, and it's super simple, it's just, it's creating a chart and it's instructors can show, Hey, you're going to do this assignment. And it's connected to this outcome and you're going to be reading this and watching this video or engaging with this PlayPosit and they can see all three pieces and they can see that interconnectedness between the different pieces. And as a student and as an instructional designer, having all of those together, it makes, it makes the assignment make more sense. And it creates that meaningful expression of the work that the students are doing.
Adam: That's great. Thank you for that. And it's nice to know that too. I think, you know, something as simple as a chart to kind of visually show that for students is maybe not all you need, but you know, the great starting point, right. That you can, you can show that and express that for your course. All right. So what are, what are some of the biggest challenges that you face within this work, you know, constructing outcomes and aligning those outcomes to activities? What are some of those speed bumps? What are some of those challenges, those hang ups that you, you get with meeting with faculty, or are there any, am I making up the fact that there are challenges?
Tiffany: I think we touched a little bit on some of them, but in calibrating those outcomes, making sure that we have those accurate verbs, that are measuring what they want the students to do, and also simplifying them. Sometimes they have these great ideas and goals that they want students to do, and an outcome can be too complex, or it could be too simple. So I think also trying to find that balance that is narrow enough and focuses what you want them to do, but it's going to lead students in the right direction. So that's kind of one speed bump I've seen just trying to work through massage, those learning outcomes for say, as a quality matters reviewer, when I'm reviewing a course, I'm kind of looking at it as a backwards outline, like I'm looking at the finished product and then you constructing it.
Laura: And one of the things that the first things that I do, and one of the first stumbling blocks, I think are making sure that the course level outcomes are aligned with the module level outcomes. So outcomes are a little bit shorter than objective naturally because they're often scaffolded within each are nested within each other outcomes can be written at the course level, but will the student be able to do after they complete this course at the module level, at the activity level or the lesson level. And so part of what I'm looking at is okay, if I have all of these outcomes in module one, for example, which course level outcomes do they align with, and is there an, a general sense of an architecture to the course? Is it building, can I see how it's building towards these course level, outcomes or not.
Tiffany: That whole idea of scaffolding, your module level outcomes. So like she said, that you build up to that course level outcome and reiterating that to faculty that it's really it's a building block process. And if you view it that way, then oftentimes it makes it a little bit easier if these are the, steps to get to the, to the top level.
Laura: Helping them understand that that takes a little bit of time, because I think that, so the thing with timeline is we, you know, we want a course done. We want it quickly done quickly. But helping them see that if they invest this time upfront to create these building blocks and build this foundation, that things will go a lot smoother as we build. And as they watch be more meaningful.
Tiffany: I also think another stumbling block is sometimes when there are outcomes that are written at the institutional level, and this usually happens to the course level outcomes rather than the module level outcomes. But if a teacher is working in a program and the director or the head of the program creates the learning outcomes for the course, and if they don't satisfy QM expectations, quality matters expectations, and the instructor can't change them. That's a little bit of assembly.
Adam: Yeah. That's actually one I'm dealing with right now. So I feel that, yes. So one of the questions I get often from instructors is they're constructing these outcomes is they ask, well, how many do I need? How many do I need for the course? How many do I need for each module? And I never know how to answer that because I feel like just throwing out a number is not really being fair to this process, you know, looking at all of it. But you know, that scaffold dig in that nesting that you talked about Nicole, and in what that looks like. And so I think, you know it's important to go back to what Reese said earlier, what do you want the students to learn? What do you want them to get out of this? And that will kind of help dictate moving forward. You know, how many outcomes you actually end up with and, and what that looks like. And, you know, like Laura said, it's that process. You have that starting point. And I always like to tell instructors that this is not a one and done type situation where you write your outcomes and you move on to the next thing where these are kind of living, breathing entities that can be changed and edited and adapted as the course design process goes through. And so I think it's important to understand that and know, you know, unless of course you're stuck with outcomes that your department or whatnot, right? You can't change them, but helping, helping faculty and helping instructors know that this is, um, you know, this is where we want to start, but this is not necessarily where we end with the outcome process and to make sure that everything lines up together and fits well by the time the course launches is the most important part.
Reese: I think that is one of the tricky parts is, you know, making sure the instructor understands that it is totally okay to revise those outcomes as we go. They're not carved in any stone.
Laura: I was just going to say, when you're trying to determine how many outcomes to include when you have the authority to decide what outcomes include. I liked how Adam and Reese are both saying to just go back and think about your students' perspective and you the course from your students' eyes. And remember the outcomes are really your way of communicating to the students, what they should be able to do. So you wouldn't want them to feel too overwhelmed, but you also wouldn't want it to be too vague. You know, you would, you would want them to feel like they're in a safe learning environment where the expectations are clear, you know,
Tiffany: Along the lines of numerical, I also have a hard time giving a number. But I also try to encourage baking too. There aren't too many that we aren't getting confused with. Here's the list of assignments and here's our learning outcomes, but making it manageable.
Brian: As an instructor, if I'm working late at night or the instructional designer I'm working with is really, really busy, where can I go as an instructor to find a list of great objectives, outcomes, goals? Do we have a resource like that that's U of A approved or DL approved?
Adam: I have a page attached to the course map that I share with instructors that talks about constructing outcomes. I know that the, the assessment office on campus does kind of workshops on that and has some materials on that.
Tiffany: I know some people like Emily and physical in our office have developed a whole module on outcome, right?
Laura: Yes. I worked on that team on my course design guide it campus that is open enrollment to anyone and has a module about backwards design and learning outcomes. And it links to a couple of websites that are outcome builders. So it will guide you through the process. You can insert a, you know, what you want to do, and it will help you create outcomes and objectives. So it's another resource. Awesome. Not necessarily ours, but smiles.
Reese: If you have the ability to engage with our course breeder workshop that is run by a fellow ID, Kathy Russell, one of the first assignments that the instructors engage in is they are building outcomes. I was actually just looking at this document and she has pages of information on that. So if you're interested in that and having some more in- depth work done, check out the Digital Learning website. digitallearning.arizona.edu.
Adam: I feel like we've kind of talked about this a little bit already, but like what, what advice would you give for constructing outcomes showing alignment? And I know we brought up kind of thinking about what the students learn, what you know, when showing alignment using a chart in there what other pieces of advice do we have for our audience out there to think about when broaching this topic?
Tiffany: I would say, just make sure you keep the start with the end in mind, really, because this is a kind of this is the first step in a backwards design, so don't necessarily think I'm going to have them do this for the final project, and then write your outcomes. As, as we said, at the beginning of the conversation, start with your learning goals, the really broad notion of what you want the students to accomplish and the, and the outcomes, and then get creative and maybe resets them exhibit some advice on creative ways to connect outcomes with different instructional technologies and things like that. But that's, that's what I would say.
Adam: That's good and I like that you brought up the idea of educational technologies, right? Because a lot of times, and you all might, you know, have similar different experiences, but we have faculty instructors that come to us saying, I really want to use this tool. How can I use this tool? And it's like, well, is it the best? Like, let's look, let's look at the outcomes first. Right, and see, you know, is this the best tool to use for what you want your students to accomplish into achieve? So I think that's important to, you know, that the tool kind of comes last. And a lot of these instances and sometimes what you think is the best tool to use might not be, and there might be something that is actually better for your purposes.
Reese: That that's something that I see a lot in working with multimedia assignments is you know, the instructors who were choosing the technology first with really no idea what it is they're using it for. When, you know, if they want to include technology, that's awesome. I'm always happy to do that, but you know, let's, let's start with the what, and then we'll figure out how to do it.
Adam: And sometimes it might be an assignment there, there might be an outcome that you want the students to reach where there could be multiple technologies or multiple ways for students to get there. Right and encouraging that student choice and encouraging that, that option. And maybe some of that multi-modality for students to accomplish that outcome is a good thing to have. You know, not only is it nice for students to be able to kind of showcase their work in different ways, but also I would think as an instructor, it might be nice to kind of have a little bit of differentiation. So you're not grading the same thing 150 times, but for maybe that's just me, I like variety.
Tiffany: Yeah. As long as the broad break allows for the variety, that’s fine.
Adam: Exactly, exactly. You know, so it's, and that has to be built in obviously, but I think it's nice to know that can be an option.
Nicole: I was going to say just one more thing, when you have those clear outcomes developed in Britain, then really it just, it all comes kind of back to the verb, the main verb. So if you're saying, you know, analyze this, whatever topic that you're working on, then you're really focusing on analysis in whatever assessment activity that you're using. So certain activities lend themselves towards analysis, whereas others don't and certain technologies allow that kind of active analysis activity and others do not. So it's kind of fun because once you have your outcomes written, you can really be creative about what kind of tools you use to help students meet those outcomes. But then you really have to focus on, well, what is the verb? What is the cognitive action or ability that I'm asking the students to achieve with this? And trying to think of a good, I don't know if anyone else can think of a good example of like a mismatch, mismatched, outcome and technology.
Adam: Well, I think it's, you know when you use a term like analyze and then, they give a multiple choice quiz.
Nicole: Yeah. Yeah. It's like a go-to for sure. Whereas with a tool like voice thread, they can comment in a variety of modalities and they can analyze something that's on their screen. So there you go. That's a good example.
Adam: Thank you for kind of breaking that down and showing that example too, of how things can easily get mismatched where, you know, we think that this quizzes is asking students to analyze something, but really it's not. And so it's just, it's real easy to make that mistake. And all it takes is either changing that assessment, right. Or altering that outcome, revising that outcome to better reflect what the assessment is actually asking of the students. And sometimes it's as simple as changing a verb to make sure that it fits. And I think that's not always understood by instructors too. They think it's, it might be a lot more work on their end, but it doesn't have to be.
Reese: Yeah. I think Nicole's example really shows how, how having a solid verb in that outcome actually helps because like analyze you have options there, but you get, you know, what, what the students are learning, what they should demonstrate. And then you're choosing, does that look like a presentation? Does that look like a paper? It could be a few different things.
Nicole: That's what I see when I'm competing courses. I didn't phrase a lot the Verde's themselves. And another sticky point is that the verbs themselves can be measurable. However, it can be unclear as to how the students are going to demonstrate that skill and having talked to instructors, but it's measurable and like, yeah, that's true, but what are they doing? And once, you know, once we have that conversation, that's like, okay, let's move on. And you know, the beautiful thing about, I mean, going back to it again, that Bloom's taxonomy will, it gives you a, you know, it gives you a wide variety of different verbs that you can use. And honestly, just being, if you get stuck Google synonyms that you want to use done it, I've done it many times.
Tiffany: Another tricky thing is sometimes with the rubrics or grading criteria, remembering the outcomes when you develop that rubric because the rubric or the grading criteria is essentially the way that you prove that the student has achieved the outcome. So they just, they have to be connected to each other. You know,
Brian: You don't want your students doing all this work for only five points
Laura: Or like if you're supposed to analyze something, but the rubric just says, well, there were no, the punctuation was good. There were no grammar mistakes that you APA or whatever, but there's nothing really related to the competency or the learning that they're obviously doing the activity. If that wasn't achieved, I would love to see the TCDs for that course. Yeah.
Adam: Just check to see if there's a future topic about rubrics alone coming up and there's not, but I think we need to add that because I feel like rubrics can be a very robust topic that we could discuss and, and go on and on about. So we'll save that for another day. But I'm just planting the seed right now. Excellent. Well, I think this has been a wonderful conversation about outcomes and alignment. I want to thank all of you for participating and joining us in this month's episode of the Digital Learning Download Podcast. Thank you. And come join us next month when we are going to be highlighting continuous improvement. So taking this conversation about, outcomes and alignment and looking at how we can continually improve throughout the course process.
Speaker 5: The Futures of Digital Learning Podcast is a production of the University of Arizona, Digital Learning. If you have any questions, comments, or ideas you'd like to share with our office, go to the contact us link on our website.