Episode 3 | Connected Assessment: Thinking Beyond the Learning Objective
Welcome to EduCast 3000. It's the most transformative time in the history of education. So join us as we break down the fourth wall and reflect on what's happening. The good, the bad, and even the chaotic. Here's your hosts, Melissa Lobel and Ryan Lufkin. Hello, everyone. We, as usual, have a really exciting episode planned for Today, we're digging into assessment. And this is actually going to be a part of a collection of assessment podcasts that we're going to do over the coming months. But in this particular area, we want to think about assessment and how we're connecting what's happening with student learning to objectives to broader course program and institutional assessment. Of course, we have a fantastic guest with us for this conversation. Someone who's lived and breathed in this space a long, time. Someone Ryan and I have known for a long time as well. Joel Hernandez. Joel is the CTO of Illumin Insights. He's been leading work again in this field for many years. We'll learn a lot more about him. Welcome Joel. Thanks guys. Great to see you. Yeah. Great to have you. Yeah. I feel like we've been crossing paths a lot lately of the different events up see and want to take credential summit, things like that, For our listeners, we always like to start off by asking our guests to give us a little background. Tell us a little bit about yourself, maybe something fun, in addition to all of the more technical aspects of what you do. Yeah, sure. Well, I definitely have a weird resume. The number of marine philosophers who running around is already a bit good. So I was a marine in the first Gulf War and a philosophy major at the University of Minnesota. I'm still a working poet, but have largely for 25 years been a software person. So I do think that that analogical thinking helps, and I can talk to English department. in demo. One interesting fact about me also is I'm a woodworker and much of the furniture in our house was built by me. So this desk I'm sitting at I built. So I can always tell a woodworker in a demo when I see some other guy who's put a finger through a band saw. I love that. You know, it's funny because as I look past my computer, my table in my entryway was made by my dad years ago before he passed away. So I have a soft spot for woodworkers. know that. What perhaps is part of getting to know you Joel? If you wouldn't mind. sharing a little bit about how did you end up with Illumine and even what is insights in the context, the larger context of Illumine? How I got involved with Illumine is a bit of a long story, but I'll keep it short. So I've been in the tech scene for a long time. The Twin Cities have been in the startups, been at the big companies like Seretian and Optum. And one day an old colleague called and said, my husband is going to leave his job as a provost at MINSKU and he wants to raise venture capital. He's never done it before. Can you help him? And I said, sure, why not? That sounds like a challenge. And I started as the outside advisor to what was essentially a MINSKU project as they tried to get their commercial legs for a year. And then said, who wants to do management consulting with that could bring the future of competency based Ed to bear. And I had known a little bit about it because I had actually worked at Capella 20 years ago on their IT side doing course development and bringing in Web CT. And so I knew Deb Bushway and the Capella model and said, what if we could make Capella in a box? Love it. I love that origin history. we're going to get a little more into assessment competency based education. Some of the things you mentioned before we go there, though, I've got one more personal question to ask you. And I think you and I talked about this in the past or around this. So I'm excited for this story. This is the greatest pedagogy story ever told. I love it. So here's the question. What's a favorite learning moment in your life? And I think this has to do with an eighth grade algebra story. Yes. So I love instructional design and pedagogy just as a nerd. And I've done it at the corporate level and I read. And one of my favorite examples came in eighth grade algebra. Right. That's the I think the year all the smarty pants kids have to take and West Duluth, Minnesota. And our teacher was Mr. Milroy, who also happened to own a bar in Superior. He let us know repeatedly. And on the first day of class, he said, welcome to algebra, young men and women. You are going to go on the beginning of what I hope is a long and fascinating journey with the beauty of mathematics. However, since I know half of you are boys, I'm going to give you two ways to get an A in my class. Do the work and get the A or I will let you cheat by writing Mr. Milroy, deserve an A in this class one million times. And he said, how many, this is the first day of algebra, maybe we haven't had algebra yet. And he says, how many of you would like to do the work? And of course two nerds and all the girls hands go up for like, we'll do the work. And all the kids already playing D &D in the back of the room on graph paper and their football jerseys say, I'll do the cheat. And he says, okay, well, here's what we're going to do. Get out a piece of paper. It's got 30 lines on it. And I want you to, we're going to go, I got a stopwatch handy and we're going to write. would like to get an A .N. Mr. Milroy's eighth grade algebra class 30 times and see how long that takes. And he basically walks us through the algebra problem to recognize how long it would take. Just do the work in the class. And says, and boys, especially this is the beauty of mathematics. Welcome to eighth grade algebra. That's awesome combination of personal motivation and pedagogy. that I will never forget it till I die. Yeah, that's amazing. That's such an amazing story. And it also is a great example of this is something that I always say on our podcast and in general, talk about awareness of the learning and how it applies. Like how can you, to your point of, okay, boys, here's a chance for you to understand why this isn't the right choice. We don't often do We focus on what's right and not necessarily how you can understand why something else might be right without having the framework to evaluate that. So I just dig that. Numbers like a million seem really abstract until you put the formula together and suddenly you're like, that's what halfway to the moon? Maybe I'll just do a homework problem. Exactly. I've had a few others. Another one that I loved is actually a teaching moment I gave to someone. do a lot of community work and I had at one point run the biggest community garden in Minneapolis as a volunteer thing in partnership with one of the high schools. And I had some at -risk youth helping me on the site. One of the kids had been in prison for beating his dad into the hospital. Like, And he's just out of prison and he's sitting with me with my truck and… I'm doing math to figure out how much dirt do I need? How much wood do I need? We're going to go to a landscaping site. We're going to go to lumber yard. I'm meeting with the principal of the school and he's shadowing me this whole time. And about three in the afternoon, his first day with me, he says, wait, you can convince people of stuff. I'm like, yeah, you're can do work. I'm like, yeah. He's like, you are a Marine. I'm like, yeah. And you know, math. I'm like, okay. He's like, you'd be an awesome criminal. And I had to explain to him like Travis. In the real world, when you can do those things, they make you CTO or CEO or CEO or head of staff. You don't have to a life of crime. You don't have to risk prison. Like competency does not equal crime. but was funny. He's like watching me the whole day. His opinion of me is fascinating. Is that I would make an amazing criminal. That's great. Well, and so his that story is actually beautiful to lead into kind of our first assessment question. And again, For our listeners, this is going to be the first of a collection of conversations around assessment. So we've asked Joel to be kind enough to help us set some grounding or some foundation as we have these continued conversations. And that story, he assessed you in the moment. He assessed your skills. I think one of the things that we're struggling with in education across the board is people use the phrase assessment in all sorts of different ways. Would you mind, you and I have talked a lot about this. When educators say assessment, what do they mean? so that we can then pick apart the pieces that we're going to dig into in this particular call. And we all are using assessment in the right way. So what do you think? When they say assessment, what do educators mean? Well, it's interesting because it's such a prismatic word, right? It can be in high stakes assessment, like your GRE or the ASVAB or some other evaluation quiz type thing. It could be Accuplacer. It can mean a quiz. It can mean an authentic assessment using a rubric and work. Or it can be the process quality of your curriculum on campus called assessment. And so one of the things that I think is interesting right now, and this is the heart of the white paper we just wrote, is those are, or many of those are collapsing in on each other as we get to things like competency -based ed and credentialing. in some pretty interesting ways and both from the side of the accreditors and the terms of say CCNE and the AACN. And then from a, if I have to evaluate on this many dimensions, I may as well just do it in the course and make it mean those things rather than doing it 58 times. So, you know, when I speak assessment, I'm always coming from the standpoint of I'm talking to the provost or the accreditation liaison officer the head of the nursing and undergrad and graduate programs. But then I have to translate it into, and here's how you're going to affect that in the way you want to accomplish your goals in Canvas. And so we're really connecting those two worlds in a way that is much more resonant with each other than they maybe used to be. Yeah. Clearly developing that common lexicon is really important. I mean, you've used the term CBE a couple of times. And one of the things we dug in a while ago was K12 has a slightly different definition of what CBE is than higher ed, right? Yeah, no, Sharla Long at CBEN and I have had a 10 year dialogue going back forever. And when I first joined Elumit 11 years ago, CBE meant get rid of the credit hour. Yeah. And I remember thinking at the time, how would I sell that to a president on an airplane? I probably couldn't. So I said to Sharla, I said, on the other hand, if I said to that president, How would you like to have an understanding of whether your curriculum was the challenge for a student or the student's challenge was where they are, what you need to do to help them, and knew how to bring the resources to bear to get them where they needed to get, and more importantly for you both to celebrate in their success. I would love to do that. Okay, we'll call that transparent student learning. And that's what CBE is trying to train people to get to. That I could sell a president. They don't have to blow up the entire institution. And I think that's where we're seeing a lot of the conversation. CBEN and Digital Credential Summit and the Trusted Credentials Framework are both being closer to that than everything is PLA and no longer have credits. And I think that one model helps certain types of learners, adult learners much better. Like if I were to go get a second degree, I don't think I would do it at Western Governors, frankly, because I could just fly through it. I don't know that I would go. But when I was eight, my daughter at 18, if I said, you like to go to Berkeley or online somewhere? I think I know that answer. And so I think that CBE, that competency -based ed, really is shifted from way to get rid of school to way to make school transparent. Yeah. Yeah. Well, Kate, you mentioned a couple of things here. One of them is that you've been working with Illumin over 11 years now. And I imagine the assessment spaced, again, to your point of originally CBE was get rid of the credit hour, right? And for those that are listening in that may not be, we're using some terms too. So competency -based education, CBE. credit for prior learning, might hear CPL or prior learning assessment. Those phrases, you know, it was about for a long time, how do we assess somebody's skills in order to give them credit to then create a different kind of pathway for them to learn. So I imagine those 11 years alone that you've been with Eluma and you've seen a lot of change over the time, especially as we look at outcome and course embedded assessment. If you don't mind sharing sort How have we gotten to where we are and where are we in particular today with outcome and course embedded assessment? So I think a lot of the work is being driven by the big community colleges, partly because for them, performance in a course is the curriculum more so than at a four year. Not everybody graduates. They stop out. They're self designing micro credentials there, right? There's lots of reasons to capture it at the course level in the big community colleges. But one of the other big drivers is the work on equity where we want to know groups of students are doing. And once you need student demographics, it's just gathering stuff in a drop box and sitting in a room with Chinese food and doing jury assessment as a group of people doesn't scale to thousands of artifacts. so I think irrespective of whether somebody wants to get to competency -based ed, other needs like annual or biannual program review. workforce alignment, equity planning. We want to get more real -time data and more granular data. And the way to do that is to use the tools that are now pretty ubiquitous in the LMSs. There's a rubric, there's an outcome. You can use it, you can get that data. Now we're working together to help make that easier, but I think it's both cultural availability, accountability drivers, and the fact that it's just good pedagogy, bringing it out of the juried assessment. still think there's some very important places for great assessment, especially around like quality of adging initiatives, et cetera. But it's not one thing. It's that it's more culturally available. The tools are getting better and the are getting sharper. And that's just slowly pushing everyone to the future. Is this where we drop our podcast mention of AI? It might be, it might be. And we just kind of take this on a left turn really quick in the role of AI. Well, I think AI is an interesting, I have a number of friends who have, how they have incorporated. AI into their pedagogy is among other things they've tried to incorporate it into, class prep, quiz prep, whatever, but is more of that I have to go meet my tutor at Oxford or the student present. So they say, you can use AI. I want you to tell me how you used it, how good a job it did, and you're going to tell that to the class. And suddenly now the demand for having to be socially and intellectually aware of what you've produced kind of actually went up a notch. Yeah. The other aspect too, I think, as we talk about CBE and these personalized learning pathways, I don't even think we've scratched the surface for how AI is going to power some of those personalized learning journeys. To me, and that's actually some of the work that we're doing, whether it's the quality of the learning outcome authoring, the alignment of them to other objectives, a thing that just amazes me doesn't exist yet in the world is I want to get these skills at this resolution, show me how I can do that on your campus. I don't even know if your governors can do that. I don't think anybody can do that at this point. I mean, that's the. Kind of a holy grail out there somewhere. know, and Matt Patinski and I have talked in the past about the what is Ed Tech's mother of all demos that famous Xerox PARC Douglas Engelbart. And I said making that technology interoperability project from 2015 work where I could get a competency from a curriculum system into an LMS, into an Ed planner, into my wallet and onto my transcript without anybody lifting a finger. And I think that's it's less than 12 months away. I think the ecosystem is going to get there. I know I got to my one at TechEd, but I think that's coming. Yeah. I'd love to believe it was in the next 12 months, but I don't know if we move that fast. So you mentioned earlier a white paper that you recently published, which we will, it's actually available right now on our community hub, but we'll make sure it's available in the show notes as well. Tell us a little bit about what prompted that and more about that. Well, I think one thing, and again, I've been very active in C -BEN for 11 years. I've been very active in One Ed Tech. Case is one of my babies. And so I'm on the steering committee of CASE and the steering committee of Coopernes of Learner Record and Open Badges. And I think one of the challenges for those of us who do all that work, right, meeting with the T3 committee at the Chamber of Commerce and we know all the acronyms, is we get all excited about what Nate Otto showed us or what the team at Territorium did or what feedback Fruits has got going on with group assessment and competencies. And we're like, it's here any second. And I have to often step back a minute and go, yeah, but the provost I'm talking to didn't go to all that stuff. And she doesn't spend all day just on that. She's got policy stuff and tenure promotion stuff and curriculum changes and accreditation and the president. And so I wanted to step back and say, okay, this is really written for provosts and their teams. How do they get an understanding of what is possible in their space in particular possible using canvas as the center of all of their assessment enterprises? And so it's really written to be taken, take a breath. I know you could be reading Toni Morrison or Tolstoy. This is just 16 pages. And I think we did a pretty good job. The team did. My name's on it, but it was a group effort. Really thinking through what is that series of thoughts I want to have to make me go, you know what? We can do this in our culture. There's a path for our culture to get to this. Not, only that other school can do it. One of the things that you commented on earlier that just made my heart sing was it's just good pedagogy to do this, right? And I think what's interesting about the white paper that I think a broad audience will really get value out of it from the real tactical, how are we thinking about doing assessment today and what does that look like to the broader, why do we do it? The thing that I appreciate about the white paper is that it's not necessarily leading with technology. It's not, here's how you use these tools to do this or to do that. It's actually leading with the practice, which I think is what makes it really compelling. for readers to start to associate, okay, I can connect my dots. It's this dot connection. That said, technology is important behind all of this. You started to talk about the potential of AI and some of those dot connecting moments. How else does technology open up opportunities for us when we think about assessment and the way we do this, particularly around outcomes course program, this assessment of what we are offering? What role does technology play in that for you? Well, I think one of the starters, and if you look at the feedback that CBEN and others have gotten, I'm sure if you met with the team at Quality Matters, they would have a similar opinion and a dog named Rubrik. And is it can be daunting. I mean, we're talking to a bunch of nursing programs that are doing reinvention for the AACN Essentials Toolkit and rethink your curriculum map when you have seven programs, undergrad and grad and 158 courses. It's just a big task. It's a big thinking task. And then say, go rewrite all your rubrics and update all your quizzes and then update all your core shells. So I think one of the things that technology, especially in the age of natural language processing and ML. graph databases, like a lot of the nerd stuff we do under the hood is to make it easier to get that stuff right. Bring me design cues. Show me hints in my curriculum map. Cause you found something in a syllabus. Show me a high quality learning outcome example. So that's one of the things we originally wrote our AI learning outcome authoring tool as a way to just understand what was in canvas. So, I found a string in canvas. It says it's an outcome. What does it mean? Does it have Bloom's taxonomy in it? And I said, you know, who'd really love that? The end user. So it was originally just a backend processing tool and turned it into a learning outcome quality recommend, like a Grammarly for learning outcomes. And about 128 people have downloaded the free AI thing on schools on the canvas thing. So people are using it. And what's interesting is I think a lot of tasks around this get to a funny place in people's psyche. I have to write a better rubric that my student is going to see. I don't want my student to judge how I judge them. I have to put this learning outcome on a comprehensive learner record. And it's not just this thing I'm doing for a pro forma exercise for data. I better clean that and they just get the task at it and they go, well, maybe we'll not have that be transparent yet. And so I think a lot of these tools, whether they're design aids, Grammarly -like functions are gonna be the bumpers that keep people more comfortable getting there, then they're not gonna replace faculty's local knowledge. None of these things know what a faculty knows. But they know how to do stuff and they're by definition pattern makers, pattern matchers. And so if you need people to follow a pattern, they'll be good at helping them get there. And so I think that's going to be, it's going to be helping with curriculum maps. It's going to be identifying where do we offer that badge or the skill that's underneath that badge and I'll go make it happen. It'll really be an accelerator and a refiner of thoughts, not a replacer of thoughts. Yep. I absolutely agree with that. You threw out a really compelling scenario earlier relating to curriculum mapping and planning. Let's talk a little bit more about the role that plays around assessment. I think the Curriculum Map is the most underused tool on campus and I wish people understood it more and of course, you know, I'm at the bottom of the rabbit hole looking up at you. But one of the most interesting things about it is it's the Rosetta stone of your curriculum. It is not just that this is a stats class. It's that this stats class impacts 78 program, right? If you have to do AAC and essentials toolkit one and two, depending on whether they're undergrad or grad, you want to know where their NCLEX ready and you want to know how your PLOs for the three different nursing programs are performing and how this course supports That's like a lot of dimensions and a curriculum app can understand all of those and give you data on it. Not only that, help you find opportunities for either assessment or learning. That signature assessment should go here because that's at the top of the scaffolding in our curriculum map. And so a lot of times it gets built as just a picture of what we did and not a roadmap for where we need to go. And I think when it becomes a roadmap for where you want to go, it's a much more powerful tool. Yeah. I mean, years ago, we spent a bunch of time with a popular curriculum mapping software. It was so hard. Most institutions. aren't looking out for six years in the future and understand what their course catalog is going to be. And so that makes the ability to create those interactive course maps very difficult. But again, that's one of those things that a lot of the work we're going to be doing here. A lot of it out of the hood has already done is start to say, Hey, we notice work happens here. That's not on your curriculum app. Would you like to put it there? Because again, I have a lot of books, a lot of records. I probably can't name them all, but the University of Minnesota has more courses than I have records in their catalog. so to hold that in your human mind and manipulate it is just a tool that things like graphs and machine learning were better at than us. Yeah. Yeah. And if I can make a, for all of our listeners out there that they may not think curriculum map impacts them or they are just one part of that contribution. have a quick little story. So, you know, I teach in a certificate program. And it makes it so much easier if I understand the context for where my course comes among the rest of the courses. And we've done some alignment activities recently and I can see a direct impact on how my students perform because I am not repeating things that are elsewhere. I'm reinforcing key things that came before me. I'm setting up things for what comes after me and I get the benefit of better work product out of students. So like that alone, so if you're a teacher listening to this, It's not that curriculum mapping and planning is just something that you contribute to, but it can really change your life. And I think to Joel's point here about it being underutilized or undervalued in some places, there's so much goodness that can come from it from every person's aspect. Yeah. Well, and I think that example you gave, like a feature we get asked for a lot is can I, and we have a faculty dashboard we add to Canvas. that shows them how they did. they're like, can you show us how our students have done in prior classes? And can you show us what I'm preparing them for? And I'm like, interesting feature, but I don't think they get that that only happens because there's a curriculum. And so as we look to, right, as anybody who's ever taught knows, you don't have enough time in the class. Everybody moves at a different pace. This is part of the Bloom's two Sigma problem, right? So everybody moves at a slightly different pace and you're always scrambling in your mind. I can't cover what everyone needs and everything in my curriculum. how do I get the most out of it? And if you knew week one, what everybody looked like, that would be so much better. could tailor. So yeah, that I think the curriculum app is also the key to personalized instruction. Yeah. And that's a really nice segue. We've got listeners that are from other technology companies, listeners from higher education, listeners from K -12. As we think about looking to the future of education, K -12 and higher ed are coming together more than ever. Much more overlap. Yeah, so much overlap. mean, and even not even just things like obvious opportunities like dual enrollment programs, but even just in the style, how they're thinking about preparing learners, how they're teaching, how they're assessing. Where do you see sort of that intersection happening and sort of what as you know, all the things we've talked about from AI to technology to just planning. How does that all come together? Well, one of them, again, another story not effective, it is my daughter, she's at South for Junior at Berkeley. It's happening so fast. don't know anymore. And she just brown knows me by getting home from Cabo and showing herself doing her physical chemistry homework saying, thank God I'm back at Berkeley. And I'm like, nobody from Cabo wishing they could be studying doing physical. No, no. love you kiddo. So she, when she was doing dual enrollment at Minnesota, she took a history of science class and she went to a very progressive school. where they got a rubric with each assignment, they filled it out, and then they got the assignment back with the rubric to do their own metacognitive development. That was how her K -12 district rolled. Two of her classes. It's impressive. Minnesota went to Berkeley. So it's a very high performing high school. And when she took this class at Minnesota, the history teacher just gave her a 78 with no comments. And it was during COVID so I could hear her screaming at him during his office hours. And she lit him up because she grew up with the Fitbit and an iPhone and game trees and rubrics in her classes and she wanted transparency. And when she just got an opaque, you got a B minus or a C plus and no criteria and no criteria were given her. Yeah. She lost her mind. She wants that data. Yeah. And so I think one thing that's happening is when I went to, when I was in college in the eighties, I couldn't even imagine the idea of a grade protest. questioning a professor. Are you kidding? They're gods, right? I thought my professors at University of Minnesota were legends. Yeah. And these kids now are like, they're self -aware in a way that I was not. And part of their self -awareness is what are the criteria on which you're going to judge me? What do I learn next? How do I learn it? My daughter always meets with her GSI, her graduate student advisor and her advisors. She does not miss an office hours because she wants to know like, what was, what does this look like? What happens next? And so I think that's bleeding into higher ed in a way. Yeah. And a parallel story, you my daughter, who's a freshman at the University of Utah, she's been in Spanish immersion since she was in first grade. And she was told that entire time that she would step immediately into a minor in Spanish, especially if she went to a Utah school. And when she found out when she got to the University of Utah, no, in fact, she's not being given credit for that. And she'll essentially have to go through the entire, all of the courses that anybody starting from scratch would. And it's hard for her because she's frustrated. And I say, look, you have the benefit of being fluent in Spanish. to her, she says, well, I don't have any way of demonstrating that. don't have a minor. I don't have a credential for that. And it's so interesting that she's, I just see the practical benefit of it. And she's saying, look, no, I should be given credit for this. And that misalignment is actually causing a lot of kind of disenfranchisement. She doesn't want to continue in the Spanish program because of that. That's frustrating. Yeah. But I think that's where both curriculum apps, where is stuff like, it's actually interesting how Illumin got started way back was actually on paper. at the University of Minnesota in the 80s. And it was to run the Bachelor of Liberal Arts and Master of Liberal Arts, Master of Liberal Studies programs, which were the self -designed undergrad and master's degrees. And you had, they built these paper worksheets for saying, well, you have to look like a U of graduate. So I need your gen ed and tell me what a Bachelor of Science in environmental philosophy looks like. What does one of those know how to do? There's your program learning outcomes. Now let's go find where in their curriculum you can become those. can get those personal curriculum map. Then they said, how do I know you're making progress on this degree consisting of random courses, undergraduate research and essays? How are we going to assess that? Let's make a rubric. And so was this toolkit that is not much different than Illumin is today. That was basically a paper toolkit that they made for building and assessing individualized curriculum at the University of Minnesota. And so when you say, do these tools are, they're useful. Why are they useful to things like personalized learning, competency -based ed, equity plans? It's the same set of tools with a different use. It's a toy it's the same Legos. And so I think that's what's so interesting is how powerful that ability to understand the context in which a student can learn and how you measure it and then to what uses you can put them. Don't reify them to a single context. Yeah and following up on Melissa's point you've seen that alignment of K -12 and higher ed but we're also seeing a massive increase in adult learning right that the reskilling and upskilling of the rise of non -degree programs credential programs what the assessment world looked like associated with that, right? That's a place where I think corporate America is actually far ahead of higher ed in the sense of when I've worked at big companies, half of my November was filling out things like succession plans and training. And so Right? If your target or your Walmart or your ADP or your Amazon or your Boeing, you know, all the skills you don't have and what those people need to have when you get it and how you're plugging it and how your career pathing people. so I think, yeah, that would be great if there was that handshake to say, I need this set of skills. Can I get that in continuing edit the university of Minnesota or continuing edit Anoka Ramsey? Or can I get that at the corporate training center? Can we partner? Can corporations kind of become the OPM for some of that stuff instead of the OPMs, especially in a local place? And so, yeah, I think that area of lifelong learning is going to be huge. And I think it's going to be a big place where the combination of having your curriculum be badge aware is also going to be very effective. Let's take that one step further. So thinking about the future of assessment, you just started to hint at that. This conversation has just opened up thoughts for me around, I don't know, I'm a bit of an organizational nerd. There's like an underlying organizational element and transparency element to what skills, outcomes, and how they're assessed needs to look like across the span of lifelong learning. But what else? What do you predict the next, let's say five years, what's going to happen in assessment in the next five years, Joel? Well, I think it'll get a lot easier because the sense of shared practice is going to very rapidly come to the fore. Like every year, digital credential summit and CBEN get more interest, CB exchange get more interesting because the conversation is less. Here's one way the conversation has shifted. Can we ever get there to how are we going to get there? That shift has been big and it's been during COVID. So I think one is the general sense of practice is going to get rich enough that the, what's the famous Bill Gates wrote ahead saying that, it always takes longer than you think, but changes more than you expect. And so I think we're getting to an inflection point on that. But I think the biggest thing is we'll be surprised at how reliant students and faculty become on having that sense of understanding once they have its power. And I'll give you a story is what during COVID, one of my poet friends was using canvas. She teaches at Utah and she posted on Facebook. I've never used a rubric before and they're amazing. Do you guys use rubric? And they're followed this raft, like hundreds of responses of poets talking about rubrics. And some of them hated them. They're the dumbest thing ever. They destroy poetry. Others talked about how useful they are even in their graduate seminars. And just watching that conversation of, know, having a rubric isn't just for your creditor. It's actually a very interesting conversational lens that changes how you have a discussion.It's not meant to just be some cheese grater or push over the student to see what they look like. It's a conversational tool about learning. And I think the technology, when it's too hard to use, I don't feel like I'm having a conversation with you, LTI tool. I feel like you're in my way, right? So as the technology recedes into the background and just becomes part of your LMS or part of your process and the practice goes those two are converging at a rate that I think in five years, we're just gonna take for granted that here's the assessment, here's your pathway, here's where you are, here's your next opportunity, here's your aligned co -curricular opportunity, here's that conditional release module just for you, because you need it. And I know that about you. That level of personalized learning and transparency is gonna come in ways that are easier than we think and more ubiquitously than we're expecting. Yeah. You gave me a walkthrough of Illuminate Insights a couple of weeks ago that was incredible. You guys are really pushing the power of assessment within Canvas. What have we not talked about on the podcast? So what did we not cover? You threw in the AI side piece. You know, one of my, and it may not fit in the theme, so you can edit it out all the way to, Joel. Thank you, Joel. And then just leave it. But here's one of the things that I think about. So I'm one of those generations where my junior high, that same junior high that Mr. Milroy gave the great algebra thing, was we had kids going on work release to the steel plants already in eighth grade in the seventh or 1980, I guess, was when I was in eighth grade. And I took lithography. I took letter I took drafting, I took wood shop, I took welding. I am Mr. ShopCraft of Soulcraft. And I think those things helped me become a better writer, a better programmer, a better leader. They taught me how to measure twice, cut once. They taught me how to estimate. They taught me how to think about risk.
And so when we look at this AI skills -based future, we want well -rounded people, things like self -awareness, risk assessment, critical thinking, creativity are going to become incredible skills. And I think they get learned in the material world in interesting ways. So one thing I hope is that our AI future brings back shop. Interesting. Yeah. Yeah. I had thought about it that way, but that is actually really compelling. But just when we think about what is that matrix of skill, like one interesting thing I know there's a startup, I can't remember the name that that Ryan Craig and team have invested in that's trying to do internships for people like philosophy majors. Cause the computer science kids all know what to do. My niece is a speech pathology major at UW Wisconsin, and she's already got a summer internship for the speech pathology K -12 system. Certain fields, there's our, everybody in nursing gets a job because there's planes of nurses coming over from the Philippines and elsewhere already. Same with teachers nowadays. And so certain fields, there's a pretty well -defined roadmap. I think a lot of the rest of the people want that serendipity would be the wrong word, but show me the experiences I could have in this and they're concrete. It's not a philosophy paper on Heidegger. It's being a UX researcher. Yeah. There's that increased conversation around experiential learning, but that can feel very internship, right? I like what you're saying, is almost, help it be part of the exploration of where I want to go. Yeah, no, I think, that's that curriculum map. Where can I learn creativity? Who has work experience available or an internship available? I think that when I said earlier, curriculum maps, shouldn't be an autopsy or a record of what we did. They should be a map to where we need to go. Yeah. It's those things. It's where can we embed these experiences? Where would they best fit? How can we deliver that suddenly now unlocks that ability for those students and those advisors and those faculty to all have one conversation about what I'm gonna go look like. That's the real vision for me is each institution, I think we say this in the white paper, is gonna have their own path and their own model, because they have their own mission and culture and values. What it all is gonna resolve down to though, is it's easier than ever to get transparency. And once you taste what it feels like, you will never go back. Awesome. So good, so good. There's so many great parts of this conversation, Joel can't thank you enough for it. And before we send everybody off on their way, I do want to make note that we will link in the show notes a bunch of the things that Joel's referenced here, and not just some information on insights and that work, the white paper, of course, but even organizations like Winnet Tech, CBEN, some of the other things that he's mentioned. We'll give you resources, especially for those of you if you're new to assessment and you're new to particularly outcomes course, you know, program, institutional assessment, that component of it. We'll make sure to provide a lot of resources for you in the show notes where you can dig in.
Thanks for listening to this episode of Educast 3000. Don't forget to like, subscribe, and drop us a review on your favorite podcast player so you don't miss an episode. If you have a topic you'd like us to explore more, please email us at Instructurecast at Instructure .com, or you can drop us a line on any of the socials. You can find more contact info in the show notes. Thanks for listening, and we'll catch you on the next episode of Educast