Outcomes: The Cornerstones of Teaching and Learning

jsailor · ‎11-09-2020

Outcomes (a.k.a. standards, competencies, objectives, learning targets, etc) are the cornerstones of teaching and learning as they lay the foundation for what is to be learned within a specific time in a student’s learning journey. We're talking about those clear, concise, written descriptions of what students should know and be able to do as a result of successfully completing a unit, course, or program. Starting with the outcomes to be addressed allows the educator to use backward design to ensure that lessons, content, and assessments are laid in reference to that outcome, tracking toward mastery of the concepts and skills to be obtained.

Oftentimes, outcomes can be quite lengthy and packed with multiple skills to learn and language that might be confusing to our students. Whether teaching in K12 or higher education, we can unpack our outcomes to make better sense of them and hone in on the specific concepts and skills that students need to learn, determine the big ideas to be covered, and write essential questions that will guide our instruction and student learning. Focusing on vertical and horizontal alignments ensures that learning is connected and that skills are built upon to deepen understanding.

Within Canvas, outcomes can be used to create rubrics that assess student learning as evidenced in completed assignments, graded discussions, and quizzes. The use of outcomes provides valuable data to analyze teaching and learning within your classroom or course in three specific ways.

First, the data makes it possible to identify levels of understanding relative to the outcomes being taught and assessed. Analyzing students’ levels of understanding helps the teacher to truly know what each student knows and what they don’t know YET.

Second, the teacher is able to be responsive to the needs of students and provide the appropriate levels of support in real time. Knowing the students’ levels of understanding allows the teacher to provide targeted intervention for students at the appropriate level to encourage further learning. The teacher may need to do some re-teaching in small groups or provide tier three interventions for those who are struggling significantly. Also, the teacher can identify students who may benefit from enhancement or extension activities.

Third, teachers and students can use this data to self-reflect. A teacher analyzing this data, can reflect on his or her own teaching practices to ensure that student needs are being met and make appropriate adjustments or plans for further teaching. Students can use this data to reflect on their own learning, focused on the skills to master rather than the grade to achieve. This self-reflection for students fosters a growth mindset as they recognize that while they may not YET have a concept or skill mastered, they still have the opportunity to do so.

With all of this in mind, it is important to set a mastery or proficiency scale that is conducive to this work and makes analyzing the data manageable. Using as few levels as possible within the scale helps students to better understand their mastery levels and what skills and concepts they need to focus on as they work to master all outcomes. It also makes providing interventions more manageable and meaningful. We typically recommend 3-5 levels. If your institution already has a scale in place, how many levels does it include? What thoughts were considered as the scale was determined?

vanzandt · ‎04-07-2021

This is unique to each department and discipline. Some are set based on accreditation standards, while others are others are a collaborative effort of the faculty within that department. While it's most common to see 4 levels of mastery used, we have heard of as many as 8 and as few as 2. We need to ensure that Canvas continues to support the flexible nature of outcomes and the varied approaches that faculty will take toward program level outcomes.

Steven_S · ‎04-08-2021

Outcomes could be used to track institution-wide goals in courses with grades based on points. However, doing so becomes difficult, because the mastery levels at the institution level may not be specifically describe the details of each rating level for every assignment in every course that measure the outcome. Every assignment measuring the outcome across the institution also may not assign the same number of points to the outcome.

In order for adding an outcome to a rubric as a criteria to work well, the points and rating levels need to be customizable within the rubric. Even if there is only meets, needs improvement, and does not meet at the outcome level, there might be 5 ratings relative to an assignment. Depending on how the rating descriptions match the mastery level descriptions, there might be 2 ratings that meet, 2 ratings that need improvement, and 1 that does not meet (for example), or there might be 1 rating each for meet and does not meet with the three intermediate ratings all applying to needs improvement. While few mastery levels may be easier to interpret at the course level, point based grading of criteria with higher point values benefit from more differentiation.

Faculty need the flexibility to build rubrics that match assignment requirements, even if the rubric is associated with an institution-wide outcome. That is not currently possible for canvas outcomes, and so we are asked every semester to fill out an excel spreadsheet related to student mastery of institution wide outcomes. Those currently have 3 levels, but the rubrics in courses vary widely in rating levels depending on the needs of each criteria in each assignment.

Ideally, outcome mastery levels could be linked to individual rating levels within rubrics, and the points for the criteria and ratings would be independent of the mastery levels. (For example, as a way to specify how a 5 rating rubric relates to the 3 mastery levels of an outcome.) In fact, for some assignments and objectives, it would be helpful to link the mastery levels of more than one outcome to the ratings of the same rubric criteria. (For example, so that course and institution objectives that are measured similarly can both be linked to the same criteria.)

rmurchshafer · ‎10-04-2021

i just finally took a look at the new Improved Outcomes Management and Account and Course Level Outcome Mastery Scales Feature Previews and they completely break everything we've been doing at my school with Outcomes. The new looking interface is fine, but removing the ability to set the scale on individual outcomes will kill us.

Below is an example of many of our outcomes currently being used.

This is just the scale we use for our General Education outcomes and as you see the descriptions for each rating is going to be different for each SLO. If get why you'd want to implement a consistent, controlled scale but that is so much different than what is available right now I imagine it's going to mess up quite a few schools. This needs to either be an option to control the scales at the account level, or maybe a completely different feature than the the existing Outcomes tool currently available.

Rick

gramos · ‎10-26-2021

I was just able to use this for the first time (having just recently heard about it) and I agree with the others who have posted here and at the Overview page for this feature: As faculty, I need to be able to set the mastery scale and calc method per outcome. Yes, *most* of my outcomes use the same scale and calculation method but some do not. Ideally:

the "course-wide" mastery scale and calc method would be set as the defaults for new outcomes
we would be able to create other mastery scales (I envision something similar to the letter grade schemes- there's a default, but we can have others)
when creating/editing an outcome, we could choose the calc method and the mastery scale to use

paxton_helms · ‎11-17-2021

@jsailor , I am not sure what this means:

Within Canvas, outcomes can be used to create rubrics that assess student learning as evidenced in completed assignments, graded discussions, and quizzes. The use of outcomes provides valuable data to analyze teaching and learning within your classroom or course in three specific ways.

When I use Outcomes, I have found that rubric elements do not roll up to at Outcome. So, for instance, I cannot have Criterion A, Criterion B, and Criterion C combine to generate a score for an Outcome. They have to be graded independently.

This is a HUGE functionality gap b/c it means that Criteria do not roll up into Outcomes which means that there is not easy to way to explain how I arrive at an Outcome score without typing it manually.

Among many enraging gaps w/ Canvas and SBG, this may be at the very top of the pile.

@BenFriedman, this may be of interest to you, too, though we can discuss further on our call next month.