Outcome and Rubrics: Accreditation and Assessment Integration with CANVAS

[weiwu]

As the department's assessment coordinator, I would dream that my colleagues could use outcomes and link them to specific assignments/quizzes/projects in Canvas. However, we've run into the following problems:

1. In terms of the connection between outcomes and assessment measures (e.g. quizzes, tests, projects, discussions, etc.), Canvas requires an outcome to have a criteria with performance levels, pretty much like a type of rubric. However, a grading rubric is usually used for evaluating student performance for a specific assessment measure. So each learning outcome may have several assessment measures (per our accreditation standards, it requires at least 2 measures and 1 has to be direct measure). In this case, assessing the learning outcome should really be the summary of results of these assessment measures. But in Canvas, if you want to associate an outcome with a specific assessment measure, it has to be part of the grading rubric, or simply be the rubric (with only 1 criterion!). Besides, in determining if an outcome is achieved, we usually set an minimum standard and target, e.g. "80% of the class has to be 80% or better for the assessment measure". This is not possible with the current settings of Outcome in Canvas.

To give a specific example, let's say I want to assess oral communication as an outcome, and use a project presentation as the assessment measure. Then I've got a whole grading rubric with 5 criteria and 5 levels of performance with a total of 100 points to assess students' presentation. In this case, I don't really want to use the "Oral Communication Outcome" as the rubric, instead, I'll use the grading rubric to grade students, and summarize their performance to see if 80% of the students scored 80% or better to determine if this Oral Communication Outcome is achieved.  

2. In terms of the mapping between program outcome and course outcomes. There is no direct mapping mechanism available for the two. I read some previous posts that suggesting the account outcome could be the program outcome, but even that, there is no direct mapping between the two. In most accreditation scenarios (e.g. ABET, ACCE), accreditation standards and learning outcomes should be mapped with course outcomes, and assessed at course level. How Canvas can help address this?

My questions are very specific and quite wordy, I apologize for that. But any suggestions will be greatly appreciated. 

[jwadec]

 @weiwu , I have noticed the same.

It seems that there are many limitations with the Outcomes tool. I hope it will get some love soon. We have some of the same goals for assessment at our institution and I have experienced the same pain points as you have described.

[weiwu]

Hello Justin,

Thanks for your comments and I appreciate your understanding of what I was describing. I feel that as more institutions and accreditation bodies are adopting outcome-based assessment standards, CANVAS really should look into this and come up with a viable solution. Let's cross our fingers and hope that's something they have on their priority list.  

[mkollman]

We have been beating this bush too and thought maybe we were missing something.  Canvas has a unique way of collecting the information, but not really doing what we want for real outcome measurement at the Course/Program/Institutional level.  Glad to see others with the same thoughts!

[david_leahy1]

We really need the ability to associate/link criteria in a rubric to specific outcomes so we can report on student achievement in relation to outcomes not just overall mark. I might have more than one criteria in an assignment that are assessing the same outcome or outcome group. The criteria might be worth different amount of marks/points. In the current set up, an outcome must have a set range of points while in my assignments, items (the same outcome) are worth varied amounts of points/marks. Totally useless for most teachers unless I am missing something?

[Boekenoogen]

We are too struggling with the usage of Outcomes. From our perspective, it is a simple rubric fix. If an Assessment can have two rubrics: One for assessing the student's achievement in the assignment and a second one that is only tied to Outcomes. If this could be achieved, I think Outcomes tool in Canvas would be a better tool. It may not have all of the bells and whistles that everyone wants, but it would at least fix the problem of how rubrics are used in association with Assessments and Outcomes.

Dr. John Boekenoogen

University of Oklahoma 

[charris6]

We'd (Weimar Institute) like to use Canvass with assessment of our learning outcomes as well. I've put in our institutional and general education learning outcomes and associated rubrics for each, and we use them to create rubrics to assess key course assignment, but extracting this information into something useful or linking Institutional level outcomes to program and course level outcomes doesn't appear possible in any useful level. I tried generating a "report" but it doesn't give any helpful information.

[learym1]

We have also worked with this system and with Canvas on development of new tools to help Higher Ed.  Long story short, over 4-5 years, there has been no development on the Canvas side.  We also use Canvas data to export and manipulate data as needed per program.  There is potential to use this data and build something outside of the system. 

One big downside that I have found with Canvas Outcomes are that they are dependent on the instructor selecting the correct criteria in the rubric for the outcome.  For us, this happened on a very low percentage with our instructors.