Early Academic Warning Indicator using data from Canvas Data Portal

[CorinnaLo]

hi, all

Washington State University has recently moved to Canvas and we are pilot testing an early academic warning system in some of our 100-level courses.  We’re looking for a consistent and scalable way to pull scores on a set of early assignments (end of week three, chosen by instructors). The sum of these scores, along with a couple of other values, would be used to flag at risk status (above a threshold the student would be on track, below they would be at risk). If a student performs below a threshold on this value, their advisor is notified so the student can receive assistance early on.

We are using the Canvas Data Portal but also are exploring the roster API as well.

We tried using assignment groups but this would require the instructor to reorder assignments outside of their preference. We also tried to set Learning Outcomes, but it is difficult as we can’t set an outcome for a group.

Current score (roster API) also isn’t too scalable since there are many variations on the use of this running score.

The lack of creating a calculated grade column (compared with Blackboard) is also proving to be limiting.

At this point we’re looking for what others have done using Canvas data.

Thanks in advance for your reply/ suggestion.

[Jeff_F]

We first created an Early Warning Report using Tableau and we now use the OBIEE system (Oracle) with our data warehouse as we can integrate SIS filters such as the academic program. Attached in the pdf is what I had posted to the Canvas Community a few years back.  Our courses each have a standard grade schema, assignments are based on the 100 pt. scale, we use assignment weighting, etc.  

Note the links at the bottom of the pdf no longer function due to a change in the community.  But this should give an idea of the approach taken.

[CorinnaLo]

hi, Jeff_F

Thank you for your reply.  I read your pdf, and have some follow up questions:

Is there a way to allow faculty to select which submissions contribute towards the early warning indicator?  There are numerous use cases where we cannot count all submissions to date.  e.g. a pre-assessment is about the student's knowledge/skill at the beginning of term, not learned material in the class, and should not be counted towards the early warning flag.  Is there a way to allow faculty to selectively tag which assignments to derive early warning?

Also, some course may require 70% score as threshold, others may require 80%.  Is there a way to put in a threshold for that?

These are issues that are course design specific.  If there is a way Canvas can be flexible to cater to these requirements, that would be fantastic.

Any advice?

[Jeff_F]

To respond - @CorinnaLo 

Sure you can do most everything you describe using a visualization tool such as Tableau.  For example, the report creator can elect to filter out first week quizzes or just those that have a keyword in the title or even quiz description such as 'pre-assessment'.  Or faculty can opt to not count the pre-assessments as a graded item. 

And varying thresholds can be established for groups of courses.  For example a course number >= 500 uses the higher threshold, while all others use the lower.  Or if there is a list of course that are exceptions, these can be specified within a calculated field within the visualization software.

You mentioned having Canvas Data.  Is that via Redshift or are you downloading the tables yourselves to your own database / data warehouse?