Assignment Submissions Report Programming

[kona]

Background

Due to Financial Aid Federal Regulations our College must show the last date of attendance for dropped students. For online students we define attendance as the last date the student submitted any type of assignment in Canvas. The Business office and Financial Aid office need this information but due to the difficulty in obtaining the information from Canvas Online Learning staff was tasked with getting the information. Online Learning staff found it very time consuming to manually locate this information in Canvas, so they worked with the College’s Internet Systems Specialist to create a better way to quickly  and easily access this information. A web interface allows you to select the student and course. “Course Information” image is an example of what shows up once you select a student from the web interface.

The submission report (Example shown below) shows every assignment in the course, whether the student submitted the assignment, and the date of submission if an assignment was submitted or the due date if it wasn’t submitted.

Submission information is now so easy to access that the Business and Financial Aid offices are able to run their own reports, saving Online Learning a lot of time and energy.

Rationale

What was the need for the submission report?

• The College needed access to a list of every assignment a student submitted in the course and the date the assignment was submitted.
• Accessing submission information, including when the assignment was submitted, in Canvas is time consuming and requires multiple clicks for every assignment the student submitted.

Methodology

Fetch the Data

There are three or four API calls that are used to obtain the report.

1. Obtain the Canvas User ID and (recommended) Canvas Course ID for the student and course if you don't already have it cached locally. I recommend storing them in your database to speed up processing. If you don't store these locally, use the Enrollments API to obtain both the course ID and student ID in one call. We cache both of these locally, so this step isn't required.
   1. If no data is returned, stop, because you won't be able to do any processing.
   2. If data is returned, then loop through it until you match the SIS ID. Then make note of the Canvas User ID and Canvas Course ID.
2. Get the student's activity using the Analytics API. This specific call is to the Get user-in-a-course-level assignment data endpoint. This involves the Student ID, which is SISable, but it does not return the Canvas User ID with the data, so you will need to have the Canvas User ID from somewhere else.
   1. If this comes back empty, then stop because there is nothing to process. This could happen when there is a problem getting information back from the API.
   2. If a "status" field exists, make sure it's not "not found". This means there is no information for that student in that course. It also happens when a student has been deleted from a course. Stop processing.
   3. Check to see if an "errors" field exists. If it does, there was a problem. However, this can also happen when a student has been deleted from the course. Stop processing.
3. Get a list of assignments for the course using the Assignments API. This is done through the List Assignments endpoint. This is necessary so we know what assignments are possible in the class.
4. Get a list of submissions for that student using the Submissions API. This is done through the List submissions for multiple assignments endpoint. This API calls takes an extra parameter called 'student_ids[]', which contains a list of students IDs -- in this case, just the one. However, this needs to be the Canvas ID for the student, not the SIS ID, which the reason for obtaining that information in step 1.

Process the Data

1. Assuming that the activities from the analytics call (step 2) doesn't contain a 'status' or 'errors' field, iterate through all of the activities. Each activity is an entry in an array. I generate a sequential integer key for each value so that I can maintain the order. You cannot sort by date because not all assignments have been submitted.
2. Obtain the date the activity occurred and incorporate this into the key. The first part of the key is the submitted_at date, if it exists, and the second part is the sequence number of the record.
   1. If there is no "submission" field, then the key looks like "-001" or "-002".
   2. If there is a "submission.submitted_at" date, then the key will be a ISO 8061 formatted timestamp with the sequence number appended at the end.
3. For each activity, store the assignment title, assignment_id, submission date*, score received*, and possible points. (* may not exist)
4. If there is assignment data (step 3 of the fetch data phase), then loop through them trying to find the one that matches the assignment_id. If you find it, then save the possible submission types into the data structure started in step 3.
5. If there is submission data (step 4 of the fetch data phase), the loop through them trying to find the one that matches the assignment and record the submission type in the data structure created in step 3.
6. Check to see if the student has submitted the assignment. Default to "no".
   1. If there is a submission_type (found in step 5) for the assignment submission, then it's a grade due to a submission, not just a grade the faculty put into the system (like a 0 for a missing grade).
   2. If there is no submission_type (missing from step 5) then check to see if "none" is one of the allowed submission types (from step 4).
      1. If "none" is allowed and there the possible points is positive, then look at the score received. If the score greater than 0, then count that as a submission. otherwise mark it as "Unknown" since we don't know whether the 0 came from failing miserably on the assignment or from not completing the assignment at all.
      2. If "none" is allowed and the possible points is 0, then just record it as "Ungraded"
   3. Add the submission summary (step a or b) to the data structure.
   4. If the submission summary is "No", then make the date to put in the key as the due date of the assignment and also record this in the data structure so we can distinguish "due at" from when it was submitted.
7. Save the data structure into an array, keyed by the date/sequence key.
8. Repeat steps 1-7 for all activity from the analytics report.
9. Reverse sort the data based on the key, which is composed of a timestamp and a sequential number. This puts all the the submitted stuff at the top with the newest first and all the unsubmitted assignments at the bottom. This allows the person reading the report to see right away what the student did last.
10. Generate the report.

API Calls

1. Enrollments: List Course Enrollments (not needed if you cache Canvas IDs locally)
   GET /api/v1/courses/:course_id/enrollments
2. Analytics: Get user-in-a-course-level assignment data
   GET /api/v1/courses/:course_id/analytics/users/:student_id/assignments
3. Assignments: List Assignments
   GET /api/v1/courses/:course_id/assignments
4. Submissions: List submissions for multiple assignments
   GET /api/v1/courses/:course_id/students/submissions

Contact Information

 @kona , Director of Online Learning

 @James , Internet System Specialist & Professor of Mathematics

[G_Petruzella]

 @kona ​ and  @BKINNEY ​​ have shared an exceptionally clear and detailed process to compile reports on student submission activity without time-intensive work in the UI. This task is likely one which most institutions have a need to accomplish in some form; and the report provides all the detail another institution would need to implement the solution. This is a great resource.

[BKINNEY]

I'm not sure why you are mentioning me here. I don't recall making a contribution.

[G_Petruzella]

Mea culpa, mea culpa! I looked too quickly over at the right column and misread the "Filter Blog" area as though it was listing the contributing authors for the specific blog post I was reading.  @kona ​, the credit here's all yours! 😄  @BKINNEY ​, props to you for the Post 'Em ...erm... post.

[kona]

 @G_Petruzella ​ I'm happy that you've found it useful. It was a major time drain for our office - especially at key drop points during the semester - so this report has saved us a LOT of time and energy.

[millerjm]

Hi  @kona ​, any chance you would be willing to share the actual report code?  I'm not sure where to even start but I need something like this!!! 

[kona]

 @millerjm , this would be the  @James ​ side of the house, so I'm tagging him to see what he can provide for you that might help.

And yes, it is an absolutely amazing tool that gets used a LOT for federal compliance with financial aid regulations.

[millerjm]

Hi  @James ​ - Is this something that you have uploaded as a resource here or in your github?    Would you be willing to share?  Thanks for everything that you have helped me with so far!!!

[James]

I do not have it posted anywhere.

The version we use is tied heavily to our locally cached database of Canvas IDs to speed up the process. I have a standalone version that I pulled out and cleaned up for someone a couple of years ago. It refers to an API module that needs a little bit of cleaning up as well (I don't think POST operations work unless you use JSON encoding).

I'm not adverse to sharing but I have a backlog of other items that need addressed first. I've also got a major project underway that will hopefully benefit lots of users not just a few, so this is not a high priority for me.

[millerjm]

No problem, thanks for the reply! 

[lmd5adm]

This is very useful!  Thank you.

We are also looking at students who have no activity in a course so we can advise them properly on dropping/withdrawing.

[rob_callicotte]

This (/api/v1/courses/:course_id/students/submissions) API does not appear to work. Any ideas why?

[James]

You must pass it something to work with or you get a 401: Unauthorized error.

This fails:

     /api/v1/courses/1894500/students/submissions

This works:

     /api/v1/courses/1894500/students/submissions?student_ids[]=all

That may not be the parameter you want to add, but try adding something and see if that helps.

[rob_callicotte]

Thanks James! This is it! Perfect. Thank you so much for taking the time and explaining this here.

[rob_callicotte]

I'm curious why the assignments_id does not work in the same way as the students_id.

It says the following works, but /api/v1/courses/1894500/students/submissions?assignment_ids[]=all does not work and /api/v1/courses/1894500/students/submissions?assignment_ids[] without the '=all' does not work. What am I missing on this assigment_ids[] argument? This is not very clear in the documentation.

List of assignments to return submissions for. If none are given, submissions for all assignments are returned.

[James]

You don't need to (actually you can't) specify assignment_ids[]=all because that's the default if you don't include assignment_ids[] at all. If you want all assignments, then leave it off. If you want some assignments, then specify it.

[rob_callicotte]

Thanks!

[rob_callicotte]

I just found that this does not always pick up all student submissions - replies to a Discussion Topic are not captured; only submitted discussions are captured. Is this because replies do not count toward the score?

[James]

Each assignment has a single submission time. For a discussion, that's the first post.

However, all of the discussion entries came along with the request for all the submission results so you can iterate over them and determine the last time one was made.

Thanks for pointing that out, I hadn't considered replies to discussions.

[TreyLeech]

Does this entire programming still work in year 2022 or is there an updated version of this by chance? @kona and @James .

Thank y'all for what y'all did!

[James]

@TreyLeech

It is still working on our end as much as it worked when the post was initially written. The only change I made since 2014 (before this post was written) was in 2020 when I added a check to make sure the Canvas course was active. I just ran the script this morning to verify that.

This post is not describing software that is available for download. It is describing a process we use.

The software could probably be sped up or improved. We're now downloading submission information nightly for an early-alert system we use so I could pull much of the information from data stored locally and then just get the changes since the last update through the API. Those changes are not likely to get made; I have a lot of other things that need done and this one works so it will probably go unchanged until it stops working. In reality, except for courses like mine that have a lot of assignments (I have 194 assignments in my stats class this semester, 68 are merely tracking whether they watched a video or not), it's pretty quick.

In the years since we wrote our script, Canvas has created their own solution for online attendance. It is explained in How do I view the Online Attendance report in New Analytics?

I haven't updated it to try and pull the results from New Analytics. I think I considered it at one time but didn't like the New Analytics solution. It may have been because New Analytics is only updated once a day and I felt bad about saying the student hadn't done anything in a week when they had done something that day. It may have been because New Analytics didn't use the REST API, it used GraphQL and I didn't understand it at the time. I still don't have anything working that requires me to use a JWT.

Even though it wasn't an option when we wrote the script, New Analytics isn't available to our financial aid people within Canvas. The person in the financial aid position moved on to a new position within the college and no one has reached out to update who should be given access. We're near the end of the term, so I may be getting that request in a few weeks.

[TreyLeech]

@James...Thanks so much for this update. I'm going to review this with my IT stakeholders and go from there.