Peer Review with large cohort

[sarah_craske]

Has anyone had any experience with using Peer Review on a class of 1500-2000 students - they will be submitting a Studio video and we are wanting to auto assign each student 2-3 reviews. 

I have no way to stress test this and we currently have issues with loading this number of students into Speedgrader so am feeling nervous...

[James · Accepted solution]

@sarah_craske 

You are right to expect problems with SpeedGrader. This is especially true with peer reviews. Canvas SpeedGrader loads all of the content for all of the students that it is going to show you. By all of the content, I mean all of the comments and peer reviews are in there as well. Since you want each student to do 2 or 3 peer reviews, then each of those adds a considerable overhead to what's downloaded.

There are a couple of ways to speed that up, though. Both involve limiting the number of students loaded.

If this was a group assignment, then you can set a course settings option to launch SpeedGrader filtered by student groups. That should speed things up, but this doesn't sound like a group assignment. I have found that less than useful, but I don't use groups that often and I don't teach large classes.

You may be able to do is to go into the gradebook and filter the data by section. This requires that you have sections, but if you do, then SpeedGrader only loads the information for those sections. If you had 15 sections of 100 students, then it would be a lot faster than trying to load 1500 students all at once. You grade 100 at a time, then switch sections to move on.

If you're not using sections, you may want to talk to your Canvas admin about getting them added automatically for you. They don't have to be official school sections, you could even have them create it by the student's last initial. Some schools set it up by TA where each TA gets their own section. Students can be enrolled in multiple sections, so you could have official school sections and then other sections for accommodation students, students who like to wear blue, students with nose rings, etc. To Canvas, a "section" is really arbitrary, but it allows you to organize how you like and offers some extra capabilities for assigning and grading.

[James · Accepted solution]

@sarah_craske 

You are right to expect problems with SpeedGrader. This is especially true with peer reviews. Canvas SpeedGrader loads all of the content for all of the students that it is going to show you. By all of the content, I mean all of the comments and peer reviews are in there as well. Since you want each student to do 2 or 3 peer reviews, then each of those adds a considerable overhead to what's downloaded.

There are a couple of ways to speed that up, though. Both involve limiting the number of students loaded.

If this was a group assignment, then you can set a course settings option to launch SpeedGrader filtered by student groups. That should speed things up, but this doesn't sound like a group assignment. I have found that less than useful, but I don't use groups that often and I don't teach large classes.

You may be able to do is to go into the gradebook and filter the data by section. This requires that you have sections, but if you do, then SpeedGrader only loads the information for those sections. If you had 15 sections of 100 students, then it would be a lot faster than trying to load 1500 students all at once. You grade 100 at a time, then switch sections to move on.

If you're not using sections, you may want to talk to your Canvas admin about getting them added automatically for you. They don't have to be official school sections, you could even have them create it by the student's last initial. Some schools set it up by TA where each TA gets their own section. Students can be enrolled in multiple sections, so you could have official school sections and then other sections for accommodation students, students who like to wear blue, students with nose rings, etc. To Canvas, a "section" is really arbitrary, but it allows you to organize how you like and offers some extra capabilities for assigning and grading.

[sarah_craske]

Thanks James ... as markers we will be using the Filtered by Sections - it's not a group assignment - I was thinking more of Issues with the student perspective and the assigning of the Peer Reviews e.g. this bit ...

[James · Accepted solution]

@sarah_craske 

The assigning of peer reviews happens as a background task, so it should (untested -- we don't have anything that big) function on a class of 1500 students, but it may take a while. I have no idea how long that will take, but my I normally figure I can make about 10 API calls per second with the stuff I do. Canvas doesn't have those same throttling issues that end users do, but they may throttle even more so that it doesn't bog down their system --- hard to know exactly. Anyway, 1500 students with 3 peer reviews each would take about 7.5 minutes to assign them all at the 10 calls per second rate. To be safe, I would start the process at least half an hour before you tell students to go complete them.