How are student numbers allocated in anonymous marking?

[kathy_scott]

Hi,

How are student numbers allocated when anonymous marking is turned on? I thought it was to do with the first student interacting with a course becoming Student 1, the second being Student 2 etc. However, we had an assignment open this week and when someone submitted late to it, they became Student 21 right in the middle of the cohort and the ones below (who'd submitted before them) all got shifted down by one Student number (e.g. Student 24 became Student 25, and the original Student 21 became Student 22). Also, we ask students to submit to two different assignments within the same course so that two markers can mark blindly/independently, and the submissions are labelled with different student numbers within each assignment (e.g. student X is Student 21 in marker 1's assignment, but is Student 42 in marker 2's assignment). What is the logic to the Student numbers please?

Thanks.

[kenneth_rogers]

Hello,  @kathy_scott ‌!

I wanted to circle back to your question about anonymous marking. I'm sorry your question has been sitting for so long without gaining any traction! I'm curious - were you able to find an answer to your question? I'll be honest, I'm not sure if the logic is out there, simply because then it wouldn't truly be an anonymous experience for the instructor to grade the student. Let us know what you've found out though!

[kathy_scott]

Hi  @kenneth_rogers ‌, thanks for getting back to me. Unfortunately, no, I've not found anything out about this - and so continue to be stumped! I would assume there must be some logic to the way the numbers are assigned, because there are a lot of similarities between assignments normally - but there just always seem to be a couple that act oddly and then ruin my theories! If you ever hear of an answer in the future, I'd be interested to know it - thanks!

[kenneth_rogers]

 @kathy_scott ‌ - just out of curiosity, is there any particular reason why you are wanting to know the logic? As I mentioned, if you know the logic of how students are added to the queue, then it will remove the anonymity of the submission.

In the meantime, I'm going to share this with https://community.canvaslms.com/groups/canvas-developers?sr=search&searchId=3a2dae7a-f38d-45b1-b3a6-...‌. They mainly focus on building LTI's or API's, but maybe one of them have come across this before. I'm also sharing it with https://community.canvaslms.com/groups/admins?sr=search&searchId=16015a69-83cc-42b2-8367-a976ef5a2a6...‌ to get additional administrator eyes on this. If you are not a member of either of these groups, you may want to join them!

[kathy_scott]

Hi again  @kenneth_rogers ‌, sorry for the delay in my reply.

The main reason for understanding this is because of the way we carry out marking (we always use anonymous marking) and thus it's the requirement to have a fixed reference number for a student that isn't a name.

We have admins who run our teaching course (and Canvas) who see the Gradebook/names/submissions, and then we have markers who are academics who come in with a direct link to Speedgrader to do their marking and who we ask not to visit any other pages (like Gradebook) and so hopefully don't see names. (Permissions around marking/gradebook is a whole other issue for anonymous marking that we'd like changed!). So, we need a reference by which we can refer to the students in Speedgrader when we talk to markers - the problem we have with the anonymous student numbers that we're talking about here is that they can change if someone else submits to the same assignment for example, so there is no consistency. We also run two parallel assignments for each essay and we ask students to submit to both, so that we can do blind double marking with two independent markers, but the anonymous student number in one assignment can be different to the number in the other.

I hope you understand what I mean?

So, essentially, I was trying to understand the logic behind the numbers being allocated differently in different assignments, and why they can change throughout the course of an assignment being open and submitted to, so that I could work out a way to stop it happening! It would be good if we could fix them and make them consistent for one student for the whole course so that we could use it as an anonymous reference number, but I don't think this is going to be possible.

I think the underlying problem is that we need to be able to feed our own fixed anonymous number into Canvas so that we have a certain, consistent reference number by which we can refer to a student by. Again, something else which we'd like changed!

Thanks again for sharing this with the developers and admins anyway - if anyone knows of a way of getting a fixed anonymous reference number for a student I'd be interested to hear!

[kenneth_rogers · Accepted solution]

Thanks for the additional information,  @kathy_scott ‌. I think I follow what you're looking for. The main issue is that you need a way for the people who do the marking to have a way to communicate about individual submissions, but they can't because the logic behind how submissions are randomized changes. In the midst of our conversation I had some back channel conversation and did find that the logic is not available in an effort to preserve student anonymity. I think you hit the nail on the head when you said "Permissions around marking/gradebook is a whole other issue for anonymous marking..." - I would think that for your use case, in order to ensure that student anonymity is preserved, and markers can still communicate, those permissions would have to be in place.

You might want to consider adding this as a new Canvas Feature Idea. For more information on how feature ideas work see the following guides, How do I create a new feature idea? and How does the voting process work for feature ideas?

Sorry this isn't better news for you! Let me know if I can be of any more help!

[kathy_scott]

Thanks, again, for your comments  @kenneth_rogers  - much appreciated!

[kenneth_rogers]

Of course,  @kathy_scott  Thank You for bringing up this question! I don't know about everyone else - but I learned something in the process. Isn't that what the Community is for!?!

[SarahFair]

Hi Team. I have found the same issue recently. to my knowledge there are no students removed or deleted from our list but the apparently solid student numbering system shifted after submission and marking was complete. The biggest issue seems to be there is no obvious way to anonymise the student names once grades are released. I changed the settings back for the course but the names do not reappear. making aligning marked work with student names a bit of a headache - luckily a small cohort. Have I missed a step somewhere? I have looked through the help pages but can't see a mention of how to restore the student names after anonymous marking is complete. 

Any help would be greatly appreciated. 

[jmarshal2]

Glad a colleague at Liverpool pointed out this thread to me. We are seeing the same issue. Towards the end of marking papers that were all anonymised the numbering changed - which makes it very difficult to sustain reliable discussions about moderation with other colleagues. If the numbers associated to students do not remain constant then we cannot rely on them. As the OP said in her case for us this also seems to be something that happens rarely and not all the time.  

[jmarshal2]

Update: I heard back from T1 support and they think the issue we are having derives from a student being given 'deleted' status then added back in during the anonymous marking process. We have discovered the same student appears in the problematic courses but that he is the only student not allocated to a group for marking. So I suspect in our case this is the cause of the problem. We now need to examine how (and when) our institution adds, removes, suspends etc students from courses and decide whether this shoudl happen at all during assessments. I hope this is the (and the only) cause of this problem as we would therefore have a hope of preventing it from occurring again. T1 were very helpful and very fast to respond btw.