Reproduce discrimination index using item analysis

[hesse151]

How exactly does Canvas calculate its discrimination index? I read this article and tried to reproduce the values myself, as the item analysis report does not include the discrimination index. The linked article describes the discrimination calculation as the "number of correct answers from the bottom group is subtracted from the number of correct answers in the top group, then the total is divided by the size of the group."

From the item analysis report, I interpreted this as the (Correct Top Student Count – Correct Bottom Student Count) ÷ Answered Student Count. However, that does not match what I see on the Quiz summary page. I tried several other variations in the denominator but to no success. I think what's going on is that I'm not sure what Canvas means by the "size of the group". 

Finally, I use randomized question pools for my exam, so I'm unsure if that's complicating the matter.

[hesse151]

I believe I stumbled across the answer to my own question. The "discrimination index" Canvas shows on the quiz statistics page matches the point biserial correlation for the correct option. I discovered this by looking at the item analysis output and observing that the number matches what's shown on the quiz statistics page.

If I'm correct that Canvas is using the point biserial correlation, they should say so instead of misusing the term "discrimination index." Otherwise they can actually calculate the discrimination index and post it to the quiz statistics page and include it in the item analysis.

[Chris_Hofer]

Hi @hesse151 ...

If you are looking at Canvas documentation here in the Canvas Community website, and you feel that anything isn't clear or could use better clarification, you can fill out a short form to make suggestions.  For example (and this is just an example I picked at random), if you felt that clarification would be helpful for this Guide, How do I create and manage filters in the Gradeboo... - Instructure Community - 577028 (canvaslms.co..., you could scroll to the very bottom of the page and click on the blue "Leave Feedback" button.  This opens up a form for you to fill out specific to that Guide.  The Documentation Team at Instructure will review your submission and make any changes they feel would helpful to people.

I hope this will be helpful to you, too.  Sing out if you have any questions about this...thanks!

[hesse151]

Chris, I have left feedback using the blue Leave Feedback button as you suggested.