After the Item Analysis improvements announced last year, the Studio Quiz Results page will adopt the same layout and metrics from Jan 16. Here are the changes applicable for Studio video quizzes:
|
|
Old Report |
New Report |
|
Report Generation |
Generated every 24 hours |
On-demand generation |
|
Quiz attempts |
All attempts are included |
Last attempt included for each students |
|
Quiz Analysis - Median Score |
Not available |
Included |
|
Quiz Analysis - Standard Deviation |
Standard deviation represents a points-based value. |
The standard deviation represents a percentage-based value to align with the mean score. This allows for a better understanding of the dispersion or spread of scores relative to the average performance. |
|
Quiz Analysis - Cronbach’s Alpha |
Not available |
Random items are excluded from Cronbach’s Alpha calculation to ensure proper measurement of internal consistency |
|
Item Analysis - Median Earned Points |
Not available |
Included |
|
Item Analysis - Discrimination Index |
Not available |
Students who haven’t received an item are excluded when calculating the discrimination index. |
|
Item Analysis - Pearson Correlation Coefficient (rpb) |
The point-biserial (rpb) correlation coefficient is used. |
Corrected item-total correlation coefficient is used. |
|
Performance by Quintile |
Focused on individual question answers rather than reflecting students' overall quiz performance. |
Not available yet. |
As you notice, one of the biggest improvements is that the report will need to be generated on-demand although the guaranteed no delay in the results will make up for the additional click. Also, keep in mind that similarly to the previous IA improvements, there need to be 3 students who fill out the quiz before the report could be generated. While there are some minor differences in the new metric definitions, we removed the Performance by Quintile section due to inconsistencies in those results. We intend to reintroduce it in the future however, there is no set timeline at the moment.
We are gearing up to invest considerable efforts in enhancing the video quiz experience. Some of the areas we will focus on is fostering collaboration between Teachers to share their video quizzes with each other, gauging student understanding with more question types (pst… open-ended is coming!) and giving more controls to Teachers over their video quizzes such as setting limits on quiz attempts and disabling the ability to seek forward. Send me a direct message or add a comment below if you want to participate in the research! We are grateful for all the feedback!
Best,
Akos
