New Quizzes' Item Analysis update

The content in this blog is over six months old, and the comments are closed. For the most recent product updates and discussions, you're encouraged to explore newer posts from Instructure's Product Managers.

TamasBalogh
Instructure
Instructure
11
3211

Product Blog Headers -- 1200 x 200.png

You’ve likely seen my recent post about the work my team has been developing the New Quizzes API and integrating the Rich Content Editor (RCE) to New Quizzes in the last couple of months. In addition I’ve had some time to dig into Item Analysis.  

Based on conversations with users, I understand that you need faster access to Item Analysis and to be able to download the report through CSV. I’ve also discovered that we could provide a more useful and intuitive interface with additional features to help you or your faculty to get an overview of the quiz and items performance. 

I’ve met a handful of customers to get great insights and based on that feedback we’ve created designs for an improved interface. I’m thrilled with what we came up with, and I hope you’ll like them too! In Q4, we will continue the discovery work around Item Analysis and vet the designs with users in order to understand where we've hit the mark and where we could use some additional fine tuning.

The content in this blog is over six months old, and the comments are closed. For the most recent product updates and discussions, you're encouraged to explore newer posts from Instructure's Product Managers.

11 Comments
YoshieRosales
Community Explorer

Your "Benchmark" label article that I reviewed is very impressive and the new eLumen+canvas has taught students like me from the post on J. Williams profile the benefits of this in-structure website. Not to mention the opportunity to continue learning and give space to our assignments as well. Navigating through my semester has been fun too!

akraft2
Community Explorer

The faculty at our college of nursing need the ability to filter an Item Analysis in New Quizzes by section. The 2023 NQ roadmap suggests various improvements for running an Item Analysis, but I don't see filtering by section explicitly mentioned. Will this feature be added in 2023?

Hildi_Pardo
Community Coach
Community Coach

One of our 4th grade teachers started using New Quizzes.   One thing we could do in Classic Quizzes was to see which students responded with which answer, but we can't in New Quizzes.  See the image below -- I expanded a Question on the Reports page to see Item Analysis -- I cannot click the number under Respondents to see which students answered, which would be incredibly helpful.  

Screen Shot 2022-11-03 at 10.52.53 AM.png

 

I can see individual student's attempts under Moderate (I have to click to open one by one), but under Item Analysis cannot see which were the group of students who answered.

 

NHawkins2
Community Member

I agree.  I need to be able to see which students are answering the questions correctly or incorrectly in the new quizzes.  I will continue to use the old quiz until this is fixed in the new quizzes.

TamasBalogh
Instructure
Instructure
Author

Hi @akraft2@Hildi_Pardo@NHawkins2  although we haven't included the details in our roadmap, we are improving the analysis with various details, filtering by section list the students under respondents are among our plans and most probably will be included in the improved analysis.

 

 

jon_mason
Community Participant

Hi @TamasBalogh, I'm glad that you mention a CSV export of the Item analysis. Can I check that you are also planning to enable a CSV export of students responses, the lack of which is currently tying a lot of our academics to Classic Quizzes? Thanks!

 

jon_mason
Community Participant

And just to mention a couple of other analytics-related annoyances in New Quizzes:

TamasBalogh
Instructure
Instructure
Author

Hey @jon_mason, thank you for feedback. Yes, we have student responses on the roadmap and we plan to make it available through CSV as well:
"Instructors can use detailed student analysis reports for deeper insights of student understanding and activity on quizzes."

Let me reach out to you in private message to talk more.

YoshieRosales
Community Explorer

Now I see what this means! Thank You. It will help me recognized more in depth writing for my classes too. To start off, calculating and rewriting is what I am attempting to accomplish at right now...

Sounds good! Is the above link the correct address for you as well?

Tasha_Biesinger
Community Contributor

Hi @TamasBalogh, we just had an instructor run into an issue with quiz analytics with Classic Quizzes because they have more than 1000 students in their course AND because Classic Quizzes Item Analysis doesn't include all questions types.

Can you talk about if/what limitations New Quizzes statistics have? Will they be able to see at least % correct values for all questions in New Quizzes stats even when there are more than 1000 students in the course?

Thanks!

TamasBalogh
Instructure
Instructure
Author

Hi @Tasha_Biesinger,

The new Quiz and Item Analysis report in New Quizzes doesn't have this kind of limitation and I believe it should support any number of students or questions. I'm sure there is a theoretical limit but it should be way bigger than in any real life scenario. 

The report currently fully supports multiple choice, multiple answer and true or false question types. Other question types have the basic metrics as well, like Discrimination Index, Item difficulty, Corrected item-total correlation coefficient, mean/median score, but don't include Answer Frequency Summary tables for now.