New Quizzes End of Quarter Update - Q1 2022

[MPioRoda]

New Quizzes End of Quarter Update

Hello there! I’m Marissa (pronounced like Teresa but with an ‘M’), the newest member of the New Quizzes team. I spent over a decade working in the Higher Ed space for an R1 University with positions in finance, central research administration, departmental research administration, and contracts.  Crucial to my development as a leader was understanding individual needs, where there might be gaps in resources, and how we can bridge those gaps using technology.  If there wasn’t an immediate solution or an easy solution, I was going to find a way to get the best resources in front of stakeholders.  I hope to contribute in that way to get us to our goal of transitioning to New Quizzes over the next couple of years.

I’ve spent my first couple of months on the team getting to know the ins and outs of Canvas, reading through the blog posts and community comments, meeting with customers, and trying to get a handle on the journey that has been New Quizzes.  From working so closely with Susan and with the New Quizzes engineers, I am so proud of the work that they dedicate themselves to every day and it’s rewarding to see the outcomes of that hard work.  With that, I wanted to give an update on what the first quarter has been like and where we are headed in the next quarter.

Q1 Achievements

For Q1, we projected two items on our roadmap: allow matching questions to include partial credit and support the Rich Content format in Fill-in-the-Blank questions.

Partial Credit for Matching Questions

We released partial credit for matching questions to production environments in the beginning of the quarter. Matching questions include the option to select partial credit, similar to the Multiple Answer questions. Partial credit is the default option for this question type. Newly created matching questions default to the partial credit option. The blog post YESSS Partial Points on Matching Questions! provides additional information.

Rich Content Editor in Fill-in-the-Blank questions

For the Rich Content Format in Fill-in-the-Blank questions work, we will release a feature to Beta environments in April that allows instructors to create fill-in-the-blank questions with rich content such as paragraph formatting, scientific or mathematical formulas, or a table. This feature supports fill-in-the-blank types for fill-in-the-table, multiple drop-down, multiple fill-in-the-blank sentences, and entering equations or formulas. Possible Answer Types include a dropdown, open entry, or word bank. A complete blog post and additional instructions will be provided once the feature is available in Beta environments.

Example of Fill-in-the-blanks question using a table from build screen in New Quizzes::

Example of Fill-in-the-blanks question using a table from a student view:

Item Bank Filter Management

As described in this blog post, we’ve made changes to Item Bank filter names for clarification and to make it easier to find the Item banks you need.  We heard from community members that Item Banks can become unmanageable for Admins at large organizations and Admins that have teaching duties.  Additionally, we heard that “All banks” as the default is not useful in most situations, so we created a “This Course” filter for item banks used in this course and this filter will be the default selection when navigating to Item Banks from within a course.

The Item Bank Filters are:

• All Banks - Every bank the signed in user has access to. The filter shows all banks the user has created as well as all shared to the user, both directly and through shared via course. *Use this in conjunction with search and sorting options to narrow
• Institution Banks - Banks shared across an account
• Banks Shared to Course - all banks that have been shared to any course. Banks shared to courses are accessible by all teachers in the course.*Use this filter when you have item banks shared to all the teachers in a course and wish to use in another course
• This Course - (“this course” is the course in the navigation breadcrumbs at the top of the page) - any Item Banks used in the course which the logged in user has permission to see (share settings of view or edit).

Q2 Objectives

Our objectives for Q2 are to focus on two big priorities that will further our work so that teachers and admins are able to maintain consistent content across multiple courses. These priorities include finalizing the work on the Bulk Migration Tool and improving the Blueprint Courses experience in New Quizzes, both of which are major endeavors that we have heard have been roadblocks for customers wanting to move to New Quizzes.  Additionally, we are beginning efforts to improve the experience around manual grading and notifications.

Bulk Migration Tool

We continued to make progress on the Bulk Migration Tool this quarter. As of 2021-12-22, New Quizzes: Bulk Migration was made available in the Canvas beta environment. In Q1, we received feedback in the community forum and conducted validation interviews with customers who had tested the tool. For more information, a screencast on how the tool works can be found here.

We realize that having the New RCE in NQ will be critical for the teacher's experience of migration to ensure what is migrated will appear as expected. While work on the migration tool is nearing completion, we are waiting for the RCE to be available in New Quizzes before releasing the bulk migration tool. Work on the new RCE feature is under way.

As mentioned in a previous post, please note these current exceptions and statuses:

• New RCE content in CQ will not appear in NQ automatically (work still underway)
• Migrated quizzes move as unpublished (fix underway)
• Images not migrating properly (fix in development)
• Quiz assignment points not moving unless quiz is published (fix underway)
• Surveys will move as CQ, so that they can still be used in the course as originally made until another solution is developed. We are looking at ​​different ways to gather feedback on course/instructors from students.
• Text No Question (such as for subsections) will need to manually be changed to Stimulus, with the next question associated to the Stimulus

Blueprint Course in NQ

In our early discovery interviews on the Blueprint NQ experience, we gathered the following insights:

• Currently, a Blueprint will only sync on the initial association to a course, users need to be able to make updates to a blueprint multiple times and to numerous associated courses/sections of a course. Some use cases include:
• K12 curriculum specialists create content and associated courses for specific grade levels and subjects.
• The main teacher works with a # of other teachers and are sharing the content with the other teachers
• Teaching Assistants are teaching sections with one main instructor managing the content
• Some customers only make major changes between terms while others need to be able to make changes during the term as well.
• Currently, item banks cannot be associated to blueprint courses so they cannot be supplied with the general course framework. Instructors need a way for item banks to be available across associated courses.

What these insights tell us is that we are on the right track with our Blueprint Roadmap which focuses Q2 on supporting additional updates/syncs and supporting item banks within Blueprint courses and associated courses.

Improving Grading Experience

We recognize that there are gaps in the current grading workflow that need to be addressed. Planning is underway to improve indication of when a New Quiz is available for and requires Manual Grading, and to remove early notifications about graded exams to students.

Related Resources:

New Quizzes Feature Comparison

New Quizzes Transition Toolkit

Migrating to New Quizzes capability coming to Beta 

Possibilities for User Experience of Classic Quizzes to New Quizzes Migration - (now Official approa...

Input Opportunity: Classic to New Quiz Outcome Alignment Migration

New RCE in New Quizzes

Current and future state of New Quizzes integration with third party vendors

Survey for feedback of API use cases

First steps of New Quizzes API and a sample documentation

New Quizzes API: Planned Roadmap 

Accessibility at Canvas, including New Quizzes

Planned roadmap for API 

[venitk]

Having the item banks default to "this course" and letting anyone with the right permissions see those banks is a huge improvement, thank you!

However, I'd like you to consider the next step. Let's say I work with a completely typical undergraduate course. The course has 14 end-of-unit quizzes, each with 10 questions. Each question pulls randomly from a group of 3 questions (so, each question can be one of three options). 

Additionally, there's three 25-question exams, and each of those questions pull from different groups. 

In other words, in that one course there are (one moment, I'm doing some math...) 215 question groups. That works in Classic because those groups live within each quiz. If I need to edit a question, I just go to the quiz, find the right group, and I can edit. Please correct me if I'm wrong, but my understanding is that in NQ, those question groups would become item banks that I could only edit by going through the item bank menu OUTSIDE of the quizzes. I'm not sure how I would easily do that with 215 item banks. 

I have another concern about how edits to item banks might retroactively adjust item banks in completed courses. It's important to have an accurate snapshot of student performance in a past course. If adjusting an item bank changes the questions seen in past courses, that's problematic. If that's not the case--if when clicking on moderate quiz or whatever for a student filing a grievance, we still see the questions as the student saw it--then I think that's ok. 

I am relieved to hear that Canvas is looking for a way to keep surveys. Graded surveys (those in which students receive points just for doing a survey; there are not "right" answers) are integral in some of our courses in promoting deeper student thought. 

[audra_agnelly]

Excellent updates, thank you!

• RCE will be in Beta in April...does this mean we are looking at a May production release if no major issues arise in Beta?
• Surveys will move as CQ, so that they can still be used in the course as originally made until another solution is developed. We are looking at ​​different ways to gather feedback on course/instructors from students. This is wonderful, thank you for listening to your customers and the numerous use cases for surveys that have been posted in the community.
• Improving grading experience. Thank you, thank you, thank you. This has been a huge point of contention for teachers who don't know a late submission needs review, and for parents and students who see a score that reflects a partially graded quiz/exam (and also for the teachers who get flooded with questions about the "bad" grades).

[Jeff_F]

@MPioRoda - I really like the list of open issues and their status. This is splendid!

Can you please add the topic of 'CQ Question Groups' to this list?  We currently have well over three thousand question groups in use in our course quizzes. When I try to use the migration tool these come over empty. We'd like a solution for these as well. Perhaps that is being considered.

Also, are there additional known issues not yet being addressed that should be included on the list? Including these on such a list will help everyone save time.

[alissaarcline]

This is excellent news, Marissa! Thank you! Looking forward to trying out the new RCE in fill-in-the-blank questions--super excited about this.

[aleveritt]

@MPioRoda 

I don't see integration with remote proctoring third party systems on the list.  I'm aware that Respondus Monitor works with New Quizzes; however, they are no longer offering this product to secondary schools (9-12).  

Is there an update on when some of the other third party companies (ProctorTrack, Proctorio, etc) will be integrated?  Without test security, all of the functionality of new quizzes is outweighed by the danger of having someone other than the student complete the assessment in a remote learning scenario.

Thanks!

[NHendleyGCS]

I'm glad to have found this hub.

I greatly appreciate how open Canvas is with its users and the fact that it is constantly adding new features.

One of the big things that curtailed me from adopting New Quizzes moreso was the lack of a "View Log" that let me know if students had stopped viewing the quiz.

Please tell me this is coming to New Quizzes!!!

[MARISSASCHRADER]

@MPioRoda @aleveritt 

I agree--New Quizzes needs to be able to be used with monitoring programs. We are a PK-12 district and use Hapara to monitor our students when on their Chromebooks and it doesn't work with New Quizzes. I hope it will be functional with Hapara soon. Thanks! 

[beavey]

I love these upgrades! I realize this next suggestion is not on the current list, but I'd really love if Canvas made correct answers transfer to a new attempt of an assignment/quiz. This feature is in ALEKs and I absolutely love it because kiddos can work on what they need more practice with! Students shouldn't have to waste their time writing down previous correct answers OR opening a new tab of the graded quiz. Please (PLEASE) consider adding this feature! 

[gemma_sinclair]

Hi Marissa, what's happening with exporting of grades? This is essential functionality and I had a question today about it being a key feature for continuing to use Classic Quizzes:

• Can export results to excel sheet for careful analysis, cannot do this on New Quizzes (NQs). We use this for grades data entry and for the questions we have to mark manually. It’s just a quick and easy way to view everything at the same time.

There were some other concerns with New Quizzes as well:

• In the Classic Quizzes quiz log, you can track when a student has looked away from quiz page to another page/program on their computer. Great for exam security. On NQs, you can only see what the learner has typed for their answer or which answers they have selected and when.
• In Classic Quizzes you can have a quiz without questions: our listening quiz has a video and slides that students must view first and then the quiz locks and they go on to answer the questions in a different quiz afterwards. In NQs, you must attach a question to a “stimulus”, which is inconvenient. If we could find a workaround in which the quiz stimuli and questions are presented on different pages of the same quiz and learners are blocked from moving backwards in the quiz to the previous page, this could work.
• In Classic Quizzes you can extend time on a quiz that a student is currently taking. This is handy for when they have internet issues, and something hasn’t loaded properly, and you just want to give them more time. On NQs, from what I can tell and have read on the Canvas community, this is not possible. We did trial using NQs for our writing exams last year for one term but for this reason, we switched back to Classic Quizzes. You can only give a student extra time before they have started the quiz, which is not what we need.  

[charlotte-miras]

Hi! Cool to read there's lots of updates.

However, I'm having a lot of trouble with this new way of working in the "blank space" type question. Every time I have multiple blank spaces in one question, the tool somehow doesn't recognize all my ` signs, even though they're all the same. (copy paste)

I am losing a lot of time removing them and trying again and again until it randomly works. Is there a way to do this more easily?