New Quizzes End of Quarter Update - Q2 2022

[MPioRoda]

• New Quizzes End of Quarter Update - Q2
• New Quizzes Resources
• Q2 Achievements
• Q3 Objectives

New Quizzes End of Quarter Update - Q2

As we approach the anniversary of the Classic Quiz Sunset Timeline post with the extension to June 30, 2024, we can reflect on a year of increased focus on New Quizzes to help us reach the milestones to move us forward.  We’ve added more resources to the effort by increasing engineering and product staff dedicated to the work. We continue to meet with customers who help us shape features with their testing and feedback.  The hard work, collaboration, and leadership show that Instructure is committed to getting us to the finish line and is willing to provide the necessary support to ensure we are successful.

New Quizzes Resources

Find all updates and the most current roadmap on the New Quizzes Hub.

Before we go into the latest New Quizzes achievements and plans for the following quarter, I wanted to make sure that everyone is able to follow along with our current priorities, what we’re looking to accomplish in the near term, and the important resources available as customers and users transition to New Quizzes.  

Roadmap for prioritized features

By now you might be familiar with the roadmap that appears on the Transparency into Quizzes Planning blog post where you can see, in real-time, what the Product team is currently working on and what they plan to do next with New Quizzes. The items on this list represent features that have been prioritized according to the current timeline towards the sunset date in 2024. 

New Quizzes Feature Comparison

The New Quizzes Feature Comparison document shows the current functionality available in New Quizzes as compared to the functionality in Classic Quizzes. We are proud to report that we have released nearly 75% of the items listed in this document. There is only one function that we have noted will not copy exactly in New Quizzes and that’s creating a survey, which we plan to use current/planned functionality to meet those needs. There may be items on the Feature Comparison document that don’t appear on the roadmap but that we intend to address once the big priorities are accomplished.   

Feature Flag blog post

We have recently updated the Feature Flag blog post with more information and release update information. As more of these self-enablement tools are available, we will update with more information. As a reminder, these features are account-level feature options and cannot be managed in sub-accounts. 

Quizzes Transition Hub

The Quizzes Transition Hub is where customers can find updates regarding the Classic-to-New Quizzes migration project and engage in discussions on forums around new features. All blog posts related to New Quizzes can be found at this hub, directly here. All forum discussions can also be found at the hub, directly here, and divided up by feature or topic.

Engineering Resources

Canvas is focusing our resources towards this important work to get us to each milestone and towards our end goals.  In Q2, we moved three seasoned Canvas engineers to our main Quizzes team and it has helped us tremendously in being able to work towards the completion of big ticket items as well as meet the monthly needs of maintenance and support tickets.  As well, we have a team working on New Quizzes integrations and specifically focused on the RCE integration work. 

Each of these resources exhibit the prioritization and hard work of many people who are centered on delivering the most important items over the next two years to ensure that we can support our customers' transition to New Quizzes.

Q2 Achievements

For Q2, we projected four items: 

• Complete Bulk Migration Tool Improvements
• Rich Content Editor (RCE) Integration work
• Blueprint Course Sync
• Blueprint Item Bank
• Improved Grading Experience

RCE Integration

Our intention with the RCE Integration work is to have full Canvas RCE in New Quizzes. For an update from the team dedicated to the RCE, please see this blog post which describes their journey so far and how the work is broken up so that we can deliver the most important features that are needed for the Bulk Migration Tool to reach the needed functionality.

Migration Tool UX improvements

In the past two quarters, the Bulk Migration Tool has been in beta and we appreciate the many customers who have tested and provided feedback on the tool. While we are still awaiting the RCE integration into New Quizzes before releasing the tool into production environments, we have been addressing issues that have come back from testers and feedback in the community on the functionality of the migration tool.

Here is a list of the related fixes or features that we have completed in the past two quarters since the bulk migration tool was released in beta environments:

• Migrated quizzes move as unpublished (fixed)
• Migrated quizzes will keep the state of the original quiz. If it was unpublished, it will copy as unpublished and vice versa.
• Quiz assignment points not moving unless quiz is published (fixed)
• Migrated question groups share with the new course (completed)
• Course Instructors have default access to the item banks shared in the Course after migration
• Multiple dropdowns with blank answers break when migrated to NQ (fixed)
• Migrated question banks used by a course should be Course Shared Item Banks (completed)
• Imported Quizzes.Next assignments are always in an unpublished state (fixed)
• NQ stuck in copy mode if importing a course export file from a course that was previously reset (fixed)
• CQ questions containing math formulas in LaTEx will automatically migrate to NQ as formulas which can still be edited (completed)
• ‘Import assessment as new quiz’ isn’t importing point value on assignment (fixed)
• Rich Fill in the Blank - The Rich FITB question type helps to support seamless transition of data from Classic to New Quizzes by providing the text formatting and tooling options of the RCE in FITB question stems. (completed)

Improved Grading Experience

As described in this post, New Quizzes that require manual grading now appear in an instructor’s To Do List. In the gradebook, instead of a partial grade, a New Quizzes icon will be displayed that shows the quiz is pending completion of grading.

When students complete a quiz that requires manual grading, they will not receive notification of a graded exam until grading has been completed. Partial grades will not show up in their gradebook, to avoid the early notification or visibility of incomplete grades.  New Quizzes will not appear in the student’s recent feedback list until after the teacher completes the manual grading.  

As soon as a teacher clicks to “Updates Scores” in Speedgrader, then Canvas will see the quiz as completely graded and remove it from the teacher’s To do list and will be displayed in the student’s recent feedback.

This Course Filter

Along with the other Item Bank Management Clarification work completed in Q1 to clarify and improve how to search for item banks, we added a filter called This Course.  The purpose of this filter is to display any Item Banks used in the course which the logged in user has permission to see (share settings of view or edit). After release, we found an issue with the filter on courses with large numbers of assignments so we had to pull it back to add operational improvements to support very large courses.  The current quizzes infrastructure and performance improvement work will address this issue and improve the overall experience.

Rich Content Editor in Fill-in-the-Blank questions

Following the addition of the Rich Content Editor in Fill-in-the-Blank questions in Q1, we received feedback from users who needed to use specific types of characters either for different languages or for scientific questions, so we improved on the work by allowing for any character within the backticks which include any special character (parentheses, math operators, etc.) and accents.

A screencast on Rich Content Editor in Fill-in-the-Blank questions can be found here.

Issues from related Release Q&A that have been completed:

• Duplicate FITB questions cause possible answer field to lose association to question
• Cannot edit FITB questions with several blanks when migrated from Classic Quiz
• Regrading FITB questions in New Quizzes does not allow modifications to answer settings

Item Bank Icon Update

Based on feedback from our community, we have changed the icon for item banks and resources within Canvas LMS and Mastery Connect from a piggy bank to a file folder. We have recently received feedback from our users from various cultural groups that the pig has negative connotations, and we want all learners to feel comfortable using Instructure products. As a global company, we want to recognize the nuanced cultural differences and avoid using imagery that is offensive. We are committed to diversity, inclusion, and belonging and to serving all members of the Instructure family.

Link to Press release

Q3 Objectives

Our objectives for Q3 are to focus chiefly on the RCE Integration work and the other two big priorities of Blueprint courses along with continuing our Infrastructure and Performance Improvements in order to ensure a more stable environment as more users transition to New Quizzes. 

An update from our team working on New Quizzes Integrations can be found here, where they provide information on Rich Content Editor, API, and Item Analysis.

RCE Integration

• Please see this blog post from Tamas Balogh on the RCE integration work.
• The RCE integration work is broken up in four phases:
• Phase 1: Core functionalities like editing, A11y checker, upload/record media, math editor, HTML editor
• Phase 2: Remaining features of RCE: insert user/course images, media, links, documents etc. 
• Phase 3: 3rd party app integration
• Phase 4: Migrate content from the current RCE to new location

Blueprint Course in NQ

Our Blueprint priorities in Q3 include completing the Blueprint Courses improvements so that courses can sync after the initial sync and item bank changes related to the course will also sync, then moving on to begin the work allowing locking on Blueprint courses in New Quizzes.

Infrastructure and Performance Improvements

We are making several upgrades to our core infrastructure to provide continued service and performance improvements.

FITB in Word Banks with multiple answers

This work improves the experience for FITB questions with multiple blanks since previously each blank was treated as having its own isolated set of choices rather than a true word bank with a common pool of answers.  On completion, the question type will support a common word bank where the answers disappear as they are used or they may be used in more than one blank.

New Quizzes Outcomes in Learning Mastery Gradebook

Our Outcomes team is working on making results for New Quizzes with item and quiz level outcome alignments available in the Learning Mastery Gradebook and in admin-level outcomes .csv report exports that provide data on student mastery over all aligned assessments. Previously these results were only available for individual quizzes in the Outcomes Analysis report.

[nathanboettcher]

Any update as to when we can expect the NQ API? This is absolutely critical to our transition planning.

[Jeff_F]

Hello! @MPioRoda  - when able, please share additional information for how the bulk migration tool handles question groups. I see from the list above that the migrated question groups share with the new course but how does this work?

Could the documentation be expanded? For example, this FAQ document does not address how to transition question groups to NQ.

https://community.canvaslms.com/t5/Quizzes-Transition/FAQ-New-Quizzes/ta-p/243540#toc-hId--644267448

This KB article does not provide sufficient details on question groups. Question groups were used with CQ to create a group of questions from which a random set of questions could be pulled. How is this to be replicated with NQ?

https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-migrate-a-Canvas-quiz-to-New-Quizzes/ta... 

Could a workflow for converting a CQ that is made up of several dozen question groups be demonstrated? What are the necessary steps and best practices here? This should help us avoid any missteps as we work to help faculty transition quizzes that use more than two hundred question groups.

[MPioRoda]

Hi Jeff! Thank you for the questions, I'm actually working on some resources to come out specifically addressing how Question Banks become Item Banks. A new screencast will be posted as well around the time that the migration tool goes into production environments. You're feedback is a big help in planning out what resources we need as users plan their transitions. Thanks!

[audra_agnelly]

Will the work on fill in the blank questions incorporate the option for partial point scoring when multiple blanks are present?

[rpeters3]

Just to make sure I am understanding correctly, is the expectation that all four phases of the New RCE integration occur in Q3?

[venitk]

@Jeff_F  I believe that CQ question groups convert to individual item banks ("Question groups with manually created questions will migrate as item banks in New Quizzes").

Can that be confirmed, and if that's the case, what is the management plan? I've asked this before, but if a CQ has 25 questions, each with their own question group, and there are, say, 5 exams, that  means in NQ it will generate 125 item banks associated with that course alone (not counting any unit quizzes). 

If this is the case, that question groups transform into individual item banks, as an admin, I could potentially have access to thousands or millions of item banks over an entire subaccount. 

This does not seem manageable. Won't that slow things down considerable if I try to access all the item banks I have access to? Won't it, like, break something?  

Can Canvas confirm or correct my understanding? 

[TamasBalogh]

Hi @rpeters3,

Phase one is our number one priority, the release dates will be confirmed as we get closer to work being completed and reiterate that we are doing this in a phased approach in order to provide benefit at earlier stages of the development instead of making it available at the end of full development.

[TamasBalogh]

@nathanboettcher,

You can find the latest update on the API here.

[hesspe]

I submitted the following request for clarification to  the Transparency into Quizzes Planning page but since that page makes it clear that a reply is not to be expected, I'm cross-posting it here:

I had thought a downloadable CSV file for the "Quiz and Item Analysis" report was listed on the Transparency page, but now I can't find it. Can you clarify if that is still on the roadmap and if so which group (Planned Next, Possible Later, Before July 2023, Before June 2024) it would belong in. 

There is also, as far as I can tell, no Instructure response on the Feature Request:

New Quizzes: Download Quiz Statistics

I would appreciate any information pertaining to this to be able to set expectations accurately.

Thank you.

[TamasBalogh]

Hi @hesspe, 

It's still there, under "Before July 2023": "Analysis downloadable csv for teacher manipulation". Although it is not closely about csv download, you can read some status update on Item Analysis in my blog post here. 

[hesspe]

Thank you!  I'm very glad to hear that.   It would make it clearer (to me, at least) if the Transparency log used "Item Analysis CSV" or "Quiz and Item Analysis CSV" (using the title as it appears in the New Quizzes reports section),  to differentiate it from "Student Analysis" report which is listed in the same column.  Thanks again for the clarification.

[crouch]

When will New Quizzes be able to handle QTI .zip file uploads?

[Nancy_Webb_CCSF]

Thanks for this great update, @MPioRoda!  Very helpful.

I would like to see QTI export available in new quizzes, for both item banks and quizzes. Any idea when that might happen? 

Current methods of sharing item banks doesn't work well for us because it allows either editing of the original or view only.  Sometimes you want the teacher to have a copy of the bank so they can make their own edits, and not make changes to your original.  A QTI export would fix that by allowing others to import a true copy to their own courses.

QTI export would also allow instructors to move quizzes from one institution to another.

[TiffanyVickers]

Can we please have rich text editing in ALL question types and answers? As a math teacher, this is frustrating that I cannot use math text in matching questions. I want to have them match the square root to their answer, or the picture to their answer even, I have more features to request, but you like them in separate posts. The biggest is this though. Other platforms allow me to have math text in all question types, in the matching portion included (like Schoology). It is something that can be done, so please do this for all of us math teachers out there who are frustrated that we cannot create questions akin to state testing. 

[TiffanyVickers]

Another suggestion, can we please have more than one hot spot? Like I want them to place two points on a number line, not just one. Again, It's frustrating because I cannot make it look like their state tests. 

[TiffanyVickers]

Another suggestion so we again, can make it look more like state tests, is to have drag and drop. If they could take a bank of things and drag them to a certain place, that would be very helpful. Again, other platforms have this option, so Canvas could do this. 

[pkellygoss]

Feels like New Quizzes are still years away from catching up to the essential core features Classic Quizzes had years ago, and still dealing with entanglements and repercussions of the LTI approach that was originally meant to make this a faster and more scalable deployment.  Our organisation has had to re-build multiple courses worth of assessment content in Classic Quizzes as issues we had thought had been fixed, such as blueprint syncing, permission/role confusion and feedback display errors, wound up causing problems yet again. Horribly frustrating and disappointing that a development on a perfectly functional and stable feature was and still is discontinued in favour of a system that is not fit for purpose and is still years away from being fit for purpose.

[Jeff_F]

@MPioRoda  @jsailor   Hello - I had posted a few questions to this discussion two months ago. I am in need of this information so we can continue planning our transition.

To see my questions you can scroll up or click here:

https://community.canvaslms.com/t5/Quizzes-Transition/New-Quizzes-End-of-Quarter-Update-Q2-2022/bc-p... 

[gwied]

I recommend we all give up on New Quizzes and use an assessment platform/LTI that works now and is feature-rich: TopHat.

[KristinL]

Hi Everyone! 

Links and content in this post have been edited to reflect the announcement that occurred on October 26, 2022. Please read We’re Listening: New Quizzes Features, Classic Quizzes Timeline for the most up-to-date information regarding New Quizzes.