New Quizzes: We're listening

SuSorensen · ‎11-23-2021

As we have been getting closer and closer to the time for Classic Quizzes sunset, we have heard many clear, articulate, and thoughtful voices talking about just how difficult this transition would be in the current climate where, to borrow a common refrain, teaching faculty are “barely able to keep their heads above water.”

We are currently exploring adjustments to the Classic Quizzes sunset that we hope will give you and your institutions more control over the transition. To help us with this effort, would you be willing to share your top three considerations or requirements for a smoother transition?

cgcamillo · ‎11-30-2021

Our top 3 items would be:

1) Make sure the question banks in Classic Quizzes can transition easily into New Quizzes

2) It would also be great if we could more easily change ownership of the question banks so that all faculty in my program have access

3) Keep the feature in Classic Quizzes that allows students to review their responses once after completing their quiz, but not allow them to see the answers after that. This is a major concern for us.

audra_agnelly · ‎11-30-2021

1. RCE and full migration of quiz questions from Classic to New Quizzes. I know the RCE updates are being worked on, but to reiterate features my teachers regularly use: the audio recorder to quickly embed recordings of a question for special ed students or audio for music and foreign language courses. I lump migration with this point because these audio files are lost when questions are migrated to new quizzes. The missing HTML editor takes away some features teachers have used to format questions and there are items lost here during migration.

2. Downloadable csv files for item analysis/student analysis.

3. You need to ceases passing back scores to the gradebook when a quiz contains questions that require teacher scoring. Currently, the sum of autograded questions are passed back to the gradebook as the student's grade, but this is often an incomplete total when there are essay or short answer questions that a teacher need to review. In addition to displaying a low grade for the assignment, the partial score rolls up into the course average and goes out as a grade notification email. This requires a lot of what should be unnecessary communication to parents when they are alarmed about their child's (seemingly) poor performance on an exam.

martinlarsson · ‎11-30-2021

Full feature parity between Classic and New Quizzes. Including RCE with images, documents, media and LTIs from the course and the user.
Seamless migration of question banks to item banks.
Erase the complications of New Quizzes being an LTI. Settings and actions like points, automatic publishing of results and theme branding must flow between the systems without hiccups.

rhenri24 · ‎11-30-2021

@SuSorensen, thank you for listening to our concerns and being willing to look into adjusting the timeline.

This is really hard to pick three because of the different use cases at our university. Of course, the RCE issue is the largest for us, but since you all are aware of that I will leave that off my list.

#1 No content lost during migration OR resources that clearly map where content might be lost

Using the same RCE in New Quizzes as in the rest of Canvas will likely address many of the current migration issues. But there are likely going to be some question types that don't migrate smoothly. There should be flags in the migrated new quiz letting instructors know that something did not migrate successfully. But just as important, Instructure should provide resources mapping out potential migration issues. These would serve as resources instructors could use before migrating to know beforehand which question types will not transfer as successfully. IF this is not provided by Instructure each organization is going to have to come up with some internal guide and that would take a lot of time and energy on our end. Also related to all this is the issues surrounding question bank to item bank migration.

#2 Improvements to Grading

As mentioned by several people above, the issue with essay questions being marked as zero will be very frustrating for instructors and students alike. Students will most likely see that zero and reach out to the instructor causing extra work and frustration for everyone involved. Also, instructors not getting notified on the to-do list and not being aware they need to grade the essays will be very frustrating for many instructors who rely on this.

#3 6-12 months with all of the key features in place before being asked to adopt the tool

We can make the transition work if we have to adopt New Quizzes that has some minor issues that impact a few use cases, but we cannot adopt it when it will disrupt all or most use cases at our institution. We should be able to see a solid version with key features fully in place a minimum of 6 months before we are asked to start the transition at our institution. We simply cannot be asked to promote a tool that we have not fully seen or vetted based on promised updates. Our teams need to be able to see the tool that they will be promoting in order to develop clear and confident change management strategies.

JMB1 · ‎11-30-2021

Our high school was new to Canvas last year. Since we already new about the inevitable sunset on the original quizzes and move toward the "New Quizzes", we encouraged and trained our teachers from the start to use the New Quizzes. We taught everyone how to import and convert their tests from Exam View and move them into New Quizzes or just encouraged them to create new exams from scratch using the New Quizzes feature. Therefore, we aren't troubled by the problems plaguing other long time users.

However, my complaint is that I wish New Quizzes actually worked fully and correctly. How is it that the New Quizzes can grade a Multiple Correct Answer select question; with penalty point subtraction for wrong or non-selected answers, but it can't grade a simple Matching or Category question correctly? I mean seriously! I can create a 10 item Matching question or Category question with 10 items for drag and drop worth 10 points and yet, the student has to get ALL answers correct to earn any points. Miss a single item, you get a zero.

The worst part, when examining the test in speed grader, it goes through all the trouble of putting green checks for correct Matching / Category answers and red X's for wrong answers, but it can't figure out how to subtract the number of Red X's from the total number of points. This requires me to hand check EVERY test to correct these types of questions by adding up the checks myself and clicking the "+ points" button to ensure that each student receives the correct scores. How does this make my life easier when I have hundreds of students to check on?

For years, Canvas has known about this problem and the complaints from it's user forum and all we get is a, "Thanks for your comment; we'll take that under advisement." Don't think about it....FIX IT! Why tout these "great" new testing features to assess student content mastery if they don't work or make assessment ever more burdensome and time consuming for the the instructors? Technology is suppose to make my work experience better and more efficient, not add even more things to my workload.

Before you worry about whether or not to "sunset" the old system from people who were clearly happy with they way things were, why not make sure the "New" system is better and more efficient in the first place. That would make it a much easier "sell" for your users to want to change over, rather than be forced, all the time knowing that they will lose functionality rather than gain any real improvements.

llj26 · ‎11-30-2021

I agree with a lot of the comments already submitted. For our institution:

1. Feature Parity - such as partial credit for Matching questions.

2. A GOOD way to bulk grade a single question item (not the way Speed Grader works - something that makes it easier to grade 100 short answer or essay questions - this is also problematic if you're not going to institute partial credit for items like Matching, Ordering, or Categorization questions - in order to grade these correctly/effectively, instructors must still grade each exam one-by-one, and in classes larger than 30, this is untenable).

3. Allowing full Stimulus sets as item bank items, grouped together. (So we can make an item back with several sets of Stimuli, and randomly draw a particular number of stimulus groups into an exam.)

Bonus: Outcomes reporting, and downloading any of the analytics reports.

KSchmitz · ‎11-30-2021

In new quizzes, when I have migrated in a quiz from old quizzes I have to click on each individual question to get it into an item bank. Why in the world can there not be a "choose all" button so I don't have to click on each individual question?

hfchen · ‎11-30-2021

I think it's a no brainer to allow for at least a year of sunsetting time AFTER New Quizzes becomes mostly feature complete to allow for migration/training.

Our priority at K-States are below

#1 New Quizzes API having a majority of features of Classic Quizzes API

We use API for our internal tool and that will need to continue to work

#2 There needs to be better course based Item Bank option

Having user based item bank is fine. But we often have instructor changes and we need a course based item banks for ease of transition as well.

#3 Migration tools needs to cover more scenarios

And when it doesn't, it needs to throw the appropriate error, preferably also with some kind of report that you can download/come back to for reference so we can handle by hand the (hopefully few) scenarios where the migration tool doesn't handle the...well migration.

srea · ‎11-30-2021

Thanks for allowing all of us to comment.

I agree with rdelp above - why take away Classic Quizzes and "force" everyone to migrate over?

Three big concerns I have are:

*ability to use Direct share to another faculty member and/or another course. without having to separately share the question banks.

*ability to Sync to SIS is a must

*Bulk migration.

finally - here at my institution we like it when quizzes copy/import over into a new course and remain UNPLUBLISHED. We dont' want them to auto publish. We want to publish each quiz at the appropriate time.

Thanks for listening. At this time I am not confident I can introduce this to the faculty here with so much that is not working - they will be extremely upset and not willing to move to New Quizzes if so much does not work.

BradMoser · ‎11-30-2021

I just have two:

1) Full parity with classic quizzes

2) After full parity, then automatically convert CQ to NQ including outcomes and item banks.

garret · ‎11-30-2021

Hey!

1) Simple Conversion (not having to import every question from a bank individually) from existing Question Banks to New Quizes (I'm sorry, I'm not sure what the equivalent name for "Question Banks" are in the new format)

2) Ongoing Conversion from Old Question Banks into New Quizes, so if I make a question bank that is compatible with the old system, it would be an easy switch to new quizes

3) When viewing a test analysis on a quiz which uses banks, give the option to look at how students did on each question, instead of each bank instance from each question.

Thanks for wanting feedback!

hesspe · ‎11-30-2021

@SuSorensen @atcarver @chriscas @lisa_wathen @milesl @rshultz @n_adamson @briddell @venitk @DeletedUser @michelle_coots @gncrum @mjennings @tarnold and the others who graciously offered thanks for the opportunity to be heard, I commend you -seriously - for your diplomacy and (perhaps) willingness to assume the best. These days "listening" is often employed as a management tactic, perhaps even more often than as sincere attempt to respond to the felt needs of others. The number and breadth of concerns voiced here make clear that discontent with New Quizzes runs deep and is neither idiosyncratic nor run-of-the mill resistance to change. The fact is that Instructure is going to be faced with some very difficult decisions if it seriously confronts the need to respond. One possible path for them is to close this discussion, to say "we heard you," and to proceed with a minor tweaks, when what is called for is something far more far-reaching. So far we have not seen an indication that Instructure is willing to accept the burden that would go along with doing the right thing, rather than passing that burden (in the form of support issues) along to its customers. So my serious thanks goes to all of those who posted here, and I guess I will wait to thank Instructure until I see whether "listening "is a pose or the first step in the serious course correction that the evidence presented here would prescribe.

philip_quealy · ‎12-01-2021

Thanks for the opportunity to contribute to the development of New Quizzes, our main concerns are,

Table comparing Classic and New Quizzes with linked Guides showing what is different and how to achieve workflows in New Quizzes. Teachers may presume that some functionality or processes are the same when they are in fact quite different (also they are currently in development and may change).
For example,
To completely hide student scores, a posting policy must be applied as well as restriction of student score visibility in the quiz settings.
Item Banks are saved to a User not a course, permissions are managed by the User.
Practice quizzes can be achieved by giving a quiz 0 point value and this will appear in the Gradebook and will need to be managed.
RCE parity and availability of the accessibility checker
LTI support for To Do list, Notification of re submission, Respondus LockDown browser etc

carljohan_loven · ‎12-01-2021

Top three from Karlstad university Sweden

Use the same RCE in New Quiz (NQ) to save all functionality and thus include all previous content that has been created in Classic Quiz (CQ) (i.e. time setting for displaying students quiz responses and that the practice quiz is not displayed in Grades).
Ownership of banks. Is there always one person who is the main owner? Or does the course own the question banks in NQ after you create a new copy or when you make an import that converts CQ to NQ (2022). What happens when you migrate the same quiz / banks (or import course parts) several times (will they be overwritten)?
Better LTI integration. It would be great if the course navigation menu and course name will be visible so that you easily can find your way (recognize) in Canvas and click back into the course. (At least clickable as in Grades and SpeedGrader).

albeck · ‎12-01-2021

I am by no means an expert nor do I know much technical vocabulary. However, as an end-user who's been a pretty faithful defender of the merits of Canvas to my colleagues who are also end-users, I have many issues with the New Quizzes.

1.) Old quizzes do not migrate nicely. I should not have to move pictures or audio over. It makes more work at a time when I have probably 500 or more quizzes to migrate. This needs to be 110% smooth without any issues to be our sole option. I am a very advanced Canvas user, and I cannot imagine that my colleagues who are still learning old quizzes will have success migrating things. It's been a huge hassle.

2.) There is no need to send a notification for each quiz that's been migrated over. I know it's being done. It stinks, and I don't need to be pestered with emails that I then have to delete. Canvas is death by 1000 clicks. Don't add to the clicks with my emails, too. Perhaps that is a setting I just haven't discovered yet.

3.) It is eternally infuriating that I am directed to the Quizzes landing page every time I finish building a quiz. I then have to click more to change any settings and instead of kicking me back out the settings page like Old Quizzes, I am forced to open the quiz AGAIN and to set the due date. I also really, really dislike setting a points amount on the "outside" of the quiz and then having each section worth a various amount of points. It's really weird.

4.) Fill ins and drop downs need the ability to put in page breaks. It is frustrating for students to have a big blob of text. The fact that a page break isn't available is very frustrating.

5.) Why can't there be media comments in essay questions like in classic quizzes? It was very easy for students to respond to a prompt with a video or sound file. New Quizzes requires me to send them to an external resource or have them try to record on their Chromebooks with a different app, save and upload. I'm trying to make Canvas my one-stop shop for tasks.

6.) For the love of all things good, please develop partial credit for categorizing questions. It takes me MORE time to go through each quiz and give credit back than necessary. Why have self-grading things if it's all or nothing??

7.) It would be awesome to have the preview button on the outside of the quiz instead of having to "build" it and then preview it. When reviewing content with students, it's a frustration.

8.) Why does changing dates take so many clicks? It would be so awesome to be able to change dates easily without having to open the quiz. I realize that there is a way to mass change dates, but it is bulky and not user friendly - having to click on the box and have the calendar pop up blocks changing numerous dates at a time. Could there be a way to click on a box and mass change the dates on a batch of assignments instead of one by one?

9.) It is very strange that there is no way to mass delete items. I have to individually delete every assignment. More clicks = more frustration. Why is it not possible to click on numerous assignments at one time and delete? Furthermore, with migrating old quizzes to new quizzes, this type of mass deleting is going to be necessary.

10.) Question banks - they work great in old quizzes. When I migrated it to new quizzes, it basically told me that it was errored out and not useable. I do not have time mid-semester for things I am depending on not to work properly. Old quizzes work fine for this - why frustrate your end-users??

There are far too many bugs in new quizzes to totally get rid of old quizzes. For teachers like me who have amassed a bunch of things over the last decade that work great, it will now result in significant amount of hours reviewing and reworking what already works. I really dislike new quizzes and am frustrated that I'm stuck with them with no option. Old quizzes work great, and we're losing a decent amount of functionality while gaining very few "new and improved" functions.

love_foy · ‎12-01-2021

@albeck -- "death by 1000 clicks" -- that is very funny. 🙂

abigail_smith_2 · ‎12-01-2021

So hard to narrow to just three. Here's my attempt to cover stuff that wasn't covered as much in other top-3's that are, nonetheless, vital.

1) Must work seamlessly with common quiz support tools like Respondus quiz builder and proctoring software.

2) We really need non-graded surveys, with an anonymous option, like Classic has.

3) NQ should behave the same way with Speedgrader & the Gradebook as all the other assignment types in Canvas. (Grading flow, notifications, muting, etc.) The UX logic from one tool to the next should always be the same. (Unless you want to become the next Blackboard?)

P.S. Why does this have to be an LTI, anyway? Can't you build it in to Canvas natively, the way other tools are? Seems like a lot of these problems would go away if you weren't trying to develop this from the outside of the main system. Since when have quizzes not been core functions in education?

rislis · ‎12-01-2021

1. Change the interface to look like EVERYTHING ELSE IN CANVAS.

2. Allow for voice recording in the RCE. Ridiculous that we can't offer this needed accessibility for our students.

3. Best choice - support both platforms through June 2023. Teachers are really struggling right now and let's hope things get better by this date.

Maggied62 · ‎12-01-2021

I really like the new quizzes -- my only suggestion is to have a place (without going into BUILD) to check the secret word for supervisors.

This feature is IN the classic quizzes - but NOT in NEW quizzes.

cannaday_ra · ‎12-01-2021

As others have stated:

1: Full features available and functional for New Quizzes for a year before Classic Quizzes are deprecated.

2: Partial Credit available for the same/similar question types from Classic Quizzes. I just had a teacher revert to classic quizzes because of this.

3: A survey option available in New Quizzes, as it is very popular with my teachers.

greenchristina · ‎12-01-2021

I like being able to assign outcomes to individual questions. However, the Gradebook does not show student progress on these outcomes. You have to check each quiz for each outcome then comparing it to the next quiz. It would be nice if the gradebook would have an option for outcomes to be tracked.

Partial credit needs to be an option for matching type questions.

bcalnan · ‎12-01-2021

Ensure no features that were available in CQ are unavailable in NQ.
Allow for survey (no correct answer) type questions in NQ.
Include the RCE and Math Equation Editor in NQ - there needs to be a consistent UI.

hochmucr · ‎12-01-2021

My top three are:

1. RCE in all question building areas, along with answer selection building. This includes multiple fill-in the blanks, multiple answer and essay questions.

2. Correct Scoring - which means partial credit, notification for questions needing grading, and easier access to speed grader.

3. Better and consistent navigation to New Quizzes. Where do you create a new quizz - under quizzes or assignments? How do you link to a new quiz - under quizzes or assignments? How do you add a new quiz to the calender - under quizzes or assignments?

My other choices would be everyone else's. Right now it seems there are more things to be fixed, than dazzling new features offered. Just give us something that works intuitively for teachers. We will embrace it.

thomas_maerke · ‎12-01-2021

Thanks for the opportunity to share.

Migration of Question Banks to Item Banks. Clear processes, with messaging for areas that will need manual attention (and minimal areas at that), on at least the Course level (not QB by QB to IB by IB, which would take forever) if not the Subaccount or Account level or even by User (for all QB by one user to IB in the same Courses). If this doesn't happen, we may get new users making new assessments in New Quizzes through the use of Item Banks, but existing users will be massively discouraged because of the loss of work that went into Classic Quizzes and Question Banks.
Student Analysis Reports that can be exported to CSV. If this doesn't happen, development of new assessments in New Quizzes will decrease, or at least not grow. The inability to pull Student Analysis Reports via a CSV diminishes the use of New Quizzes in shared courses where teachers collaborate in Professional Learning Communities and strategically apply RTI practices. In every instance in our institution where teachers are sharing data and applying RTI practices, they are using Classic Quizzes.
Rich Content Editor. It's for consistency. Learning a different feature set is discouraging to teachers. And teachers go in expecting to use image features, math formula features, or other LTI integrations, but they can't. The question types and current features aren't enough to attract the teachers, and the surprise of the NQ editor not being the full RCE is enough to drive the teachers back to Classic.

Maybe this is stupid question, but is there a planned method to follow-up to this feedback? It would be great to have a response from @SuSorensen or @shaun_moon or anyone as a follow-up to this post after some designated length of time.

Thanks again.

TroyPhillippe · ‎12-01-2021

I do not think New Quizzes is functional enough to replace Classic Quizzes.

Please listen to what others have said.

Please let Old Quizzes stay for awhile longer, until such a time that New Quizzes works properly and the old quizzes can be automatically ported over into the new system.

I will add one additional piece of feedback to help New Quizzes be better.

Matching Questions don't work. They aren't graded with partial credit (incredibly frustrating). In OLD Quizzes, if I wrote a matching question with 4 answer choices, the grader would weight each correct match for 25% of the allotted total points for that question. NEW Quizzes will automatically give the student a 0 for the question even if the student gets 3/4 correct.

sarah_jarboe · ‎12-01-2021

1: Allow partial credit for ordering and matching questions.

2: For the word bank fill-in-the-blank type: Right now, if you have a sentence/passage with 3 blanks, you have to identify optional choices for all blanks, resulting in multiple words that aren't necessary. I want to be able to have only 3 word bank items; 1 word goes into each blank.

3: Make TurnItIn available in NQs.

ProfessorBeyrer · ‎12-01-2021

Thank you for the direct opportunity to comment and the encouragement to list 3 priorities:

Rich Content Editor - There will already be enough change, especially if perfect replication of features is not in the plan. Let's give our instructors the comfort of using a familiar editing tool.
Surveys (anonymous, graded, and ungraded) - Restricting an anonymous survey to enrolled students is too important to leave at the whim of security settings for a third-party provider. Having students complete an early-semester, low-stakes assessment via a New Quizzes survey is a great way to give students practice at using a tool that will matter more later in the semester. And it is a nice way to collect feedback throughout the semester, again using a tool that students are familiar with. If we can have ungraded discussions, why not ungraded quizzes?
Clear navigation for students - The inclusion of the Previous and Next buttons is good for when a New Quiz is linked from a module; better would be clear language about what the "Return" button means when a student is "inside" a New Quiz, especially when viewing results.

marita_bird · ‎12-01-2021

1) Full RCE capabilities, the RCE needs to be comparable to existing RCE.

2) They must work seamlessly with Blueprints, until this happens we cannot even start thinking about transitioning. We must be able to update a quiz that has previously been synced to an associated course.

3) A 12 month transition period once all features are fully implemented. Classic quiz updates need to included in this. Currently we cannot update after December 2022 but because of the blueprint timeline cannot start transitioning until July next year, 6 months is not sufficient time to undertake this work.

DanBurgess · ‎12-01-2021

I'll put my "If I had my wish, it would be..." thought out first.

Give me all the goodness of New Quizzes in Classic Quizzes.
Alluding to what @abigail_smith_2 wrote, aren't most of the problems caused by the decision to architect NQ as an LTI along with some other design choices? Can we get the new question types and the enhancements to moderation added to Classic Quizzes? I like our Classic Quiz home. Okay, so maybe it is a bit out of date, but can't we do some remodelling instead? Do we have to move?

If we do have to move, then our NQ home needs:

Parity in all respects; others have done a great job of providing details. We don't want to downsize.
A migration process that works. We don't want lost or broken items after moving day.
Reliability. Who wants to move to a new home that has plumbing issues?

siri_horrigmoe_ · ‎12-02-2021

If NQ is supposed to me a real improvement om CQ, for me and my institution, they need to be self-graded, meaning that studentes not only see a point or percentage score, but an "approved" message or similar, if they hav reached the set criteria.

Opportunity to use partial point/credits.

Completely seamless and easy migration from CQ to NQ.

KimFergus · ‎12-02-2021

Make features of Old Quizzes available in New Quizzes before doing any transition. Here are the things we can do with Old Quizzes that we can't do with New Quizzes. We NEED these features!

1. Posting something students can see but don't have to reply to (text only)-

2. Make item banks click and drag.

3. Make recording a prompt for question response. This is ESSENTIAL for my field of study.

4. Make returning to the items settings page possible from the build stage.

5. Make item categories and matching, etc. points per answer instead of all or nothing.

6. The load time and import issues are problematic- fix that.

7. Make seeing the responses once after submission an option.

8. We need real time (immediate) statistics for student feedback instruction- not 1- 2 day lag time.

9. We need individual item analysis when a item bank is utilized.

Basically, New Quizzes has nothing to offer people teaching in my field and Old Quizzes has the features we need. These are all features we currently use with Old Quizzes.

sale0012 · ‎12-02-2021

In our college and in many other colleges at the university we have Canvas support staff who support upload and editing of quizzes for hundreds of faculty. If we move to New Quizzes, we lose the ability to provide this support. Also most of our faculty teach collaboratively, then pass on their content to other faculty. They cannot pass on New Quiz content and new faculty would have to recreate quizzes each semester. Can Instructure give us editor sharing access on New Quizzes and if not, can we retain Classic Quizzes? The loss of quiz content editing and sharing is a huge roadblock for us and many other colleges around the U.

addisoncombs · ‎12-02-2021

1. Allow instructors to Grade after student access is closed. Currently we are not able to grade quizzes after student access is closed, even while the Term is open and the Instructor access is open (Course set to Term participation). This makes any essay question due at the end of a term impossible to grade without manually re-entering the grades in Gradebook.

2. To-Do list for Grading

3. RCE for all parts of the quizzes.

2 and 3 have been stated in previous comments, agree with those above.

jpryor3 · ‎12-02-2021

Before institutions are forced to go to new quizzes it needs to have the same features as classic quizzes. For example partial credit needs to be available, reports need to be returned in a timely manner. Faculty should not have to wait 24+ hours before reports and analytics is returned. I have faculty now that are test driving new quizzes and they hate this. Test banks in classic quizzes should be able to roll over into new quizzes. The RCE editor needs to have all the capabilities as in classic quizzes and accessibility of features is a must. Faculty are overwhelmed with Covid and how that has affected teaching . Presenting them with a quiz tool that is not robust and requires lots of training to implement at this time is not in the best interest of those that utilize your LMS.

megankoontz · ‎12-02-2021

I'll repeat what I've said multiple times and I can see others have said

Assurance that the teachers can easily tell which quizzes need grading, that partial credit is available, and that ungraded quizzes are displayed on the teacher's to-do list when using new quizzes.

This is a DEAL BREAKER for us and new quizzes and maybe even Canvas in general. We do not have due dates for our assignments and we need a way for faculty to quickly and easily identify what needs to be graded. Going into every quiz every day to see if there is a submission is NOT workable.

woodk · ‎12-02-2021

1 - ability to bulk download submissions from NQ that contain file upload questions

2 - availability of "student analysis" CSV file that currently exists with NQ, and preferably one that doesn't take 24 hours to generate

3 - 12 months transition time after NQ is fully functional, all while continuing to fully support CQ.

MaskeM · ‎12-02-2021

Same RCE as in all tools within Canvas
Easy transition from CQ to NQ, to include question banks
Better and more training and support on a more frequent basis

BrandiGranett · ‎12-02-2021

I would recommending extending the deadline for migrating from Classic Quizzes until after all of the bugs are worked out with the bulk migration or course copy migration option is complete. Since both tools exist currently, consider if there really is a need to sunset Classic Quizzes so soon.

ppoet · ‎12-02-2021

I am definitely repeating comments from other contributors as they are so important.

Full feature parity with classic quizzes including bulk migration from Classic to New.
Extend the timeline for sunsetting Classic until everything is actually functioning properly in New Quizzes, then make that time at the very least one year (preferably 18 months)
RCE - functionality and feature parity with classic quizzes.

The recent announcement about the December updates not being available as planned put our training on hold. We are in limbo because there are important questions not answered. We made such great strides with the use of quizzes in the last 18 months, and this feels like we will be taking a giant step backward without proper implementation. Teachers will lack confidence in Canvas if they are given a muddled product release.

Thank you for asking for feedback.

clare_davies · ‎12-02-2021

As an instructor (I note that most of the comments here seem to be from admins), with over 5 years of intensive usage of CQ under my belt, I thought I'd be just fine transitioning to NQ this semester as demanded by my institution. But it. Is. A. Nightmare. All this hassle and so many 'intentional' (ha, right!!?) bugs, supposedly to give us a couple more question types? Are you kidding? I agree with the commenter who said this is probably about rolling out a la carte licensing, but it's a daft idea imo. Integration = good. Piecemeal modularity with inconsistencies everywhere = might as well use Moodle for free.

So here is my top 3.

1. Just no sunset at all. Personally I would call it all off. I have no idea why anybody thinks NQ better. There are SO MANY bugs and missing features, NQ shouldn't even have been released yet - let alone imposed on us.

2. Sort out grading above all else - in my institution we have extremely short turnaround times for grading, and it's very pressured work. Instructors absolutely MUST be able to do the things we've come to depend upon, as below. I added the interactions to try to give some sense of how much more cumbersome it's all become. We need to:

Know which students as well as questions remain ungraded (in SpeedGrader, never mind the 'todo' list which doesn't work well in our team-teaching institution anyhow).
Click, click, click, click, click, click...
Quickly navigate to the grade-requiring questions (because unlike in CQ, the 'unanswered questions' notification at the top in SpeedGrader isn't hyperlinked).
Scroll, scroll, scroll...
Grade by question, to facilitate self-calibration by grading one question for everyone before the next.
Scroll, scroll, click click scroll...
Either have grades and feedback auto-save as we work, OR move the 'Update' button so that it's accessible all the time or via a shortcut keypress. Or both.
Set each question's grade. Click to open feedback box, type in it. Click to close the feedback. Scroll to the bottom. Hit Update. Scroll back up. Repeat...
View and annotate file-upload questions within Speedgrader or (e.g.) Word online, AND be able to bulk-download them (named automatically by Speedgrader student numbers, if anonymous).
Scroll, download Student1 and then move on click click click, scroll, download Student2 and then move on click click click... Open, work offline, save... Find correct student in SpeedGrader again (double-check... mistakes far too easy with anonymous marking...), upload click click click...
Access full documentation from the very first use. E.g., explain what the heck those red and green buttons are for, and whether it's OK to ignore them when most of your manual question gradings are inevitably partial? And many similar stress-inducing mysteries.

Overall, grading is much less efficient, less ergonomically healthy and more frustrating than it used to be, and it's reached deal-breaker for me. Having been a Canvas (and previously Moodle) quiz pioneer in my department, I'm changing my assignment for next year to avoid like the plague. I CANNOT go through this year again.

3. If there has to be a CQ sunset, it must be at least a year after NQ reaches genuine parity and beyond with CQ, in usability and functionality. Completing partial grading, restoring text-only questions (or just let a stimulus appear without questions - that'd do), and and restoring surveys and practice quizzes (kept out of the Gradebook) are just scratching the surface - the whole UI doesn't make sense anymore.
'Return'ing to where I don't want to be; time-wasting formative grade-awarding (when students have passed a quiz already but Canvas no longer lets them progress without all essay questions being graded); unexpected manual-release expectations (I discovered in Week 5 that the students couldn't yet see their first 4 weekly formative quiz results... aaaaaaarrrgh!!)... What's with hitting the word 'Build' to get a report? Hasn't anyone at Instructure heard of usability principles? And why have to switch from the Assignment screen to the Gradebook to sort out unwanted 'grade policies' and to open Speedgrader (i.e., the things we use before the Gradebook is populated)?

Enough ranting. I have to get back to my maddeningly slow marking. No more NQ for me after this. But thanks for reading - Instructure seems to be full of very nice people, at least.

tammi_brandon · ‎12-02-2021

We recently began using New Quizzes and two BIG things stand out.

1. When a new quiz is used in a course blueprint and the quizzes are published in the Blueprint, the associated courses do NOT have the quizzes published automatically. The teacher must go in and republish each and every new quiz for every associated course. As you can imagine, this creates an extra step for each course. For institutions like ours that allow enrollment every two weeks, this means opening each new course and publishing 20+ quizzes per course.

2. From the student perspective, the quiz is less user-friendly. There are a lot of extra steps. Submit, Confirm Submission, Return, Next to get to the next assignment. The quiz always asks if the student wants to retake it even if they meet the minimum score - this is confusing to some students. It feels like a lot of extra steps and the screen is too busy.

rmcochran · ‎12-02-2021

Make sure features from classic are present in new quizzes, especially partial scoring for matching, etc. as well as the ability to grade by question.
Fix issue with essay questions automatically getting scored as a 0. This also means making sure they get added to the To Do List for instructors.
Respondus 4.0 compatibility

lenzeb · ‎12-02-2021

1. The ability to copy content across instances easily. This includes quizzes with and without test banks. The test banks themselves should also be able to be easily copied across instances.

2. New rich content editor available when building quiz questions.

3. Making sure the transition is easy for teachers. Having to manually export and import quizzes and or files would be a huge ask for a majority of our educators.

Mboyde79 · ‎12-02-2021

Yes to what everyone has said, and please add fully developed partial credit to New Quizzes.

Please add the feature to record in the RCE, not just embedding a recording. The teachers need this feature for oral testing.

ShiraGreenberg · ‎12-02-2021

I am agreed with what is mentioned above (especially RCE), although I am not sure that we need a 12 month turn over period since we have been informed of the existence of NQ for so long.

1. I would like to add the issue with stimulus questions- they do not allow for PDF, making it very difficult to insert a reading passage and attach relevant questions. Right now I have students opening a link in a new window. Also, stimulus questions are not grouped together in the question bank, making it difficult to go back and select the questions that go with each stimulus.

2. It has already been mentioned, but the "reports" are a major issue. No reason for it to take 24 hours or longer to get a breakdown of a simple ten question multiple choice assessment. I administered one assessment almost a month ago and still no report available. After reporting to Canvas, I was told someone would follow up and get back to me...still waiting...the Quiz Statistics item in CQ gave great data and was very user friendly.

3. Make SG available through the assignment window, right now we have to go through "grades." Yuck.

Thank you, and I hope "the powers that be" are listening because there is a HUGE difference in Canvas utility for elementary, upper, and post-secondary education! It needs to work for us all.

SuSorensen · ‎12-02-2021

We have been collecting and analyzing these responses. We have some ideas about beneficial changes and are going to validate possible approaches. You can expect to hear more from me before the end of the year.

spipitone · ‎12-02-2021

More time for the transition
Fully Functional RCE
Individual credit for multiple answer questions

bdiamond · ‎12-02-2021

I know these have been said by various people, but I really tried to start using New Quizzes and I actually gave up because of these:

1. Partial credit, especially for matching. I gave my students a matching set with 10 questions and a couple of extra distractors, and if they got 1 question wrong, they got them all wrong? It makes matching useless.

2. Speed grader and Quiz Statistics are much less user-friendly in New Quizzes, and I consider those (especially Quiz Stats) to be a main reason I use Canvas for assessment. The fact that Quiz Stats take so long to process, when they are instant in Classic, is an absurd downgrade.

3. This is less important than the others, but in Classic Quizzes, I was able to insert images into more places, such as in matching options. Again, I have to manually change the entire structure of these questions to use them with New Quizzes, even though I loved that feature.

elovitt · ‎12-02-2021

Like many of the rest of my colleagues I would encourage the following for priority.

1. More time for the transition (without all the pieces in place it is hard to push our faculty to have everything migrated and ready for students by Fall 2022)

2. Fully Functional RCE (all question types need to be supported with the same RCE as we find in the other areas of Canvas)

3. Easy Migration of Question Banks to Item Banks (I know the theory of this but we really need to be able to kick the tires so to speak so we can look for problems)

thank you for listening and look forward to any feedback!

Ed