Wanting to randomize question order from multiple question banks

[CatherineFollis]

Does anyone know if this would work?  (Apologies for the long question!)

I have question banks set up for each chapter that I am teaching from the textbook.  On my quizzes, I have been having Canvas select questions from the question banks randomly so that each student gets a different quiz.  Let's say, for example, that my question banks for Chapters 1, 2 and 3 each contain 50 questions.  I would like to create a 30 question quiz, with 10 questions randomly selected by Canvas from each chapter.  In the past, I have set up the question groups on my quizzes and asked Canvas to select for the first question group 10 questions from Chapter 1, the second group 10 Q's from Chapter 2, and the third group with 10 Q's from Chapter 3.  The quiz then will show the student 10 random questions from the Chapter 1 Bank, then 10 Q's from the Chapter 2 bank and then 10 Q's from chapter 3.

I am wanting to see if I can mix the order up the questions, so that the students are not getting all 10 questions from one chapter question bank in a row, and then 10 in a row from the next chapter, etc.  Is there a way to randomize those questions, without having to combine the individual chapter question banks into one large one for the quiz? (I'd rather not do that because I can't guarantee the student would be asked an equal number of questions from each chapter if I combine the  individual chapter banks into one).

Does anyone know If set up my quiz like this, would Canvas know not to select the same question twice on a single student's quiz?

• 2 questions randomly selected from Chapter 1 question bank
• 2 questions randomly selected from Chapter 2 question bank
• 2 questions randomly selected from Chapter 3 question bank
• 2 questions randomly selected from Chapter 1 question bank
• 2 questions randomly selected from Chapter 2 question bank
• 2 questions randomly selected from Chapter 3 question bank
• and repeat until I have all my questions...  

Would this work, or would there be a risk of Canvas repeating the same question on a student's exam?  Perhaps there is an easier way besides combining all the individual chapter question banks into one large bank?

Thanks so much!  If I am not being clear in what I'm asking for help with, please let me know!

[Chris_Hofer]

@CatherineFollis ...

Thank you for the detailed explanation of what you'd like to accomplish in a quiz.  I think I understand what you are going for...but I don't think it's going to work...at least not in current Classic Quizzes.  When you create question groups in a Classic Quiz and specify how many questions to show to students from that grouping (let's say you want to show 2 questions from Chapter 1 out of 30 possible questions), you'd have to put all 30 of those questions in the question group.  Then you'd do the same for Chapters 2 and 3 question groups.  After you've set this up for the first six questions, you're going to run into a pretty significant problem.  If you were to create a 4th, 5th, 6th, etc. question group (each pulling two questions), you're not going to have questions to put into those question groups because they are already included in the question groups you created for the first six questions.  So, you'd have to have an identical set of 90 questions (30 questions x 3 chapters) in the same quiz, and then repeat the question group process again for creating the next six questions.  However, I don't think there would be any guarantee that students wouldn't get the same question again with this process...which as you've probably guessed would take a lot of time.  If you wanted, you could try out this scenario in an unpublished Classic Quiz in your course just to see what kind of work would be involved...but again, I think it would be a very long process for you.

At the moment, this is the only thing I've thought of...and there may be other ideas that I've not thought of from other Community members.  So, I would welcome their thoughts, too.

I hope this helps in some way.

[James]

@CatherineFollis 

With classic quizzes (I don't use new quizzes, but I suspect something similar would apply), each question group is completely independent. If you had two question groups that pulled from the same bank, you could get duplicate questions. New quizzes doesn't use question groups, but it sounds like the process of adding random questions is similar. 

To do what you want, you would need to split your question banks into smaller ones and then select from those smaller banks. In your case, you could take the 50 questions in chapter 1 and create five banks with 10 questions each: banks 1a, 1b, 1c, 1d, and 1e. Then do the same for chapters 2 and 3. Then pull 2 questions from bank 1a, 2a, 3a, 1b, 2b, 3b, 1c, ..., 2e, 3e.

This is a lot of work to go through. I would think about your reason for doing this and whether it accomplishes that goal sufficiently to put the effort in.

Adding all questions to a single bank and is not a solution for this. You could end up with 30 questions from chapter 1 and none from chapter 2 or 3.

[blangston]

Hello,

I have successfully been able to pull questions at random from question banks into a quiz as long as the questions in the question bank has a unique number. This prevents duplicate questions from being used. What this means is I have had to go through and renumber all questions in a question bank. For example if I am pulling questions from Lesson 7.2, I would number the questions 7.2-1 , 7.2-2, and so on. I hope this helps. Because of all the time and effort I have put into creating these question banks I cannot see myself using anything other than classic quizzes. The new quiz version doesn't seem to offer pullig questions from banks and that is very disappointing. 

[James]

@blangston 

I did some testing pulling from the same question bank and it appears you are onto something. I never link because of all the negatives associated with linking instead of including, but this does appear to be one benefit of linking.

I took a question bank with 30 questions, each having an unique name, and created a quiz from it. I created two question groups, each of which linked to 10 questions from the same bank. That means I'm pulling 20 out of the 30 questions and there is a high likelihood of duplication if the question groups are independent.

I generated 4 different versions of the quiz and never had a repeated question.

Interesting.

I then extended it to ask for three groups of 10, so 30 questions in total, the same that is in the bank. Once again, no duplicates.

The chance of that happening by random selection alone when the groups are independent of each other is so tiny that I'm willing to accept that it's not random. I haven't verified it by looking at the source code, but it appears to be the case that when you link from the same bank multiple times within a quiz that the questions are not duplicated.

This obviously fails if you ask for more questions than are in the bank (I ask for 40 questions total but there are only 30 to pick from). In that case, the first 30 were unique, but then it started reusing questions for the last 10.

When I look at the source code, I still don't see anything that refers to the name of the question in the de-duplication process, though. I see references to the ID, which would be the Canvas ID. I went back into my question bank and renamed the first 10 questions to all have the same name.

There was still no duplication in questions, even though the names were duplicated in the question bank. That is, you do not have to use unique names for your banks.

Congratulations, you taught me something new today.

What @CatherineFollis wanted to do would work provided she linked to the question banks instead of adding the questions to the quiz itself.

I'm still not going to use linked question banks because of the other issues (linked questions are hard to get out of Canvas when it comes time for analysis and you cannot regrade a linked question), but it seems that Canvas pulls the questions from the question bank once rather than pulling the required number of questions from the bank with each question group.