Testing every question within a randomized Quiz pulling from several question banks and groups

[JunoKim]

Hello, I am designing an exam using a question bank for randomization within Canvas, and we are wondering if you have any insight into how Canvas randomizes its questions. We will have 13 different Quizzes of approximately 80 questions each, with an undetermined number of testers who will each testing some number of these 80 questions in each Quiz, and we would like to ensure that at some point within the testing, all questions will be viewed by at least one tester. The testers will be vetting the questions for implicit or explicit biases, and thus we need to ensure that every question is seen at least once. However, because of the large number of questions, we unfortunately cannot ask the testers to look at the entire question bank.

Thank you, and apologies if this is a confusing question! 

[ColtonSwapp]

Hello @JunoKim

Thanks for posting this in the global canvas community.

I was not able to find any official documentation on how Canvas actually randomizes questions that are placed within a bank on a quiz. It seems to handle those by each users individual attempt. If I had to guess, it most likely uses some sort of backend API to randomize the questions within the bank on each users attempt.

If you need to ensure that each question was viewed at least once by the users, you might be able to look at the quiz statistics for that. One thing worth noting is that quiz statistics will only generate for quizzes with 100 or fewer unique questions or 1000 total attempts. The item analysis report will offer some more detailed information in CSV form for quiz questions that were selected and pulled.

Hopefully this helps a little bit!

-Colton