[ARCHIVED] Identifying which question a student has received from a question bank - how?

[Mattias_]

I am generating a question bank using a computer program that automatically creates a a question bank with a set of applied mathematical problems of the same kind, but with different numerical values for the problem variables. The idea behind this is to reduce potential cooperation by having a quiz to draw a unique question at random from the question bank.

At the same time as the program creates the question bank it also generates problem solutions in order to make it easier to check student solutions. Each solution is saved in a separate file which is linked to the problem through an identifier in the filename. The same identifier is used in the question ID of the question bank.

Upon grading I would like to know which problem each student has been assigned from the question bank as I want to compare the student solution with the computer generated solution to the problem.  For this I need to relate the particular problem drawn from the question bank to the correct file containing the solution. 

I would therefore like to ask about how I can extract the question identifier of the problem (as it has in the question bank) from the particular problem extracted and assigned to a student quiz. Having such an identifier at hand I can then directly find the computer generated solution.

Best wishes,

Mattias

[Ron_Bowman]

@Mattias_ 

What you are doing sounds very intersesting.  I am interested in how you are generating your question bank questions from a computer program. 

If I understand it correctly, you have something automated that generates the questions for you so that you do not have to do it yourself.  If that is the case, can you add the information that you want into the general comments for the question?  That way when you grade the question, the general comments would have the answer there for you (students cannot see that information until you release the solutions).  I am speaking from knowledge of Classic Quizzes.  I am not sure about New Quizzes.

Ron

[Mattias_]

Hello Ron,

Thank's for your advice on using general comments for a question - I will look into that to see if this is a solution.

What I am doing is basically this:

1. I use a Matlab script that generates as many questions as I would like in the question bank. The questions are formatted with HTML and each question is written to its own textfile. All questions are of the file upload type as the student should solve the problem by hand and submit a document containing photos of the handwritten solutions.

2. I created a question bank that contained two questions of the file upload type and exported it. From this I could deduce the structure of a question bank consisting of the kind of questions I am interested in.

3. A second Matlab script creates the question bank by using the deduced structure from step 2 and by adding the questions generated in step 1 in a loop.

4. After compression into a .zip-file the question bank is ready to be imported. 

What I need to do, I think, in order to find out if general comments to a question is a solution is to check how such comments will look like in a question bank of file upload questions and add this to the Matlab scripts I use. I am also using Classic Quizzes, so your response has made me hopeful about finding a solution.

Thank's and best wishes,

Mattias

[Mattias_]

Hello again Ron,

I have now tried your suggestion, and it works.! I can add the question ID as a general comment when generating the question bank.  When grading, the question-ID is shown and I can then find the correct pregenerated solution for the problem the student has received.

Before, I hadn't noticed the availability of general comments ( I am quite new in Using Canvas). Now that you have pointed this out and that I can use them to provide the reference for the question a student has received it implies that the problem is solved.

I am grateful for your suggestion and that you understood what I needed in order to find a solution.

Thank's again and best wishes,

Mattias

[Ron_Bowman]

@Mattias_ 

You are welcome.  Glad that hint helped you out.  I am curious about how you are generating your quiz questions.  It sounds like something I may be interested in doing.

Ron

[Mattias_]

@Ron_Bowman 

I'd be happy to tell you more on what I do, but I am not sure if using this forum is the best for this; and it also takes time to write these posts. I am not sure, however, how we can proceed. I would not like to share contact information etc. in public.

Best wishes,

Mattias

[James]

@Mattias_ 

If you click on the email icon in the top right, you will get your Private Messages Inbox.

Then you can click on the create a new message icon and send a private message to @Ron_Bowman 

[James]

@Mattias_ 

The low-tech way I do it is to go to SpeedGrader and look at the student's quiz to see which question they got.

To automate the process, it depends on how you created the quizzes.

Are the questions linked to a question bank or are the questions included as part of the quiz by finding them in the bank and adding them to the quiz?

• How do I create a quiz with a question group linked to a question bank?
• How do I create a quiz by finding questions in a question bank?

Question banks are difficult to work with and difficult to extract information from, so including the question in the quiz itself makes it easier to get the information that you want.

If the questions are included in the quiz, then you can generate the student analysis from the quiz statistics screen: Once I publish a quiz, what kinds of quiz statistics are available?

The top of each column contains a code, colon, and the text of the question. The code is the question ID that is an unique identifier for each version of the question.

That question ID can also be used with the Canvas API to fetch information about the question.

If you're finding the questions in the bank and importing them into the quiz, then you could use the Canvas API to do everything without messing with the CSV file that you get from the student analysis.

If you linked to a bank instead of including the questions in the exam, then the only way (that I remember) to extract the information is to download the assignment submissions and tell it to include[]=submission_history. When you do that, there is a property called submission_data for quizzes that contains the question ID for questions that are part of a bank. They do not include the text of the question, though, and there's no API for querying the question banks like there is quizzes, so you will still need to find the IDs some other way.

[Mattias_]

Hello James,

Thank you for your advice and discussion. First, the low-tech solution works even though it is not very practical. Currently I have used rather small question banks of about 40-50 questions. A manual comparison of the question the student has received with the questions in the question bank requires me to go through the question bank. Having found the question in the bank I then need to find which order the question has in the bank as the unique question ID is basically a running number. Then I can find the computer generated solution based on this running number. This I would then have to repeat for each student taking the quiz...and for each question bank I use in this way....

I am using a quiz that links to the question bank with random selection of questions from the bank. If I would add a question from the bank into the quiz I would not have a problem as I then know which question I have added. But what I want to do is to be able to present students with the same type of problem but with different numerical values of the parameters of the problem in order to reduce the risk of cooperating with other students during exams. For this reason I think linking to the question bank is the only option.

The download assignment submissions suggestion seems like a possibility as it appears that I can find the question id:s of the questions taken from question banks in this way. Before trying this, however, I will investigate the suggestion of Ron with general comments.

Best wishes and thank's,

Mattias

[James]

@Mattias_ 

Randomly selecting a question from a collection of questions has absolutely nothing to do with a "question bank." It is controlled by a "question group." You can randomly picking a question from a pool of questions without ever using question banks by adding questions directly to the quiz. But if you happen to have a question bank, you can either link that question group to a question bank or find questions in a question bank and include them in the question group.

Including the questions, as opposed to linking to the bank, makes reporting and data collection much easier. Including them as part of the quiz allows you to select some of the questions in a bank for a question group. Linking to them requires that you link to the entire bank, so you have lots of little banks, one for each type of question, as opposed to one large bank that you select questions from.

Without question group, there is no random selection from a pool of questions.

I will acknowledge that is probably not how many people understand question banks. With New Quizzes, Canvas is changing the terminology and not having question groups anymore. I don't work for Instructure, but I imagine some of the confusion people have over question groups vs question banks might have had something to do with the change in names.

[Mattias_]

@James

Thank's James for your clarifications regarding question bank/question groups.

What I have done is, as you describe, to use question groups, and to link to question banks within these groups.

To clarify, I would like reduce the risk of unaccceptable cooperation between those who are taking a quiz. One such strategy is generate different problems of the same kind. The idea is that each quiz instance shall be as unique as possible eventhough it still attempts to measure the same knowledge as all other quiz instances. For this reason I am creating question banks where each bank specializes in one type of problem by containing variations of the same type of problem.

I would like to automate the process of generating a quiz as much as possible. For this reason I am carrying out an automated production of specialized question banks. It is then very convenient to create a quiz with, say, 5 questions by creating five question groups that link to 5 different question banks for 5 different problem types. I find it much less convenient to have one question bank that contains, say, 50 questions of the first problem type, 50 problems of the second problem type and so on. To automate the generation of such a bank is also more difficult than automating .

Moreover, I believe (but I am not sure it is correct) that creating the actual quiz the way I want it by including (rather than linking to the question bank) questions from the question bank requires quite some manual labour. Creating a question group for a particular problem type, find the associated questions in the question bank, include them one by one and then repeating this for the four other problem types. Perhaps there is a better way to do it but regardless of this I also believe that it will be difficult to get an overview of a quiz that contains 5 question groups of 50 questions each (i.e. in total 250 questions). Disregarding the automation issue I would rather prefer to have five question groups linking to five different question banks.

Best wishes,

Mattias

[Mattias_]

I did not finish the sentence below:

To automate the generation of such a bank is also more difficult than automating the generation of the specialized banks.

[James]

@Mattias_ 

I'm glad you've figured out at system that works for you. It looks like the general comment approach that Ron ( @Ron_Bowman ) suggested worked for you.

The finding / including does require some additional steps of selecting and including, but you can select more than one at a time. You can also still create separate banks and include all of the questions from that bank into the question group. As you said, it is more work, but it helps with the analysis later on. I was approaching the challenge from an automation perspective and linking to question banks makes it hard to identify things.

I appreciate the desire to mix things up to avoid cheating. On my statistics final, I had 100 questions in a bank and the students were going to get 50. There were 10 types of questions with 10 questions for each type and the were in sequential order (questions 1-10 were type 1, 2-20 were type 2, and so on). I created four different versions of the final and, because I hate repetitive work, used JavaScript to select the questions for me. Version 1 of the final got all the odds, version 2 got all of the evens, version 3 got the first half of each of the 10 questions for each type, and version 4 got the last half of each of the 10 questions.

Then I randomly assigned students to one of the four versions of the final. The first time this happened was in a course that met face-to-face until COVID hit and then went online. I had a sense from being face-to-face who was friends with whom, so I made sure they got different versions of the final.

[James]

@Ron_Bowman 

Ron, you can use the Quiz Questions API  to add comments to a quiz question when you create a question or when you edit an existing question.

[Ron_Bowman]

@James 

thanks for the information with the API's.  i am really going to have to learn API's and work with them - not to mention javascript.  Of course that is probably a rather deep rabbit hole that I do not need to go down right now.

My question was directed back at @Mattias_ because it sounded like he had an automated way of creating the questions and question bank - and if that was the case I figured he would be able to also add in the necessary information into the general comments when he created the questions.

Ron

[James]

@Ron_Bowman 

I wasn't sure if Mattias was using a package or programming quizzes directly. It sounded like he might be using R/exams or something similar.

I've used the direct approach through the API and as part of the question creation process, you can also create comments. When I tried to import questions through a bank (using Respondus), I was not able to get satisfactory results. I tried telling it to use the correct / incorrect comments and it didn't. In fairness, there was a warning in their online help to that effect.