@James suggested that I should file a blog post about my recent findings/results.
Background
I just recently started with Canvas because Uppsala University has decided to use it as its upcoming LMS platform after a failed attempt with another product. Therefore I had already spent some time with Blackboard and was quite fond of the calculated questions type in quizzes. I quickly found out that Canvas offers essentially the same functionality but a bit less comfortable.
Problem
A calculated question or Formula Question as it is called in the interface of Canvas is based on a table of pre-generated variable values and corresponding results. In the general case the variables are defined and the target function is entered using the web interface, then Canvas calculates random number values for the variables and the resulting answer value. However, as the designer you have no possibility to influence the variable values afterwards (unlike in Blackboard where you have a spreadsheet-like interface). Also, in Canvas, the equation cannot be altered once it has been entered - and the supported syntax is not very convenient for more complex problems.
I was also missing the ability to give a relative tolerance for the correct answers in a question, however, I found out that entering a percentage-sign exactly gives this behavior even though it does not seem documented anywhere.
Solution or problems?
My hope was then for the API, since it seemed to support the creation of questions. But even though there is a Python library for the purpose of controlling Canvas, many of the functions are not very well documented. My first tries failed miserably but finally I was on the right track.
The cause of my problems was that the Canvas API uses different field identifiers and structures when creating a calculated question as when you retrieve the contents of an already existing question, as I of course did in my attempts to reverse-engineer the interface.
Working solution
Here is now an example for a working solution to give you full control over the generation of Formula Qeustions using Python and the canvasapi library. The example is in Python 3 and creates a question from the field of electronics - the voltage in a voltage divider. The Python script generates the variables, fills the variables with random numbers from a set of predefined, commonly used values. I tried to write the script more for readability than any pythonic optimization.
from canvasapi import Canvas
import itertools
import random
API_URL = "https://canvas.instructure.com"
API_KEY = <your api key here>
canvas = Canvas(API_URL, API_KEY)
# create a calculated_question
# example of a potential divider
#
# U2 = U0 * R2 / ( R1 + R2 )
#
E3 = [1, 2, 5]
E6 = [1.0, 1.5, 2.2, 3.3, 4.7, 6.8]
E12 = [1.0, 1.2, 1.5, 1.8, 2.2, 2.7, 3.3, 3.9, 4.7, 5.6, 6.8, 8.2]
coursename = 'test'
quizname = 'test'
# define the input variable names
# each variable has its own range, format and scale
#
variables = \
[
{
'name': 'U0',
'unit': 'V',
'format': '{:.1f}',
'scale': '1',
'range': [1.2, 1.5, 4.5, 9, 12, 24, 48, 110, 220]
},
{
'name': 'R1',
'unit': 'ohm',
'format': '{:.1f}',
'scale': '1',
'range': [ i*j for i, j in itertools.product([10, 100, 1000], E12)]
},
{
'name': 'R2',
'unit': 'ohm',
'format': '{:.1f}',
'scale': '1',
'range': [ i*j for i, j in itertools.product([10, 100, 1000], E12)]
},
]
# how many sets of answers
rows = 30
# create an empty list of lists (array) for the values
values = [ [ i for i in range(len(variables))] for _ in range(rows)]
# create an empty list for the calculated results
results = [i for i in range(rows)]
# fill the array of input values with random choices from the given ranges
for i in range(rows):
for j in range(len(variables)):
values[i][j] = random.choice(variables[j].get('range'))
# and calculate the result value
results[i] = values[i][0] * values[i][2] / (values[i][1]+values[i][2])
# format the text field for the question
# an HTML table is created which presents the variables and their values
question_text = '<p><table border="1"><tr><th></th><th>value</th><th>unit</th></tr>';
for j in range(len(variables)):
question_text += '<tr>'
question_text += '<td style="text-align:center;">' + variables[j].get('name') + '</td>'
question_text += '<td style="text-align:right;">[' + variables[j].get('name') + ']</td>'
question_text += '<td style="text-align:center;">' + variables[j].get('unit') + '</td>'
question_text += '</tr>'
question_text += '</table></p>'
# format the central block of values and results
answers = []
for i in range(rows):
answers.append(\
{
'weight': '100',
'variables':
[
{
'name': variables[j].get('name'),
'value': variables[j].get('format').format(values[i][j])
} for j in range(len(variables))
],
'answer_text': '{:.5g}'.format(results[i])
})
# format the block of variables,
# 'min' and 'max' do not matter since the values are created inside the script
# 'scale' determines the decimal places during output
variables_block = []
for j in range(len(variables)):
variables_block.append(\
{
'name': variables[j].get('name'),
'min': '1.0',
'max': '10.0',
'scale': variables[j].get('scale')
})
# put together the structure of the question
new_question = \
{
'question_name': 'Question 6',
'question_type': 'calculated_question',
'question_text': question_text,
'points_possible': '1.0',
'correct_comments': '',
'incorrect_comments': '',
'neutral_comments': '',
'correct_comments_html': '',
'incorrect_comments_html': '',
'neutral_comments_html': '',
'answers': answers,
'variables': variables_block,
'formulas': ['automated by python'],
'answer_tolerance': '5%',
'formula_decimal_places': '1',
'matches': None,
'matching_answer_incorrect_matches': None,
}
courses = canvas.get_courses()
for course in courses:
if course.name.lower() == coursename.lower():
print('found course')
quizzes = course.get_quizzes()
for quiz in quizzes:
if quiz.title.lower() == quizname.lower():
print('found quiz')
question = quiz.create_question(question = new_question)
Since this is mostly the result of successful reverse engineering and not based on the actual source code of Canvas the above example should perhaps be used with care, but for me it is what I needed to create usable questions for my students. Perhaps this could also serve the developers as an example on how the interface for calculated questions could be improved in the future.
How does it work?
The dictionary variables (lines 26-49) contains the names and ranges of the variables, as well as formatting instructions. The ranges are given as lists. In lines 61-66 the random values are generated and the results calculated from these values. Lines 70-77 create a rudimentary table to be included in the question text containing the variables and their values as well as physical units for this particular question. Lines 80-93 finally assemble the variable/answer block and lines 109-128 put everything together into the dictionary to create a new question.
The script then inserts the question into an existing quiz in an existing course in line 140.
After running the script
This screenshot shows the inserted question after running the script, obviously this would need some more cosmetics.
And when editing the question this is what you see:
Be careful not to touch the variables or the formula section since this will reset the table values.
Cosmetics
In order to be presentable to the students the above questions needs some cosmetics. What is to be calculated? Perhaps insert a picture or an equation? More text?
After updating the question and leaving the editor it now looks like this in the Canvas UI:
Seeing and answering the question
When you now start the quiz, this is how the question looks:
Summary
- calculated_questions can be generated using the Python canvasapi library
- answer values have to be provided with the key 'answer-text'
'answers': [
{
'weight': '100',
'variables': [
{'name': 'U0', 'value': '9.0'},
{'name': 'R1', 'value': '5600.0'},
{'name': 'R2', 'value': '5600.0'}],
'answer_text': '4.5'}, - when querying an existing calculated_question through the API the answer values are found with the key 'answer'
answers=[
{'weight': 100,
'variables': [
{'name': 'U0', 'value': '110.0'},
{'name': 'R1', 'value': '82.0'},
{'name': 'R2', 'value': '8200.0'}],
'answer': 108.91,
'id': 3863}, - when supplying an equation for the 'formular' field this has to be done in a list, not a dictionary
'formulas': ['a*b'], - when querying an existing calculated_question through the API the equations are found in a dictionary like this:
formulas=[{'formula': 'a*b'}],
