Batch API requests

[bbennett2]

I'm working on a proof-of-concept project which pulls in specific assignments and their submissions for a list of Enrollments. This means that I'm making repeated API calls in a loop (I'm using a Python backend and the

I know you can batch requests for SIS imports and other large migrations, but is there a genera batching parameter for the API?

[James]

 @bbennett2 ,

By batch, do you mean supplying a list of IDs and getting results for all of them with one call rather than making a separate API call for each? If so, most API calls do not support batch.

However, the submissions API does support it for the List submissions for multiple assignments endpoint. Despite what the title suggests, you can get all submissions for all assignments for specific students, get submissions for specific assignments for all students, or get submissions for specific assignments for specific students.

For other API calls, look at the documentation. Generally, any parameter that has a [] at the end is an array and can accept multiple values. For example, with student_ids[] or assignment_ids[] or just ids[], you can do batch processing with that parameter. The student_ids[] only appears in the documentation twice (once for the submissions and once for moderation).

If you mean something else by batch, can you clarify?

[bbennett2]

Thanks for the reply James.

I'd found the multiple submissions endpoint separate from your response,

which is what I'm looking for. I can get a valid response, but the

'grouped' parameter doesn't seem to nest responses by user like it says,

even when just testing with Postman.

There was another post about grouped data not working, so I'm wondering

there's an issue with the API itself. Any thoughts or experience with that

option?

On Friday, March 15, 2019, james@richland.edu <instructure@jiveon.com>

[James]

 @bbennett2  

I don't normally try grouped=1, but I tried it to see what happened and it did change things. Without the grouped=1 in there, the submission information was at the same level as the submission_id, assignment_id, and user_id. With the grouped=1 in there, it added an extra property called submissions. This is an array with just a single item in it, but almost all of the information except for user_id, section_id, integration_id, and sis_user_id have been moved under that.

It's poorly worded, but I guess technically, if you want to stretch it, it is grouped by user as information about the user is at the main level and then all the submission data is grouped under that.

I suspect what you're wanting is to get all of the submissions for a student. To do this, it's not grouped=1, but include[]=submission_history that you need to look at. That returns more than one submission and it also gives you a property called submission_data for each of the submissions when the assignment is a quiz.

By the way, that API call also has an exclude_response_fields[] property that can reduce the amount of information delivered. You can repeat this with attachments, discussion_entries, and preview_url to save a little bit on the amount of information downloaded. There may be some others that you can exclude, it looks like there's isn't a whitelist; those are just the ones I got when I watched the network calls being made by the gradebook. There are other tricks like that for assignments and assignment_groups and maybe some other places.

[bbennett2]

Thanks again for the followup. I have something else going on because if I pass more than one assignment ID in, I only get the first returned. If I make `assignment_ids = all`, it groups and returns everything correctly, so I'm not sure if it's formatting or something else.

This is definitely the route, though. Thanks for the tips.

[bbennett2 · Accepted solution]

I finally found the solution.

I was trying to string together assignment IDs into one parameter, but the Canvas API expects each ID separately in the URL string. For example, this is the wrong way to send the request:

https://yourinsitute.instructure.com/api/v1/courses/12345/students/submissions?student_ids[]=all&assignment_ids[]=118888,118889&grouped=1‍

The API expects multiple assignment_ids[] params specifying a valid ID. So, the correct string formatting is:

https://yourinsitute.instructure.com/api/v1/courses/12345/students/submissions?student_ids[]=all&assignment_ids[]=118888&assignment_ids[]=118889&grouped=1‍

Creating the string manually is tricky, but the fantastic canvasapi Python library from UCF Open has a simple method that takes the arguments as a list and makes the API call. To get the specific assignments, get a Course object and then pass in the arguments:

# See https://github.com/ucfopen/canvasapi for install and instantiation
from canvasapi import Canvas

course = canvas.get_course(12345)

assignments = course.get_multiple_submissions(student_ids='all', assignment_ids=(123, 456, 789, ...)

print(assignments)

<PaginatedList of Submissions>‍‍‍‍‍‍‍‍‍‍

Note that the current version of the library disables the grouped argument because it cannot (yet?) parse the returned nested object. I really wanted the student submissions grouped because we're building a table/row UI and grouping assignments makes that much easier. To get around it, I edited the source file (for dev only...) and used the Object.to_json() method to convert it to a JSON string and then loaded it back into the list as a JSON object for parsing.

from canvasapi import Canvas
import json

json_data = []
course = canvas.get_course(id)

submissions = course.get_multiple_submissions(assignment_ids=(123, 456, 789), student_ids='all', grouped=1)

for sub in submissions:
 json_data.append(json.loads(sub.to_json()))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The project isn't quite finished yet, but this is a big problem out of the way. When the whole thing is done, I'll write up a longer blog post.

[caiyunlong]

@bbennett2  Thank you for your post.

I have another question.  I tried to get scores of all assignments for one specified student, but it failed.

course = canvas.get_course(course='ECE2300J',use_sis_id=True)

submissions = course.get_multiple_submissions(student_ids='5410' ,assignment_ids=(7370,6574), grouped=False)

Error:
raise CanvasException(
canvasapi.exceptions.CanvasException: Encountered an error: status code 500

Have you tried this?  Any idea about the problems?

[MisterPerkins]

I'm a relative noob, but I think your assignment_ids should be enclosed in square brackets.

https://canvas.instructure.com/doc/api/submissions.html

[JArom]

For the last few months obtaining a valid structure via

modified_keys = ["assignment_ids[]="+str(key) for key in cloned['assignments'].keys()]

has been working.

I'd then make the request call within

submissions_students = canvas.get('students/submissions', all=True, course=course_id,
    params={
        'student_ids[]': 'all',
        'assignment_ids[]': modified_keys,
    }, data={
        'include[]': ['visibility', 'rubric_assessment']
})

Now I get:

<Response [403]>
{'error': 'invalid assignment ids requested'}

as a response.

Any insights into why this suddenly is not working? It doesn't look like the API changed.

Thanks!