How do we *USE* API?

[mgudites]

We are new to Canvas, coming off of Blackboard. I've got to say, I'm kind of shocked and dismayed at how little can be accomplished via the GUI. It seems like just about every thing I ask our implementation team about doing, the answer is always "you'll need to figure out how to do it in the API." 

Batch apply/change the Grade Scheme? Do it via the API.

Batch course-copy material? Do it via the API.

Batch publish a bunch of courses? Do it via the API.

I've been sent to this page a number of times: Canvas LMS API Documentation

We do have a programmer on staff who I know can figure this out. However, he's a busy guy and has limited time to be able to spend on this. The Canvas LMS API Documentation link above doesn't really give any information on where to begin...what do we need installed on a PC to make these calls, for example?

We've asked Canvas how to do this stuff, for some baseline instruction on how to setup a PC to run the scripts, only to be told "Instructure cannot tell you that, we're not a scripting company." I don't know how they can tell us "we can't tell you how to do this" when the software seems to rely on scripting for damn-near everything.

Can someone point me to some instruction please? Is there nothing out there, some kind of baseline instruction on how to prep a PC to run a script that can run against a Canvas environment, and how to build one of these scripts, so we can make some of these batch changes that came SO EASILY in Blackboard? Surely there are schools with Canvas who don't have an army of programmers at their disposal, but still have a need to accomplish tasks that the woefully inadequate GUI can't do. What do they do?!

[James]

 @mgudites ,

Start with Stuart Ryan's document: Canvas APIs: Getting started, the practical ins and outs, gotchas, tips, and tricks 

[agschmid]

https://insomnia.rest/  Insomnia is a nice tool to practice making API calls

[robotcars]

Simple browser based API tutorial. Developer Tools for the Canvas User 

Andrea mentioned Insomnia, also Paw – The most advanced API tool for Mac 

What languages does your programmer like?

I can provide pretty simple snippets in Ruby, Python, and Node.

[ember_ludwig]

Here's an updated link to Developer Tools for the Canvas User by robert-c:

https://community.canvaslms.com/t5/Developers-Group/Developer-Tools-for-the-Canvas-User/ba-p/270668

[jsavage2]

Hi  @mgudites , 

The transition from Blackboard to Canvas can definitely be a challenge. The advice to go to the API docs is given because Canvas (the system) is platform agnostic. A REST API is, at the end of the day, just a collection of web pages: you go to a particular, defined, url; Canvas performs a particular action and gives you a response. You can use any language or tool that's convenient for you. The examples in the docs use curl, which is included on nearly every linux/unix server and desktop, and can be a easily installed on Macs and Windows PCs (curl for Windows ). Most people who do much API work have a scripting language of choice, though, and there are good examples out there in almost every scripting language sysadmins use daily for configuration management on different platforms: Perl, Ruby, Python, PHP, .NET, Powershell, Node.js, etc. Even Bash for the particularly old-school.

That's a bit of a departure from Blackboard's "my way or the highway" approach to automation, but in an ideal world, it should make it easier to fit Canvas into whatever your school's existing IT ecosystem is. 

Stuart Ryan's Getting Started guide is definitely the place to start.

If you don't have particular preference and/or your school isn't an "X" shop, I'd suggest using the Python bindings from UCF (GitHub - ucfopen/canvasapi: Python API wrapper for Instructure's Canvas LMS ). Python seems to be the "it" scripting language at the moment and there's a lot of good information out there to get up and running quickly. 

But it doesn't have to be all scripting, either. Well-formatted JSON isn't difficult to read, and sometimes it's faster and/or better to just look at things with human eyes. There are also good API tools out there like PAW, which someone mentioned above, and Postman (Postman | API Development Environment ) that make it easy to manually construct urls with different parameters and read the responses, both for testing and for frequently-performed manual tasks.

The thing to remember is that Blackboard was cryptic and unintuitive at first, too, with that thing you do once a semester in a setting buried ten levels deep in some menu somewhere that you have to look up in manual the first 10 (or 20) times.

Having done this same transition, I'd also advise against putting the cart before the horse. Canvas is a very different product from some other LMS, and is designed to function in very different ways. Trying to replicate Blackboard's behavior is probably a recipe for frustration. A lot of the standard practices in Blackboard come from a very different time in LMS development, when the LMS was "it" as far as electronic materials we're concerned. So in addition to actual teaching and learning, Blackboard was designed to manage a lot of administrative "stuff" like posting syllabi, functioning as a student portal, and being a sort of general-purpose document distribution system. Canvas, though, is designed around the exception that it exists in an ecosystem with a dedicated student portal, which most places now have, and also probably a much simpler document distribution system like Google Drive or OneDrive or Dropbox.

Canvas is designed, pedagogically, to embrace a particular constructivist theory of education in general, and distance education in particular. The assumption is that faculty should be taking responsibility for their courses and be actively engaged in their design, management, and distribution, because that leads (in theory) to better student outcomes. Canvas resists central administration at almost every level, beyond basic enrollment, in favor of authority devolved to departments (sub-accounts) and ultimately faculty. So some of the things you're asking about--like publishing courses in bulk--are actually difficult by design, because the the expectation is that faculty should be making their courses available at a point that makes sense, pedagogically, for a particular course. Everything doesn't need to be published at the same time.

Similarly batch copy: direct batch copy isn't intuitive, but Canvas offers the Commons, for faculty to pull content from other courses, and Blueprint Courses for instructional designers to push content to courses, in addition to the normal course copy function that is available to faculty in all their courses. It's ultimately the same result as Blackboard, it's just initiated from a different point in the hierarchy.

Administrators *can* do it vis the API, but it's not really the way the system is designed to be used. It's kind of like offroading in a Porsche. Sure, you *can*, it's got 4WD, but the dealer isn't going to suggest it, and the people who do it modify them heavily for rallying.

Anyway, I've done several transitions from other systems to Canvas, and I'm happy to chat offline about the process, and about learning the API as somebody who isn't a professional programmer.

[wchang]

Hi @jsavage2,

Are you still available to chat with people offline? I'm asking because this is an old post. I am ok with the Live API but, I'm not a programmer.

--Wen

[kblack]

 @mgudites  - In addition to the ideas above, for some reason Instructure has not been very good about promoting the Live API tool that is a somewhat more user-friendly API.  While it is not good for anything where there are multiple requests, it may be handy for simpler, one-off API calls that may need to be done. 

There is information about this tool at a presentation that was done at InstructureCon 2016 here:   .  Basically, you can get there by your institution's base Canvas URL followed by /doc/api/live .  You still need to generate an access token, of course, but afterwards it's fairly easy to navigate once you figure out (based on the API documentation) where to go!  Not every single API call is there, but as I say it's a bit of a user-friendlier start for those times when you can't get to your programmer.  I still can't figure out why this is not more publicized out there, unless Instructure is simply no longer devoting resources to developing it.

[qnguyen]

Do you think this would be done for any tasks that found in the API Live page? By using the URL only without using any API interface?

I am looking for a way to see the total hours of each student within a term that he/she has been spending in a particular course?

Thanks

[mgudites]

Thanks everyone for the responses so far! I sent our programmer some info on "Insomnia" and we're also checking out the Live API thing. That's definitely helpful and yeah, not sure why Instructure wouldn't promote that. 

[pklove]

If you are looking at Insomnia, you might also want to consider Postman (Postman | API Development Environment).

There are some articles on the web comparing them.

[robotcars]

Also, https://community.canvaslms.com/groups/canvas-developers/blog/2016/09/11/api-testing-postman 

[penny_christens]

Any updated page - this page no longer there

[mgudites · Accepted solution]

We're still trying to figure out how to script things. For example, scripting course-copying or batch-publishing courses.

My main problem with the "Canvas Universe" is that Instructure does not do anything to help on-board people with this API thing. They send you to the API Documentation page and say "here you go." For new clients there should really be at least a little bit of hand-holding because again, not all of us are programmers.

"Here's what you need to install/configure on your computer to run this stuff."

"Here are some sample scripts that will help you accomplish some basic tasks across multiple courses at once."

I realize it's all "open standards" or whatever but that doesn't mean jack to someone like me. If Instructure is going to rely so heavily on the API for even the most basic of tasks then there should really be more out there for beginners.

[jsavage2]

Hi  @mgudites ‌,

I think you may be misunderstanding the advice being offered both from Instructure and from the people who have responded on this forum. While some people have suggested favorite tools, there is nothing you need to install to use the API. It is designed to work with whatever you already use to manage your technology.

I realize you aren't a "programmer", but as an experienced system administrator I'm sure you're familiar with system automation and configuration management. Whatever toolset you use to accomplish that, now, will work out of the box with Canvas. If you're not a Linux user, it may not be clear, but the examples on the API pages are working examples. 

curl 'https://<canvas>/api/v1/courses/<course_id>/content_migrations' -F 'migration_type=common_cartridge_importer' -F 'settings[question_bank_name]=importquestions' -F 'date_shift_options[old_start_date]=1999-01-01' -F 'date_shift_options[new_start_date]=2013-09-01' -F 'date_shift_options[old_end_date]=1999-04-15' -F 'date_shift_options[new_end_date]=2013-12-15' -F 'date_shift_options[day_substitutions][1]=2' -F 'date_shift_options[day_substitutions][2]=3' -F 'date_shift_options[shift_dates]=true' -F 'pre_attachment[name]=mycourse.imscc' -F 'pre_attachment[size]=12345' -H 'Authorization: Bearer <token>'

That is a complete, working command** that will migrate content from an existing course cartridge (a common export format from other LMS) to a canvas course and update the start dates.

If you are a Windows admin, you can pass the same parameters to PowerShell's standard "Invoke-WebRequest" tool instead of the cUrl tool that Linux admins favor. To do a batch of courses, you'd just loop through the ids, as you might for any PS script you wanted to run on a list of users or list of machines.

If you want to copy from one Canvas shell to another, you can use "course_copy_importer" and "settings[source_course_id]" instead of "pre_attachment[name]". So to copy from course <XXXX> to course <YYYY>, something like:

curl 'https://<your_canvas_url>/api/v1/courses/<XXXX>/content_migrations' -F 'migration_type=course_copy_importer' -F 'date_shift_options[old_start_date]=1999-01-01' -F 'date_shift_options[new_start_date]=2013-09-01' -F 'date_shift_options[old_end_date]=1999-04-15' -F 'date_shift_options[new_end_date]=2013-12-15' -F 'date_shift_options[day_substitutions][1]=2' -F 'date_shift_options[day_substitutions][2]=3' -F 'settings[source_course_id]=<YYYY>' -H 'Authorization: Bearer <token>'

If you didn't see the suggestion above to check out  @stuart_ryan ‌'s Getting Started guide, one of the links is to  @rosec ‌'s instructional video on avoiding traditional automation systems entirely, doing it all by storing the parameters in Google Sheets: https://www.youtube.com/watch?v=g8XRYqzj6Qo

I hope this clarifies things a little

**You need to substitute your own url and login information into the <variables>, and obviously update the dates to whatever is relevant for you...or just ignore those options.

[mgudites]

Yes, this is the kind of thing I needed to see. We are a Windows shop and I'm very familiar with PowerShell. I use it all the time to manage values in Office 365 and SharePoint, among other things. This is making more sense now -- thank you!

[robotcars]

Matt,

I agree, it's a little overwhelming but there's support here.

Our SIS integration was built on the Bash import found here, kajigga/unsupported-canvas: Unsupported Canvas Tweaks, with ideas and solutions in many languages (it's older some tweaking may be required)

Which was forwarded to me by our CSM at the time.

Our grade book report was built in Ruby by ProServices, with a Ruby script to download it from the API... I have learned everything I know about Ruby since supporting that 1 script. That's not a lot... but's adding up.

I also spent a considerable amount of time performing tasks with Python scripts after InstructureCon Hack Night'15, where a very nice group of people were able to tell me how I could accomplish a task, and then pulled their code down and emailed it to me.

I realize it's all "open standards" or whatever but that doesn't mean jack to someone like me. If Instructure is going to rely so heavily on the API for even the most basic of tasks then there should really be more out there for beginners.

I full agree. While the community is nice, and there are those here always willing to offer their support. Others will get help, solve their problems to the point where they don't come back and share. Those who share may not do so here and indexing and keeping things up-to-date is hard. There are a lot of solutions on Github, you just have to search for them and then support them. I often feel like what we need are Recipes, so people have a common starting point.

Had a conversation with  @James  awhile back we kinda settled on the idea of a Rosetta Code for Canvas.

The thing to remember is no matter what programming language you feel comfortable supporting the API is the standard. If you can find the code or ask here on the community you can most likely start mixing and building what you need.

Another perspective after 6 years on Canvas is... no matter how complex and uncomfortable this feels now, once it's all setup and automated for you, you will likely take a break, reflect on what you've learned and then start looking at the API to figure out 'what can I solve now'. The greatest thing about Canvas is its openness. The API is the tip of the iceberg.

You mentioned batch publishing courses... I just copied my batch purge and here's a batch publish in Python.

This is done via, Update Course - Canvas LMS REST API Documentation 1 course at a time.

But you can also batch/batch 500 courses at a time, Update Courses - Canvas LMS REST API Documentation, which would require a little more programming than I've provided here.

import requests
from multiprocessing import Pool, freeze_support
import sys
import json

# declarations
token	= "26~asdf"
url 	= "https://edu.test.instructure.com/api/v1/"
headers	= {'Authorization' : 'Bearer ' + '%s' % token}

# courses to update
courses = [123,345,567,890]

# loop through the courses
for course_id in courses:
	req = requests.put(url + 'courses/' + str(course_id) + '?course[event]=offer', headers = headers)
	res = req.json()
	print (res)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Here's a version in Ruby with a little more of the boilerplate code I use for handling 'jobs'.

#!/usr/bin/env ruby
require 'httparty'

courses = [123,456,789]

class CourseUpdater
  include HTTParty
  base_uri 'https://asdf.test.instructure.com/api/v1'
  headers 'Authorization' => 'Bearer 26~asfd', 'User-Agent' => 'asdf-canvas-api'
  #debug_output STDOUT

  def update(course_id, event)
    self.class.put("/courses/#{course_id.to_i}",
      :verify => true,
      :query => {
        :course => {
          :event => event
        }
      }
    )
  end
end

begin
  course_api = CourseUpdater.new()
  updated = 0
  total = courses.count
  
  courses.each do |course_id|
    update = course_api.update(course_id, 'offer')
    if update.code == 200
      updated = updated+1
      # print progress to screen
      printf("\rworking: %d of %d", updated, total)
    else
      puts "course #{course_id} - status #{update.code}"
      puts update.request.last_uri.to_s
    end
  end
rescue => e
  puts e
  puts 'something went wrong'
end

# when complete
if updated == total
  puts "\nall courses updated"
else
  puts "\nfinished with errors"
end‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Remember to always test your code against TEST or BETA instances first.

[jsavage2]

I just want to second everything carroll-ccsd just said, and also put in a plug for perl if you're still looking for a scripting language. The canvas-specific documentation is thinner, but there are great newbie lists and communities to support learning the language. 

As a scripting language for LMS administration its __END__ and DATA keywords make perl uniquely useful to administrators. Essentially, you can include an ad-hoc data file at the end of the script and not be constantly editing variables to accomplish one-off tasks. The data doesn't clutter the code, and when you need to run you can just paste the data into the end of the file from whatever spreadsheet or report your users or enrollment department send you.

A lot of ruby and even python code can also be used without much modification, since many of the most popular ruby gems are straight reimplementations of perl's original CPAN modules, and the python modules are based on the ruby gems.

Perhaps most importantly, Randal L. Schwartz, biran d foy, and Tom Pheonix's "Learning Perl" (O'Reilly) is, bar none, the best introduction to programming, both concepts and practical techniques, ever written. 

[robotcars]

Nice comment. I have not tried PERL in a long time.

I actually posted this months ago, but just noticed that (and edited) Ruby#L17, because it was 'courses' and not 'course'.

[James]

I've been using PERL for a long time and had never used __END__ and DATA. Thanks!  The best book I found on PERL is the Perl Best Practices by Damian Conway. It wouldn't have been good for a beginner, but it was good when I wanted to clean up my code and start writing it the correct way.

I started all of my Canvas API code in PERL and migrated to PHP and am now moving towards Node JS. The problem with PERL and PHP was lack of easy asynchronous calls. There was Guzzle for PHP, but I never got into the whole Composer thing, probably because I had been using PHP for 20 years or so and it's hard to learn new tricks. JavaScript was new to me, so I learned the modern way without all the baggage of not keeping up with the language. I haven't had time to mess much with Python yet, but it seems promising.

[jsavage2]

It definitely depends on what you’re trying to accomplish. Perl and NodeJS (and to a certain extent PHP) definitely fill very different niches. I don’t think today I’d start a new UI project with mod_perl.

It’s still the secret weapon of many sysadmins, though, for scripting boring stuff that happens often enough that you don’t want to keep typing it but not often enough or in a consistent enough way that building out a full-fledged application makes sense. The idea that everything doesn’t always have to be an object can be a powerful tool for automation and data transformation.

(And both Net::Curl::Multi and Net::Async::HTTP make asynchronous calls easy, although personally I don’t find much use for them in the Canvas admin part of my job.)

[amber_batten]

Hi Matt,

I know this is a delayed response, but I just came across your post.  I can possibly help with the course copying.  I've created a process (a bit involved) for our needs.  All of our courses are copied from "master" courses so that all teachers teach the same stuff.  I'm in a K12 virtual school.  Our teachers do not build their own courses.  Anyway, every semester we have to copy over 3000 courses from about 135 "masters".  We are slowly moving to Blueprint.  So, after manually copying these courses one semester, I figured out a way to initiate copies in bulk via the API.  I am not a programmer, either.  

I'm happy to help!

Thanks,

Amber

[mgudites]

Hi Amber,

Thanks for the offer! Since my original post, I do have some scripts that we use to batch-perform some settings. It's starting to make a little more sense to me now though, I still think it's dumb how little you can do in the GUI. Like, if I wanted to delete 20 users or delete 20 courses, it's strange to me that I can just check the boxes in the GUI and hit "delete." Some things just shouldn't need a script.......

My next challenge is figuring out a way to delete inactive users/old courses. Like if I wanted to script finding users who have never logged in, or, finding users who haven't logged in for say, a year, and then deleting them. 

[mgudites]

(Also, hi from Asheville)

[qnguyen]

Hi @Amber

I would love to see the sample API that you have for copying Master to 1000 courses, that is one of the solutions that we are going toward. I wonder if it would be a hard task since I am a newbie to API. Thanks

[holmer]

I have all of my institution's major API batch scripts written in Java. I use the APIs to make schedule changes (add and drop users from courses), download current grades, export / import grades from Canvas into our Oracle database, etc. Your programmer can begin by experimenting with API endpoints just in a web browser and then pick a language they are comfortable with to write some batch scripts.

[qnguyen]

Hi everyone,

I am a newbie to API but recently got a request for getting total log-on hours for all students within the current term.

Is there any already made API script can do that task since GUI report is very limited.

Thanks

[asergay]

Well, I agree that it's thin gruel that Canvas provides for API support. And all the people saying I can provide a sample script, please do. I am too stupid to find out how to contact people through this bulletin board, and if I want a sample, so do a dozen others. 

So how about posting examples here? Apparently you can attach a file. What I need is a Python script that includes the business about pagination and looping API requests so I get all the pages of needed information, or all the changes I need to make. Currently I want to change a bunch of courses' subaccounts. 

Also, as long as I'm ranting, how about an updated list of CCS classes Canvas uses? Why is that kept secret?

Signed, The Interface Curmudgeon
(emphasis on the curmudgeon part)

[JamesSekcienski]

@asergay 

I highly recommend taking more time to explore the Canvas Community, especially in the Canvas Developers Group.  There are a lot of examples people have shared in various posts and resources to help with getting started and understanding APIs.  Here are a few that may be of interest:

• Awesome CanvasLMS 
• Canvas APIs: Getting started, the practical ins and outs, gotchas, tips, and tricks 
• Handling Pagination 
• Pagination makes endless api calls when retrieving page_views 

If you still need assistance beyond these resources, it would be helpful to make a new post in the Canvas Developers Group to help make your request more visible.  Comments on old posts, especially after a solution is accepted may be missed.  It is also helpful if you share what you have tried to do so far.

As for the CSS classes, I don't think they share them because it would restrict their ability to make necessary changes/updates to the interface.  They also don't want users to become dependent on a certain CSS class existing and then taking it away or changing it in a new update.  You may also notice that some of the class names appear to be randomly generated which is likely due to a system/package they use to generate content.  While you can utilize CSS class names that you discover through inspecting elements, you are doing so at your own risk, which is why it is important to continue to monitor the Beta server for upcoming changes that may impact your code.

[memrys]

Making REST calls can be difficult without coding skills. However, the SIS integration is relatively easy to use with some basic Excel know-how. If you want to do batch updates, start here. You can upload CSV files (simple tables with a header row) to add hundreds or thousands of courses, accounts, enrollments, etc at a time.

And if you add the same account/course/enrollment again (with the same unique SIS ID for the course/student/etc) it won't add a duplicate, it'll update the existing item. So you can also UPDATE thousands of entries at a time.

The things you can manage with SIS files includes Users (teachers, students, admins), Accounts (sub accounts, don't confuse with users), Terms, Courses, Sections (of courses), Enrollments (linking users to courses), Groups and Group Memberships (dividing course enrollments into smaller sets for collaboration and group assignments), User Observers (allowing parents/employers to keep tabs on their student), Admins (to grant additional powers to staff), Logins (only needed if you manage multiple auth methods for users), and Change SIS ID (in the rare case you need to manipulate the normally fixed SIS IDs).

This covers the most common mass-insert and mass-update needs, and just requires some basic spreadsheet skills. Can you do more with the REST API? Absolutely. Of the things you listed, this can manage batch publishing. If you use blueprint courses (which I highly recommend) then you can also batch create courses from the blueprints which removes the need to batch-copy content. Batch management of the Grade Scheme is still an API-only task as far as I'm aware though.