Canvas APIs: Getting started, the practical ins and outs, gotchas, tips, and tricks

stuart_ryan
Community Novice
‎03-26-2018 02:18 PM
31
149066

Overview:

Welcome one and all, this document is designed to assist everyone from those that have never heard the API acronym before, to seasoned programming veterans who may be looking for tips and tricks that are specific to the Canvas API. I have worked to structure this document to get more complex as it goes on, so you should be able to find your comfortable starting point easily. Most of all, if something doesn't make sense, please leave a comment, if you have some ideas for inclusion, or even if you would like to challenge something that I have proposed, I encourage you to leave a comment!

 

API Foundations - Let's go through the basics

So, you may be asking 'What is this API acronym'? API stands for Application Programmer Interface, and in this section, I will cover off some basics about what APIs are. When you access a website (such as Canvas), you are accessing an application. The application has a web interface that is written for humans to interpret information and interact with that information. These interactions may be adding data such as discussion posts, pages and module items or they might be changing data, such as updating student or staff information or enrolments in courses. This list is not extensive, and there are many other functions available. However, these are just some examples I will reference for highlighting the differences between the web interface of Canvas and the APIs.

 

The key difference between the Canvas web interface and the APIs is that the Canvas web interface is designed for a human to interpret and interact with the application, it presents information in a visually appealing way, using things such as iconography and visually logical structures to users such as staff, students, and administrators. The APIs differ in that they are designed for consumption by a computer. You can think of the APIs as 'just another website', but a website that computers can access to interact with Canvas and the information contained therein. For a computer to most effectively communicate with another computer, the visual elements and structures of a web interface such as Canvas' web interface, only get in the way. Therefore, the APIs present functions and data in formats that other computers and programs can call and consume in a standardised way.

 

Canvas Web Interface Example API Interface Example
Image displaying the Canvas Web Interface with Two Test Courses Canvas API Interface displaying only the first of the Test Courses on my user account

Important points:

  • Displays a variety of information including the courses the logged in user is enrolled in
  • Sources and displays information from multiple courses such as what is 'Coming Up'
  • Provides visual guides and references to guide you through the Canvas application
  • A user can easily bounce to different areas of Canvas by clicking buttons such as their inbox, calendar, Courses, Commons and so on

Important Points:

  • The image above shows only a representation of the first item on my dashboard the 'Test Stu' course. To obtain this information I used the API for 'Courses by User' instructing Canvas to provide me with a list of courses based on my user ID.
  • From the image above, you can see this is verbose, and only represents a small portion of what the application pulls together for users in the Canvas web interface
  • The information presented includes information generally not exposed to users such as internal system IDs, Account IDs, Term IDs, enrolment IDs and so on
  • There are no visuals included. However, you may see that there is a logical defined structure to the information. The structure displayed is known as JSON and is the structure Canvas utilises to tell a computer (via scripts and programs) to interpret the data that Canvas sends back.

 

What can the APIs Do?

The APIs can achieve most things that are possible within the Canvas web interface. It is important to note that the APIs are for Canvas itself. Therefore tools installed such as LTIs will not be accessible through the Canvas APIs and are out of the scope of this document. The concepts herein, however, could be applied to any LTI vendor's APIs if they publish an API specification similar to the Canvas LMS API documentation. 

 

The functions listed in the Canvas LMS API documentation are extensive, and some of the more common applications of the APIs include:

  • Integrating with other systems at your institution to provision user accounts (for systems not supported by direct SIS integration)
  • Enabling LTI tools greater levels of access to information to interpret more about a user or context
  • By savvy admins who wish to automate elements of their work, setting up new courses, copying content and more
  • This list is by no means exhaustive and represents just some of the ways you can use the APIs, though, in all honesty, your imagination is the limit

 

Can anyone use the APIs?

Indeed they can! Canvas publishes the full API specification, and any user can interact with the APIs. The key to note here is that a user will only be able to access as much information in the APIs as they are permissible to within the Canvas Web Interface. Therefore, if a student tries to query information about their account, they will have the authorisation to get the details back. However, if they attempt to run a query for another users' account, they would get an access denied error. Equally, I will reiterate here that a user must have an authenticated account or access token for your institution to access the APIs, without which, the API calls will fail.

 

A great way to illustrate this is the Canvas Mobile App for iOS and Android. The mobile apps use the same APIs that are available to institutions and users, to interact with Canvas and present information to users on the run based on their user account. The mobile apps take the information that Canvas presents in the API Interface Example above, interpret that information, and then re-present it to the user with visual elements much like the Canvas Web interface.

 

What should I know before getting started if this is my first rodeo?

I am glad you asked! :smileygrin: There are some incredibly important things you should know before you get in and start playing.

  1. It is incredibly easy to change things in your environment with the API, and this means it is also incredibly easy to accidentally do something wrong or unintended when you are learning. For this reason, start out using your Beta environment (i.e. yourinstitution.beta.instructure.com ). The Beta environment refreshes weekly, therefore, if you break something on a large scale, you only have to wait for the next week to get a clean slate.
  2. Consider leveraging your Test environment also (i.e. yourinstitution.test.instructure.com ). After you have played around and have a script or integration working in Beta, I recommend testing it against your Test environment. You may think of it along the lines of User Acceptance Testing of your script or integration before migrating it to your production environment. When developing something that modifies data at scale, I highly recommend this progressive process.
  3. Don't think you have to start from scratch! With the Canvas APIs being completely open, and with such a great community here, it is highly likely what you are trying to achieve has been done by someone else at some point. The Canvas Developers‌ group is a great place to look through discussions on the API, examples that people have posted, and issues they have faced as well.
  4. Don't try to do too much, too fast. Start small, get your authentication working with a simple script (say, pull the user details for a single hardcoded ID), once you have that, build on it further, and so on. I would also recommend, save versions of your script every so often (you may wish to use Github.com). That way, if you accidentally break something, you can go back to a known working version.
  5. Be mindful that you will need programming skills. If this is your first time programming, you may need to step back and look at some basic programming courses first. I would probably suggest something like Python (despite it not being my language of choice, give me PERL any day).
  6. Lastly, and most importantly, NEVER EVER EVER EVER (EVER) run a script you have downloaded from the internet against your production environment without first reading the script through fully and completely. You need to ensure you understand what every line or block of code is attempting to do before you run the script. Often scripts are provided best-effort, some may have errors, some might have testing code left in them, some may not quite meet your exact needs, and hence they should be taken as a starting point. You should also be mindful of the potential for malicious scripts, though unlikely, it is still possible. I stress that it is good practice and policy to only ever run something in your environments that you fully understand yourself. 


When are Canvas APIs most appropriate?

What are the available alternatives?

There are several alternatives you may consider before you look at the APIs. I won't cover these in great detail, as there is a wealth of information already available, hence, I suggest you check out these links for more information.

 

Which alternative should I choose?

So, now you know that Canvas has multiple ways to get data in and out of the system, you may be wondering which one you should choose!

  • Prebuilt SIS integrations – These are great for automatically hooking into supported systems with minimal effort. For institutions that are looking for a holistic, out of box solution, pairing Canvas with one of the supported SISs can lower the time and ongoing administrative investment over both the short term and long term.
  • Canvas Data – The Canvas Data platform provides a read-only view into some normalised data about your Canvas Instance. It is great for understanding ‘what HAS happened’ (with a delay of up to 36 hours). Primarily it is designed for reporting. Some institutions choose to leverage Canvas data to trigger when they should do something with the APIs (though, this is far less common). Another important note for Canvas Data is that 'deleted' data will not show in the Canvas Data exports, you will need the API to see any deleted records.
  • APIs – The APIs provide Real-time, access to both active and ‘deleted’ data. The APIs are great for ‘what IS happening’, 'what HAS happened' or ‘I want to change something’. You may wonder why you would use the APIs to ask the 'what HAS happened question' as Canvas Data provides this as well. Canvas data can answer this question rapidly en masse for large sets of data (hence, great for reporting), the APIs can answer this question for small subsets of data, usually far more quickly than setting up a Canvas Data environment. Also, you would pick the APIs should you wish immediately up-to-date (i.e. real-time) information.

 

Design concepts and considerations:

Oauth with Developer Keys vs User Access Tokens

One of the first things you must decide is how you connect to the API (i.e. with which type of authentication method). The method you choose will largely depend on your needs.

  • User Access Tokens (generated from within an individual user's settings page) are useful for tinkering, or a single-user run script. They will only provide the equivalent permissions of the owning-user.
  • OAuth. OAuth is a little different as it enables user logins for an application, then the application can interact with the API as that user. You can read more about this at OAuth2 - Canvas LMS REST API Documentation. The best way to describe this would again be with the Canvas mobile apps. The apps use a shared key and secret (developer key) to permit a user to authenticate to Canvas and log in, without the app itself ever storing the password, Canvas generates a session key and provides this to the mobile app for that user. For any user-facing apps or development, you must use OAuth to protect your user's credentials.

 

Understanding API rate-limits: a practical guide!

Canvas outlines the API rate-limiting policy in the API Rate Limiting document. However, when I first read this, it took me some time to get my head around the best way to approach designing my solutions to ensure I wouldn't run into any issues. I recommend having a read of that first, then come back here for some of my additional tips.
 

There were a few of things that we came up with as possibilities to ensure we did not run into any issues:

  1. You can avoid the rate-limit entirely by using sequential processing only - by doing things in a sequence, such as a loop, waiting until one call had finished to continue, you avoid the rate-limiting entirely. The 'bucket' (as described in the official Canvas guide) is to stop a flood of requests from a single 'application' or 'user' so to speak. Hence, sequential is not an issue. With regards to user tokens, these operate in the same way. The documentation suggests a user token as, in theory, if you had 1000 students who needed to do something via the API concurrently, they each have their own API rate-limit 'bucket' and therefore would not get throttled. The best example I could give is the way the mobile apps work. Rather than a single API key for a mobile app for an account, when a user logs in, Canvas returns a user session token which is used for the API. That way when each student uses their mobile device, it is parallelising requests from all students on their individual session tokens. 

  2. Spread the load across multiple API keys - The second option you can opt for is using an API token per request type (as one example) - you could (if you had a significant need to do parallel processing) use a different API key per API request type. For example, if you need to get course details, you use one API token, if you need to set course details you use a second API token. There are other implementations of this (such as having a pool of keys and using round-robin to select a key for each new call), but I would anticipate this is the most common.
  3. Do minimal parallelisation - with only a small number of parallel threads (less than 5) the likelihood of hitting the API limit lessens, more on that below.
  4. Parallelise and gracefully handle the rate-limit - if you are planning to run parallel queries, you can (and I dare say should, regardless of which solution you select) gracefully handle the rate-limit, set a 'sleep' time and then try again when the rate-limit has gone away. Gracefully handling the API limits ensures you implement your solution in a programmatic, logical, and supported way.

 

The option we have used for our major integrations are three and four. Personally, if you are doing anything other than sequential processing, I think it is good practice to gracefully check for, and handle the rate-limit error (i.e. option four). In our instance, we decided to run five threads in parallel, with the rest of the operations in those threads running sequentially. That gave us a good hybrid of performance, along with ensuring we would not hit any API limits as the most we have running at once is five API calls on a single token. 

 

Pagination concepts

The next item you should be aware of is the need for pagination when you are using the API. Pagination is not overly difficult but is often encountered as one of the first stumbling blocks when getting into the Canvas APIs. 

 

The Canvas API Pagination mechanism is documented in the Pagination - Canvas LMS REST API Documentation. So, what is pagination exactly? When you have large amounts of data that you are pulling out with the API, it would be incredibly system intensive to pull all that data out in a single call. One example includes using the API to list all courses in an account. You could conceivably have hundreds, if not thousands (if not possibly tens of thousands) of courses returned, depending on the size of your institution. If you made a single call and that retrieved all results, it could result in significant delay in even getting the results, and this is where pagination comes in!

 

Canvas (by default), only returns the first ten items by default in an API call and will also add an item in the 'header' of the return message indicating there are more 'pages'. While you can increase the per_page_limit, be aware that (as per the documentation) the per_page_limit maximum value is undocumented, so be aware, your mileage may vary if you choose to tinker with this. What you need to be most aware of with pagination is that no matter what you are doing with the API, no matter how simple, one-off, or just playing, you will need to handle pagination.

 

Once you implement pagination, things work a little differently. Using our previous example, when you call the API to list all courses in an account, Canvas will return the first ten and also tell you there are more pages. Therefore, your script/integration must recognise that, and then call the Canvas API asking for the next ten, and the next ten, and so on. For those starting out with programming, this is a great way to get into playing around with looping in your chosen language. 

 

When you don't handle pagination, you will find you may struggle as you seem to get small result sets, you may think there is no real rhyme or reason to the result sets you get back from the API, and you may think data is missing. Therefore, if you run into any of these, I highly recommend ensuring you have handled pagination in some way, shape, or form, and that it is working correctly.

 

You may want to check out this thread Handling Pagination which has some interesting discussion on, and code examples for, handling pagination. I stumbled across this recently and it is a great starting point that I wish I had found earlier.

 

Developer Keys and User Access Tokens in Beta and Test Environments

All righty, if you only read one section of this document (surely you want to read them all right?!?), please make it this one! I can not stress the importance of this concept, especially for those starting out with the APIs as things can go very very wrong, very VERY fast.

 

One of the great things Instructure provides for hosted clients is several copies of their production environment including Beta and Test. There is a slight downside of these the environments' regular cloning to be aware of, if you haven't been made aware of the better alternative (kudos to Deactivated user for calling out the update to this one). Historically, one had to manage developer keys for the Test and Beta environments manually. As Brett has now made me aware, that there is now an option when you edit a dev key to mark it as "Test Clusters Only" so it will only work in Test and Beta and not Prod. To use it, create the key in your Production environment and it will then sync down to the Test and Beta environments on refresh and be available for testing but wont work in Production!

 

The best practice option ensures that when you think you are playing around with a Test environment if you only have a key that works in the test environment, there is no risk of it connecting to your production environment, it would simply give access denied. When you have the same keys sitting on the production and test environments, you run the risk of accidentally thinking you are working with Test (or Beta), when in fact you are modifying production data.

 

Remaining Concepts

There are a few more concepts you should know about in getting started in the APIs, for these I will refer you to the existing documentation already available, though if there are any that people would like me to expand on, please let me know:

  • SISIDs - Canvas allows you to store unique, institution/school provided identifiers for things like courses, terms, students and so on. These are known as SISIDs (Student Information System IDs). Utilising these can mean integrating with the API is significantly easier (and requires less logic). Check out Object IDs, SIS IDs, and special IDs - Canvas LMS REST API Documentation  for more information.
  • Masquerading - When using a developer key, you can have a program masquerade as another user and have an API call appear that the specified user made it. See Masquerading - Canvas LMS REST API Documentation for more information.


Planning, as you have to crawl before you walk:

Tips for getting started and some useful resources

When you want to get started, the best thing I can suggest is start small, don't try to build Rome in a day, build your foundational column and work on that until you get your head around it. Equally, don't think you are alone, there are some great resources out there to help you get started:

 

Picking something to tackle

Figuring out your first task to tackle might seem simple, you have an immediate need, and you want to dive into that. I would propose something a little different. If you are starting out completely fresh, consider doing something you already know about, to simplify the learning curve. For example, you may need to automatically enrol some users with the API. Before jumping into this, I would propose taking the time to put together a script in your favourite language that does something more simple. By starting with functions and features you know well, the learning curve of how the API works will be easier to grasp. Some possible starting places you may want to consider include:

 

Tools to help you on your way

  • Canvas Live API - This one will blow your mind (well, OK it blew my mind, so hopefully it does for you too)! When getting started with the API, you should visit yourinstution.instructure.com/doc/api/live. Please note, you will need a user access token or a developer key to utilise the Live API. The Live API is a great way to tinker and learn about the data, the data structures, and how to send and receive responses to and from the API. Remember, this is exactly the same as making changes on your live systems, so when you are playing around you may want to use yourinstitution.beta.instructure.com/doc/api/live for safety as you learn.
    screenshot of the Canvas API with arrows to the access token
    Remember to put your access token in and press 'Save Token' to ensure the API has an authentication key to work with. Otherwise, all you will get back is access denied.

    Screenshot of the Canvas Live API with Courses functions displayed
    With the Canvas Live API, you can see some great detail on what each field in an API response contains, for every one of the API functions.

    Screenshot of the Canvas Live API with Courses functions for sending data displayed
    The great thing is you can also prepare a call in a visual interface that helps understand the process as you build a query and get a response back. The Canvas Live API is a really great way to troubleshoot when things aren't working as expected.

  • Postman - Check out this awesome blog post  @garth ‌ put together on what Postman is and how you can use it API Testing: Postman.
  • I will add more here as I come across them, or please feel free to make recommendations also!

 

Useful Video Resources:

John Raible from UCF talks about getting started with the Canvas APIs.

 

 @matthew_emond  introduces canvasapi (Formerly known as PyCanvas) - A Python Wrapper for the Canvas API

You can check out the source code for canvasapi at the official Github page --> GitHub - ucfopen/canvasapi: Python API wrapper for Instructure's Canvas LMS  


API Calls Made Simple -  @rosec ‌

Thanks so much for the recommendation of this video  @audra_agnelly ‌!

 

Optimising – The evolution from tinkering to a future-proofed and production ready script:

Once you have the basics of using the APIs under your belt, the next steps are programming for robustness. A lot of us start out with simple use-cases, looking to achieve a specific purpose. In such cases, a simple script that assumes a best-case scenario is often appropriate. 

 

Moving beyond the simple scripts, however, you get into the realm of developing for reliability and robustness. I have prepared this section with some insights into things you should keep in mind when you start developing solutions for more than just yourself, or very simple tasks.


Covering your bases – adding robustness to your scripts and integrations:

Logging

This may sound simple, and this may even sound incredibly logical, but when you start out, I implore you to log your actions and what your integration/scripts are doing. It is great practice to build in both basic and debugging logging. You may want to look at a high level of detail such as 'something happened that I didn't expect - log it', or you may choose to go down to the level of 'I am about to try something', 'I am trying something', 'I tried something and this was the result'.

 

By adequately implementing logging, this will (by its very nature) force you to implement excellent error handling. More on that below! As an example for Python, check out https://fangpenlin.com/posts/2012/08/26/good-logging-practice-in-python/ (Python 2) or The Pythonic Guide to Logging  (Python 3, recently updated) which covers off some good logging practices. The reason I use this as an example is it shows the usage of a proper logging module, which does a lot of the heavy lifting for you. Once you have your language chosen, jump over to Dr Google and search for something along the lines of '<language> logging best practices' or '<language> logging modules and error handling'.

 

Timeouts and error handling

Something I have seen come up often surrounds API timeouts. If you work in an enterprise environment, you may have worked primarily with self-hosted systems, rather than something in 'the cloud' (or as the formal term goes, Software as a Service, or SaaS for short). When you have an application or system hosted by your institution, rather than over the internet, there are a lot more known factors, especially surrounding the network. When you jump over to a SaaS application such as Canvas, however, you have hundreds, if not thousands of network devices between you, and the system, therefore, you should develop your integrations with additional robustness.

 

A common misconception is that 'the API should not timeout, it should be more robust and I shouldn't have to handle timeout errors'. Today I am here to bust that myth! Any number of issues can cause timeouts of which Canvas is only one small piece of the network puzzle in the mind-bogglingly colossal chain of devices you rely on to connect between the two. Sometimes in the ether of the internet, something takes a little longer and some of what you sent over the internet goes astray. Most of the time, the internet is robust enough to handle this, sometimes, despite the best-laid plans of mice and men, you get a timeout.

 

Therefore, in my opinion, correctly handling timeouts is not even optional, it is a necessity. One great implementation is that if you get a timeout, you should try again, if you get another timeout, implement a mechanism to sleep and then try again after X period (e.g. 60 seconds). You may want to re-try at increasingly longer durations for up to 24 hours, then implement a hard failure. Not only does this cover you for occasional glitches in the world wide interwebs, but will also cover you for any outages if your institution loses internet connection for a brief period, or if Canvas experiences either scheduled or unscheduled downtime (as two examples).

 

Now for the rest of error handling! Rest APIs employ standard HTTP error codes in the first instance, and then application specific error codes in the second instance. I highly recommend checking out HTTP Status Codes on Rest API Tutorial's site. You may not initially want to handle every single one individually, 200's indicate success (that's good), 400's and 500's indicate a problem (that's bad), now you know these exist you can handle them (that's good), having to handle them can be time consuming (that's bad). Kudos to anyone that can pick the pop culture reference/joke here!

 

I recommend looking at implementing an error handling module/subsystem/routine/procedure in your code, you write it once, can pass any results through it, and it will ensure consistent and robust error handling.

 

Finally, application errors! An application error is something that you try to do that Canvas itself says 'sorry, not going to happen'. This may be because you don't have permissions/access to execute your desired call, it may be because some of the required data is missing in your call, it may be because you have some flags incorrectly set, and so on. In these cases, it is essential to log what you tried to do, the calls you sent (ensure you do not record your API authentication token in logs), the result the API returned, and importantly the exact time and date (and timezone if relevant) you sent the call. Capturing all these details will make things far easier to diagnose if you need to contact Canvas support, and will greatly aid the team helping you to narrow down the problem.

 

Rate limit handling and rate limit optimisation

While I have covered off most of this in the earlier sections, it is imperative to reiterate the concepts when discussing robust solutions. When you put together a script/program/integration that your institution comes to rely on for day-to-day operations, you should ensure you have accounted for these. The last thing you want to happen is an integration to work until you reach a level of scale where you start hitting the API limits, which could catch you off-guard.

 

Assuming you have already read the 'Understanding API rate-limits: a practical guide!' Section lets build on this further! One thing I did not touch on further is the detail on the best way to handle API rate limits. Rather than simply detecting, and backing off, you can go a step further, and your script/integration can be aware of the API limits.

 

If you have a read through Throttling - Canvas LMS REST API Documentation, it states: 'To assist applications with planning, every request will return a X-Request-Cost header that is a floating point number of the amount that request deducted from your remaining quota. If throttling is applicable to this request (it could be disabled on your Canvas installation, or you are whitelisted and not subject to throttling), there will also be a X-Rate-Limit-Remaining header of your remaining quota.'.

 

Therefore, if you are starting out, and wish to go down the path of parallelisation of your queries, I would strongly recommend you consider logging the X-Request-Cost in your debug logs, as well as the X-Rate-Limit-Remaining value. Knowledge is power, and ensuring you are aware of the maximum scale of your solution ensures you understands its limits.

 

You may find that you can run numbers (with some assumptions) and ascertain the acceptable throughput of your solution. If you find you are exhausting the API rate limit repeatedly, you may need to optimise your code (to reduce API calls if possible) or follow some of my earlier suggestions to use different API keys per call (as one example).

 

Troubleshooting – what you can do when things don’t go to plan:

Things won't always be rosy and when you are programming that is OK, especially when starting out. For me, it is half the fun of learning a new system or programming language, as it helps to understand the inner workings. The below section covers some basic troubleshooting tips to help you out when things go wrong.

 

Step 1: Check you connected to the right environment with a valid access token

It may seem logical, but sometimes this will snag you, make sure you connected to the right environment. If you have multiple environments, this is an easy one to miss. Equally, once connected to the right environment, ensure your API token is valid and correct.

 

Step 2: Check the call in the Canvas Live API

When you run into problems, first and foremost I would go to the Canvas Live API. The graphical interface is incredibly useful for walking you through API parameters, calls, and responses can be invaluable for helping to walk you through your logic. I know from personal experience that the Canvas Live API tool has helped me catch a few logic errors in calls that I was making (or expected returns).

 

Step 3: Search the Canvas Community

The Canvas Developers‌ group, along with the rest of the community holds a wealth of information and examples. When I started out, I spent a significant amount of time looking through examples, and hence I am an advocate of exploration.

 

Step 4: Ask a question in the Canvas Developers group

If you haven't had any luck thus far, jump in to the Canvas Developers‌ group where you can ask a question. There are many knowledgeable members of the community that may be able to help point you in the right direction.

 

What to include:

When you ask a question in the community related to APIs it is great to ensure you give as much information as possible. You can use this template to help you get started with the sorts of details that can help:

 

  • What you are setting out to achieve (in plain English, rather than code): 
  • Programming Language (including version): 
  • Operating System: 
  • Has this ever worked before, or are you writing something new:
  • Related API endpoint (with a link to the relevant documentation/examples you have used): 
  • Relevant snippet of code (ensure you remove any API tokens, and URLs etc.): 
  • Example input data (if available, again, ensure this is dummy data): 
  • Are you hosted by Instructure (SaaS) or self-hosted: 

 

Contact support (if you are an Institution on the SaaS hosted Canvas product)

If your institution hosts Canvas with Instructure (Software as a Service), you can drop a request to Canvas Support. Also, if you are at the beginning of your implementation, your Canvas technical implementation staff member may be able to assist you (chat with your CSM). Please note that Canvas Support's responsibility is to help if the Canvas APIs are throwing application errors, if you are having issues with your particular language you are programming in, that is where you would need to fall back to the forums.

 

Should you contact Canvas Support, there are some things that you should include. The list is a copy of the list you would usually post to the Canvas Developers group, with some notable additions:

  • What you are setting out to achieve (in plain English, rather than code): 
  • Programming Language (including version): 
  • Operating System: 
  • Has this ever worked before, or are you writing something new:
  • Related API endpoint (with a link to the relevant documentation/examples you have used): 
  • Relevant snippet of code (ensure you remove any API tokens, but *keep* URLs): 
  • Example input data (if available, again, ensure this is dummy data): 
  • Exact date and time of a failed attempt to make the API call: 
  • Copy of any returned error message including HTTP status code: 
  • Canvas user ID that the authentication token belongs to (do not include the token itself): 
  • IP Address of machine/terminal/system that the API call originated from: 

 

Providing support with all this information will let them check into the error you are receiving, and ensure that they have the right information at hand to assist you as best as possible. 

 

Specific cases that come up from time-to-time:

Below is a list of specific cases that come up from time-to-time, these are things that seem to come up semi-regularly, therefore, I have added this section to cover them off and point you in the right direction.

 

When removing an enrolment, it only seems to be concluded and not deleted:

When you are removing an enrolment, be sure you set the task parameter to deleted. If you do not pass the task parameter (or it has a syntax issue) the default behaviour of the API concludes the student enrolment. Check out the following question for some screenshots https://community.canvaslms.com/thread/23387-using-the-content-migration-api-for-blackboard-content#....

 

Appendix A: Resource List

Most of these I have laced throughout this document, and are included here also as a complete list of resources:

31 Comments
a_craik
Community Contributor
‎04-12-2018 08:13 AM

Hey  @stuart_ryan  , just wanted to give you some early feedback as requested:

  • First (and very minor point) - the two embedded videos say "not authorized to view this content", so maybe a quick permissions tweak required there?
  • Second -

    WOW. just.... WOW.

    I've been thinking about writing up my own API learning journey to date - all the scouring of the community, finding different pieces of the jigsaw and wondering how they fit (and what the picture is), making notes of best practices/things to avoid... but I don't need to write up this journey because where you've currently written up to is where I'm at: what's next when you can make simple calls in Postman. How fortunate for me!

I'm very excited to see the next section appear Smiley Happy 

Bravo sir. May many learn from this.

stuart_ryan
Community Novice
Author
‎04-12-2018 03:37 PM

Hi  @a_craik ,

Very interesting RE the videos, I will ask around and see what I can drum up on this because it would be great to get that sorted. I will keep you posted when I hear back, so thank you for raising this.

Hehe and thank you! Looking forward on getting into the next section, it should be right up your alley. 

Cheers,

Stuart

Boekenoogen
Community Contributor
‎04-12-2018 04:46 PM

This is a great resource! I will share this with others at the university. 

audra_agnelly
Community Champion
‎04-13-2018 07:25 AM

Thanks  @stuart_ryan ‌, this is great! I haven't built a full tool from the ground up but have dipped my toes in the API's. For users new to APIs, I found the API Calls Made Simple presentation from last year's InstructureCon incredibly helpful for retrieving data from Canvas, which demos a simple Google Sheet you can copy and use to run GET calls. I have encountered pagination issues with some calls when I use that sheet and we've largely moved over to Canvas Data to retrieve the same information. You did mention that Canvas Data doesn't show deleted data, but I haven't run into that issue, and in most of my queries, I need to filter out deleted items. I'm thinking in particular of courses, where the workflow state is available, completed or deleted. Just curious if you've run into certain data fields where deleted data is unavailable? And finally, circling back to APIs, I'll say that one lesson learned after spinning my wheels is the point you mention about access to making those calls. We have APIs in our custom JS that have worked fine when I test and fail when I masquerade as a user. After hours lost, I now know to first check an API call to verify that the targeted user will have the permissions use that call. 

stuart_ryan
Community Novice
Author
‎04-17-2018 01:21 AM

Hi All! 

 @a_craik ‌, I have done some tweaking of how the videos are embedded. With some help from erinhallmark, I think you should now be able to view the embedded videos. It had to do with the specific 'type' of youtube URL I put into the embed. Please let me know how you go!

Thanks so much for the additional recommended video @audra_agnelly ‌, I have now put that into the document as well! This is what I love about having sessions recorded, so much of the information is timeless!

Cheers,

Stuart

0 Likes
a_craik
Community Contributor
‎04-17-2018 08:38 AM

Hi Stuart

Embedded videos are now playing fine in the page - thanks for that! Just as an update from a current 'API student', I have recently followed John Raible's recommendation on learning python the hard way, but should note to others that this resource is no longer free.

For newbies who don't have a coding language behind them I think it's hard to know which way to even start out... it certainly was for me! I've seen people who are in the know advise that you can use 'the language of your choosing'... but picking was proving a nightmare that just resulted in more questions than answers (e.g. "Canvas is built on Ruby, so isn't this the 'smartest' choice for API also?", or "will my choice be affected if/when the GraphQL API comes about?").

Anyway, I'm sticking with Python (holds breath in fear of someone advising against this!)

colin_lowe
Community Explorer
‎04-17-2018 07:01 PM

Thanks for putting this together Stuart, an incredibly well written and useful introduction to API's.  I didn't know about the API live function which is a great way to learn and understand the API's.

stuart_ryan
Community Novice
Author
‎04-18-2018 09:24 PM

Very welcome  @colin_lowe ‌, never far away so if you have any questions (as I am guessing you are deep-diving on that at the moment), always happy to chat.

0 Likes
James
Community Champion
‎04-18-2018 10:58 PM

 @a_craik ,

I couldn't tell if the questions were rhetorical or not. It sounds like you might have the answers, but I'm going to try to provide them for those who come along after you and may be wondering the same things.

"Canvas is built on Ruby, so isn't this the 'smartest' choice for API also?"

No. Neither should you use a robot to drive your car just because robots were used to build your car. You do not have to consume the API in the same language that generates it. You do not have to be Italian to enjoy Italian food. If you want to really appreciate Italian food, you may go spend a summer in Italy and immerse yourself in the culture, but that's not necessary to eat the food. You may learn Ruby so that you have an appreciation for what Canvas is doing, but it's certainly not a requirement to consume their information.

That's the beauty of standards. The API exposes it in a way (JSON) that makes it possible for this to happen. If it exported a Ruby data structure, then you would need something that could understand Ruby, so Ruby might be the best choice. But it exports JSON and common modern languages either support it natively or have a library or package you can include if it doesn't. Some languages have better support for JSON than others. For example, PERL lacks a Boolean (true/false) type, so you have to fake it with 1 or 0.

"will my choice be affected if/when the GraphQL API comes about?"

Unlikely. There are GraphQL libraries for a lot of programming languages.

Support for JSON and GraphQL are so widespread that it is unlikely to be the driving force behind what language you use. I started using the Canvas API with PERL when I was writing back-end stuff. Then I switched to PHP when I was writing web interfaces. Now I'm using JavaScript since I want things that run inside the browser. I've got to say that JavaScript has been simplest to accomplish API calls with, but then I tend to use built-in functionality rather than external libraries or frameworks. PERL and PHP both have a lot of overhead when it comes to making requests and support for multiple connections that can be used to speed up the throughput is poorly implemented or documented with those clients. JavaScript lets me set up 30 API requests, send them off, and wait for them to finish (if I want) before I continue.

Yet JavaScript hasn't been the easiest thing to learn, especially for people like me who are used to something else. And just when I thought I was getting it, I discovered the more modern version scopes variables like I thought it should because I came from PERL and PHP. Then JavaScript became even more power, cleaner to write, easier to understand in one sense, but harder since there was more powerful. Those changes make it not supported in MSIE 11 and less help available since it's newer. I wouldn't necessarily recommend JavaScript for someone who doesn't know any programming language, unless they were needing to run something that worked inside a browser.

People are correct that you can choose about any language to access the API in. It definitely doesn't have to be the language that the original software was written in. Now, if you are planning on self-hosting Canvas and tweaking the software, then I would definitely have some understanding of Ruby / Ruby on Rails.

I know that answers that don't answer the question the user thinks they have aren't helpful for someone who has no clue about what to even ask. I would suggest that starting with the programming language you're going to use isn't the right question at that point anyway. Spend time getting to understand the API and play around with the live API or using a REST client like Postman or Advanced Rest Client. You're going to need to know how it works before you can ever hope to make it work within code and debugging API calls within a program is much more difficult than through a dedicated piece of software.

The language that you choose because you think it's easy to and quick to learn may not be the one that's best suited for the task you want to do. Just like you need to familiarize yourself with the API and learn how to do it, you'll want to figure out what it is that you want to do with the information after you have it, where you're going to run the program, etc. I teach math, not programming, but I know that having a clearly defined goal is important in both. Know what it is that you want to do before you start to do it and there will be a lot fewer headaches along the way. Knowing where you're headed will help you pick the best tool to get you there.

Another thing to do before picking a language is to search for some of the questions you're asking and see how good the answers are for that language. One of the Stack Exchanges is where I often end up. Look at the quality of the solutions; are they understandable or leave you going off on additional searches to use them? Are there lots of solutions or are solutions hard to find? How clear is the official documentation of the language? How easy is it to get started?

Finally: Unless the person telling you what language you should use is going to be there with you to help you out when you have problems, then any advice should be considered just a recommendation.

jperkins
Instructure
Instructure
‎04-23-2018 08:39 AM

This is a fantastic resource, and for anyone who is interested... as part of my Masters Program, I created a self-paced course on Canvas Network about getting started with Ruby & the Canvas API. See more information in my blog post, or sign up directly here.

stuart_ryan
Community Novice
Author
‎04-28-2018 03:54 PM

Hi  @jperkins ,

That is awesome thank you! When it moves over to a self-paced course, I will add it to the resources on the document here.

Cheers,
Stuart

0 Likes
cgaudreau
Community Contributor
‎04-30-2018 09:21 AM

A man, sitting on a couch, clapping his hands together.

jperkins
Instructure
Instructure
‎07-19-2018 08:34 AM

The course is now available as a completely self-paced course. Hope people find it helpful!

Sign up for Ruby & the Canvas API - Self-Paced

kmeeusen
Community Champion
‎07-19-2018 03:19 PM

Hi  @jperkins 

I tried registering for the self-paced course, and the registration button is unresponsive.

Kelley

0 Likes
jperkins
Instructure
Instructure
‎07-20-2018 12:48 PM

Kelley,

I see you are in the course now. I'm assuming that you got it to work.

Let me know if you are having other issues,

0 Likes
s_hols
Community Novice
‎10-18-2018 02:05 AM

Very insightful and very well written, thank you for taking the time to make everything easier for the rest of us  

pklove
Community Champion
‎12-11-2018 03:17 PM

Hi Deactivated user‌,

I just had a query from someone who referred to your above section "Oauth vs Developer Keys vs User Access Tokens".

You list three distinct methods of accessing the API: user access tokens, developer keys, and OAuth.

I thought it was only two: manually generated token and OAuth2.  As you mention under OAuth2, you use a developer key as part of the OAuth flow and the permissions the token has are those of the user (and can now be limited by the new scopes if they are in use with the key).

You seem to be saying there is some other way to use a developer key and that it gives you "complete root access".  How do you use a developer key without involving the OAuth process?  In the OAuth case you would only get root access if you logged in as an admin.  Have I completely missed something?

Peter

stuart_ryan
Community Novice
Author
‎12-18-2018 03:04 PM

Hi Peter,

Indeed you are correct and that is a mistake on my part. Lemme fix that right up! (Not sure how I got my mind around that when I wrote it #myBad)

Stuart

bdalton_sales
Instructure
Instructure
‎02-18-2019 04:00 PM

 @stuart_ryan ‌ it might be worth noting that in the "Developer Keys and User Access Tokens in Beta and Test Environments" section that there is now an option when you edit a dev key to mark it as "Test Clusters Only" so it will only work in Test and Beta and not Prod.  To use it create the key in Prod and it will then sync down to the test and dev on refresh and be available for testing but wont work in prod.

stuart_ryan
Community Novice
Author
‎02-19-2019 01:02 AM

Awesome, thank you for the update Deactivated user, I have put in an update to the document, appreciate the extra info!

Cheers,

Stuart

lars_vemund_sol
Community Contributor
‎03-15-2019 08:14 AM

Hi,

First: What an awesome resource!

Second: I miss the "Hello World"-chapter (or a link to it):

  • Where do i start?
  • Do i need a designated server?
  • Can i run scripts from a browser extension?

Those questions might be hopeless, but i believe this is where a lot of people crashes even if they have some coding skills.

Maybe a small example script and how you get it running?

Lars

awilliams
Instructure Alumni
Instructure Alumni
‎03-15-2019 10:24 AM

Hey  @lars_vemund_sol , 

Short answer, no you don't need a dedicated server and yes you can run scripts from the browser, browser extensions, and browser-based apps, as well as native applications. I think the jumping in point of this content that is most relevant to your questions would be "Planning something to tackle." That's where Stuart starts talking about beginner tasks and tools you can use to accomplish them. 

Here is another useful post about a specific browser extension / browser app that people like to use: API Testing: Postman.

Also, the replies to this recent discussion provide a lot of other useful tools for interacting with API's https://community.canvaslms.com/message/138999-how-do-we-use-api 

0 Likes
dschuma
Community Participant
‎05-03-2019 01:41 PM

Nice job,  @stuart_ryan !

0 Likes
tlampley
Community Contributor
‎05-06-2019 02:14 PM

Greetings  @stuart_ryan  Smiley Happy

Thank you for this!

Have a SUPER Day!

0 Likes
tom_likes_wine
Community Explorer
‎04-02-2020 08:02 AM

WOW. This is the best introduction to an API I've ever seen, and I'm only half-way down. I'm interrupting the read to go play with the Live API - genius! 

ember_ludwig
Community Explorer
‎10-01-2020 11:54 AM

Hi @stuart_ryan,

Thanks for posting this.  I was really interested in the developer keys and the new beta/test option that @bdalton_sales mentioned, but our consultant at Instructure is recommending Access Tokens for our internal development.  And the Access Tokens are a LOT easier to create.  What are your thoughts on which to use for internal development, not to be accessed by users? 

We actually already use the APIs to provision our courses and do some internal reporting, but we're looking to re-write things and expand it some.  We were hoping to get the scope & beta/test cluster option with the Access Tokens, but that doesn't look like an option.  So, I created a idea conversation to see if they could be "merged" for internal development, if you'd like to chime in.  🙂  (or anyone else, we'd love to know how others are doing this).

https://community.canvaslms.com/t5/Idea-Conversations/Add-internal-use-Access-Tokens-to-Developer-Ke...

Thanks!

cre
Community Member
‎02-09-2022 07:00 AM

I am an administrator who has full account access.  I am trying to determine if I could get assignment group percentages with the API for all users for all courses for each grading period (you can view this info in the Canvas gradebooks, so I imagine it should be possible...)

Alas, I looked through the documentation, but it doesn't seem like this is something I can get.  Has anyone else tried for the same information and succeeded?   I just want to know if it's possible.

If it's possible, my next step is to meet with a developer to get them to build me something for the report I need (eventually I want to learn how to do this myself, but I need this info ASAP).

0 Likes
RonniTyger
Community Participant
‎05-22-2022 11:01 AM

Ryan, please give me the URL to where I should get started to learn about API and Canvas.  We have some needs that our IT folks are not prioritizint.  I have been a Canvas administrator for many years, but at my current job, I am only a sub account administrator for our School.  We have needs for customized reporting and we want to create a dashboard.  I should try to learn what I can.  Thank you for this fabulous information.  Can you recommend next steps?

0 Likes
Doug9
Community Participant
‎03-30-2023 07:04 AM

Thanks @stuart_ryan and thanks for the tip on logging.  I hadn't thought about that.  I'll add logging to my scripts.

0 Likes
rungun_nathan
Community Explorer
‎12-29-2023 07:41 AM

Stuart - nice guide.

I would add how to generate the access code.

To generate "User Access Tokens": In canvas Account->profile->Settings->Approved Integrations->New Access Token.  It will open a dialog box, you can name it "Testing" and give it an expiry date or leave it open.  Remember to save this to use it in the future.

RupertRussell
Community Contributor
‎04-01-2024 10:47 AM

Thanks for this article,
It may be worth updating to remove the reference to Canvas Data

Which as far as I can tell was deprecated on December 31, 2023. 

 

0 Likes