Fetching enrollments limited by sis_course_id

[swaldie]

Based on this guide, the following works to return assignments limited by sis_course_id

/api/v1/courses/sis_course_id:A1234/assignments

And I've confirmed that this works to return just the course object: 

/api/v1/courses/sis_course_id:A1234

 However, I've not been successful in getting the concept to work for returning enrollments using this:

/api/v1/courses/sis_course_id:A1234/enrollments

I could make it a 2-step process by first getting the canvas course ID, then using the following endpoint:

/api/v1/courses/:courseID/enrollments

But I would rather do it all in one shot if possible.

Any suggestions would be appreciated.

[dgrobani]

Hmmm. https://samuelmerritt.instructure.com/api/v1/courses/sis_course_id:2020-1SP-01-NABSN-120-LEC-SAC-01/... returns enrollments. [edited to remove trailing comma from URL]

[swaldie]

Thanks for the quick response, dgrobani. The notice went to my junk folder, and I just found it today.

So that works for you. Interesting.

Using Postman, this works and returns the course object:

However, using this...

gives me the following message:

{

    "errors": [

        {

            "message": "An error occurred.",

            "error_code": "internal_server_error"

        }

    ],

    "error_report_id": 9585

}

[dgrobani]

@swaldieHave you resolved this? If not, I wonder if the encoded slash is a factor. I know it wasn't a problem in getting the course, but still, how about requesting enrollments for a course whose SIS ID doesn't have a slash?

[swaldie]

@dgrobani, that's it!

Using a different sis_course_id that does not contain the '/' worked to return enrollments.

Using this endpoint must need different encoding than I'm using for just returning the course_id.

Any suggestions?

[dgrobani]

Oof, that sounds like an API glitch to me. There might be some tricky way of getting around it, but I'm not aware of one. Perhaps contact Canvas support and report it? @James?

[James · Accepted solution]

@swaldie , @dgrobani 

Hex encoding works for the time being (it has been deprecated and undescribed since 2013). See end of post for details. The details of how I discovered this follow.

In the equation editor, they use double URL encoding to work around the case of a slash (for example, division). They encode the / to get %2F and then that gets encoded again to get %252F. But the routes aren't set up to handle the double encoding. I tried.

I also tried escaping it with a backslash \/ and that didn't work either.

One solution is to not use RFC 3986 reserved characters in your SIS IDs: Those are ! * ' ( ) ; : @ & = + $ , / ? # [ ]

Now, some of those are okay because the URL doesn't use them and Canvas doesn't use them. However, the / is used to separate the path components and Canvas uses the : to separate the prefix sis_course_id from the actual id. The : will work because Canvas splits on the / and then looks for sis_course_id: and takes everything after it, including the : as part of the ID.

I also stumbled across this thread from 2019 that had the same issue and the solution there was to get the Canvas ID and then use it.

After a lot of looking at source code, there was a comment about hex encoding that said to refer to the API documentation. At the bottom of the SIS IDs documentation, there is a section on Encoding and Escaping

The first paragraph of that section says:

SIS IDs should be encoded as UTF-8, and then escaped normally for inclusion in a URI. For instance the SIS ID CS/101.11é is encoded and escaped as CS%2F101%2E11%C3%A9

The / is clearly encoded as %2F there. But we know it doesn't work.

If you go down a little further, it talks about proxypass and mentions the double encoding.

Also beware that if you use ProxyPass, you should enable the nocanon option. Similarly, RewriteRule should use the NE, or noescape flag. Other modules may also need additional configuration to prevent double-escaping of %2f (/) as %252f.

It wasn't until the very last paragraph that things became useful.

Prior versions of this API documentation described using a hex encoding to circumvent these issues, since the proper Apache/Passenger configuration was not known at the time. This format is deprecated, and will no longer be described, but will continue to be handled by the server for backwards compatibility.

Then I went to the source code for the API documentation and went into the history to see what "prior versions" described.It says 

SIS IDs can be URL escaped as usual, for instance the ID "CS/101.11" could
be escaped as "CS%2F101%2E11". However, various releases of web servers and
Rails environments have bugs related to escaping of characters such as
"/" and ".". So it is recommended that SIS IDs be encoded using a hex
string notation, similar to a hex digest, where UTF-8 bytes are
encoded to hex digits and displayed as Ascii, high nibble first.

For instance, the string "CS/101.11" would be encoded as
"43532f3130312e3131". To perform this encoding in Ruby:

"CS/101.11".unpack("H*")[0]

The SIS ID is then included in the URL as usual, but prefixed with
"hex:", for instance:

/api/v1/courses/hex:sis_course_id:43532f3130312e3131/assignments.json

So, if you must use a /, then hex encode it and then use hex:sis_course_id instead.

But it says that format is deprecated, so you might want to consider moving away from slashes.

However, it still works for now.

It sounds like Canvas has a bug though since the way they say to do it doesn't work.

[swaldie]

Very informative.

Thank you, @James!