List active courses in account not paginating?

[ek]

Accounts - Canvas LMS REST API Documentation 

I've been using the API endpoint: 

GET /api/v1/accounts/:account_id/courses?per_page=250

For a while now to get all courses within a specific Canvas account. It has worked great, up to 100 courses. Now today I have 102 courses, and the last 2 courses after 100 are not showing up. Per the API documentation, pagination is in place where I should expect Link headers to give me another URL to call to get the remaining courses. This isn't happening.

Here are the Link headers I'm getting from my initial API request:

[link] => ; rel="current",; rel="next",; rel="first",; rel="last"

There is no URL being provided for rel=next or any rel for that matter

I also ran this test: I added request parameter of published=true, unpublished one of my old courses, and the 101st course showed up. So the request is definitely being limited to 100 courses.

GET /api/v1/accounts/:account_id/courses?published=true&per_page=250

I have also tried this:

GET /api/v1/accounts/:account_id/courses?per_page=10

Even when I set pagination to 10 results only, the Link headers are all still blank (no URLs provided) even though I have 100+ published courses. My results only bring back 10 courses.

Has anyone experienced this before? Coding in PHP on a wordpress site.

Sample request I'm doing:

 $get_courses = wp_remote_get('https://myuniversity.instructure.com/api/v1/accounts/6/courses?per_page=250', array(
 'headers' => array(
 'Authorization' => 'Bearer ' . $authToken
 ),
 ));

 print_r($get_courses);

The above returns the headers, body etc with the Link headers being blank and only showing 100/102 courses

[ek]

Oh, just realized this line in the documentation: These links should be treated as opaque.

I'm assuming that is the issue I'm facing, but not sure how to make them "not opaque". More learning to do!

[muh]

Hi Eric,

First, the cap of the per_page is 100. Second, for handling the pagination, you may refer to  @James  's comments here (Handling Pagination ).

Best,

Mu 

[ek]

Thanks Mu! James' comment will surely help me

[j_causby]

Hi Eric,

One thing I noticed is that you are trying to use ?published=true parameter on your API call, and a bit of testing and looking at the documentation I don't see that parameter being supported. However I get a list of published courses by using the ?state[]=available parameter instead, which is supported as shown in the courses api.

[pklove]

Eric is using the account course listing endpoint (not the courses - list your courses endpoint).

The account course listing (Accounts - Canvas LMS REST API Documentation) does have published as a parameter.

[pklove]

Have you tried doing the get in your browser and using developer tools to look at the headers?  Or using command-line curl to see what comes back?

Getting the link headers works fine for me with that endpont, so I'm wondering if it has something to do with wp_remote_get.  If you try one of the above, that might help sort out if this is some Canvas issue, or some code issue.

What other headers are shown when you dump the response?

[ek]

Hi Peter, good call. I will try getting the headers in developer tools to check. I'm thinking you may be right that it's something with wp_remote_get. The full headers I'm getting with wp_remote_get are (i removed some values, not sure if any of this is sensitive. sorry about formatting, can't seem to get it to show normally):

Requests_Utility_CaseInsensitiveDictionary Object (     [data:protected] => Array         (             [cache-control] => max-age=0, private, must-revalidate             [content-encoding] => gzip             [content-type] => application/json; charset=utf-8             [date] => Thu, 23 Aug 2018 17:02:18 GMT             [etag] => removed             [link] => ; rel="current",; rel="next",; rel="first",; rel="last"             [p3p] => CP="None, see http://www.instructure.com/privacy-policy"             [server] => Apache             [set-cookie] => Array                 (                     [0] => _csrf_token=removed; path=/; secure                     [1] => log_session_id=removed; path=/                     [2] => canvas_session=removed; secure; HttpOnly                 )              [status] => 200 OK             [strict-transport-security] => max-age=31536000             [vary] => Accept-Encoding             [x-canvas-meta] => removed             [x-canvas-user-id] => removed             [x-content-type-options] => nosniff             [x-frame-options] => SAMEORIGIN             [x-rate-limit-remaining] => 700.0             [x-request-context-id] => removed             [x-request-cost] => 0.6317221110005821             [x-request-processor] => 02b03309b595cf65f             [x-runtime] => 0.702321             [x-session-id] => removed             [x-ua-compatible] => IE=Edge,chrome=1             [x-xss-protection] => 1; mode=block             [content-length] => 11956         )  )

[ek]

Hi Peter, put the request in my browser and looked at headers and I indeed see the links. So must be something with wp_remote_get. Thanks for getting me on the right track!

[ek · Accepted solution]

UPDATE:

Was finally able to retrieve the next URL using the below code. Targeted the Link header and then specifically grabbed the URL between the "<>" characters. Thanks everyone for your comments! Specific thanks to James Jones in this thread: Handling Pagination. Hope this helps someone in the future.

     $linkHdr = wp_remote_retrieve_header($get_webinars, 'link');
     $Hdrlinks = explode(',', $linkHdr);
     $nextpage = NULL;

     foreach ($Hdrlinks as $link) {
         if (preg_match( '/<(.*?)>; rel="next"$/', $link, $matches )) {  
           $nextpage = $matches[1];
           echo "nextpage<br />";
           echo $nextpage;
         }  
     }‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍