Canvas cURL pagination: doesn't include "last" page link?

[sam_ofloinn]

For my project I need to get all the pages of data for a project.

I can make cURL requests on data endpoints and get all the pages I want. For example, see this DD of some page data that includes all four links: "Current", "Next", "Last" and "First":

array:4 [▼  0 => "Link: <https://ucc.test.instructure.com/api/v1/accounts/1/courses?page=1&per_page=100>; rel="current""   1 => "<https://ucc.test.instructure.com/api/v1/accounts/1/courses?page=2&per_page=100>; rel="next""   2 => "<https://ucc.test.instructure.com/api/v1/accounts/1/courses?page=1&per_page=100>; rel="first""   3 => "<https://ucc.test.instructure.com/api/v1/accounts/1/courses?page=218&per_page=100>; rel="last"\r"]

However, sometimes when I get page link data, they will not include the "last" field, I'll only have the first three to work with: "Current", "Next" and "First":

array:3 [▼  0 => "Link: <https://my.instructure.com/api/v1/accounts/1/courses?search_by=course&search_term=2019&page=1&per_page=100>; rel="current""   1 => "<https://my.instructure.com/api/v1/accounts/1/courses?search_by=course&search_term=2019&page=2&per_page=100>; rel="next""   2 => "<https://my.instructure.com/api/v1/accounts/1/courses?search_by=course&search_term=2019&page=1&per_page=100>; rel="first"\r"]

To clarify, in that above example, I can confirm there are far more pages than just two, there are at least seven. So I know there are many more pages to fetch, but for some reason I don't understand, there isn't a "last" link available.

For my project, I want to be able to always link to the last page, so having that "Last" link would be very valuable.

I have read the Canvas API documentation on Pages, on Endpoints, and on Courses and Users among other pages, but nothing there seems to have the solution.

Here are two URLs that make sure to include the "last" page:

"https://ucc.test.instructure.com/api/v1/accounts/1/courses?per_page=100";

"https://ucc.test.instructure.com/api/v1/accounts/1/users?per_page=50";

Here are some which don't.

"https://ucc.test.instructure.com/api/v1/courses/$id/enrollments?page=$pageNumber&per_page=50&sort=name;"

"https://ucc.test.instructure.com/api/v1/accounts/1/courses?search_by=course&search_term=$sis_course_id&per_page=100"

Does anyone know how I might be able to get it? Or have any ideas that might point me in the right direction? Any help at all would be greatly appreciated

[l_osullivan]

I have recently come across this problem too. API calls which previously returned the "last" url are now not doing so. I can only assume it is a Canvas bug.

[lmsstaff]

Dude!  Me too!  Apparently the Link header was changed to return a dictionary instead of an array or string.  So now, my course archiving project is undergoing a rewrite.  Was this an announced change?  If so, I totally missed it!

So now the Link header looks like this:

{'current': {'rel': 'current',
'url': 'https://everettsd.beta.instructure.com/api/v1/accounts/1/courses?page=1&per_page=100'},
'first': {'rel': 'first',
'url': 'https://everettsd.beta.instructure.com/api/v1/accounts/1/courses?page=1&per_page=100'},
'last': {'rel': 'last',
'url': 'https://everettsd.beta.instructure.com/api/v1/accounts/1/courses?page=62&per_page=100'},
'next': {'rel': 'next',
'url': 'https://everettsd.beta.instructure.com/api/v1/accounts/1/courses?page=2&per_page=100'}}

[l_osullivan]

I've checked with Canvas support and they let me know that you can't rely on the "last" link being available as it won't be computed if it's too "expensive".

https://canvas.instructure.com/doc/api/file.pagination.html