Obtaining and using Access Report data for an entire course

[James]

May 27, 2020, update (version 15):

Another week, another update. This time I installed a new linter to check for crud and subtle issues with the code. It turned up a lot of little things and at least one place where I was using an array when I thought I was using an object. Here are some of the changes for version 15

• The file is now sorted by the sortable name, when present. Version 14 sorted it by the Canvas User ID.
• Promises now work in the code. Version 14 relied on Bottleneck's idle function to kick in, but it might have triggered before all of the fetches were done.
• The script can now handle students enrolled in multiple sections of the course. You can generate a delimited list of sections, duplicate the information for each section, or just return the first section found.
• The script now only tries to fetch usage information when there is a last_activity_at date. That is no guarantee that there is an access report, but I never found an access report without having a last activity date.
• The script can now hide empty columns with no data. Previously, this was only for SIS data unless you edited the code.
• The script can be run correctly without reloading the page. Previously it appended the data, so if you ran it twice without refreshing the page, the data would be there twice.

May 19, 2020, update (version 14):

I've done a major rewrite of the script.

• It now uses a throttling library to play nicer with Canvas. Large courses were getting inconsistent results because the requests were failing. Requests shouldn't fail, but if they do, I throttle it even more and then try to repeat the failed requests.
• I added the Last Activity and Total Activity Time from the People page to the report.
• I added the current course score and current course grade to the report (the grade is only available if you're using a grading scheme).
• I modified the quiz views to match what Canvas gives. In the data supplied, they count taking a quiz as a view, but they subtract that off in the online access report. My data now matches theirs.
• I check the permissions of the user to make sure they can generate a report and don't add the button if they cannot. Previously, students could add the access report, it just didn't give them any information. Now it doesn't even show the button.

This is a major rewrite and updating will show a lot of changes. There is a good chance that any local customization you have done to the script will be lost. Those using a non-instructure.com domain for Canvas may need to reset their // @include line on line 5 (see the custom URL section below).

Synopsis

Canvas provides an Access Report for each student, but you have to click on each student's name from the People page to get it. This document will show you how to install a button on the People page that will create a .CSV file of the Access Report data for all students enrolled in the course. It will then show some ways you can obtain useful information from that data using Microsoft Excel. Although much of this data could be obtained through Canvas Data, this script makes it available to the instructor in real-time.

Quick Install

For those power users who are impatient, here are the quick install steps.

1. Install a browser userscript manager Tampermonkey for Chrome/Firefox/Safari
2. Install the Access Report Data user script.
3. Navigate to the People page and click on the "Access Report Data" button

If you run into problems, be sure to go back and read the instructions.

Note that the Tampermonkey extension for Safari requires payment. This is for the author of the extension that allows the script to run. I am not asking anything for the script itself.

Introduction

Canvas will give you an Access Report for each student that tells you how many times a student has viewed or participated in a particular content item. To obtain the list, you go to the Course Roster by clicking the People navigation link, then click on a student's name. Once you do that, you will get an item on the right-side navigation bar that says "Access Report for student's name". When you click on that, you get something like this:

It gives you the type of content (as an icon), the name of the content, the number of times a student viewed the content, the number of times a student participated in the content, and the last time the student viewed (or participated) in the content. It is sorted by the time the content was last accessed so you can see what they have been working on most recently.

Unfortunately, this information is available for just one student at a time, so answering questions like "Who viewed the PowerPoint presentation I told them to read?" becomes difficult -- you need to go into each student individually to see that information.

I've written a User Script that solves that problem. It fetches a list of all of the students in the class and then obtains the Access Report for each one, compiling all of that data into a single Comma Separated Values (.CSV) file that can be opened with a spreadsheet like Microsoft Excel.

This document shows you how to install that script and then analyze the data that you get from it.

Installing the User Script

A user script is a JavaScript that is ran by the browser on the user's machine. Rather than a Canvas-supplied script, it's one that the user installs and runs on their own. The installation is per user and per machine, so you will need to install it on each machine that you want to use the script with.

Custom URLs

The script automatically runs on any page that matches https://*.instructure.com/courses/*/users. This is the main People page if your site is hosted by Instructure without a custom instance. If you have a custom URL, like canvas.university.edu, then you will need to modify the script to get it to work.

The specific steps to do this vary depending on your browser add-on. In Greasemonkey, click on the Greasemonkey pull-down, choose Manage User Scripts, find the Access Report Data, right click and choose Edit. In Tampermonkey, click on the Tampermonkey Icon, choose Dashboard, and then click on Access Report Data.

In either case, you need to change // @include statement on line 5 to match your instance. In the case of canvas.university.edu, you should change it to https://canvas.university.edu/courses/*/users. The * is a wildcard that will match any course.

Customization

There are two configuration variables that can be set within the source code to alter the functionality of the script.

• showViewStudent will include the participations where a student clicked on a student's name from the People (roster) page and viewed their profile when it is set to true. The default is false, which will remove this information from report before the CSV file is downloaded. This information was included prior to February 3, 2017, but faculty were getting confused when they did a pivot table and the names of their students showed up as titles along with assignments and content pages. Now users will have to explicitly set it to true to get that information.
• quizParticipation is a flag to make the data in this report match the data displayed in the Canvas Access Report. Canvas counts quiz participations as views, but then subtracts them off when displayed. When this is true, it makes this report match Canvas. If false, then it displays what was in the JSON data, which was the previous behavior.
• enrollmentStates is an array that says what kind of enrollments should be included. The default is 'active' and 'completed', but you may also use 'invited', 'rejected', and 'inactive' (inactive probably won't help much). See the List users in a course endpoint for additional information.
• analytics is an array that provides an easy way to include the student analytics (last activity and total activity) or grade (current grade and current score).
• disableMissing is a Boolean value that will remove empty columns from the output when true.
• headingSpaces will allow you to remove the space in the headings and replace it with something else (empty string or underscore are common). This can cause "User ID" to become "UserID" or "User_ID". Some users reported problems with programs, like R, when there were spaces in the variable names (headings). The default is ' ' (a space), which is to leave the spaces.
• multipleSections determines how students enrolled in multiple sections are handled. Setting the value to 1 will duplicate the information for each section, so a student enrolled in three sections will have three times the data. This is important if you are using the section in your pivot table, but will throw off the counts if you are not. Setting the values to 2 will take the first section that it finds and use it and ignore all of the other sections so that each item is included only once with no duplication. Setting it to a non-numeric string will use that string as a delimiter but not duplicate any items. The default is ', ' (comma followed by space) so that multiple sections will show up as a comma delimited list.
• maxConcurrent is the maximum number of concurrent calls to make to Canvas. This is part of the throttling system. The higher this is, the faster the calls can be made, but the more likely you will abuse Canvas in a way that causes them to block your requests.
• minTime is a delay in milliseconds injected between each call. If you make a lot of calls that the same time, Canvas imposes a penalty that can quickly exhaust the available limit before the maxConcurrent is reached. It is likely that this value will have more impact that maxConcurrent will.
• debug is a Boolean flag to output debugging information. Included in the debugging output is the minimum x-rate-limit-remaining value and the maximum x-request-cost value. Those can be used to help tweak the maxConcurrent and minTime settings.
• csvFields contains the headings, source field, any special formatting required, and allows you to disable columns that you do not want. The three sources are u (user data), s (section data), and a (access report data). This is under the advanced configuration section, which typically means don't mess with it if you don't know what you're doing, but any user should be able to change the name property.

Export the Data

In the June 4, 2016, production release, Canvas consolidated the buttons that previously appeared on the People page under the administrative cog.

The script was updated on June 6, 2016, to reflect that change and move the Access Report Data into that same cog. Note that the demonstration videos have not been updated to reflect the change.

When you expand the menu by clicking on the cog, then you can choose the Access Report Data item.

Click on this and wait.

There is now a progress bar that appears once the list of students has been downloaded and it is fetching the access reports.

It has to make an API call for the list of students and then generate the Access Report for each student in the course. This only took 8 seconds for my class with 46 students and up to 112 content items, but we're only one month into the semester and it will probably slow down later. It took 12 seconds on a course with 81 active students and 126 content items. It took 15 seconds for a class with 35 students but 470 content items.

Limitations

Early on, really large classes were having timeout issues in Chrome. Switching to Firefox helped. Now I wonder if the problems were related to the requests failing and so I hope the changes I made with version 14 have improved this so that it is no longer an issue. I've made some improvements to the code with version 14 and let Chrome run for 4.5 minutes in a class with 29,615 items without issues

If you have a really large class, you may need to tweak the settings. The call to get the enrollments is relatively expensive compared to the call to get the usage (access report data). In a class of 300 students, it was fine, but in a class of over 10,000 students, it was timing out. It is supposed to retry failed attempts, but there was some data loss that I haven't tracked down yet. It's best to avoid exhausting the x-rate-limit-remaining header. Increase the minTime or decrease the maxConcurrent values. Normally, the minTime is the more important one, but if your requests are taking a long time to process (like the enrollments were), then you may decrease the number of concurrent requests. Reducing maxConcurrent from 40 to 20 avoided the lost data issue in the 10k+ course and only added 4 seconds to the download time for the class with 300 students.

The course with 81 students actually had 233 students in it, but it was a resource course for faculty and only 81 had bothered to go into the course to do anything. That brings up an important note about the Access Report.

The Access Report only provides information on students who are doing something in the course.

Access Report data does not return information about who has not done something, but Excel has an option that will allow you to see this.

Another thing to note is that additional information is available through the Page View data, like the exact times when students did something and which browser they were using at the time. However, faculty don't generally have access to this information and you have to load the information for each student and sift through it to see which applies to the particular course. This takes way more than 8 seconds for little gain.

The Page View information is available through Canvas Data. However the data there is not current, running about a day or two behind real-time. It also has limited availability and requires additional resources to analyze. Page view data still occurs when a student does something. If they're not doing anything, there is nothing to record.

Finally, links to URLs that are contained within assignments or pages are not included here. The External URLs are just those that are linked to items in modules, not a link in a page full of instructions.

Raw Data

Okay, you've installed the script, clicked the button, and opened the file in Excel. Now what?

Well, you get a bunch of raw data that will need manipulated before you can tell anything useful. It looks pretty intimidating, you have a report that looks like this (and this is just the first 10 rows).

Note that the SIS Login and SIS User ID may not be there, depending on the permissions you have within Canvas.

Where did all this come from?

It turns out that Canvas actually provides more information than it shows on the page you get when you ask for the Access Report. Canvas displays the HTML version with just the highlights and in a form that is easy for humans to read. It also provides the data in JavaScript Object Notation (JSON) format that is much easier for a computer to deal with. Canvas internally calls these items assets, and that gets reflected in the naming of the item.

Here's what that first entry looks like in JSON.

{
    "id":123456789,
    "asset_code":"discussion_topic_11319205",
    "asset_group_code":"topics",
    "user_id":1278402,
    "context_id":1785810,
    "context_type":"Course",
    "last_access":"2016-02-12T21:31:11Z",
    "created_at":"2016-02-10T20:12:45Z",
    "updated_at":"2016-02-12T21:31:11Z",
    "asset_category":"topics",
    "view_score":2,
    "participate_score":null,
    "action_level":"view",
    "summarized_at":null,
    "display_name":"Discussion 4: Hypotheses and Errors",
    "membership_type":"StudentEnrollment",
    "readable_name":"Discussion 4: Hypotheses and Errors",
    "asset_class_name":"discussion_topic"
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

What's being displayed in the HTML version of the report is contained are the asset_class_name, readable_name, view_score, participate_score, and last_access. The User's name is displayed at the top of the report, but the user_id (1278402) can be used to identify the student (in this case Cubic Dream) and the user script merges the user information and the access data together into a usable format for the user. By the way, all of the user data have been anonymized, but the assignments and access data are from my actual class. Cubic Dream isn't my student's real name and 1278402 isn't Cubic's real Canvas User ID.

Data Dictionary

In statistics, I emphasize the importance of having a good data dictionary. You need to know what the values represent.

• User ID is the Canvas User ID. Everyone has one. Most people won't find it extremely interesting, but in the case of multiple students with the same name, it can help tell them apart.
• Display Name is name of the student that is displayed in Canvas. It's normally in First Last format, like Cubic Dream or Skinny Record.
• Sortable Name may not be available to you. There is a feature that your Customer Success Manager can enable that will allow you to list names in Last, First format like Dream, Cubic or Record, Skinny. If it's enabled, this provides you with a quick way to sort alphabetically by last name.
• Category corresponds to the asset_category_type of the Access Report data. It includes things like announcements, assignments, collaborations, conferences, external_urls, files, grades, home, modules, quizzes, roster, topics, and wiki.
• Class corresponds to the asset_class_name of the Access Report data. It includes things like announcement, assignment, attachment, content_tag, discussion_topic, google_docs_collaboration, quizzes/quiz, student_enrollment, teacher_enrollment, and wiki_page. Generally, there is a one-to-one correspondence between category and class, but some things like the roster category are broken into student_enrollment and teacher_enrollments for the class name.
• Title is the name of the content. In the Access Report data, it's called the readable_name There was a display name as well, but it was sometimes blank and when it was there, it mostly matched the readable name.
• Views is the number of times the student viewed the content. For discussions, this would be the number of times they went in and viewed the discussion. This is the closest most people will get to the number of messages they read, but it's the number of times they went in, there is no guarantee they actually did anything past opening the page.
• Participations is the number of times the student participated. For discussions, this would be the number of posts they made.
• Last Access is the last time the student viewed or participated. It is converted to the local time according to the browser.
• First Access is the first time the student viewed or participated. It is converted to the local time according to the browser.
• Action is either view or participate. I'm not exactly sure on this, but I think it corresponds to the last access.
• Code is a unique identifier for each piece of content. It contains the type of content plus the Canvas ID. For example, assignment_8556874 identifies this as assignment with a Canvas ID of 8556874. That is not very useful for faculty, but if someone was trying to link all the data together for advanced reports, it may be.
• Group Code is another way of categorizing the data that most people will ignore. The group doesn't refer to Groups in the normal sense, but is some way to organize the data. For example, there is an assignment_group_### group code field that appears to represent which assignment group the item belongs to. That means you could, with additional information, break it down by whether the assignment was homework, exams, projects, etc., depending on your assignment groups. On the other hand, every single code beginning with wiki_page_ belonged to the same wiki_ group code (at least for my course).
• Context Type should be Course or Group and relates to the Context ID to determine exactly which course or group.
• Context ID is the Canvas Course or Group ID for the course. It is used in conjunction with the Context Type.
• Login ID is what the user uses to log into Canvas with. For us, it's their NetID, but it could be an email address.
• Section Name is the name of the section the student is enrolled in. This may be a delimited list if the student is in more than one section. See the multipleSections configuration variable for more information.
• Section ID is the Canvas ID for the section the student is enrolled in. This may be a delimited list if the student is in more than one section. See the multipleSections configuration variable for more information.
• SIS Course ID is the Course ID supplied by your Student Information System (SIS). This column may not be there if the person requesting the Access Report doesn't have access to the SIS information from Canvas. It's a permissions issue and the script runs as the person calling it.
• SIS Section ID is the Section ID supplied by your Student Information System (SIS). This column may not be there if the person requesting the Access Report doesn't have access to the SIS information from Canvas. It's a permissions issue and the script runs as the person calling it.
• SIS User ID is what your SIS knows the person by. For us, it's an integer that uniquely identifies the user. This column may not be there if the instructor doesn't have access to the SIS information inside Canvas.
• Last Activity is the last time the student accessed the course as explained in How do I use the People page in a course as an instructor?  It is converted to the local time according to the browser.
• Total Activity is the total time in decimal hours as explained in the Canvas Instructor Guide. Instead of giving hours:minutes:seconds, 3:12:15 becomes 3.21 hours.
• Current Score is the current numeric score for the student in the course.
• Current Grade is the current letter grade for the student in the course. This is only available if a grading scheme has been set.
• Final Score is the score the student would get if all ungraded assignments were given a 0. This is disabled by default.
• Final Grade is the letter grade the student would get if all ungraded assignments were given a 0. This is disabled by default.

Note that section information was added after the original script was released and are not included in the videos.

Drilling Down to Specifics

Some questions can be answered using just the raw data.

Format as a Table

If you're going to examine the raw data, you will want to turn it into a table first. To do this, go to Home > Format as Table or Insert > Table. This allows you to filter or sort on a column, which will greatly increase your productivity later.

Who participated in a discussion?

Let's say we wanted to know who participated (and how many times) in Discussion 3.

I purposely started this one off in a novice way, to show what people who may not be familiar with Excel can do. There are more efficient ways of doing this.

• You can sort by the title by clicking in the Title column and going to Data > Sort > A-Z. An easier way is to click on the down arrow on the Title heading at the top of the table and choose Sort A to Z.
• Doing that may overwhelm you with information so you can use Ctrl-F to find the one you want. But once I did that, it turned out that Discussions are in there twice, once as a discussion and once as an assignment. This is a problem when you have multiple contents with the same name, so sorting by title may not be the best solution.
• You can filter the Category (choose topics) or Class (choose discussion_topics). To do this, click on the appropriate heading at the top and un-check the ones you don't want.

Who viewed an external URL?

You'll want to sort by title to group the content together. You could use the Code to group, but it wouldn't be alphabetical.

I then used filters to select the external URLs. Those can be found under Category (external_urls) or Class (content_tag) and found the one I was looking for.

Show Me!

Here is a video walk-through of the three items mentioned here.

Summary Reports

Sometimes you want a broader picture than just what a student did on a single assignment. To accomplish these, you'll need to create a Pivot Table.

Create a Pivot Table

To create a pivot table, you need to go to Insert > Pivot Table and click OK.

Once you're there, you will probably want to drag the Display Name or Sortable Name down to the Rows so you can break things down by students. Every one of the items in this section starts off the same way.

Pivot tables allow you to insert slicers to quickly filter the data or time slicers to view data over a particular time period (you must have a date/time field to do this, but we have first access and last access to pick from)

Analyzing Discussions

This whole Access Report project grew out of a desire to know how many times students had gone into the discussions and at least viewed them.

What I'd like to know is how many times did a student view or participate in a discussion this semester.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows
3. Drag the Participations to the Values. Excel wants to do "Count of Participations" instead of "Sum of Participations". You can change this in several places, but double clicking on the heading of the table is probably the quickest. Change it from Count to Sum and then change "Sum of Participations" to something else. You might want to use "Participations", but that's already used, so you have to pick something else.
4. That gives the participations for the entire course. You could add a filter on the right side, but a faster way is to choose Insert Slicer from the top. This gives a nicer interactive menu where you can immediately click and limit your data.
5. After looking at Participations, you can do the same thing with Views. Drag it to the Values box, where, thankfully, it comes through as a sum. I would change the title from "Sum of Views" to something else, like "View"
6. You can sort the data by clicking on the Row Labels pull down. The default A to Z and Z to A are for the data in that column, but you can choose More Sort Options and tell it to sort by another column
7. To break the report down by the discussion, you can add another slicer for the Title and then look at participations and views one discussion at a time.
8. If you decide you want to look at all of the discussions, then drag Title to the Columns selector on the right side. You'll need to turn off the filter on the Title if you do this.

Here is a video that shows all of that in action.

Filtering by Time Period

The Course Roster (People) page shows you when the last activity of a student was and the total amount of time spent in the course. It does this in alphabetical order. Unfortunately, last activity could be just logging into the course, it doesn't mean they did anything else once they got there.

As a side note, if you have installed my Sort a Roster Canvancement, then you can click at the top of any column to sort by that column. You can find sort by section, by the time they last accessed the course, or even by the total amount of time spent in the course.

You can filter the information in the Access Report Data spreadsheet by time.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows, and both Views and Participations to the Values. As before, change the heading "Sum of Views" to just "View" and the "Count of Participations" to Sum instead of Count and then change the title to "Participation".
3. Click on Insert Timeline and choose Last Access.
4. Change the slider from Months to Days and then highlight the date range to restrict the report to.
5. You could add filters (through the Slicers) to limit what kind of activity you want to look at.

Remember that only students who have data for that time period will show up. There is no easy way to get a list of students who aren't doing a particular thing.

Here is a video that walks you through the report.

Quick Tables

A quick table is a table that shows you information about a particular area. What we're going to do here is break down the information by the Class. You could just as easily choose the Category and some might find it more useful.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows, the Title to the Columns, and Class (or Category) to the Filters.
3. Draw Views and Participations to the Values. Rename Views to be V (yes, just a single letter). Change "Count of Participations" to Sum and then rename it to just be the letter P.
4. Rotate the titles in Row 4 so that they are vertical. Do not just click on Row 4, it doesn't work; you need to select the cells and then do the rotation. To rotate the text, click on the Orientation icon from the Home screen and choose Rotate Text Up. You may also want to Right align all the text from columns B on, although it may not really matter if you use a single letter for Views and Participations.
5. Be prepared to be wowed!
6. Click inside the pivot table, then at the top click on PivotTable Tools > Analyze. On the left, choose the pulldown menu next to Pivot Table Options (don't click on the word Options). Then click Show Report Filter Pages (this won't be available if you forgot to put something in the Filters box on the right). Then choose the filter(s) to use and click OK.
7. What you get is a page for each type of content. The name of the student is on the side and the name of the content is across the top. This allows you to quickly (hence the name Quick Table) look at discussions or quizzes or external URLs or anything else in the class that is available.

Here is a video that shows all of this in action.

Viewing Who Has Not Participated

As mentioned above, what we are looking at is the Access Report data and it doesn't include information about students who are not engaging in your course. Luckily, there is one checkbox in Excel that we can check to get that information. The student will have needed to do something, anything, so that their name is in the Access Report data, but then we can see what they have not done.

1. Choose Insert > Pivot Table
2. Drag the student's name to the Rows
3. Drag the Participations to the Values. Excel wants to do "Count of Participations" instead of "Sum of Participations". You can change this in several places, but double clicking on the heading of the table is probably the quickest. Change it from Count to Sum and then change "Sum of Participations" to something else. You might want to use "Participations", but that's already used, so you have to pick something else.
4. Choose Insert Slicer from the top and add slicers for Category (or class) and Title. Choose the content you want to view. What you currently have is a list of those who have participated.
5. To get the list of those who have not participated, click on the student's name from the Rows field and choose Field Settings. Then click on Layout & Print. Check the "Show items with no data" and click OK.
6. If you like, you can go to Data > Sort > A to Z to bring those who have not participated to the top.

Here's a video showing how it works.

Updating the Script

Tampermonkey should attempt to update this script automatically for you. In case you have turned off that functionality or would like to update sooner, you can click on the Tampermonkey icon and choose Check for userscritp updates.

The update process keeps your local settings and shows the changes to the code for you to decide whether to upgrade.

Canvancements

This script is a Canvancement -- a Canvas Enhancement. The links in the document point to an installable version of the code, but there is an Access Report Data project page as well that contains the source code as well as a version that you can use to anonymize the names like I did for the videos. Other projects, like the Roster Sorter that was mentioned here can be found on the Canvancement website as well.

[cjford]

Fair enough :smileysilly:

Thanks for the info; I will do some snooping in the network tab.

[d_ellis]

An update: It turns out that the script was not busting the rate limit and being throttled by Canvas. I monitored the X-Rate-Limit-Remaining header (as documented in Throttling - Canvas LMS REST API Documentation) and it never dropped below 600. Given the rate that this replenishes, you could probably only deplete the limit with a large number of parallel calls.

Instead, it looks like the Google Chrome browser timeout was the culprit. Switching to Firefox, which has a more forgiving timeout limit, resolved this issue.

[James]

Great research  @d_ellis ​. I do all of my development in Firefox, which might explain why mine would take 15 minutes but still work.

[d_ellis]

It looks like the latest production release has moved the buttons on the People page to a drop-down menu: https://community.canvaslms.com/docs/DOC-6857#jive_content_id_People​

This prevents the "Access Report Data" button from displaying. I'm sure  @James  will be along shortly to find a new home for the button, but in the meantime you can add the button to the Home page instead with these code changes:

// @include     https://*.instructure.com/courses/*/users

becomes

// @include     https://*.instructure.com/courses/*

And this

$('#right-side-wrapper div').append('<a id="jj_access_report" class="btn button-sidebar-wide"><i class="icon-analytics"></i> Access Report Data</a>');

becomes

$('#course_show_secondary .course-options').append('<a id="jj_access_report" class="btn button-sidebar-wide"><i class="icon-analytics"></i> Access Report Data</a>');

[James]

Thanks for pointing that out  @d_ellis ​. I don't use most of the scripts I write and I definitely haven't gone in and verified that they continue to work after Canvas changes stuff, so I would have missed this until someone said something.

I'll see if I can find a new home for it.

Unless someone has a better idea, I think the most logical place would be under the pull-down cog.

Your @include change makes it run on almost every page within a course, not just the course homepage

[James]

I've updated the script and the documentation at the top. It's now under the cog with the rest of the items.

This also marks the release of a progress bar that I had in development that has been released to everyone.

It also uses the FileSaver.js script to assist in the file creation and making sure the application-types are correct. I couldn't get it to work right when I required it as part of the script, so it fetches it as you use it, but it is cached so the delay isn't too bad.

[lindalee]

Hi  @James ​​,

Thank you so much for making this tool and your thorough documentation available! This is just brilliant. It's super-simple to install and use, and it saved me an enormous amount of time this morning!

Thanks!

[nealc62]

Jim, You're a magician! This is so helpful to instructors and instructional designers!

[James]

 @nealc62 ​,

I'm glad you found it useful.

However, my name is James -- Jim Jones died on November 18, 1978.

[nealc62]

Hi James,

Thanks for the clarification. Realized my error as soon as I hit Add Comment! Force of habit; my father and brother were both James but called Jim.

Cheers,

Jan

[alyson1]

Love the access report - exactly what I need.  Is there a way to set dates for a report - for example 10/1/2016 -10/31/2016??

[James]

alyson​,

You can use Excel to filter that information after it is downloaded. There is a video and instructions in the original post about how to select a time period. There is no way to set it ahead of time.

[alyson1]

Got it  - thank you.  I will go back to the video.   Appreciate the support

[ProfessorBeyrer]

Yeah, that's awesome. I was in a meeting with another professional developer and this question came up. During the meeting I searched the Canvas Community and found your page. So good on so many levels -- easy access to this data, example of Instructure's openness, and an inspiration to share the possibilities. Thank you,  @James ​!

[alyson1]

James

I installed this script ~ 2 wks ago and it worked perfectly and got the data I needed.

I went to get the data reports today -- and I am lost!

I made sure I changed this line: /^https://.MYORGANIZATION\.instructure\.com/accounts/[0-9]+$/

On the courses page in the admin panel - tampermonkey shows a 1 when I am on the course page.  But when i go to the subaccounts - nothing and I cant figure out how to access the instructor report.  I am on a mac in chrome.

I have no idea what happened from 2 weeks ago to now.  thanks!

[James]

alyson​,

I'm not sure what you mean by you changed that line -- that line doesn't appear anywhere in the code for the Access Report.

Also, it has nothing to do with subaccounts, it's from the People page.

Are you talking about a different script?

[alyson1]

Hi James

Not the access report  - those work amazing!!

I am taking about the instructor access report - line 5 - I added my org in place of the *.  I was using this script in the admin per instructions posted: It adds a button to the Managed Accounts > account name > Sub Accounts page called "Instructor Access Report Data". I put it there so that you could limit it by subaccount if you wanted to.

I am not getting the button to appear

I appreciate your support.

[James]

Ahhh -- now I got you. That was more of a proof of concept for the person who asked about it. It took a really long time to run and people can probably get more useful information and a lot faster out of Canvas Data or from programs like Dropout Detective. I never updated it when Canvas changed their layout, so yes, you're right, it won't work as is. One user also found that Chrome imposes a time limit on scripts, so it worked on Firefox, but not Chrome. I basically consider it orphaned code. It was not updated to use FileSaver.js or provide a progress bar like the main one was.

The good news is that getting buttons to appear and with the new formatting is probably just a matter of changing just a few lines around line 17.

[alyson1]

Hi James

Thank you for the additional clarification.  I will check to see if someone in IT can tweak the code.  It worked so well a couple of weeks ago - and was perfect for our needs.  I will also check out Dropout Detective.   Thank you again for you support.

[James]

If your IT figures it out, shoot me a message with the changes and I'll see if I can incorporate it. I guess I should have said Instructor Insight instead of Dropout Detective. Dropout Detective is for students.

[Chris_Munzo]

Hi, alyson​ -- We work with James & Kona Jones (and many others) with our Dropout Detective and Instructor Insight analytics.  Very strong in the world of online education.  Feel free to reach out to us via our partner page Alliance Partner - AspirEDU​ if you are interested in learning more.

[James]

alyson​,

I found an hour tonight so I went ahead and updated the code that adds the button. Again, this was never really meant to be put into production, but if it was working for you, then this may once again.

It turns out that Canvas changed the way that they named the classes, so it wasn't able to find the proper place to insert it. I modified that, but there are now multiple divs with a class of rs-margin-bottom, so I had to do some work to make sure I got it into the proper place. There's probably a fancier way to do it, but it seems to work.

If you had to change the include line to match your Canvas instance, then you may need to do that again.

If you customized it in other places, you may find it easier to replace the function that adds the button and keep what you have. One note about that, I did rename the button id, so there is one place later on that needs changed as well. Viewing the history on GitHub will show you what those were.

I didn't work on any other improvements -- just getting the button to reappear.

[alyson1]

James -

WOW -- thank you so much for doing this.  It worked perfectly.    You are so kind to do this and i so appreciate it.   I can't thank you enough.

I will also follow up with AspirEDU for a longer term solution

With much gratitude!

[mpilati]

It's crazy that anyone would have to go through all of this to pull such basic and critical data. I should be able to to go in and sort students by date of last access and then be able to easily message students who have not accessed the course for a specific time period. Has anyone suggested that functionality? If not, we need to make it happen.

I LOVE the ease of contacting students via the Gradebook who have not completed a certain assignment, but assignment completion is only one course-involvement metric. We all have experienced the student who logs in as early as they can and completes all the initial work - and then disappears.

[ross2]

Our student overview gives you all this data of all of your students for your courses on the emotuit dash: 

 

[kona]

Where are the actual individual pages that were accessed? The main purpose of this work-around is that you can see every student that accessed a specific page/item. Canvas already provides the information above in various places and in one place on their new context cards.

[Stef_retired]

For anyone interested in this topic, be sure to RSVP to the https://community.canvaslms.com/community/ideas/canvaslive?sr=search&searchId=1ba40099-8471-4c0b-862...  eventObtaining Access Report Data for an Entire Class , which is coming up this Friday, February 3, 2017. RSVP “yes” if you will be there--and if you’re interested, but your schedule doesn’t allow you to attend in real time, RSVP "no" or "maybe" to receive all event updates. Your RSVP ensures that you will receive a notification should the event be cancelled or changed.

[Shar]

First, a huge thank-you and now I am a Pivot Table convert. Sweet! The event Obtaining Access Report Data for an Entire Class‌ opened my eyes to Greasemonkey and the scripts! Wonderful job showing just how easy it is to install (the browser add-on), run (the script), manipulate (the excel data)!:smileygrin:

Now, I'm trying to see how often students are seeing the Front Page-- we have our course home set to be a specific front page. In the example below the front page is called ECFS 303 Course Home (in the Android App the name of the page shows when the course opens). I'm comparing the "Title" field on Course Home and ECFS 303 Course Home to display the sum of views -- these should be the same thing, right?? What is Course Home -- when students press Home from the Course Navigation area or when students enter the course from the Dashboard?

Image 1: Pivot table of Title with sum of views

Why is there a different number between Course Home and that named front page? How can some students have a view number on that page name-- is that them entering the course through mobile and specifically pulling up that page?? Wouldn't that be a treat as a backhand way of who views through mobile!

We are in the 6th week of course now, does that mean the 5th student in the above image has only been in the course 14 times?? to be fair, I think that student did drop the course, heh.

Thanks in advance any help you can provide  @kona ‌ or  @James ‌ in deciphering the titles of things.

Here's another one I'm not sure of: is wiki the category for pages that show up on View All Pages so like Lesson 01 Overview, Lesson 01 Materials, et cetera?

Image 2: Pivot table of Category with sum of views

Cheers - Shar

[James]

ishar-uw‌,

Those questions aren't specific to this script, as all that it does is pull the information that Canvas provides for every student and puts it in one place. That mans that any help related to "Access Report" is applicable here, although this does provide some additional information that Canvas does not display.

That also means I'm not an expert on what the values are. Now that I've established that, I would suggest looking at more than just the title.

For instance, I have a title="Course Home" with a category="home", but I also have a title="Front Page" with a category="wiki". Based on the counts, I'm going to guess the first is visiting the home page and the second is when I click on that page. I changed my home page from modules to be a content page named "Front Page" and the "Course Home" incremented when I clicked on Home, but the views to "Front Page" did not. If I click on Pages, then category="pages" and title="Course Pages" is incremented, but "Front Page" is not incremented until I go in and actually click on that page from the Pages list.

The beauty is that it's live data, so click something, reload the report, and see what changed.

There may be issues with mobile apps showing up, if it doesn't normally show up in the access report for a single student, then it won't show up here, either.

As for the dropped student, no, it doesn't mean that. Every time they click Home counts as a view. So, they could be in once, click Home 5 times, and get 5 views. Do not try to relate that to number of logins.

Wiki is the category for Wiki Pages, which is now more fashionably being called content pages.

[Shar]

Thank-you James. I'll play around a bit as I try to discover the distinction between pages and wiki category as well.:smileycool:

Cheers - Shar

[Shar]

*happy dance* When a mobile (iOS or Android app) user views the "Course Home" either by entering the course from the dashboard, or navigating back to Home from within the app, it counts as page view if a Front Page is set to the course home.

Image: Access report after viewing "home" on mobile.

I created a brand new page A A Page and only looked at it from my phone - count 1. Then I set A A Page as the front page and set my home to Front page. Viewed it from browser - Course Home went up, Viewed it from phone - A A Page went up count 2 and Course Home stayed the same. I was so excited I took that screen shot shown above.

I navigated away in mobile to discussions, nothing showed up. Back to home, went up to 3. Oh boy! Had to wait for iOS to update but same thing when I entered the course A A Page count went up. Oh but now I updated my iOS Canvas app and the A A Page count only goes up when I navigate specifically to that page. aww man

❤️ Love this script for the access report for the entire course at once!! As an instructional designer, I keep telling these folks that stuff needs to be mobile friendly and I think they humor me, now I've got proof that I can fetch myself w/o making a data requisition. As a designer, I like knowing what features are useful and seeing (at least) the views by students. Wonderful good stuff!

Meanwhile I suspect the difference between page and wiki category is page refers to group space pages, and wiki refers to course content pages. I unsuccesfully hunted around in the community for something that would explain the categories to me as nicely as the Data Dictionary on this page.

Cheers - Shar

[James]

Something else to consider is how the information is loaded. I suspect that mobile apps load their stuff through the API, wiki (content) pages are getting counted when you view the course home that has a content page set. On the other hand, the web interface dishes the content out as part of the page that's delivered, so it doesn't register in the same way.

That's conjecture on my part. A more definitive answer is found in the source code under the /app/controllers folder and sub-folders. The function that actually logs the entries is called log_asset_access and is defined in application_controller.rb. You can search for log_asset_access in the source on GitHub. As I write this, there are 23 hits.

That may or may not be useful depending on how easily reading code comes. But for the wiki_pages_controller.rb file, the show page API call has a call to log a view level access. Show the front page just calls the show page function, so it would log a view access as well. This is when it is done through the API, which is what I think the mobile apps do.  It's also done if you make an API call to fetch the content of the page for some program you're running.

It would be interesting to have a definitive list of codes, though. And then if those codes could be tied into Canvas Data so that people could search there to get the same information, it would be really great.

[mcclarens]

This is amazing -- Thank you for sharing! So happy I found this. It was very easy to install and run. This will definitely reduce the time associated with downloading all the individual activity logs and combining them to apply search filters.

The TITLE column is much more useful and user friendly over the URL column from the original activity log. I noticed the REMOTE_IP and USER_AGENT data was not included in this report. With my very basic programming abilities, what would be the best (ok easiest) way to edit the script to have it also pull this information for me? Any suggestions would be greatly appreciated.

Best,

Stephanie

[James]

Remote IP and User Agent are not supplied with the Access Report data. This script pulls the same information that Canvas provides, it just gathers it for every student in the course rather than forcing people to look at it one at a time.

Access report data is a summary or overview and often includes more than one view or participation per student and asset. That means that you might have multiple Remote IP and multiple User Agent strings involved because the student could have used different devices or browsers (or apps) to access the information.

To get that information, you're going to need to look at Page Views through the API or the requests table of Canvas Data. Both of those require elevated permissions that teachers are unlikely to have. They're also both involve a lot of data that will require filtering after downloading. Besides not fitting in with the report, it's best handled in an offline situation rather than within the browser while the user waits for something to happen.

[mcclarens]

Hi James, Thank you for the response. I am an admin within my college with elevated access and I’ve been using the “User Account Details” report to download the csv file located under Page Views. While this method has been tedious, this script should help me pin-point which student csv files I need instead of all having to download all 120 of them after every review session. This is progress, again thank you for sharing the script!

Best,

Stephanie

[d_ellis]

Hey  @James 

I remember that this code uses (used?) RequireJS for the file saver - will this be impacted Canvas' change to Webpack?

[James]

I updated the code 27 days ago so that it doesn't need a require(). Be sure you grab the latest version.

[kroeninm]

 @James ‌, I'm skimming through a lot of resources today so might have missed it but just to confirm, if a course is concluded then the People->Access Report and your script won't work right?  It should be on an active course? (note - I don't think it needs to be available in concluded courses, just doing some documentation for a short presentation).

- Melanie

[James]

 @kroeninm .

I've not heard this before, but we never conclude our courses manually so wasn't sure. I can say that if the Canvas call made by People > Access Report doesn't work, then my script won't either. That's not to say that if you can't get to the call through the UI that it doesn't work -- read on.

I went into my Fall 2013 stats (on the beta instance) and the script worked there. There was a full set of students still in it, but it was one of those automatically concluded courses by the end of term soft concluding and didn't even offer me a choice to conclude like shown in How do I conclude a course at the end of a term? 

While still in beta, I went in to a current course that I hopefully won't need to test anything on this weekend and manually concluded it.  Now I see this:

But, if I go to the settings cog and choose "Access Report Data" (this script), it still returns a list of the data to me.

I don't know if that's a bug that will be fixed in a later version or if it's because I'm an admin as well as a teacher or what, but it appears that the script still works even if the course has been manually concluded (at least in beta).

Unconcluding the course did not restore the users, but the script still works after the course has been concluded.

I would say this myth has been busted.

[fosterl]

Hi James, what a great tool! I have used it and recommended it here at Oregon State. But today, it's not working for me. On Chrome, I get the "Fetching Access Reports" screen, but never any progress on the progress bar. So I thought I'd just go install it on Firefox with greasemonkey. That's all installed now, but I get the same exact behaviour. I definitely have permission to run an Access Report in the course (tried that, and I'm the Admin anyway). Is there something in the latest Canvas release (Saturday) that has broken this?

--lindy

[fosterl]

Never mind, I figured out the problem! 😉

[James]

I just tested it in my summer course and it worked fine, but I have a small course. I went into a course with 99 students and it worked fine, too. It paginates with 100 per fetch, so I dropped that number in my version down to 50 and it still worked.I then dropped both API calls down to only 5 per page to try and force it to mess up. At that point, it acts like it downloaded all of the data, but it never came back. It's possible that one of the calls failed. It took a lot longer and the progress bar went to 100% and I thought it had crashed, but after a while, it finally came up with the file.  In other words, I'm not able to duplicate what you're seeing right now with my courses.

It needs to download the entire list of students before it will start moving the progress bar. Do you have a really large course? If so, they might have started enforcing request throttling limits, or there could be another issue going on. Try checking your developer tools console in the browser (press F12) and see if anything is getting reported.

[fosterl]

What it was was a training site, where all of the participants are teachers. And the leader of the site just wanted to see what all the other TEACHERS had accessed during a specific timeframe. So I thought of your tool, but it apparently just does students. Operator error on my part! Thanks for checking!

[James]

I always love it when my scripts aren't broken -- it's easier to fix that way.

There was a proof of concept script I wrote to check what instructors were doing in their courses, but that's not the same thing as what you're wanting.

To get what you want, you would need to modify the way it gets the list of students. Currently, I'm using the get a list of sections and including the students on line 71. That particular call won't return a list of teachers, but it was a change from the first version of the script I had in response to people asking for the section of the student to be included. You could get a list of course users and and modify how that information is processed in the getStudents() function and get a list of teachers. Or you could make those teachers students in the training course (that's what we do), then you should be able to pull up the access report for them from within Canvas, but you would have to do that one at a time if you're not into modifying the script and losing some of the functionality.

[scottdennis]

Just a reminder that the shift from requirejs to webpack goes live with this coming release.

[a1222679]

I had a quick query about the Access report.  I've noticed a difference between an individual Access Report on Canvas and the access shown for the same user in the Course Access Report from Tampermonkey.  All of the rows without participations tally perfectly, however those with participations look like this...

COURSE ACCESS REPORT			INDIVIDUAL ACCESS REPORT
Content	Times Viewed	Times Participated	Title	Views	Participations
Week 9 Online practice questions	2	1.0	Week 9 Online practice questions	3	1
lecture test 3 mc	9	8.0	lecture test 3 mc	17	8
Lecture Test 2 Multiple Choice	7	3.0	Lecture Test 2 Multiple Choice	10	3
Week 8 Practice Quiz	3	1.0	Week 8 Practice Quiz	4	1
Week 7 Practice Quiz	4	2.0	Week 7 Practice Quiz	6	2
Week 6 Practice Quiz	4	2.0	Week 6 Practice Quiz	6	2
Week 5 Practice Quiz	3	1.0	Week 5 Practice Quiz	4	1
Week 4 Practice Quiz	2	1.0	Week 4 Practice Quiz	3	1
Field Trip - Safety Information	3	1	Field Trip - Safety Information	3	1
RSVP for Optional Field Trip	3	1	RSVP for Optional Field Trip	3	1
Lecture Test 1 Multiple Choice	9	4	Lecture Test 1 Multiple Choice	9	4
Plate Boundary Practical Test for marks	6	2	Plate Boundary Practical Test for marks	6	2

As you can see, Views in the first 8 rows of the Individual Access Report differ to Times Viewed in the Course Activity Report.  It seems that Views in the Individual Access Report include the participations as well.  When you get to the last 4 rows this stops happening.

Does anyone know what might be happening here?

Thank you in advance for your help.

Daniel.

[James]

 @a1222679 ,

In looking at my data, it looks like Canvas is subtracting the number of times they participated in a quiz away from the number of page views. That kind of looks like what you're seeing, but I'm not sure about "Lecture Test 1 Multiple Choice". Is there something different about that one? Can you look at the group code on the ones that are not adjusted and see if they're different from "quizzes"?

There is a note in the /app/models/asset_user_access.rb file that says:

# For Quizzes, we want the view score not to include the participation score
# so it reflects the number of times a student really just browsed the quiz.
def corrected_view_score
  deductible_points = 0

  if 'quizzes' == self.asset_group_code
    deductible_points = self.participate_score || 0
  end

  self.view_score ||= 0
  self.view_score -= deductible_points
end‍‍‍‍‍‍‍‍‍‍‍‍

If the asset_group_code is "quizzes" then it makes the correction.

That code was placed in there in November 2013, which was before I wrote the script. It's been different this whole time and you're the first to say anything about it, so good catch.

The place that calls it is inside the view that generates the HTML page, but the JSON returned does not apply that correction. What I'm doing is dumping the information without applying the correction.

So the question becomes -- do we report the corrected number or the uncorrected number? Technically, they did view the page when they took the quiz. Either way, I guess we should document the decision that we make.

[a1222679]

Hi James,

I don't know where to look up the asset_group_code for these (can you advise?) but I can see that Lecture Test 1 Multiple Choice (not corrected) and Lecture Test 2 Multiple Choice (corrected) are both assignment quizzes in Canvas.

[James]

It's the column in the spreadsheet called "Group Code" or maybe "Group_Code".

[a1222679]

Got it.  Curiously all 12 rows have the Group Code "quizzes".