🌊 Deep Dive into LTI Usage 🌊

teryn_odom
Instructure
Instructure
6
1513

Impact.png

Hopefully if you’re reading this blog post, you’ve already read the previous It’s an LTI (R)Evolution post. (If not, please start there and come back to this one afterward.) In that post, we discussed what was included as part of the launch of the new LTI Usage feature set, as well as what is rolling out soon and in the coming months. In that post, we didn't have space to fully explain the mechanics of LTI Usage. This post will provide those details and as a disclaimer, intentionally be much more technical in nature. In this blog post, we’ll plunge into the data sources, data processing, onboarding of data, (un)expected data, examples, and recommendations/best practices.

🌊 Unchartered Waters: Data Sources

The data source(s) of LTI Usage is/are somewhat new and are truly unchartered waters. Let me explain… LTI launch (event) data from Canvas is now sent to an internal event streaming service, which we store at Instructure. The events are processed every 4 hours in all regions and environments. This data is matched and combined with separate tool information, including the Global Product ID and the new Unified Tool ID (among some other metadata like Tool Name, Parent Company, etc.), of which there are 870+ as of the day of this post. The Global Product ID represents a product that comes from the LearnPlatform library of 20k+ products. Each product may have one or more LTI Tool, which is represented by the newly created Unified Tool ID. The LTI Tool registration must have a Unified Tool ID in order for launches of that tool to be surfaced inside of the new LTI Usage feature set. We use a powerful query engine to access and surface these data points from the different data sources.

Did you know LTI launches, in and of themselves, are messy, and for a variety of reasons?

  1. Different launch locations: First, there are many different ways to “launch” an LTI inside of Canvas and depending on how it’s launched, some or all data may be sent/provided. In previous efforts, this was a huge barrier, as launches from the Rich Content Editor (RCE) weren’t included.
  2. Different LTI tool installations & configurations: Depending on how an LTI tool is configured when installed, it may have/use different LTI launch URLs. Some LTIs are relatively simple and have one launch URL; others have dozens potentially. Trying to capture all of the possible configurations of LTI tools is a laborious task to ensure that the data is as accurate as possible. Knowing what configurations we are capturing and what configuration(s) you use can help us troubleshoot. To do this, we’ll be sharing mapped configurations very soon inside of the product providing this information! That way, if you aren’t seeing launch data appear for an LTI tool that you know is being launched, you can see which configuration details we’re capturing and then also directly report a new configuration to us to enhance our dataset. It is worth noting that the new Dynamic Registration process (blog post and technical documentation) is the gold standard for installation and automatically associates a Unified Tool ID. Tools not registered using Dynamic Registration are handled as historical installations and subject to our mapping process.
  3. Evolution of LTI tools: LTI tools evolve over time, whether it's their names, their functionality, who owns them, who works on them, etc. This creates quite a task of maintaining up-to-date information on LTI tools. Fortunately, this task is much easier for us as we are able to combine the power of LearnPlatform with the power of Instructure’s Partner Program. 
  4. Redirect Tool: The Redirect Tool makes our lives much more complicated. The Redirect Tool is an LTI tool by itself. It's historically been used as a workaround to place high priority or frequently visited sites or tools in the navigation menu. But, that’s just it… it could just be a website or another Canvas course, or an actual LTI behind an LTI. As such, it’s likely that you may be interested in some, all, or none of the data behind the Redirect Tool and each of you (and your use of Canvas) is different. So finding a way to handle the Redirect Tool in a way that meets everyone’s needs is, well, not easy. Currently, we have started pulling out/surfacing what I call the origin LTI tool for Redirect Tools, but work is still continuing to determine the best approach in handling and surfacing this Redirect Tool data. As such, if you currently use the Redirect Tool, please know that you should not expect to see this data right now, but will be forthcoming in the near future. In the core Impact product, we’ve already solved this problem, so we are taking that knowledge and experience and working to find an optimal solution where you can slice and dice your data how you’d like, Redirect Tool or not. Stay tuned for more details on this soon!
  5. DIY/Homegrown LTI tools: Many of you have created and/or work at institutions which have created their own LTI tools. This is exciting, amazing, and equally a part of the challenge as we look to provide you with the same ability to track usage of all tools! Currently, DIY/homegrown tools are not surfaced in LTI Usage, but it’s on our radar and internal roadmap. This is another area where Impact currently fills the void and we’ll look to reconcile in the coming months. 

🧭 Navigate the Depths: Data Processing

The sheer quantity of LTI launches is staggering, millions and millions of records! It truly takes a village to make sense of, map, ingest, partition, store, query, process, and surface the data. Fun fact: We have four separate engineering teams who collaborate to make LTI Usage a reality! Because LTI Usage is a global feature set, we make sure to store data regionally all over the globe to maintain the strictest standards in data privacy. Speaking of global, LTI Usage now detects the language of the Canvas instance and adjusts accordingly. As mentioned above, processing is approximately every four hours, so data should be slightly older than that when it reaches your screen. 

📋 Charting the Course: Onboarding of Data

As institutions purchase either Impact and/or Intelligent Insights, engineering is notified on at least a daily basis and data is onboarded manually based on specific region. Additionally, there is a backfill process to process data for a given start (and end date). The backfill process aggregates raw LTI launch data into a format optimized for faster querying and filters launches with Unified Tool IDs. Additionally, it collects and associates metadata from the LearnPlatform (e.g., tool names) and Canvas (e.g., course, user, and sub-account information).

Pressure Test: (Un)Expected Data

Based on what we’ve already discussed above, we want to make sure that we are all aligned on what you should be seeing and should not be seeing. If you are a current customer with Impact or Intelligent Insights who has been onboarded, you should start to see data flowing for any/all of these 870+ tools based on the availability start date listed, assuming that you have the same configuration(s) as those have. We are hard at work to extend both the amount of tools represented, as well as the configurations of the tools represented using a mapping process. We will regularly update the list with additional tools as we continue to add them. We’ll also be adding a link soon on the Overview page where you can request for LTI tools to be added. 

On a related note, we’ve heard reports of “Canvas LMS” showing as an LTI tool and some other Instructure LTI Tools showing up incorrectly. This is due to how the LTI tools were previously categorized in the LearnPlatform Library. Please know that we are aware of this issue and are actively working to resolve it. We appreciate your patience, as we work through these growing pains and expect to have a resolution soon. 

Anchor: (Instructure) LTI Tool Example

Let me drop anchor for a moment and bring the highly technical details above into a relatable reality. To provide a tangible example of the complexity of what we’ve discussed and how we are approaching that complexity, let’s look at the LTI Tool of Badgr (Credentials). Badgr used to be a part of a different company (Concentric Sky) before being acquired by Instructure. That has now been updated in our source database. The name of the LTI tool has evolved over the years, just like its LTI tool version (of which it has 1.1 and 1.3) and launch URLs (of which we have three confirmed). We want to enable tracking of historical LTIs still in use (with the goal of trying to get everyone to move to the gold standard of 1.3 and clean up their Canvas instances). Currently, inside of LTI Usage, you would see this tool listed as “Canvas Credentials (formerly Badgr)” showing an aggregation of all associated LTI tools (Badgr, Canvabadges, Credentials) and you can filter on the version. 

༄。° Currents: Recommendations & Next Steps

The winds and currents are strong, but don’t let them crash your boat! While most of what we discussed above is on us, we do have a few strong recommendations and best practices for you to follow to better manage and track the usage and adoption of your edtech ecosystem:

  1. When you install an LTI tool, do so using Dynamic Registration! This is the gold standard for LTI installation as it is much more streamlined, efficient, and will also automatically associate it with a Unified Tool ID in the process! 
  2. Install, upgrade, and use 1.3 tools! 1.3 is the most up-to-date standard and comes with a lot of added benefits, notably including upgraded security. When searching for new or replacement tools, we are excited about the upcoming new Canvas Apps page, which will allow you to filter by version when searching! We also want to make sure that you review the LTI tools already installed in your instance, migrate from the 1.1 to 1.3 version for them (if you haven’t already), and encourage/push your edtech vendors to move to the latest 1.3 specification (if they haven’t already).
  3. Stay up-to-date! To do this, make sure to stay tuned to the Release Notes for new LTI Usage features added and bugs fixed, ongoing Product Blog posts for regular updates, and check the Known Issues page of the Community prior to submitting a support ticket regarding an issue that you have. 
  4. Reach out to Support if your question/issue is still unanswered or unaddressed after reviewing the above and cross-checking our mapped tool configurations inside of the product with the configuration(s) that you have! Some examples of when you should absolutely submit a Support request include (but are not limited to):
    1. Not seeing LTI Usage from the Impact navigation menu (if you are an Impact customer).
    2. Not seeing LTI Usage from the Analytics Hub.
    3. Users seeing LTI Usage who should not see it.
    4. Not being able to search, filter or filter selections not being applied.
    5. Links not working.
  5. Once launched (soon), submit LTI Tool Mapping Requests inside of LTI Usage! This will ensure that the LTI Tools that you need and expect to see are there and correct.
6 Comments
adam_marshall
Community Explorer

How does one get one's tool added to the 870+ registered LTIs which are included in your usage reports?

teryn_odom
Instructure
Instructure
Author

Hi @adam_marshall! Thanks for the question! As mentioned above in #5 of Recommendations & Next Steps, "Once launched (soon), submit LTI Tool Mapping Requests inside of LTI Usage! This will ensure that the LTI Tools that you need and expect to see are there and correct." Let us know if you have additional questions or feedback!

ctjune
Community Explorer

Hi Teryn, 
We have Intelligent Insights up and running on our instance, but I cannot find the LTI Tool Mapping Requests within LTI Usage. Could you please point me to where it is located? Thanks.

teryn_odom
Instructure
Instructure
Author

Hi @ctjune! Thank you for reaching out! As mentioned above, it is not out yet, but will be soon. "Once launched (soon), submit LTI Tool Mapping Requests inside of LTI Usage! This will ensure that the LTI Tools that you need and expect to see are there and correct." We look forward to getting those requests soon! Let us know if you have additional questions or feedback!

Chris_Hofer
Community Coach
Community Coach

@teryn_odom ...

I really wish this LTI data wasn't an extra cost.  There's some really useful data in here that I'd love to get on who is using LTIs in our Canvas environment and where they are being used.

teryn_odom
Instructure
Instructure
Author

Hi @Chris_Hofer! Thanks for reaching out! We are currently analyzing our pricing and packaging approach for our EdTech Effectiveness solutions, including LTI usage reporting.  Stay tuned for more details soon via the Product Blog!