Smart Search Technical Update

[feathj]

First, I want to give a shout-out to all of our beta users. We really appreciate the time you are spending to test and give us feedback. Your input has helped, and will continue to help make this a great feature for Canvas and all of our users.

If you have been testing with us from the beginning, you have probably noticed quite a few changes in the smart search feature:

1. Added scope - We started out with just indexing wiki pages. This allowed us to test the technology and infrastructure front-to-back, and give us a foundation to add more data types. We have since added more data types, and are continuing to expand the scope. We are currently working on indexing text content from course files! We know this is something that has been asked by many of our users, and we are excited to get this rolled out.
2. Keyword highlighting - Although we believe that a context aware search using AI is the right approach to building a robust search feature, people have become accustomed to a more "keyword" style search. To give users the best experience, we decided to perform keyword highlighting in the search results. This gives a "best of both worlds" experience, and is in line with expectations set by other industry search features. Currently, the highlighting can be a bit noisy with "stop words" such as "and, is, but, or" etc. We have added functionality to filter those words out of highlighting results. That change is rolling out soon.
3. Better UI / UX - We set the expectation that as a "closed beta" product, the UI / UX would be pretty basic on initial launch. We have spent time to address some of the more frustrating elements of the UI / UX for smart search and will continue to iterate on the experience. Some of the change we have rolled out include:
   1. Better visual UI experience
   2. Open search result in new tab
   3. Save search term to allow link sharing and back-button persistence
   4. Removing relevance score from visual preview (score is helpful for calculating which results should be shown, but confusing to users)
   5. Adding "how it works" to help with user education

We hope that these changes have helped to make this a more useful feature. Along with the additions we have outlined above, we are also working on prototyping some generative AI features to smart search. As we have monitored the types of search queries that are being tested in the feature, we have noticed that more "direct" questions are being asked of the data such as "What percentage of my grades is assignments?" or "Can I use AI tools?". Although we currently return results that are the most relevant to the search queries being asked, we want to go the next step and answer these questions while also citing source information. Stay tuned for updates on this exciting enhancement.

Thanks again for your help testing and giving feedback.