[ARCHIVED] Good way to search content within a course?

[mlattke]

Does anyone have a good way to setup a search feature within a course?  I know that the Files tab has a search feature, but I'm looking more for content pages than files.  Here is the scenario.  We have a curriculum course with hundreds of pages of resources organized by grade level and subject.  A teacher can click through a series of buttons to browse to their desired resource, but the curriculum department wants teaches to be able to search for say "Poetry" and find all the pages on poetry.  I know this is not a built in feature, and I know I can submit a feature request, but I'm looking for an external tool or some other work around/clever idea that would let me do this in the short term.

Ideas I have thought of:

-Create an index page that lists all pages in a table of contents and use the browser search command on that page to locate a page. (downside -- hard to keep up as pages are added over time).

-Make sure pages are named with a clear naming convention and open the pages tab and use the browser search command on that page to locate a page. (downside -- when many pages are present, not all appear until you scroll, so you would not be able to search the ones that do not yet appear on the page using the browser search).

I don't think either of these ideas are particularly good, so I'm hoping that someone out there has an amazing idea for how to do this!

Thanks in advance!

ANOTHER THOUGHT:  Is it possible to use Google Site Search somehow?  I googled this, but couldn't find an answer.  😉

I thought maybe there was some custom javascript that could be embedded that would let site search crawl the pages.....anyone? anyone?

Message was edited by: Michelle Lattke

[orwinr · Accepted solution]

I have successfully integrated Google Custom Search into my Technology training resources site by creating an html file that includes the GCS javascript with a note telling users all results will open in a new tab. Once you have created the html file upload it to your course files area and use a module item to create a link to the html file you just uploaded. When you click on the link in the module you have a nice custom search page embedded in your course. If you want to take it further use the redirect LTI tool to create a link in the left navigation menu. You can see our example at: iSchool Technology Training and find the search link in the left nav menu.

The code that is currently in my search.html file is:

<p style="color: grey;">All results will open in a new tab or window.</p>

<div>

<script>

  (function() {

    var cx = 'your search engine ID goes here';

    var gcse = document.createElement('script');

    gcse.type = 'text/javascript';

    gcse.async = true;

    gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +

        '//cse.google.com/cse.js?cx=' + cx;

    var s = document.getElementsByTagName('script')[0];

    s.parentNode.insertBefore(gcse, s);

  })();

</script>

<gcse:search linkTarget="_parent"></gcse:search>

</div>

I hope you find this useful until we get search functionality in courses themselves. Just a note that only the pages and announcements areas appear to be searchable via GCS, or at least that is what I am finding, which totally makes sense. I also believe that your course has to be public for Google to crawl it, which also takes some time to occur on new pages. You can change the behavior of the custom search script through your GCS management console.

Randy

[orwinr · Accepted solution]

Hi  @kevinw ,

My experience with the Google search on public sites is that it can take 48 hours or more before anything gets indexed. It takes a long while. The other thing to test is to make sure that your html file really works.

If you are interested in a bit more of an enterprise class solution that searches within all course content, even in private courses, take a look at "Atomic Search" by a company called "Atomic Jolt". Their CEO is  @joel_duffin ‌. We have a sandbox course configured at the moment and have been quite impressed with the search results and the responsiveness of the dev team when we have run into issues. It is a bit pricey for our small organization at this point but worth a look if you want to use it across an organization. We did a survey of our students and faculty about search within Canvas sites and 91% of faculty and 96% of students said they would like to see search in all Canvas courses.

Good luck!

[ahayson · Accepted solution]

Hi  @mlattke ‌,

Have you had the opportunity to check out Atomic Search? They have an lti‌ with Canvas, it might be worth a look!

- Am

[orwinr · Accepted solution]

Hi  @rislis ,

We have been using Atomic Search for almost the last year and really like it. As a company they have been quite responsive to our needs and our faculty and students love being able to search individual courses as well as across all of their enrollments at the same time. 

[kona]

Unfortunately I have no wonderful wisdom on this other than I'd love to have this feature and if you haven't already vote on this feature idea - !

[mlattke]

Thanks for the link, Kona.  I hadn't seen that request and didn't find it when I searched the community, so I just voted it up!  That will be helpful for the future for sure if it turns into a real life thing.  I'm hoping maybe someone out there has a hack or creative idea.  It is a pipe dream, but there are so many smart people out there, I thought it was worth a shot!

[James]

I'm not exactly sure what you're looking for. I thought I knew (the title says search content, but your ideas are for searching page titles). Anyway, let me try throwing some ideas out and see if one sounds doable.

If you want to search the content of the pages ...

What if you downloaded all of the pages in a course and stored them in a single, really large, Word file. Then you could open up the word file and press Ctrl-F for find.

As part of my efforts to replace all the links to the old Guides with links to the new pages in the Community, I've already written the code that will obtain a list of all of the pages and then cycle through and download them one at a time. Right now, it just processes them internally, replacing the links, and then optionally replaces the existing page with the updated one. But that last portion of code could be modified to write them to a file.

If someone had the tools, there are indexing programs out there, and someone could populate the search tool with the pages rather than putting them into Word. I was just trying to make it (relatively) easy to do.

Another option would be to save the pages to your own local server and then have Google (or another search engine) index those pages.

You might also be able to do a course export into the common cartridge format. That's a zip file with a different extension. You could unzip it and have all of your files. However, someone posted earlier (I haven't verified) that you get random looking page names, so there might be more programming there to figure out everything than there would be to use the API.

If the list of words was predefined (like tags), then you could even create the index yourself as you download each page and not need to store the content of the pages.

If you just need to search the names of the pages ...

Life just got a whole lot easier. We can use the API call to get the list of pages and then write their titles with links into the course to a file. It could be Excel or Word or even an HTML file.

[mlattke]

Hi James,

Sorry for my confusing wording!  Ideally, I would like to search the contents of the course/pages.  My very limited ideas (that didn't involve much intervention) were only ways to search titles, which is part of why I discarded both of them.  I was trying to list things I had already thought of so people would know where I was at.  But it sounds like that I made things less clear.  😉

You are awesome, because you have a lot of really amazing ideas for creatively solving the problem, which is exactly the kind of thinking I was hoping to find with this question!

I think some of the ideas may be beyond the scope of what I would be able to do in the short term, but it gives me a lot to think about!

Thanks again for sharing these ideas!

Michelle

[mpoole]

I was able to create a google custom search that will search my course and return based on keyword and it seems to actually work well.  I just created the search and put my top level course URL as the site to be searched.  It returned files, page content, and assignments.

But, I don't know enough about the code side to know if I can then embed the search into my site in a way that is helpful.  I tried using the embed code generated and that doesn't work but I don't  know if it is something that could be converted to work within Canvas or not.

This is what the code looks like.

<script>

  (function() {

    var cx = '010804999352776858981:17kqd-2skl8';

    var gcse = document.createElement('script');

    gcse.type = 'text/javascript';

    gcse.async = true;

    gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +

        '//cse.google.com/cse.js?cx=' + cx;

    var s = document.getElementsByTagName('script')[0];

    s.parentNode.insertBefore(gcse, s);

  })();

</script>

<gcse:search></gcse:search>

Here's the link to custom search engine

[mlattke]

 @mpoole ,

This is amazingly good news, and exactly what I was looking/hoping for.  I think if I could get that to work like you have described, I would be a seriously happy camper.

Before I added the Google bit to my question, I had tried to setup a custom search and to list the course URL  (as you describe), but it keeps returning no results when I search using the custom search page.  Did it take a few days for Google to index your course and for you to see results or did you get results right away?

Was there anything special you had to setup in your Google Account to make this work, or did you just add a new custom search, and it just worked?

The embed code you listed doesn't work b/c it is Javascript, and so it would get stripped out of a page, and you could only use it if you somehow incorporated it into the custom javascript file at the account or sub-account level.  However, you can totally link to the custom search page from within your course and pop it open in a new tab.

Thanks again!!!!

Michelle

[mpoole]

​I was able to search my course content right away And I didn't do anything special. Our canvas organization is pretty simple and my course is in the top rather than a sub, so that may have played a role? 

[G_Petruzella]

Hey  @mlattke ​​, I ran into the same issue you did when I tried this - no results when I search for stuff I know is there. 😕 But this is an awesome enough tool that I'm going to keep poking at it to see if I can get it to work the way  @mpoole ​ has it - and I'll share here if I discover anything.

[mlattke]

Thanks,  @G_Petruzella ​!  I'm glad to know I'm not the only one! 

I agree -- knowing it works for Melissa gives me hope to keep trying!

I tried making the course public to see if that would work, and I still get no results.

I'm going to keep at it too, and I will let you know if I figure anything out as well!!

[G_Petruzella]

 @mlattke ​, no answer yet, but I've noticed that

1. Creating a Custom Search Engine on my entire site ( mcla.instructure.com/* ) yields results;
2. Creating a CSE on my specific course ( mcla.instructure.com/courses/1572734/* ) does not yield results.

[mlattke]

 @G_Petruzella ​, more to report:

1. CSE on my entire site also works.

2. Got CSE to work on one course

3. CSE still doesn't work on other courses

I have NO idea what is causing it to work on one course and not the other.....

-Did nothing different on the CSE setup

-Tried setting both courses up to have same public/privacy settings in Canvas

Here is a screenshot of one that works and one that doesn't.

CLASS THAT WORKED:

CLASS THAT DOESN'T WORK:

[kevinw]

Did you ever figure out why one worked, and not the other? 

[orwinr · Accepted solution]

I have successfully integrated Google Custom Search into my Technology training resources site by creating an html file that includes the GCS javascript with a note telling users all results will open in a new tab. Once you have created the html file upload it to your course files area and use a module item to create a link to the html file you just uploaded. When you click on the link in the module you have a nice custom search page embedded in your course. If you want to take it further use the redirect LTI tool to create a link in the left navigation menu. You can see our example at: iSchool Technology Training and find the search link in the left nav menu.

The code that is currently in my search.html file is:

<p style="color: grey;">All results will open in a new tab or window.</p>

<div>

<script>

  (function() {

    var cx = 'your search engine ID goes here';

    var gcse = document.createElement('script');

    gcse.type = 'text/javascript';

    gcse.async = true;

    gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +

        '//cse.google.com/cse.js?cx=' + cx;

    var s = document.getElementsByTagName('script')[0];

    s.parentNode.insertBefore(gcse, s);

  })();

</script>

<gcse:search linkTarget="_parent"></gcse:search>

</div>

I hope you find this useful until we get search functionality in courses themselves. Just a note that only the pages and announcements areas appear to be searchable via GCS, or at least that is what I am finding, which totally makes sense. I also believe that your course has to be public for Google to crawl it, which also takes some time to occur on new pages. You can change the behavior of the custom search script through your GCS management console.

Randy

[naren_salem]

Hi Randy,

I love what you have done to embed the search. Your solution is perfect and I wanted to add "Search" to the sidebar using the Redirect. How were you able to get the redirect to open the search.html file you have uploaded to the course? I.e. how do I get the URL to enter into the Redirect plugin.

Thanks for the great solution!

Naren

[orwinr]

Hi Naren,

It is a two step process after you have created the html file and uploaded it to your course files area.

1. The first thing you need to do is go to the modules page in the left nav bar and add a new item that links to the html file in your course files area.
2. After adding the link in the modules area, click on it to confirm that it works. If it works, right click (Control click on a Mac) on the link and copy the url of the link.
3. Now that you have the link of the module item, go to the redirect tool and use that link in the configuration.

If all is well with the world, your search link will show up in the left nav bar 🙂

[naren_salem]

Thanks Randy! That worked. Just for the record, I am adding this to a course I have on the Instructure hosted instance of Canvas. Initially my searches kept returning 'No results'. I then requested Google to search my course using this site and then it worked:

https://www.google.com/webmasters/tools/submit-url?continue=/addurl

Also, Randy, I am not sure if you preferred to have your results open in a separate window or not. But I wanted to have the search result links open in the same window. I was able to get that by adding the linkTarget attribute in my search.html like so:

<gcse:search linkTarget="_parent"></gcse:search>

[orwinr]

Glad it worked Naren! Thanks for the tip on the results, I hadn't even looked for a solution so you saved me some time 🙂

[wills004]

Hi everyone,

I'm not sure what the problem might be. I tried creating some general search engines (erau.instructure.com, .com/*, etc) and would get no results.  I tried it before asking Google to search, and afterwards (thanks to Naren's idea).  No difference.

I put the search.html into the Files of a course template, created a Module item, linked to search with FILE, and when I run the search it finds a lot of the "out there in the wide world" items (well, 15 of them with the word "network" in canvas pages, that is), but nothing in the course itself.  IN FACT... the only site I see is canvas.uw.edu and subpages...

Any ideas?

Could there be something that my school's site administrator has done that blocks this google search engine from looking internally?

Thanks,

Mike

[naren_salem]

Does your search work by itself? As in, if you ran your search.html without embedding in Canvas, does it work?

I am guessing it is not, because it sounds like you are having trouble with the indexing part where Google creates your custom search engine.

You may be right in thinking this may be an issue with the site admin settings. Also when I was trying this, it only works if the course was set to be public - Google can't see the lessons if there is a password.

Ultimately I gave up on this - our course needed a password, and also the way Google was doing this was a bit mysterious. I ended up writing my own search engine for the course.

[orwinr]

Mike, if your search.html file is configured correctly and you have a Google Custom Search ID, your results should only contain items from your course. In the course that I shared when I created the scenario above, I just did a search for the term "Jing" and got 17 results. All results were from this course, with the obligatory ads at the top of the results list. As I pointed out and   @naren_salem ​ pointed out, the search only works if your course is public. If it isn't public Google can't crawl it and maybe we can get  @naren_salem  to tell us about the custom search tool he wrote for his course.

[wills004]

Thanks, Randy, for the reply. You’re right, our courses are not public, and policy prevents us changing that for many good reasons.

I shall keep thinking and keep researching…

Cheers,

Mike

[naren_salem]

I'd meant to reply to this, but I just realized that I never did. Here is a quick write up I made on how I created the custom search:

Creating a search engine

The process is pretty involved and technical. I'm sure the workflow can be a bit more automated. I guess this approach is viable depending on how badly you want the search option . We were submitting our course for state board approval and having search was a key requirement. So we bit the bullet. 

[scalesj]

@naren_salem - Thanks for sharing the documentation. I'm getting an error though when I click on your link.

[kevinw]

I seem to be having the same problem some users have had. 

I made a copy of a course, made sure it was public, created my search engine, and no results. Could it be that on the back-end of Canvas, it takes time to convert everything to public, so that google can crawl it? Or is there another issue that I could be missing. 

[orwinr · Accepted solution]

Hi  @kevinw ,

My experience with the Google search on public sites is that it can take 48 hours or more before anything gets indexed. It takes a long while. The other thing to test is to make sure that your html file really works.

If you are interested in a bit more of an enterprise class solution that searches within all course content, even in private courses, take a look at "Atomic Search" by a company called "Atomic Jolt". Their CEO is  @joel_duffin ‌. We have a sandbox course configured at the moment and have been quite impressed with the search results and the responsiveness of the dev team when we have run into issues. It is a bit pricey for our small organization at this point but worth a look if you want to use it across an organization. We did a survey of our students and faculty about search within Canvas sites and 91% of faculty and 96% of students said they would like to see search in all Canvas courses.

Good luck!

[kevinw]

Hi Randy, 

Thanks for the response. I am in 100% agreement with you there, Canvas needs a way to have a course content search. It's been in the radar for a long while but I haven't heard any updates recently. I suppose it's just a waiting game at this point, to see if the course gets indexed, and how long it takes it to do so. It couldn't have been the HTML file because I was search directly from the Google CSE and getting no results. 

[allan_trautman]

I've gotten good results (as mentioned above) by downloading an export of the course, renaming the file extension to ".zip," opening the zip file and then just searching the resulting folder for whatever content I'm looking for. Many pages are named exactly the same. Other items like discussion instructions are in xml format, but always toward the top of the page there is a <title> tag that identifies it. No scripting required! Just a plain-text editor.

[ahayson · Accepted solution]

Hi  @mlattke ‌,

Have you had the opportunity to check out Atomic Search? They have an lti‌ with Canvas, it might be worth a look!

- Am

[rislis]

Noticing the dates of these comments and then Deactivated user‌ encouraged the look at  Atomic Search.  Anyone using that? Is the Javascript answer working for others? orwinr - your iSchool Learning Technologies page is incredible and just what I needed to see this morning!  Thank you for sharing!

[orwinr · Accepted solution]

Hi  @rislis ,

We have been using Atomic Search for almost the last year and really like it. As a company they have been quite responsive to our needs and our faculty and students love being able to search individual courses as well as across all of their enrollments at the same time. 

[richkingsford · Accepted solution]

You might try Atomic Search - a Canvas plugin that provides basic and advanced search capabilities. 

[AnnaHelme]

The solution is definitely not a solution.

I am a bit shocked that Canvas doesn't have some kind of Search widget or powerful built-in search functionality - if only for native content types e.g. Pages & Files? 

People having to kludge export into an obscure data format or integrate Google is surely unreasonable?