UDOIT Version 2.3.5

[tr_jbates]

This is a very tiny update that fixes an issue with false positives and Vimeo videos.  Previously, UDOIT was only detecting Vimeo videos embedded in an iframe because the URL was of the form "https://vimeo.com/player/########".  However, when you simply link to a Vimeo video, it's in the form "https://vimeo.com/########".  UDOIT was detecting those Vimeo video links, but not properly checking them for captions, resulting in a false positive.

This release does not require any changes to your localConfig file, migrations for your database, or updates to packages.  However, you should still test it on a test installation of UDOIT before moving it to production.

To download, visit Release v2.3.5 · ucfopen/UDOIT · GitHub 

[Chris_Hofer]

Hi tr_jbates...

I have been working with  @mmutz1 ‌ to upgrade our UDOIT installation from version 2.3.1 to 2.3.5 this week.  Matt was able to get it installed in our Canvas "test" environment.  Yesterday I was able to scan a two or three very small courses (not much content) and two very large courses (lots of content).  All of those course scanned fine.  However, I also scanned my own "sandbox" course and a "Resources" course that I made for our full-time and adjunct instructors, and those two courses came back with a red message: "UDOIT failed to scan this course."  I'm not sure why my own sandbox course and this "Resources" course would be any different than the other courses I scanned.

Matt had e-mailed me the following:

Should I try to reset the code to make sure everything is the same as the release on GitHub.

I was thinking maybe I’d accidentally changed a file at some point, troubleshooting or something, but I just checked and our repo is in sync with the Github repo for 2.3.5. So I’m not sure what else might be an issue.

Any thoughts on what to do here?  Is there any report on the back end that we can give you to look at and help troubleshoot?

[Chris_Hofer]

Hi tr_jbates...

Quick update for you.  I'm still testing out the 2.3.5 installation in our "test" environment.  A recent posting from rgibson1 prompted me to also try selecting only certain parts of a course to scan with UDOIT.  For my "sandbox" course and the "Resources" course I mentioned in my above post, all parts of the course scanned fine except the "Pages".  I'm thinking I have a page or two in the "Pages" view that UDOIT doesn't like?  Question: Is it safe to have the "#" sign in the page title?  What about other special characters like "?" (question mark), "&" (and), "-" (dash), "," (comma), ":" (colon), or "( )" (parentheses)?

Thanks.

[tr_jbates]

I apologize for taking so long to reply.  I have been out on vacation and am just now catching up.

He haven't had any issues with special characters in the page title before.  As long as Canvas allows it, it should be fine.  You could try removing them and see if it works after the change.  It's also possible some of the HTML is tripping up UDOIT.  It doesn't happen often, but it definitely happens.  If you can isolate the code that UDOIT doesn't like, that would help me a ton.

[Chris_Hofer]

Good morning, tr_jbates...

I spent a few hours troubleshooting things on my end.  I think I have some information for you that should be helpful.

Here's a summary of my entire process that I carried out in our "Test" environment of Canvas:

1. In my own "sandbox" course (let's call this "Sandbox 1"), I scanned it with UDOIT.  I scanned each category (Announcements, Assignments, Discussions, Files, Pages, Syllabus, and Module URLs) separately to determine that the Pages category was the one that was producing the error message shown below.  I have about 70 Pages of content in "Sandbox 1".  Each page has a variety of content such as embedded websites, plain text, images, linked documents, LTIs, videos, etc.
   
2. Created a brand new, empty "sandbox" course shell (let's call this one "Sandbox 2").
3. Used the Course Import Tool to individually copy (select specific content) a single page from "Sandbox 1" to "Sandbox 2".
4. Once the page was imported to "Sandbox 2", viewed the page to confirm that it looks the same as in "Sandbox 1".  Also checked the "Files" area of "Sandbox 2" to ensure that any associated documents/images also copied over.
5. Scanned "Sandbox 2" course with UDOIT (keeping all categories selected).
6. If scan completed successfully, rinse an repeat starting with step #3 to import an additional page to "Sandbox 2" until the above error message was found.

Thus, you could potentially have about 70 separate UDOIT scans...once for each time a page was imported from "Sandbox 1" to "Sandbox 2".

After about 65 scans, I was able to determine that a page in my "Sandbox 2" course called "Vimeo Embedded Video" was the one causing the UDOIT error message.  Here is the HTML code from this page that you can paste into a page in your own sandbox course:

<p><iframe src="https://player.vimeo.com/video/129482676?color=e649bc&amp;title=0&amp;byline=0&amp;portrait=0" width="800" height="450" allowfullscreen="allowfullscreen" webkitallowfullscreen="webkitallowfullscreen" mozallowfullscreen="mozallowfullscreen"></iframe></p>
<p><a href="https://vimeo.com/129482676">2015-06-06 New Feature Screencast</a> from <a href="https://vimeo.com/canvaslms">Canvas LMS</a> on <a href="https://vimeo.com">Vimeo</a>.</p>‍‍‍‍‍‍

It should look something like this:

As soon as I deleted this page from "Sandbox 2", the course scanned fine.  After I determined this page caused the error, I carried out steps #3 through #6 above to ensure all remaining pages I had not yet imported from "Sandbox 1" to "Sandbox 2" scanned successfully.  Once all those pages scanned successfully, I re-imported "Vimeo Embedded Video" to "Sandbox 2", and I again received the error message.

Just to make absolutely sure this was the page causing the error, I created a third "sandbox" course (let's call it "Sandbox 3".  I imported only the "Vimeo Embedded Video" page into this course from "Sandbox 1".  UDOIT produced the error shown above.

Finally, in my above reply from October 19, I mentioned I had also scanned a "Resources" course that we have available for our instructors.  This course also has embedded Vimeo videos in it...which is probably why UDOIT is erroring out.

Maybe a patch can be released soon?

Hoping this will help you, Jacob!

cc:  @mmutz1 ‌

[tr_jbates]

Wow!  Thank you for doing such thorough debugging!  I tested the HTML out in a page in one of my courses, and the scan completed correctly, and detected the video as not having captions.  I am using version 2.3.5 hosted on AWS.

Have you tested out your Vimeo API key manually?  It should be failing gracefully if no API key or an invalid API key is provided, but there's always a chance we missed an edge case.

Speaking of edge cases, my instance of UDOIT also picked up https://vimeo.com/ and https://vimeo.com/canvaslms as not having captions, which is erroneous and would be confusing to users.  I'll be creating an issue for that.

Again, thank you so much for running UDOIT 70 times to discover this code snippet!

[Chris_Hofer]

tr_jbates...

No worries!  It took me a long time, but I wanted to get to the bottom of why UDOIT was erroring out on my end.  I have no idea of we have a Vimeo API key, to be honest.  I wasn't the one who did the install on our own servers.   @mmutz1 ‌, do you have any feedback on this?

Will you be posting a patch based on my findings?

[tr_jbates]

It's hard to say if a patch is needed at this point.  If your Vimeo API key is invalid or nonexistent, the quick fix for you would be to create a Vimeo API key and add it to your localConfig file.  In that case, we would look into why it wasn't failing gracefully and incorporate that bug fix into a future release.

[Chris_Hofer]

Would the Vimeo API key be found in the admin side of Canvas under "Developer Keys"?  If so, the only one we have listed under the "Account" tab is for UDOIT.  I also do not see it listed under the "Inherited" tab.

[tr_jbates]

 @Chris_Hofer ‌ The Vimeo API key is set in the localConfig.php file on the server.  The README has instructions on how to generate the key and where to put it in the localConfig.php file.

[Chris_Hofer]

Good morning, tr_jbates...

I have been working with  @mmutz1 ‌ here at MPTC, and it sounds like we didn't have the Vimeo API key set up.  He has now done this, and courses appear to be scanning properly.  Just wanted to check with you on one thing...  When UDOIT indicates a Vimeo video doesn't have captions, we are seeing this:

This particular video is one of the Canvas Guide videos (111 - User Settings & Profile Picture on Vimeo), and you can see that it does have captions.  Should we still be seeing this error?

(Note: the two other errors listed as #1 and #2 are just part of the footer text for the video.)

[tr_jbates]

You should not be seeing that error.  In fact, I don't see it when I embed the same video in my test course.  I recommend using Postman or a similar application to test out your Vimeo API key on that video.  Here are the details I used (be sure to replace "yourapikey" with your actual API key):

GET https://api.vimeo.com/videos/74243735/texttracks 

Headers:

Key:  Authorization, Value: Bearer yourapikey

And here's my result:

{
    "total": 1,
    "data": [
        {
            "uri": "/videos/74243735/texttracks/4820873",
            "active": true,
            "type": "captions",
            "language": "en-US",
            "link": "https://captions.cloud.vimeo.com/captions/4820873.vtt?expires=1544230800&sig=e4f58d37fceb753a3715f6d...",
            "link_expires_time": 1544230800,
            "hls_link": "https://captions.cloud.vimeo.com/captions/4820873.vtt?expires=1544230800&sig=e4f58d37fceb753a3715f6d...",
            "hls_link_expires_time": 1544230800,
            "name": "111 - User Settings  Profile Picture.en.vtt"
        }
    ]
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Let me know what you get from the same test using your Vimeo key.

[rlzhang]

Hi Jacob, Our institution had similar question regarding the Vimeo key.   In the past, we didn't setup vimeo api key, if there's error if some canvas course did have vimeo content.  We use some Kaltura for video content, first is there some parameter on the config which we could disable scanning on vimeo video?  Second, instead could we twist on config to point to scan on Kaltura for media content instead of Vimeo?  Thanks!

Ruiling