Mandatory Orientation Programming

[kona]

Background

At Richland Community College all students must complete an online student orientation before they are able to access their courses in Canvas. For more information on our Orientation itself see the following article - http://files.eric.ed.gov/fulltext/EJ1011381.pdf and resource in the Community - Canvas Instructor Training and Student Orientation

This process starts when a student registers for a class. Students who have already completed the orientation are enrolled in the proper section of their courses, but students who have not completed the orientation are added to the orientation and to a special section of their course. This special section is called "Need to complete Orientation" and the start and end dates for that section are set to the end of the course. This allows the student to see that the course exists, but not be able to access it. To access their course the student has to complete the orientation. The orientation is self-paced and includes 10 modules. Students must work through the Modules in order and complete the final cumulative exam with an 85% or higher to complete the orientation. Once students have completed the orientation they are removed from the orientation course, added to the Canvas Student Resources course (FAQ), and switched from the “Need to complete Orientation” section of their course into the section in which they are enrolled.

Rationale

Why have a mandatory orientation?

• We saw a 7.7% increase in retention for our online courses after implementing the mandatory orientation.
• Helps improve students’ confidence in their ability to successfully use Canvas for their courses.
• Gives faculty back their first week of class since they don’t have to troubleshoot and play tech support
• Faculty teaching traditional courses are more likely to use Canvas and use more advanced Canvas features because they know their students have completed the orientation and should know how to use it.

Why do we add students who have not completed the orientation into a separate “Need to complete Orientation” rather than not adding them to the course until after they have completed the orientation?

• Faculty can easily see who has completed the orientation and who hasn’t. If the students weren’t add to the course until after they completed the orientation the instructor would have no easy way of seeing who still needed to complete the orientation.
• Faculty can still take attendance and enter grades for students who haven’t completed the orientation. All of this information stays with the student when they move from the “Need to complete Orientation” section to their regular course section.
• Students who haven’t completed the orientation can see their course(s), but when they click on them are not given access.

Concluded, deleted, vs. active?

• Concluded - students have access, but can’t do anything in the course. If there is any student data it is retained and is accessible to the instructor. [In Beta right now: It appears that the concluded state will prevent students from accessing their course.]
• Deleted - students do not have any access to the course. All student data is wiped, including that they were ever signed up for the course.
• Active - students have access to the course and student data is available/accessible to the instructor.

Databases

Our Canvas database systems are maintained separately from our Student Information System (SIS). There is no requirement for this, but the employee working on the Canvas stuff is in a separate department and doesn't like to alter the SIS database unless necessary. There is a lot of duplication that could be removed if they were integrated, but for now, it means that our SIS contains no information about which course sections use Canvas and how they are configured.

Our Canvas database systems are running on a MySQL server, but again, there is no requirement for that. MySQL is widely supported by programming languages, including

PERL and PHP, which are the ones used by the integration. The SIS database is running Informix and compiling PERL and PHP with support requires compiling from source since precompiled packages are generally not available.

When we were developing the system, we found that it was easier to access information through the API using the Canvas IDs for values, so we have a system that contains all of the information that is in Canvas for terms, accounts, courses, sections, users, and enrollments. Many of these have a last_updated field, so that we can tell when they were last changed and reduce the information being sent to Canvas.

Helpful Hint: Store the Canvas IDs in your database. Some of the API calls return Canvas IDs and not SIS IDs and this will save extra API calls when you need that that information.

Canvas API

The system makes some calls to the Canvas API to obtain user information and assignment submissions. We use PERL and PHP but there were no sufficiently developed libraries at the time so we wrote our own libraries to handle the API calls.

PERL is used for almost all of the backend processing. The API calls are performed using the  LWP::UserAgent module. Although lib-www-perl (LWP) is part of CPAN, it was available through the system's package manager and had been used for other programs so it was familiar. PERL itself was chosen because an existing PERL codebase that handled the creation of student accounts. One major issue I had with PERL is that I tried to implement a schema to validate the input before making the call. The CPAN module Data::Schema was one of only a few available without a lot of overhead, but was deprecated in 2011 (and since removed from CPAN) in favor of a new module, Data::SAH, by the same author. Unfortunately, that newer module isn't fully compatible with the old one and no tool is provided for automated migration. A replacement validation module should able to work in both PERL and PHP but the search returned nothing useful, so Data::Schema is still in use.

PHP is used for all pages with a web-interface as well as a few of the backend processes. In PHP, I am using the cURL library that comes with PHP. Unlike PERL, there is not a library built-in to PHP that accomplishes that. There are some packages written, but there doesn't appear to be a clear winner like there is in the case of PERL. There was an HttpRequest module available in PECL, but my experience has been that packages that come precompiled work better than those available through either PEAR or PECL.

At the 2014 InstructureCon, I learned that most programming languages come with a REST package. CPAN includes REST::Client for PERL and there are several available for PHP. However, none of these have as much support as LWP::UserAgent or cURL.

Registering Students

Not every course at our institution uses Canvas. Faculty must request that their courses be created inside Canvas and whether multiple sections of the a course should be combined in Canvas or created separately. There is no requirement that faculty decide that their course will use Canvas prior to the start of registration for a semester or even before the semester begins. Some divisions shuffle teaching assignments the weekend before classes begin and courses that didn't require Canvas under the one instructor now need those added into Canvas.

In short, there is no way to definitively know which students are going to need Canvas before the semester starts. But there is Board of Trustees policy 4.1.2.4 (adopted May 2012) that states, "All students enrolling in a course utilizing a learning management system are required to successfully complete the mandatory technology orientation." The procedure we use to implement that policy is that students are required to complete an orientation course inside Canvas prior to taking courses that use Canvas.

Since students aren't allowed into their courses until they complete the orientation and we sometimes don't know which students will need the orientation until right before classes start, we have a potential problem. The solution was to put all students into the Canvas orientation, not just those who are enrolled in courses that will be using Canvas.

The process to create network accounts and assign emails is ran every 10 minutes. The process to add students to Canvas is ran every 20 minutes, but only processes those students who have already had their accounts created. The longest delay a student may have between signing up for classes and being added to Canvas is 25 minutes.

Completing the Orientation

There is a mandatory orientation to Canvas for students. Students must successfully complete the orientation before they are allowed into their courses. The orientation undergoes regular revisions as part of a quality improvement process and to reflect changes in Canvas, but students do not need to retake the orientation.

Inside the Canvas orientation, there is a "final review" assignment. Completion of this assignment means that they have successfully completed the orientation. That is the official end of the orientation.

However, we added an external tool titled "Click here to complete orientation" after the final review. When a student clicks on this, it calls a small PHP script that records that the student has completed the orientation. This LTI does not make API calls to remove the student from the orientation as that might be too strange for the student to be kicked out of the orientation while they are in it.

We also correctly anticipated that not all students would not click on the external tool, so we use the Submissions API to get a list of the course submissions. There is a course id and assignment id that is unique to the final review, and we ask Canvas for that information and then record the completion for any student it returns.

GET /api/v1/courses/:course_id/assignments/:assignment_id/submissions

However, the Submissions API does not return the SIS code for the student, it returns the Canvas User ID. We would need to call the Users API to get the students SIS code so that we could record which students have completed the orientation. We maintain a cache of the Canvas information in our local system, so we just lookup the student's SIS id locally in the cache. If we find that a student's Canvas ID isn't in our system, then we'll go ahead and look it up using the Users API Get Profile command.GET /api/v1/users/:user_id/profile

Looking up the users that are missing their Canvas ID in the local cache is also a regular part of the process, so finding someone who has completed the orientation before obtaining their Canvas user ID is rare.

On our end, either through the LTI or Submissions API, we record the date, time, and version number of the orientation completed. We only use the version number for reporting purposes. We also use the date and time to process recent changes rather than processing all of the information.

Background Processing after Completion

If a student clicks on the "Click here to complete the orientation" external tool assignment, the only processing that is done is to write a successful completion to our database. No other processing occurs at that time.

We have a process that runs every 20 minutes between 6:05 am and 2:55 am. It takes a break between 3:00 - 6:00 am for additional processing, including a full batch upload in case something fell through the cracks during the day.

Here are some of the items that occur during that process.

• The list of submissions for the final review is obtained and any students who didn't click on the "Click here to complete orientation" link are noted as having completed the orientation.
• Any students who have completed the online orientation are deleted from the orientation and placed into a Student Resources course. This course is essentially the orientation with the quizzes removed and allows students to access the information in the orientation after they are removed from the actual orientation.
• Any students who completed the orientation and are enrolled in a course that is using Canvas are deleted from the "Needs to complete Orientation" sections and placed into the correct sections.
• Any courses that have had all students complete the orientation have the "Needs to complete Orientation" section deleted.
• Any new courses that have been added into Canvas in the last 20 minutes are processed. This includes enrollments and might involve the creation of a "Needs to complete Orientation" section.
• Changes in faculty are automatically processed as well. This might include changes in faculty due to teaching assignment changes, but it might also include existing faculty who have recently completed the Canvas Faculty Training course. That course has three levels of training: enhanced, hybrid, and online courses, and a faculty who has only gone through enhanced training won't be added as the instructor in an online course.
• The SIS imports are sent to Canvas. This is done incrementally through the day, with changes within the last hour being sent. The overnight processing sends a full upload to catch anything that slipped through the cracks.

Contact Information

 @kona , Director of Online Learning

 @James , Internet System Specialist & Professor of Mathematics

[jnuckles]

Great write-up Kona, as you know we do something nearly identical and have had similar results. Are you doing this for students in face-to-face courses as well? We are still only doing this for online and blended course sections.

[kona]

jnuckles​ per a Board Policy our orientation is mandatory for ALL students using the LMS, meaning traditional, hybrid, and online students. In addition we have mandatory faculty training that is set up similar to the student orientation in that faculty don't get access to their courses in Canvas until they complete training.

PS good to see you on Jive!

[hallmans]

Nice article- we are doing a similar thing with undergraduate orientation across all areas. Instead of herding them all into the auditorium and firehosing info down their throats, we do the same modules to quiz to modules format. The self paced works much better, the retention is up and we learned from the quiz results that we need to explain some concepts more clearly.

[kona]

hallmans​, that's awesome! Is your orientation mandatory or have any type of programming attached to it? I'm always looking for Institutions who might be doing something similar so we can compare notes!

[hallmans]

Yes, our orientation is mandatory and we use the quiz results to track individuals progress. Our basic set up is video recorded content followed by the mandatory quiz. If they don't score 100%, they can't open the next module. The ability to email students directly from the grade book who haven't finished a quiz is perfect for us. We can see at glance who is progressing and who isn't.  They typically only have about a week to finish all the modules.

At the end, the students have to rate their choices for areas of study in the program and that helps us form break out groups for the actual onsite.

A very nice touch is the final question- they choose what they want for lunch on their onsite session.

One great thing for us is to see how many times they took each quiz and what questions they got wrong. We realized we had to redo our section on IT authentication and logins for different areas because so many people were failing it so often...

[gtaylor]

We ran into this problem,

Once this section is created(in this example the start and end date is set to Dec 18th), Assignments are not longer able to be created/changed, due to restrictions on dates being outside of one of the course sections.

"We implement this orientation by adding a section to each canvas course that is called "Need to complete Orientation" and setting the start and end dates for that section to the end of the course. "

The only workaround was to assign not to everyone but the main section with the students.  Is that what your doing or am I missing something?

[kona]

gtaylor​, that's interesting... and no, that doesn't happen to us with the way we've got things programmed. I just double-checked this with my Spring Math 113 course and it worked fine. I'm going to provide the settings for my course and our sections and maybe that will help trouble-shoot why you're seeing something different from us.

Below are the course detail settings for my course:

Here's a list of the course sections in my course. Notice I have the Need to Complete Orientation and my regular Math 113-Y1 section both listed. The third section is the Instructor section that I'm enrolled in.

These are the specific section dates for the Need to complete Orientation section:

Notice for the regular course section we don't have section dates set, they are blank and are determined by the course details/term dates. We also don't have dates for the Instructor section of the course.

This shows my example test assignment that I created in my course. It's listed for Everyone in the course and it has an available and due date that fits within the normal semester open dates (per my course details/settings), but definitely doesn't fit with the Need to Complete Orientation section.

I hope this helps and let me know if you have any follow-up questions!

[gtaylor]

hmm, One difference was date i used was the same date for the section start and end.  So I did later try it with a 24 gap in the section start end date.(but there was no difference)

The other is we don't use a separate section for the instructor.  They are in the Main section with the students. I don't see how that could effect it. 

But I'm curious what is the benefit of the separation?(separate section for the instructor.)

Still similar problems. Here is a video(3m 15s)   Which I'm sending to support for review.

Section_date_inconsistency_-_20151202_153348_17

Yes the video is still in silverlight so it will not play with chrome.

[James]

You mentioned having the start/end on the same date, but it also looks like (based on your image above) that the start and end times are the same. In other words, that section is available for 0 time. You might try changing the ending time for the orientation section to be different (after) the starting time. It's possible that it might be a check somewhere and if in one place it checks start <= end and another it checks start < end that it might work in one place and not in another. I'm guessing and I don't really know, but it's worth a shot.

The other thing you described is that you're adding dates for the main section but not for "everyone". We don't differentiate dates for our orientation sections, they just can't do anything in the course so there's no reason to set up sections for them. You said it wouldn't let you add everyone as long as there was the orientation section, but I didn't see that in the video (poor resolution or wasn't there). That might be related to the start/end being the same.

So, I would start by changing the before or end date for the orientation section to be after the course and see what happens. Also, our start date for the orientation is one day before the end of the course, so we don't have any dates outside of when the course ends. So the other thing, when your changing the date, is to make sure that all the dates on the orientation section are within the course dates.

If course ends Dec 14 at 12am (effectively the end of the day on Dec 13), then I would set the start date for the orientation section to before that (maybe even Dec 13 11:59 pm, but try Dec 13 at 12 am and see if it works and then if it does, you can go to 11:59 pm) and leave the end date at Dec 14 at 12 am.

The other is weirdness, but I would try changing the dates for the orientation section before filing a bug report.

[kona]

James provided a lot of very helpful information and about the only thing I can add is for this question: "But I'm curious what is the benefit of the separation?(separate section for the instructor.)"

I double-checked with James (because he's the one that programmed it) and he said it has to do with how we create courses and manage things with our SIS. So no real benefit that I'm aware of other than that's the best way to do it with our SIS.  @James ​ is there an actual benefit for other schools doing it this way?

[James]

That's a variation of what I said.

The enrollments.csv has two columns you can enroll someone into -- a course enrollment or a section enrollment. If you put someone into the course enrollment, then Canvas automatically creates a section with the same name as the course when you SIS import an enrollments.csv file and stick the course enrollments into that section.

In our school, we don't use master courses with multiple instructors, but each instructor gets their own course. Instructors may have more than one section inside a course (we don't do cross-listing) and so it makes sense that the instructor belongs to the course. If you have TA's, then you would definitely want to put them into the section so you can take advantage of limiting them to their section only.

It never crossed my mind, or if it did it was only for a fleeting moment, to put instructors into a section rather than into a course. Strangely, I don't program the things I don't think of.

[gtaylor]

I changed the section start and end time of the orientation section to the last day of the course as well as a 24 hours block of time within the course start and end dates.  It gave the same result.

The chances of this being a problem are slim.  I don't think it is likely that a faculty member would choose to assign work specifically to the Orientation students.( I guess it is nice that Canvas would not allow them to do so).

[James]

That's where a little education of the teachers as to the purpose of the section goes a long way. The instructor wants to know who is in their course (possibly to take attendance) but they don't want them to be able to do anything until they've completed the orientation, and this is the best way to accomplish that. Then, as part of that explanation, just tell them not to use differentiated assignments with the orientation section because they won't be able to do anything anyway.

[gtaylor]

Hello Again.

I've run into another snag.

With the Needs to complete Orientation Section and the student view.

When an instructor uses student view, they can't participate in the course, due to being enrolled(the Test Student) in the Needs to complete Orientation Section.

Are you also experiencing this?

Here is what I got back from Instructure:

Hello George,

The Test Student enrollment is populated to all sections upon using the Student View button in the Settings. While this is unavoidable as a Teacher role, being an Admin I have found a workaround method you're able to use.

Start by going to the Sections tab and open the section you wish to remove the Test Student from, now delete the Test Student from that section and return to the Sections tab. Open the section the Test Student is still enrolled in and click on the "Test Student" name to view their user profile. Masquerade as them by using the Become link and you'll be able to proceed through the course "in Student View" however the Test Student will only be listed in the one intended section. As long as the Student View button is not clicked the section enrollment will stay consistent.

?Please check out our Canvas Community site for guides/answers to frequently asked questions and contact us should you have other questions regarding Canvas.

Thank you,

  Sam Anderson
L1 Support
Support@Instructure.com

[James]

We have not experienced any problems with this in the 3+ years we've been using it. Is this a recent problem (like Canvas just changed the way something worked in a recent update) or could you just be noticing it because classes just now started back up? Was it working before and it's not working now?

[gtaylor]

The issue was first reported to us early June 2016, it's possible it could be a new thing, but I can't say fore sure.

It did not come up during the Jan-May Spring Semester.

It's also appears to be only Quizzes.

The student view can post to discussions or submit assignments.

Here are the results of testing it out.  The behavior is weird.

The student view does not get this.  (Masquerading as a student in Needs to Complete Section)

Also (Masquerading as a student in Needs to Complete Section) Notice Quiz is locked

This is as Test Student. (No locked and no button to take quiz.)  << of course faculty can preview the quiz.  Really they are checking that the quiz is open to students.  Which they can do by checking the dates, but they don't always trust them.

This is a Masquerading as a regular student.

[James]

Based on what the Canvas tech said, this sounds like something out our hands unless they change how it works. It seems weird that the test student would be added in multiple times, but I haven't done any looking into it so I won't rule it out.

I will mention that I delete the "Need to Complete Orientation" section once all of the students have completed the orientation. Depending on how Canvas is used, we encourage our faculty to drop those students who haven't completed the orientation within the first 10 days of the class, so the inability to access the test student turns into a temporary one. But we've still never had anyone complain about this issue, so I can't confirm that it's happening to us.

[kona]

gtaylor​, I just checked this and we are not having this problem. I went into a fall course where there is one student that hasn't completed the orientation. I tried using "Test Student" and it worked with no problem. In addition, the test student got added to the main section and not the Need to complete orientation section.

Hope this helps!

[gtaylor]

In the short term, I will adjust the programming to delete the Need to Complete Orientation Sections, As that will certainly help.

I also came across this,

It's been over a year I think I'll try and revive it. Next week when I have time to write it up well.

[jomontuori]

Once again,  @kona ​, you're my hero. I'm not sure if I'll get administrative support for making mandatory a Canvas student orientation course in my K-12 institution, but I'm going to push for it! I just imported your course from the Commons (and successfully completed it!). Thank you, thank you, thank you, for your generosity. Even if I had the time to create one, it wouldn't be nearly as thorough and well-designed.

I don't have any suggestions for improvement. I'll of course modify it for our institution, but it's everything our students need to get off on the right foot. AND, best of all, this is going to push our faculty to raise the bar in our second year of using Canvas.

In fact, I'd suggest you post this blog — with perhaps a new introduction — in the Canvas Engagement Strategies group. Here's why. If, as a result of taking a Canvas training course, all students are well versed in using the various Canvas tools, instructors can raise expectations for what students can accomplish in their Canvas courses. Removing the student learning curve from every instructor's shoulders allows those who may otherwise hesitate to implement that first Discussion, Group activity, or formative assessment Quiz.  And as you know, increased usage tends to ripple outward.

I already knew you were a proponent of mandatory instructor Canvas training (which probably won't fly at my school, except for new hires). But a student course could be the next best tactic to further instructor engagement with Canvas.

Thanks again!

[kona]

Wow! Thanks for the great feedback!! It's much appreciated! 🙂

[Bobby2]

Joe How did you go with pushing for mandatory training? And how has the rest of your Canvas introduction gone?

[rtyger]

Good afternoon.  Is this the most current 

Student Orientation you are sharing?   I'm updating ours today and have always liked your designs.

[James]

 @rtyger  

I didn't know that Kona was sharing that one. That one that you linked to is the current live orientation that we are using with our students. She has another one called "Public Student Orientation" that I figured would be the one that she was sharing. I did a side-by-side comparison of the modules page and they appear to be the same.