Building a Faculty Practicum Rating Sheet in Canvas

[jmarrapodi]

I've been asked to recreate a rating sheet that is used by field supervisors to assess student performance during their practicum. I've attached the form. It has fields for tracking supervisor names, hours student logged, info about the space for the practicum, then it has a rating sheet with multiple areas of competencies. The form is filled in by the supervisor, who isn't on our faculty, but we can get them temporary access to Canvas to do this if I can figure out how. I've built it in Microsoft Forms, which dumps the data into an Excel sheet. It eliminates getting the temporary access but it doesn't add the data to the outcomes being tracked in Canvas. 

My goal is to have this data work with the outcomes that are already in Canvas, since they are the program outcomes, created at the institution level.

I'm not sure how to do this. The form is set so they have sub-outcomes that are rated, then they add the scores to get a final score for the competency. I started building a rubric, then added in the outcome and it adds more points. I had to put directions for the scoring to determine if they passed the competency since I can't do subtotals. Here's a snapshot. That seems to be close, but it doesn't work because of the directions and the extra points the competency adds. Do I do 9 of these, with directions above each for scoring? That doesn't seem right.

Any suggestions on a way to make this work?

[ProfessorBeyrer · Accepted solution]

Thank you @jmarrapodi for including the review sheet used by your supervisors. I think you already have an elegant solution for collecting the observation data (Microsoft Forms) and readying that data for analysis (Microsoft Excel). I would dispense with the stress of extending Canvas access to the evaluators. If it's temporary, that means it has to be enabled and then disabled, which adds administrivia to your workload.

To connect your data to the Outcomes tool in Canvas, I suggest creating one assignment with a rubric with 9 criteria (1 for each outcome). Hopefully you can make each of those outcome criteria worth the points that are in the attached document (i.e., Competency 1 is worth 20 points, Competency 2 is worth 12 points, etc.).

Match the levels for each criterion within the Canvas rubric with the titles from the observation sheet and make the range of point values match as well. Here's how I would do it for Competency 1:

• 16-20 = Excellent 
• 11-15 = Meets expectations
• 6-10 = Needs improvement
• 1-5 = Does not meet expectations
• 0 = No evidence

The data entry for Canvas begins with opening the rubric inside SpeedGrader for the assignment in your browser and your data sheet in Excel. Then enter the points for each criterion reported by your supervisors. If you set up the grading rubric as ranges, the corresponding criterion level will "light up."

At the end you will have both sets of data for each student: A score out of 124 for their letter grade and a value for each of the outcomes. 

This was a great question, and I hope my suggestions work for you. I'm going to mark my response as a solution, but hopefully other members of the community will also share their ideas.

[jmarrapodi]

Thanks @ProfessorBeyrer! That's really helpful. This will work well to tie into the existing competencies.

Now that I have this wonderful form built in Microsoft Forms, the Dean informed me that it has to be editable by 2 people (the two reviewers) AND has to have a field for their signatures AND the student's acknowledgement of receipt and agreement or disagreement. Oh let's just add a new layer of complexity to this. Currently they have been using the Word form and passing it around then printing it for signatures, and scanning and uploading it into Blackboard (going away 8/22). They want the whole thing to be digital. I can't figure out if there's a way to have multiple people edit the content of a form or if Microsoft has a signature option like Adobe does with Adobe Sign or if using DocuSign would work. I wish I'd known these specs at the beginning. I can recreate the form in Acrobat as a form to fill in but it looks like their form data management involves a bunch of extra work. https://helpx.adobe.com/acrobat/using/collecting-pdf-form-data.html . The Microsoft option dumps into Excel, where I'd have to fiddle with putting in formulas but it would auto calculate things on the back end.  Google Forms might work with this but we are a Microsoft shop and Google isn't allowed. 

I'm stumped and have time scheduled with IT to see if they have any ideas. Brilliance and insight welcome if anyone has ideas out there in Canvasland. And if I can't ask this because it falls outside of Canvas, I'll understand. Thanks. 

[jmarrapodi]

Three other questions, @ProfessorBeyrer .

1. Would you set that rubric up as a no-submission assignment?
2. How do I get this form into Canvas, uploaded by the faculty member. I suppose I could use the completed digital form as a submission, but I need the faculty to upload not the student. Would you attach that in the comments or is there another way to store this in the system?
3. I already have these outcomes set up at the program level with the standard 5-3-0 the default setting in Canvas. Am I creating an additional set of outcomes with this new rubric or am I mapping this new rubric to the existing outcomes? I only have the comprehensive exam set up with the existing outcomes and only had 7 students take the exam so I could rejigger the program outcomes to have these 4-3-2-1-0 criteria, which I suspect might make the most sense. They didn't work with the exam anyway, since I created an Item Bank and attached the Outcomes there, and when I imported the items from the bank at the root level into the course, they didn't connect to the scoring in the course, which is a known thing in the new quizzes/item banks that I didn't know about. I have a different question out there about fixing that . 

Thanks for any insights.

[ProfessorBeyrer]

Yes, the assignment would be a no-submission assignment. The form could then be attached as an assignment comment by the person doing the evaluating. I do not envy your quest for a process to efficiently capture multiple signatures, so scanning and submitting a paper form might be the best option.

For the point value for the outcomes, I suggest editing them so that they have different values as multiples of 4-3-2-1-0 (Competency 1 would be 20-15-10-5-0) and that the evaluators enter whatever score the students earned for that competency. They would have to do those sums themselves, and then the scanned-and-uploaded document would be evidence in case anyone asked questions.

[jmarrapodi]

Thanks @ProfessorBeyrer! I appreciate that reply. I found out from my IT friends that if I use the same form I built in Microsoft forms, but redo it in Sharepoint, I can add signatures and process steps so it goes from the supervisor to the faculty then to the student for acknowledgement. Pretty nifty! It can also add up the scores.

I've added fields to summarize the numbers for the 9 competencies in the end so whoever has to enter this in Canvas has that all in place. I like the idea of printing the final form as a PDF and clipping it to the comments so the details are available if there is ever an audit.

Phew! I'll be glad to be on the other side of this project. 🥳