So You Want to Have Team Based Learning Quizzes

mjennings · ‎09-05-2017

Good Grief! Wait, can grief be good? Can struggle and toil produce something good? Absolutely. This is the story of my current good grief.

Many semesters ago, in a previous LMS, I had a faculty member introduce me to the concept of Team Based Learning (TBL) and they wanted to offer a quiz that each student would take individually and then as a group. I am sure the look on my face was somewhat like this:

My initial reaction was, the LMS that shall not be named is not designed for students to take a quiz as a group the same way they can submit a group assignment. As she continued to press on what she was trying to accomplish, I learned more details of her course (we would call it a supplemental course, meaning that they met live but used the LMS to provide resources, materials, assignment drop boxes & administer exams) and we discussed what ways they could maybe bend the system to their meet their objective.

Here is where we landed that worked pretty well:

The student came to the testing lab and look the exam (50 questions randomized, one at a time, 60 minute time limit¹).
Students remained in the testing room after they submitted the exam until the full time was completed.
The faculty had the student get into their groups, gave a paper version of the same exam to each student, and gave them 40 minutes¹ to work through the questions.
Students were sent back to their computer station and competed a "group" version of the same exam in the LMS using their notes from the breakout session (50 questions, not randomized so they matched the paper copy, all questions at once and 20 minutes to answer the questions & submit¹).
The paper copies & notes were collected as students left the lab to prohibit the questions from being shared with other students.
The LMS was set up so that the two exams would average together, however the instructor would review the original submission which needed to be a minimum score (75¹). If the minimum score was not met on the individual exam, the group exam was thrown out and the student just received the individual grade as the final exam score. It would look some thing like this:

In the example above, these are 4 students in a group that individually got an 86, 92, 74, & 78 respectively. Then after meeting as a group, re-took the Group Exam and scored an 98, 100, 98, & 96 respectively. This highlights a benefit to them working together and then taking the exam individually - that if they disagreed with the group on what the answer should be, they had the freedom to still answer as they wanted. This paid off for Student 01, but hurt Student 04 in this example. Notice also that because, Student 03 did not meet the Individual Exam minimum score, the faculty member has manually changed the grade to the Individual grade (noted by the asterisk).

If you are still with me, you deserve something. Here are some cute pandas:

Okay, let's bring this up to the present. As our faculty have continued to explore TBL, we have used the method above in several courses and have since moved to Canvas (we are now starting our 4th year!!!). About a semester ago I worked with one of our faculty to work to incorporate some TBL assignments into her fully online course. Now she would like to include iRATs & tRATs. While those terms were foreign to me, they are essentially what we were doing in our blended courses described above. The iRAT is an Individual Readiness Assessment Test and the tRAT is a Team Readiness Assessment Test. These are typically given prior to the content being taught, think pre-test. We would then follow up with a post-test once the module has been completed. (This is my limited understand based off of working with faculty, feel free to show me the error of my ways in the comments - just be nice about it .)

My initial reaction to her request was to see how we could translate what we did in the live courses to a fully online course. We created a pretty nice timeline/event calendar that would work something like this:

This gives a good idea of the cycle but we are still working out some of the details and would love some thoughts on the approach we are planning in case we are missing something.

Here is the schedule breakdown with some of the thoughts behind them:

Friday - iRAT open from 8am - 5pm. (10 random questions)
Saturday to Tuesday Morning - tRAT group discussion board opens with the questions from the iRAT posted. Groups are instructed to work together to provide an answer & rationale for the questions in the discussion board. (This is the biggest concern. We are informing the students that the quiz questions are only going to be used here and will not be repeated on future quizzes. We also have no real means to know if the students are interacting or just working out the question/answers on their own. We are asking them to "show their work" by asking them to post rationale & are encouraging them to discuss, but you know what they say, "you can lead a horse to water, but can't make it drink.")
Tuesday - Discussion board closes & tRAT quiz open from 8am - 5pm.
Wednesday - a virtual meeting is held (we use GoToMeeting) to discuss the most missed items from the iRAT & tRAT

I believe there are a few other items such as the module post test and possibly another virtual meeting prior to the next cycle beginning - I just don't have that schedule in front of me.

So if there are any TBL experts out there, I would welcome your thoughts on this cycle and any suggestions that might help make this an excellent learning exercise for students.

¹ Denotes the best I can remember and are not intended to represent actual settings & configurations.

taylorbs · ‎03-02-2018

Full disclosure: More of a TBL expert than a Canvas expert, although I think I was able to follow everything you were saying regarding both aspects. We have a lot of heartburn here about having to do the workaround that is merely due to a limitation in Canvas. Basically, if I understand your workaround reasoning, the second individual submission is still going to undermine the core purpose of the team completion phase, which is to require cooperation among the entire group, and to offer an incentive to continue trying to succeed collectively even after a first failure (because they will eventually get partial credit for trying til they get it right, and for developing negotiation skills). That’s the Team part of TBL, to get people to interact and trust each other, and to be able to gauge success in doing so. The way I understand it, taking an average of individual scores or letting individual people override the group decision and do whatever they want regardless makes the team part a waste of time and effort. The point of cooperative learning is to put stakes on collaboration: you can’t succeed unless we all succeed.

Also, IMO and TBH, as a practical matter, anything more than about 6-10 questions on a RAT or asking people to show their work means the TBL designer needs to rethink their TBL lesson. If designed well, assurance of preparedness for TBL shouldn’t take that many questions to level-set a team, and the collective correction will incentivize the cirrect derivation of an answer. Even if they are perfect strangers. If the teacher wants to give a full-blown assessment to validate induvidual students’ content recall comprehension, call it a quiz and call it a day.

My apologies if you read this whole comment and it turns out I misunderstood the TBL reasoning used at your school. Also sorry for the number of edits. I am typing this on my iPhone. (Don’t judge). But thank you for the pandas !

mjennings · ‎03-02-2018

@taylorbs , Thanks for taking the time to read my post and give a thoughtful reply. I agree that many of the decisions that the instructor has made seems to negate the purpose and spirit of TBL and I have been working to have these conversations since we started this. This term we are trying some different approaches to solving the problems. Hopefully we can present on it at InstructureCon 2018‌. :smileygrin: