Quiz's mean elapsed time is large multiple of available time

[arencambre]

I have given three quizzes in my class.

Each of these quizzes are exactly 10 minutes, enforced by the quiz's start and end times. It is impossible for a student to go longer than 10 minutes. (Exception: 1.6% of my students have a disability accommodation and get 15 minutes on their quizzes, but that is such a small effect, I doubt its relevance.)

Here's the mean elapsed times for these quizzes, as per the Quiz and Item Analysis report:

• Quiz 1: 08:20 (5000% of max time!)
• Quit 2: 01:18 (780% of max time!)
• Quiz 3: 0:03 (30% of max time!?)

I am unclear how to interpret these numbers. Is this field mislabeled?

I just double-checked quiz 1's settings in its Assign section, and they are all as expected: every quiz begins and ends on the same calendar date, and the start and end times are all within 10 minutes of each other (except for the 1.6% who get 15 minutes).

[Ron_Bowman]

@arencambre -

Well that is an iteresting statistic.  If you figure it out, the percentages are acting as if the time to take the quiz is 10 seconds (3 seconds/10 = 30%, 1:18 = 78 seconds => 78/10 = 780%, 8:20 = 500 seconds => 500/10 = 5000%)

So, I believe there may be a math issue in the actual calculation - somehow they are mixing units of seconds and minutes in the calculation.

That is all I have.  I may play around with it and see if I get the same results as you do.

Ron

[Ron_Bowman]

@arencambre -

Are these new quizzes?  

I do not see the %time in parentheses next to the average time in classic quizzes.  In my trial quiz in New quizzes, I should have had an average time of around 1 minute, but it showed 0 for the average time(I am not sure why that would be the case).

Can you provide a screen shot of the quiz analysis showing the percent you are talking about.

Ron

[arencambre]

---

@Ron_Bowman wrote:

@arencambre -

Are these new quizzes?  

I do not see the %time in parentheses next to the average time in classic quizzes.  In my trial quiz in New quizzes, I should have had an average time of around 1 minute, but it showed 0 for the average time(I am not sure why that would be the case).

Can you provide a screen shot of the quiz analysis showing the percent you are talking about.

Ron

---

Yes, they are new quizzes.

Here's a screenshot of the output from quiz 1:

[Ron_Bowman]

@arencambre -

Now I am confused.  In your original posts you showed for the 8:20 average time a 5000% value.  I thought the 5000% is what you were referencing.

In that screenshot, the average time is 8:20 which is 8 minutes 20 seconds which is under the 10 minutes allotted for the exam.  So I do not understand what the problem is.  What am I misinterpreting?

Ron

[chriscas]

I think there may be some ambiguity in the current time notation.  I'm guessing @arencambre interpreted 8:20 as 8 hours and 20 minutes, when I do believe it's actually meant to be 8 minutes and 20 seconds.  It may be beneficial for the UI to display the hours unit when it's zero (0:08:20) or have some other way to indicate the time units to avoid this ambiguity.

-Chris

[arencambre]

There is no consistent interpretation of that number that makes any sense.

If 08:20 means 8 minutes and 20 seconds, then this means my 113 students completed a similar quiz (also 20 question, some of which require thought) in one minute and 18 seconds! 😱

But wait, there're more! The third quiz had 19 instead of 20 questions. Just omitting that one question caused them to answer all 19 questions in only three seconds!!! 😱

Strangely, when I reduce to 12 questions, they go back to taking 7 minutes and 14 seconds?

[Ron_Bowman]

@arencambre -

That is interesting.  One thing I noticed is that the format you see for the statistical information is different from mine - maybe window size as I am on a computer and you may be on a tablet (that is beside the point at hand).

I decided to look up the difference between mean and average and found the following: "Average, also called the arithmetic mean, is the sum of all the values divided by the number of values. Whereas, mean is the average in the given data. In statistics, the mean is equal to the total number of observations divided by the number of observations"  

That has slightly different meanings.  Just out of curiosity, how many students took the quiz for each quiz for the results shown and how many were in the class?  I am wondering if there were (hypothetically) 100 students in the class, 1 student took the quiz in 3 minutes (180 seconds) so the mean that Canvas calculates is 180/100 or 1.8 seconds.  That calculation is actually an average whereas the mean would be the 180 - I am assuming that a quiz not taken is still counted in the sample with a time of 0.  Just throwing something out there to see if we can find any semblance of sense out of the calculation. (I think I have the mean and average determined correctly)

Ron

Ron

[arencambre]

---

@Ron_Bowman wrote:

@arencambre -

That is interesting.  One thing I noticed is that the format you see for the statistical information is different from mine - maybe window size as I am on a computer and you may be on a tablet (that is beside the point at hand).

I decided to look up the difference between mean and average and found the following: "Average, also called the arithmetic mean, is the sum of all the values divided by the number of values. Whereas, mean is the average in the given data. In statistics, the mean is equal to the total number of observations divided by the number of observations"  

That has slightly different meanings.  Just out of curiosity, how many students took the quiz for each quiz for the results shown and how many were in the class?  I am wondering if there were (hypothetically) 100 students in the class, 1 student took the quiz in 3 minutes (180 seconds) so the mean that Canvas calculates is 180/100 or 1.8 seconds.  That calculation is actually an average whereas the mean would be the 180 - I am assuming that a quiz not taken is still counted in the sample with a time of 0.  Just throwing something out there to see if we can find any semblance of sense out of the calculation. (I think I have the mean and average determined correctly)

Ron

---

Um, that is one wild quote. It appears you got it from https://www.cuemath.com/data/difference-between-average-and-mean/ ? It's bizarre as it's pretty much saying the same thing twice while falsely alleging it's saying different things.

A mean is a form of an average. So are median, mode, and mid-range. But in common language, mean = average = sum of all items divided by count of items. If Canvas means something other than mean, it needs to change the word it uses.

For quizzes 1-3, all 113 students took it. For quiz 4, 109 students took it. I have a good sample size for all quizzes.

There is no way 113 students took quiz 3, a 19 question quiz, and averaged 3 seconds each. Two of the 113 students had a technical glitch, so I reassigned them an opportunity to retake the quiz. Their grades were fine, so it impossible for them to have warped the mean.

I am still coming up with no rational explanation for what this number is showing me. Even if we say "mean" is mislabeling the number, I am not clear what useful statistic this would possibly be.

[Ron_Bowman]

@arencambre -

I agree.  I sort of figured out that you had several students taking it when I noticed(or payed attention to the fact) that you had high scores, low scores and average score.  I looked at the instructor manual page and they state there that the mean value is an average value.

I have no idea why the calculations are off.  I was trying to see some results and found out that the statistics for a quiz can take up to 24 hours to be available(from the instructor guide page).

My recommendation would be to send the information off to canvas support with the 4 pictures you show, and for each one state number of students taking it and number of questions.  I would be interested in knowing what is going on with that.  I don't use new quizzes and I usually don't look at the stats for my quizzes.  I did go and look at the stats for some of my quizzes from this semester, and they all appear normal (i.e 17 minutes for the average time on a 30 minute quiz).

Good Luck

ron