Will creators embed an AI detector into Canvas?

[dubliners099]

I've just busted my first student for using ChatGPT to generate an assignment. Clearly this will become more and more prevalent. My question is this--will Canvas embed an AI detector into their LMS? I would love to see this. 

Currently, I'm using https://openai-openai-detector.hf.space/ to catch this, but any LMS which embeds an AI detector would clearly have a leg up on the competition. (See screen shot of the kind of quick report that openai provides). 

Ideally, it should stop a student from submitting something with a certain percentage (or higher) that their work is fake, and ask "Are you sure you want to submit this?" with an option to back out of the assignment submission before they face real consequences.

At my college, our student handbook has a general statement that indicates submitting any work that isn't the student's own is academic misconduct. I strengthen that by making my own course policies which state that using any AI, including ChatGPT to generate or write one's assignment will result in a zero and a write up with the college. A second situation will result in a zero (F) in the class and a second write up with the college. At my college, three write ups for academic misconduct results in suspension or expulsion. (Or at least that what our student handbook claims.)

Oh, and yes, I do think that Elon Musk and his buddies are evil.

[Chris_Hofer]

Hi there, @dubliners099 ...

Your posting sounds a lot like a Feature Idea to help enhance the Canvas LMS (Learning Management System).  At this time (until the beginning of February), all Feature Ideas have been temporarily locked down (creating new ideas, commenting on existing ideas, and giving a star rating to existing ideas) so that some maintenance can be done on the entire Feature Idea process.  But, when the Feature Ideas space re-opens, you should be able to submit this as a Feature Idea.  Also, since you are a New Member, you may need to do a few more things like post replies to "rank up" (similar to ranking up in video games) so that you can submit new Feature Ideas.  You can read more about what is happening with Feature Ideas by going to: Idea Conversations: The Path Forward.

So, if you can hold on for a few more days yet, then the Feature Idea space should open back up.

Hope this helps a bit!

[chriscas]

I just want to add a quick personal reply about the detectors...

While I definitely do not want students to cheat or have an AI generate work they are supposed to do on their own, I think the detectors can be very problematic.  Plagiarism detectors offer some "hard" evidence that exact content was found in a book/webpage/etc...  A student can say they didn't get wording from those sources, but there is no question that it exists.  The AI detectors, as shown in the screenshot attached by @dubliners099, are giving somewhat of a confidence level that the text came form an AI.  On the surface this seems great, but there's really no hard evidence, and the numbers are not straight 0 or straight 100%.  In the screenshot, it's very likely an AI wrote whatever was being checked, but there is still a small chance it did not.  How would students be able to protest if they actually did write the assignment themselves?  There's also the fact that ChatGPT and other services like it are going to be constantly improving.  Things that detectors look for today will not be good markers tomorrow.  It seems that it will be a constant arms race, and setting up a confrontational environment between teachers and students.

I'm sure there will be external partners that ill offer AI detection just like they offer plagiarism detection today for schools that are interested, but I would rather that Instructure did not go down that path themselves.

-Chris

[TrishaMeyer1]

A low-tech solution to this issue is to create assignments that require references to content that is accessed in the class and cannot be replicated by anyone - or anything - who/that wasn't present in the class. An AI can search all the content in the world, but only your students know about that one discussion that happened in class or that unique perspective you used when designing the lesson. 

[holimoch]

I used the AI checker you referenced (https://openai-openai-detector.hf.space/), testing it out with two different texts. 

The first text is a poem I wrote 25 years ago, never published. The AI checker result was 99.38% Real.

The second text is a ChatGPT AI response to the question: "How can an aviation tech use drawings?" The AI checker result was 70.51% Real.

I'm thinking the AI checkers might improve over time, but I'm skeptical of them currently.

For now, I think teachers might ask their students to write during class. I've also learned that some teachers are asking their students to analyze an AI response regarding their content.

[AllysonStanford]

So have been using Turn-It-In in my HS - ENG. I also teach a Dual Enrollment Freshman Comp. class using CANVAS. The Turn-It-In program has added an AI feature, but it is sensitive to almost any internet language (AI). Two of my students used Grammarly for their essays. The whole paper was fed through Grammarly, it was then flagged in Turn-It-In as 100% AI. These students are not the type to cheat, and I was startled at the result. After discussing this score with them we realized that it must have been flagged with Grammarly. Not sure how any program would be able to be accurate in this process. 

[mewitt]

Grammaly now has AI text generation built into it. I had a similar issue with a student that was flagged through Turnitin and said they only used Grammarly. I did a little digging and found they just implemented this into their system. Here is a link to more info: https://www.grammarly.com/grammarlygo

[dubliners0999]

Turnitin.com's rep came to my college and indicated that use of Grammarly Premium (paid) does come up as 100% AI even if the student didn't use all the writing assistance used in the premium package. 

[mewitt]

Interesting. Thank you for clarifying. That's helpful to know and I fear this will yield a lot of false positives as many of my students use Grammarly.

I wound up giving the student the benefit of the doubt. I recently installed Grammarly's Chrome Extension to test it and the AI text generation built into it is impressive and I wonder how many students will wind up using that without thinking through it.