The Latest on the Known Issue Related to Course Files in Item Banks

[AllisonHowell]

Update: 11/13/2024

Great news! The long-term fix has been deployed as of November 12. 

Over the next few weeks we will continue monitoring this fix to ensure everything is stable. Once we are confident in the stability, we will begin the datafix that will go back to June 2023. We'll keep this blog post updated of the progress on the datafix as we know this is something you are anxiously awaiting. 

There is a Canvas account feature flag named "Use file verifier authentication for New Quiz file links" that we recommend stay enabled for the time being. We plan to remove this feature flag once we are confident in the stability of this fix. 

Update: 10/30/2024

Since our last update in late September, our engineering team has continued working diligently on the long-term solution for Image File issues identified in IA-100. Although we were hopeful for an October release, unforeseen technical challenges have pushed our anticipated release date back to November 12. We understand this is not the news you were hoping for, and we truly appreciate your patience as we work to get it right. Please know we’re committed to delivering a fully vetted and reliable solution that meets the needs of all of our customers, as quickly as possible.

We’ll continue to keep you updated as we approach this final release date. Thank you again for your continued patience and partnership as we work through this issue.

---

Thanks to all of you who have raised the importance of the course files in item banks issue and for following along in our previous blog post as we rolled out the short-term fix. We’re excited to share that the long-term fix for this issue is nearing code completion. However, after careful consideration, we’ve decided not to push for our original end-of-September production release date.

Given the size and complexity of this update, we believe thorough testing is essential to ensure a smooth rollout and a high-quality experience for all users. To allow for comprehensive testing, we’re adjusting the release timeline and now plan to move to production at the end of October. Once it is in production, we will also run a datafix for all affected files dating back to June 2023. 

Thank you for your continued patience and understanding as we finalize this important update.

[KathyPalm]

Appreciate the update!
Is there any possiblity for the fix to go back further than February 2024? Our institution first noted this issue in January 2024 so my cocern is it won't fix all the instances of this issue for us. Thank you!

[dbrace]

Resolving this issue needs to need to go further because not every Canvas course shell is reused every semester and that is because of a rotation/sequence that exists for certain pathways/certifications/degrees.

[AllisonHowell]

@KathyPalm @dbrace Thanks for the input! I'm talking with the team responsible about the possibility and timeline for a more comprehensive datafix and will update once we've got a resolution. 

[leward]

@AllisonHowell ,

Ideally the data fix should go back to June 2023 when the standard RCE was reintroduced in New Quizzes.  I too have found examples in question banks dating back as far as September 2023, although the largest number of cases occured in February 2024 and later.

[AllisonHowell]

Thanks everyone! After discussing with the team, we will do a data fix that goes back to June 2023. 

[KathyPalm]

Great news, thank you!

[James_Kocher_UF]

Is this now being pushed to after November, based on this just released annoucement?

https://community.canvaslms.com/t5/The-Product-Blog/Updated-Canvas-Deploy-and-Release-Schedule/ba-p/...

[AllisonHowell]

@James_Kocher_UF No, we still anticipate releasing this in October.

[dbrace]

Thank you for the update, @AllisonHowell.

I have two questions:

1. Will this blog post be exempt from the Canvas Community comment posting policy that does not allow comments after a period of time?
2. Is the new "Use file verifier authentication for New Quiz file links" preview flag documented in either a Deploy Note or Release Note at https://community.canvaslms.com/t5/Canvas-Releases/tkb-p/canvas-release?

-Doug

[leward]

@AllisonHowell 

Is this fix on beta so we can test it?  At the moment, when I try to launch NQ from beta I am seeing: 503 - Service Temporarily Unavailable. 

Also, @AllisonHowell,  I've noticed that when a Canvas team member adds an update to the original post, the update is not included in the Product Blog daily digest , which means they can be easily missed.  The only reason I saw your your update was because I had commented on this post, so I received a notification when @dbrace commented today.  If there's no way to force updates to the original post to be included in the daily digest, it might be helpful if Canvas team member posted the same text the original post (so it appears at the beginning), and also as a comment (so it is included in the daily digest).

[dbrace]

@leward,

I have been experiencing problems with "New Quizzes" in beta since (and I have reported it) the morning (USA ET time) Tuesday, October 29th.

I received a notice on Tuesday, November 5th that engineers were looking into it.

Earlier today, I tried again and received the same error message that you mentioned and requested an update.

-Doug

[isabel_anievas]

@AllisonHowell 

Any updates on the known issue related to course files (images and media files) in Item Banks? 

Are we going to be notified when the issue gets resolved? 

I have multiple (unpublished) new quizzes with this issue. I am still relying on the old Classic Quizzes version of these quizzes. I cannot create any more New Quizzes until this issue is resolved. I teach Spanish and I rely heavily on media files. 

Please, advise. 

Thank you!

[AllisonHowell]

Apologies for not being aware that the updates weren't coming out in the daily digest. Here is an update on the comment:

Great news! The long-term fix has been deployed as of November 12.Over the next few weeks we will continue monitoring this fix to ensure everything is stable. Once we are confident in the stability, we will begin the datafix that will go back to June 2023. We'll keep this blog post updated of the progress on the datafix as we know this is something you are anxiously awaiting.There is a Canvas account feature flag named "Use file verifier authentication for New Quiz file links" that we recommend stay enabled for the time being. We plan to remove this feature flag once we are confident in the stability of this fix.

[isabel_anievas]

Thank you @AllisonHowell and @leward 

I missed the update even though I check all the Forums regularly. 

I usually click on the link in my notifications and that brings me directly to the most recent comment. 

Posting the update as a comment is very helpful for those of us relying on notifications for updates.

Thanks again!

[AllisonHowell]

@leward @dbrace There was an issue with New Quizzes loading in beta for the IAD and DUB regions. The team deployed a hotfix and everything should be working as expected.

[dbrace]

Hi @AllisonHowell,

Thank you for the update.

Unfortunately, as of the morning (East Coast in the USA) of 2024-11-15, there is still a problem and it is the same problem (different than what is documented in this blog post so I apologize for bringing it up but I am in IAD or us-east-1 / cluster99) that I have been experiencing (which has been reported to Canvas Support) for almost three weeks.  In short, it is not possible to make a new New Quiz and build it. 

The process creates the "shell" of the new New Quiz but attempting to build it results in the same error message.

-Doug

[leward]

@AllisonHowell ,  Like @dbrace , I am still unable to access the NQ Item Banks tool and the NQ Build interface.  Can you let us know when it is available again for our Canvas instance (iu.instructure.com).

Many thanks!

Lynn

[MARISSASCHRADER]

I'm sorry to post this here as well since it doesn't directly relate to the specific topic, but I cannot test New Quiz/cannot access any New Quiz I have in Beta. I get a "503 - Service Temporarily Unavailable" error message. It tells me "Scheduled maintenance is underway. Check back soon." 

I have received this error for a while now and cannot test things I've been asked to test in Beta surrounding New Quizzes. I submitted a ticket earlier in the month, but it's still an issue.

[cms_hickss]

@AllisonHowell  Regarding the datafix that will be run after you are sure everything is stable, will you be announcing that before you run the fix?

[leward]

@AllisonHowell,

I just tested the replacement for file verifiers in quiz bank images and I noticed two significant problems.

1. When you save the question, the following pop-up still displays:
   "You are attempting to save content containing course links which may not work when used in other courses."
   
2. If you upload a file of the same name to course files with the intention of updating the image where ever it appears, the image is not updated in the question bank.  The ability to update a an image, or indeed any file in the files tool and have the update be reflected in all content in which the file is referenced is a behavior that end users expect.  It looks as though you are copying the image to a separate file store, and in the process, breaking the tie with the original file.  

[leward]

@AllisonHowell 

Another issue with the fix.  As far as I can tell, when a file or image is added to a quiz, students can see the image regardless of its publication status in the course.   I tried adding an unpublished image to a question and the image was visible to students when the bank was used in quiz. 

This doesn't bother me particularly because it seems unlikely that an instructor would add an image to a question and not intend students to see it.  But my understanding is that the verifier was removed from item bank images in February of last for this very issue.  Personally, I think that was a horrible decision.  Even more horrible was Instructure's willingness to allow broken image URLs in image bank questions to persist for almost a year.  

Add to this, the fact that the images in bank questions are updated not when the originals are updated and its hard to feel that this fix is not a step backwards.  What am I missing?

[TrinaAltman]

@AllisonHowell The guidance above states, "There is a Canvas account feature flag named "Use file verifier authentication for New Quiz file links" that we recommend stay enabled for the time being. " However, I am no longer seeing that Feature Option in our instances. I know it was there on on 11/18/24 when we checked our settings after the 11/16 Release, but it no longer appears and I haven't seen any notices about its removal (unless I missed something). Can you let us know what is happening with that? We had been using it in an attempt to test the core fix in our development environment. 

[leward]

@TrinaAltman It looks like the feature flag has been removed and the new approach to file storage for bank images is being applied to new bank images.  I don't see evidence that the data fix for broken image has been applied yet.

[leward]

@AllisonHowell,

Can we please get an update on when the data fix for broken file links will be run?   

[TrinaAltman]

Thanks @leward. I asked our CSM about the removal of the feature flag. It is apparently hidden but retained the state of the feature flag as set by the institution.

My understanding (again from our CSM) is the engineers are monitoring and verifying the new link style is stable before starting the migration process to move all existing file links to the new style. (I appreciate the caution in this regard.) If existing items are modified, they should have the new link style. (They also temporarily have a data-old-link reference to aid with rollback if a critical problem is encountered.)

That said, in our testing (which was earlier this month), our technician was able to see that an existing item bank item that was saved again appeared to have the new link style (plus the temporary data-old-link reference). However, when creating new item banks with new images, she was still seeing file verifiers within the src attribute, which shouldn't have been the case since our understanding from the engineers (via our CSM) is that the new style of file links (no verifiers) should appear in any newly created quiz/bank entries. We are currently waiting on clarification about that and how we can actually test the fix/new links.

It would be great if someone from Instructure (@AllisonHowell ?) could update this page to let us all know where things are at.

[leward]

@TrinaAltman,

We had the flag set to continue to use the old-style verifier, but when the flag was hidden, the links to images and files started reflecting the new format, where end users are given URLs with tokens that expire eventually (not sure how long they remain valid).  So, at least in our case, we can test the links in production.  I was able to test them in beta as well, but I only have time to do very superficial testing.  My testing seems to suggest that the links in rendered exams work for anyone who has the link (including unauthenticated users), but they do expire.  So, I guess it's the expiration of the token that provides some level of security.  As I noted above, once the question is saved, the file appears to be copied to a separate file store and no longer respects the permission of the file uploaded to or stored in the Course Files tool.  Also, if the original stored in Course File is updated, the version in the question is not updated, which is a different than the current behavior as well as the behavior for classic quizzes, which also stored a copy outside of the course.  

Hopefully,. @AllisonHowell will eventually chime in on whether these new behaviors are bugs or just a limitation that we'll need to learn to live with.

[AllisonHowell]

Hi everyone, my sincerest apologies for my absence on this thread. I have been moved to another team and was not aware that these had not been answered. I am looking into your questions now and will either answer them myself or have someone from the team chime in with more info. Thanks so much for your patience - I know this has been frustrating! 

[TrinaAltman]

@leward  Thanks so much for all the details. This was different than what our technician saw when she previously tested, so I asked her to test again in case things had changed (unannounced by Instructure to my knowledge, ahem). Indeed they had and we are now seeing the token you referenced. I encouraged her to do additional testing for some of the new concerns you reported as well as impact on course copies.

[TrinaAltman]

@AllisonHowell (or whoever from Instructure is now assigned to this),

We anxiously await your update to this post. It seems there have been behavior changes since the 11/12 go live. Please see the information/questions from leward.

Can you please tell us more about:

• The tokens, e.g., who can access them, how long they are valid, what happens after they expire, etc.?
• The changes in behavior reported by leward (e.g., file updates to existing bank/quiz resources)
• Impact of the changes on course copies
• Anything else we should know about how changes will impact previous instructor workflows and student access to quiz resources.

We really don't want to end up in another situation where students are unable to access the images in a live exam like we did when the file verifiers were removed earlier this year.

Thank you.