Course Storage Report and Quotas

[vanzandt]

I hadn't found anything in the community that aggregated the information, so I thought I'd share some details from an interaction I had with Canvas Support while researching which content applies to which quotas. The info is all out there in different places, but my hope is this will provide a summary in one place.

The Course Storage report, offers two different values, "storage used in MB" and "sum of all files in MB". The tool tips for the Report offers this basic description:

"The storage used reflects all files that go against the courses quota. The sum of all files in the course is the sum of all files that are in the course regardless of impact to the courses quota."

So, "storage used" in a course is generally:

• All of the content that the instructor has uploaded to the course
• This is the number counts against the course quota that has been set

While the "Sum of all files":

• Also includes all of the student submissions, or documents uploaded in discussions, etc.
• This is the number that counts towards your overall account quota for your Canvas instance

In case you're now wondering how this impacts individual user quotas, since these files also appear in their My Files area...

• Files that have been submitted by students as attachments to assignments and graded discussions are not counted against their user quota. 
• Files that are uploaded to a course by a student that are not related to assignment or graded discussion submissions will count toward their user quota.
• Files that are uploaded to a course site by an instructor do not count towards their user quota.

If anyone has encountered other important information, or has found their experience to be contrary, please feel free to leave comments, so that we can make this a useful resource for other admins.

[kbickell]

Todd, very helpful, great resource!

Then there are institutional level quotas, which are slightly less clear...

[tdelillo]

I don't know if this helps or muddies the waters, but I've been musing for a month over a comment erinhallmark made in the Canvas Beta Release Notes (2017-09-05) Specifically:

"Files from previous course imports or course copies aren't included in the storage quota or files count. However, a course will need to have enough space to process the import file (the course should have enough free space that is at least double the size of the file). So only files directly uploaded to a course in course files have any significance to the course quota. If you were to download the course import file and directly upload it to the course files for safekeeping, that file would go against the course quota."

We had quite an internal debate about what this means, but the way I read it, you can copy content in but still have the full quota available to you. Because a freshly copied course will say "0% of 500MB used" at the bottom of the Files screen, and in Settings>Course Statistics>File Storage will also only register directly uploaded files. I've never quite understood it.

[vanzandt]

Thanks, Tracey. I don't think it muddies the waters at all. I think it's an important distinction, and helps clarify why someone may see a discrepancy from what I've stated above. I agree with you that her statement makes it sound like choosing "Copy a Canvas Course" when Importing Content will not apply to your course quota, but if you export the previous course and then choose to import the "Canvas Course Export Package", or any of the other options, that those would apply to the course quota. That's not what her quote says exactly, but it's what I think is intended. I guess we could test the theory fairly easily.

And hopefully erinhallmark‌ can respond here to offer a follow-up or add some clarity.

[vanzandt]

Thanks, and you're welcome!  Do you mean it's less clear what content/data applies to the institutional quotas, or it's less clear what the institutional quota actually is? And by institutional level quota, I am interpreting that as, the quota for the entire top-level account for the instance. Is that right?

In case it's the latter, that top-level account quota is defined, for us, within Internet2's Business Agreement that was made with Instructure.  I'm not sure how non-Internet2 groups have this defined.

Todd

[erinhmcmillan]

 @tdelillo ‌

The way I read it, you can copy content in but still have the full quota available to you. Because a freshly copied course will say "0% of 500MB used" at the bottom of the Files screen, and in Settings>Course Statistics>File Storage will also only register directly uploaded files. 

Yes, you can create a new course and then import a course export file. When used for the course import, the export file contents do not count against the quota.

vanzandt‌

I agree with you that her statement makes it sound like choosing "Copy a Canvas Course" when Importing Content will not apply to your course quota, 

Correct.

but if you export the previous course and then choose to import the "Canvas Course Export Package", or any of the other options, that those would apply to the course quota. 

No. Export packages used for a course import do not apply to the quota. My example was relating to if you were to upload an exported file directly into course files. 

Let's say you created a new course called History 101. You need to add content to History 101, so you choose either the copy content from another course option OR or the import a Canvas course export package option. With either option, History 101 statistics will show that the course is using 0% of your MB quota—both options are considered to be a way to copy a canvas course.

However, once you start adding content to course files, your quota will start to be used. Per my example (because it's a real example and people do this), let's say that you go to Course Navigation and click the Settings link, then click the export course link because you want to create a new backup of your course content. The export creates an IMSCC file, which would probably be at least several megabytes (depending on the size of your course). Let's say the file is 20 MB and it's called history-101-export.imscc. Then you decide you want to upload that file in course files—go to Course Navigation and click the Files link, click the Upload button, and drag/drop and/or search for history-101-export.imscc on your computer. After the file is uploaded in the course, your Files page will show you have used a small percentage of the MB quota for your course. The quota is also applied if you upload any other files into course files, such as PDFs or images or media files or any other content that wasn't already included for free when you imported the course.

Let me know if that information helps!

Erin

[kbickell]

Todd, our institution's storage quota is defined in our contract, so that's clear. I don't know how it works for Internet2 institutions.

When you click on Statistics at the admin level, it lists "uploaded" and "media" under file storage - yet neither is all inclusive. So you must contact your CSM to find out where you really stand. Workable but not ideal, especially when there are contract terms defining storage levels. Why we can't accurately track our own usage is a good question.

Then there's managing storage levels...we've been with Canvas since 2012, and as you can imagine our usage is high. Unfortunately there's no way to bulk download or archive courses to stay within our storage limits. Deleting courses is simpler, but not a real solution. Downloading courses one by one by one - well with more than 21,000 courses that's not an option either. I've asked repeatedly for some type of bulk archive or download solution, but so far nothing has developed. So we had no choice but to purchase more storage from Instructure. Not a horrible solution, but why we can't manage our own space is also a good question.

[vanzandt]

Thank you for the follow up. This is very helpful! We're only about year and half in, and we haven't had to pursue these issues yet, so I didn't know/realize that those figures for file storage under Statistics were not accurate. It's probably something like the Users number, where it's the number active within the last 30 days (or something like that).  I'm so glad I posted this thread, and really appreciate you, and everyone, sharing their knowledge on this topic.

I suppose you could get closer to your total with a course storage report for All Terms and then summing the "sum of all files in MB" column, but this still wouldn't include the user and group storage usage.

I also agree with you, that it would be great to have a bulk course archive/download tool. For us it would be helpful to have it by term, and/or by sub-account, as a zip of all the .imscc course exports.  I'm sure it would take forever to run, but for old data, that isn't changing, the process could even be run against beta or test instances if there was a concern around performance of the production environment.

Thanks again for your contribution to this thread!

[vanzandt]

Thanks Erin! This is a very helpful clarification, and it's great to hear that importing Canvas Course Export Packages will not count towards the course quota. Will this imported content count towards the overall Canvas instance/institutional quota?

[erinhmcmillan]

Hi, Todd,

It doesn't count against any quota you need to be worried about.

Thanks!

Erin

[gouldt]

Erin,

Thanks for explaining the quota thing. I am having a little problem importing content right now due to quota. I have a course which happens to be a Blue Print association. The Instructor had to add content (quizzes) directly in the live course because they wouldn't sync correctly. I need to replace our master course with all of the content from the live course. I wasn't sure how this was going to work, being a BP course and all. So I thought it would be best to export/import. When I went to import into our master course I couldn't, it said the course import exceeded the 250 MB limit. This confused me because our quota limit is 500 MB. So I tried migrating directly from the live course. At first it said it couldn't do that, due to the 250 MB limit, but it just completed the migration. So I think, when importing large course files, it works better to migrate than to import. I was getting ready to change the quota limit on this course when it started working. I did see that to use the import feature it is best to have more than double the amount of quota available in the course. So it's possible that the file I was importing was just over 250 MB. Yea, I know, ridiculously large:) Luckily I only have one of these to do, and all day to fix associated problems. 
Thanks for the information.

Tracey

[straussi]

In our conversations with our CSM, we were told that deleting courses will not reduce the total storage. Makes sense when you realize you can restore any deleted course with SIS csv or API calls. 

Our "Uploaded" File Storage number is close to what our CSM reported as our current storage, so the question is, how does the "Media" number fit in with that? Is that in addition to 'Uploaded', or a part of 'Uploaded'?

[vanzandt]

Thanks  @straussi , for contributing the additional detail that you received from your CSM.  

Your other question is a good one, and I don't know. I just posed it as a Comment on the https://community.canvaslms.com/docs/DOC-10803  guide. We'll see if they can provide a response.

[vanzandt]

HI  @straussi , Here is the response they posted in response to my question on the How do I view statistics for an account? guide...

https://community.canvaslms.com/people/jivedocs@instructure.com" style="color: #d54f37; border: 0px;...

Thanks for your question, and apologies for our delayed response as we've tried to find a comprehensive answer for you. As far as we understand, the "Media" number is included in the "Uploaded" number and contributes to the account storage limit. 

Hopefully that helps answer your question. Please let us know if we can help with anything else. 

-Allie 

[straussi]

Todd, thanks for the update!

- Ira

[bgibson]

I've had it explained to me that Canvas keeps track of files and doesn't actually make a 2nd or 3rd copy of that same file to place in other courses, so in essence there is only one file that is referenced multiple times.  Therefore there would be no additional storage required of Canvas for copying the content of a previous course, except for a minimal admin fee (in the database) to keep track of the file locations in the new course.  *Say I've got a 1 GB video file that is used across multiple sections, and multiple semesters and even over several years.  I'm guessing the total storage would be only slightly above 1GB no matter how many courses actually referenced or used that one file.  But I think Canvas would never be able to get rid of that file, because with the possibility of restoring it, they would still need a copy.  **If you could use that same file across multiple schools and keep track of it, wow (1 GB storage for 100 institutions using it multiple times x...)  ***If you could do that (keep track across institutions), it would be to Instructure's advantage to have content producers selling the same content to multiple institutions because there would only need to be one copy of the content.  

[swilson34]

How does this relate to students who need to submit large files for assignments? I occasionally hear that our digital media faculty find that large files can't be submitted, but I'm not sure that increasing the quota is a solution or that it might cause an unintended consequence elsewhere. 

Stephanie Wilson

[vanzandt]

When I have asked in the past, I have been told there is a 2GB individual file upload limit, though I've never tested this theory.

My understanding is that if it's a graded assignment submission or graded discussion forum post, it would just get tracked against the overall account storage quota. It should not bump the course against it's quota, or bump the user against their user quota.  If the student is uploading it to a non-graded discussion forum or another part of Canvas, then it would likely count against their user quota and that might restrict them. There are also Group quotas, in case that happens to be where the students are trying to upload their Files into Canvas.

I guess it really depends on the particular scenario where the problem is occurring. With a file that large, it may take a while, so I'd even say that the stability of the internet connection could come into play when uploading a submission.

Maybe others have more direct experience with troubleshooting these large file uploads, that they can share.

[swilson34]

Good point about the stability of the connection, it doesn't take much to foul up an upload. 

[erinhmcmillan]

Hi, Todd!

Just FYI, we published a Files Quota PDF that has a lot of this information in one place.

The PDF originates in the following lessons:

How do I view statistics for an account?

How do I view course statistics?‌

https://community.canvaslms.com/docs/DOC-10734-67952724475 

Thanks,

Erin

[kmatson]

Todd,

I like your summary but I think it is not correct. Any file a student uploads counts against their file storage. We now have students that can't submit to their assignments and graded discussion boards because they are out of room. How can a file I have to submit for an assignment be counted so I can't complete the course because I am out of room.

Canvas please fix this or tell us what we need to change.

Karen

[Stef_retired]

 @kmatson ‌, as outlined on page 3 of the Files Quota PDF, students' submissions to assignments and attachments to graded discussions do not count against their quota. If, for some reason, students are uploading those files to their personal user files first, they would quickly run up against the 50mB limit—but they don't need to do that. Have you asked Canvas Support to investigate the matter of students running out of room?

[Ray_Butler]

But even in the Files Quota PDF, it’s still unclear as to whether student submissions eat into the course or course owner space. All it says is this:

• Student Submissions: Any attachments added as part of a graded assignment submission are also copied to a student’s user files but are not counted against their user quota.

This refers only to a student’s storage space, rather than the course’s or the course owner’s. What about them? Surely as the submission is into the course, it has to consume some of the course quota (and if not, what other quota is it consuming instead?).

[nwilson7]

@Ray_Butler If the assignment is graded, it does not eat into anything, it is just Canvas space and not directly tied to any course or user. 

Ungraded assignments like a discussion can use up the student's space.  We have had faculty change discussions or quizzes to graded just so it did not count against the student so larger files can be shared (we have several digital art classes that upload large Photoshop files into discussions and were quickly filling up student space when they are ungraded).

Hope that helps!

- Nick

[TrinaAltman]

In relation to "Files that have been submitted by students as attachments to assignments and graded discussions are not counted against their user quota.", we found that the new RCE causes student files to count towards their user quotas even for gradeable items if they used the ‘Upload Document’ or 'Upload Image' button on the RCE toolbar to attach the files. We reported this to our CSM last fall and he has been pursuing it with the Project Manager over the RCE since then. There has been a bit of a delay as the Project Manager changed earlier this year after some initial progress with the former PM.

Trina