How to Archive Courses via Python

[lmsstaff]

This blog describes how to archive courses using a Python class and a mapping file.

Start thinking about archiving your courses and rolling them over into an Archive sub-account with several archive terms.  What if you could make the terms more dynamic so the names reflect some sort of offset from the current school year?  What if each archive term could roll into the next one until the courses are so old that they just exit the system?  Oh, and if you are using Blueprints, what if those associations could be dropped during the archive process? That would be so awesome.

Below are the instructions and code for making this happen.  You will need to download the files from here.

Let's begin with the configuration file. 

Open up the EpsITCanvasArchive.ini config file.  Some explanation is in order for clarity:

The first thing you will want to do is create an Archives sub-account.  Get that account id (from the URL) and assign it to the archiveAccount parameter.  This is where all archived courses will reside and is a much cleaner approach than having archived courses strewn throughout all sub-accounts.  Once that is done, create your archive terms and give them a SIS id that you will use for mapping.

There is a setting for what school year you wish to archive, as well as other limit values that can be changed if desired.  You may choose not to conclude courses when they are being archived.  Set the concludeCourses parameter to reflect this.  NOTE:  We have found that concluding courses is very bad practice so use at your own risk!  The mapFile parameter is the physical location of the very important JSON mapping file.  It is central to where archived courses will be moved.

Only courses with a SIS id will be archived.  As well, if the course SIS id matches the noArchivePattern regular expression pattern, then the course will not be archived.  In this example, all courses that end with the word .STUDENT will not be archived.

The archive process includes a debug mode as well.  Setting this to 1 (True) allows you to test against your test or beta instance. 

Finally, the runStatus parameter must be set to 0 (False) for the script to fully execute.  This is a safety measure, as running this process more than once per year is probably not desirable.  After the script finishes the archive process (barring any exceptions being thrown), the script sets this parameter to 1 (True).

Secondly, we need to configure the mapping:

Open up the ArchiveCanvasMap.json file.

All sections are required, except for the comments.  The term_map section is for making your term names dynamic.  These terms must be set in your Canvas instance first with the same SIS id.  In my example, all of our archive terms are A1 - A4.  The number stands for the year offset from the current school year. 

	"term_map": {
		"A1": "Archive <YYYY1>",
		"A2": "Archive <YYYY2>",
		"A3": "Archive <YYYY3>",
		"A4": "Archive <YYYY4>"	
	},

So, A1 is current year - 1, A4 is current year - 4 years.  You can have as many archive years as you like, as long as they are also configured in your Canvas terms.  For the term names, there are several patterns that you can use to ensure the name reflects the current year offset.   For example, if I wanted ARC 19/20 as my archive term name for last year (assuming we are archiving the 2020 year), then my pattern above for "A1" would be ARC <YY/YY>.  I don't need to provide a number as an offset because the script knows that this format would only make sense with those years.  This same logic applies for 19-20 (<YY-YY>).  You must include the '<' and '>' symbols in the pattern so the script knows that it is a replacement value and not a literal value.  Hopefully this will clear up any confusion:

            <YYYY> = 4 digit year
            <YY> = 2 digit year
            <YY-YY> = 2 digit school year start-stop
            <YYYY-YYYY> = 4 digit school year start-stop
            <YY/YY> = 2 digit school year start/stop
            <YYYY/YYYY> = 4 digit school year start/stop
            <YYYYn> = 4 digit year with negative year offset (YYYY3 = current year minus 3 year offset)

Use the patterns to enable a dynamic name each year that the archive process is run.  Choose what works best for you.  The archive_map section allows you to map the "from" term (left) to the "to" term (right).  You can map any term to another. 

	"archive_map": {
		"A4": "",
		"A3": "A4",
		"A2": "A3",
		"A1": "A2",
		"YEAR": "A1",
		"S1H": "A1",
		"S2H": "A1",
		"S1M": "A1",
		"S2M": "A1",

Of course, each term must exist in your Canvas instance.  The only exception is that the oldest archive term maps to an empty value.  This map tells the script to simply delete these courses. 

The accounts section is simply a list of the accounts that get their respective courses archived.  The account id must exist in your Canvas instance.   You get the id from the URL or my hovering over the sub-account name. The associated name is only for logging purposes and can be any value really.  During the archive process, the log file will write the accounts id and value to the log.

	"accounts": {
		"44": "CHS",
		"46": "EHS",
		"58": "JHS",
		"67": "SHS",
		"82": "OHS",
		"122": "ONL",
		"124": "SUM",

For the archive naming patterns to work, this function needs to be included in the code somewhere.   The classEpsUtility.py class file that is included in the download package has this function included.  For the sake of brevity, here is the function:

    def ArchiveConfig(self,archive_year):
        """
        Sets variables for the Canvas Archive class.
        Doing it this way so variable substitution will work without hardcoding into the main class.
        To add new variable substitutes:
            1. Create a variable that is configured off of the archive year
            2. Add a key to the dictionary below and also the variable as the value
        @Param archive_year: Int
        @return: Dictionary
        """
        try:
            YYYY = archive_year
            YY = int(str(YYYY)[2:])
            YY_YY = f'{YY - 1}-{YY}'
            YYYY_YYYY = f'{YYYY - 1}-{YYYY}'
            YY__YY = f'{YY - 1}/{YY}'
            YYYY__YYYY = f'{YYYY - 1}/{YYYY}'
            return {'<YYYY>': YYYY, '<YY>': YY, '<YYYY-YYYY>': YYYY_YYYY, '<YY-YY>': YY_YY, '<YYYY/YYYY>': YYYY__YYYY, '<YY/YY>': YY__YY}
        except:
            EpsException(__file__)

The archive year that is set in the configuration file, is passed to this function at run time.  From that year, the variables are set within the class so they can be referenced and substituted later when the mapping file is loaded.  It's a pretty efficient and loosely-couple way to approach the custom problem.

The classes classEpsConfiguration.py and classEpsException.py classes are included so you can see the entire package as a whole.  They are not required to run the archive class but if you wanted to build out your own package, the code is already there for you.

Lastly, look at the main class:

If you open up the class EpsItArchiveCourses.py, the most important portion of code is probably this one:

# set the authentication header
self.header = {'Authorization': f'Bearer {cfg.canvas_token}'}

After all of the archiving is done, the final act of the script is to ensure all Blueprint course associations are disassociated.  This is quickly accomplished by having the Archives sub-account so it is a single call to the API to find all associations.  Quick and simple.  There is even a log file generated that details all of the archiving actions.

That's it.  At the end of the road will be a nice log file that details all of the actions that occurred during the process.  If any warnings were generated (for example, bad course data or a failed API call), they will appear in the log.  Reach out if you read this article or have questions.