CANBASH Part 2 - Resty and Jsawk for Simple BASH Scripting using API
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
LAST TIME ON CANBASH -- Kent shows how to interact with Canvas using BASH scripting with an introduction to curl, json and APIs. Nanananana CANBASH!
CANBASH Part 2
INTRODUCING RESTY & JSAWK
Hello! It's been a while since I've posted in the Developer's Community, but today I would finally like to show part 2 to my CANBASH series that makes scripting much simpler. Using two tools found on GitHub, we can use the advantage of open source programming to make maintaining BASH scripts easier, and faster to write to perform specific tasks. These two tools in particular are Resty and Jsawk.
Resty is a tiny command line REST interface for bash ( and zsh ) that provides an easy to understand wrapper for curl that we used last time. This way we have something doing most of the heavy curl work for us. You can check out the actual source code and find out it's just a bunch of functions that are added to your shell. This is great because it's not something separate we have to deal with, it's just "built-in" to our BASH shell once we source it. This can be done in your ~/.bashrc or called in the script itself. Jsawk is a fantastic solution to deal with JSON documents using a familiar AWK versatility. You can do a ton with this tool and when combined with RESTY, you can do some really cool things. And this is what this tutorial is about.
Once you have installed Resty and Jsawk from github or some package manager ( like homebrew for OSX using command "brew install jsawk"), we can begin to work on building our scripts. Since I will be sourcing Resty inside my shell script for this example, I am also going to show the visual hierarchy of the file structure so that you can produce something similar and can understand how to source RESTY correctly.
I am using Jsawk v1.4 and Resty v1.5 for this demo.
~/Desktop
└── CANBASH
| File Permissions | File Name
|- - - - - - - - - - - - - - - - - - -
├── 644 | -rw-r--r-- | resty
├── 700 | -rwx------ | make_modules.sh
└── 644 | -rw-r--r-- | modreport.txt
I added the file permissions numerically and in their normal form. If you want to list your permissions numerically too, then you can use this command ( I recommend making an alias with it to use it all the time 😞
ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/) \
*2^(8-i));if(k)printf("%0o ",k);print}'
You can change your file permissions using the chmod command to give a script execute privileges for example.
chmod 700 your_script.sh
Now that your have your structure ( with jsawk installed ) we can write a simple script (that is heavily commented) that will initialize Resty with our API token and correct canvas domain...
#!/usr/bin/env bash
# Initialize Resty with domain "ttps://your.domain.edu" & Your Token "XXXX~XXXXXX~"
# Syntax is important for this. Don't forget the /api* at the end of your domain
# Don't forget Authorization: Bearer, as this is your token header
. /path/to/file/resty -W 'https://your.domain.edu/api*' \
-H 'Authorization: Bearer XXXX~XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
# Prompt the user for input
echo "WHAT IS YOUR COURSE ID?"
read -p "➜ COURSE ID: " ID
# Return Course Name given Course ID
echo "➜ COURSE NAME: $(GET /v1/courses/${ID} | jsawk 'return this.name')"
# Return Enrollment Status given Course ID
echo "➜ ENROLLED: $(GET /v1/courses/${ID} | jsawk 'return this.enrollments' | jsawk 'return this.type')"
Once you have navigated to the approbate directory where you made your file structure with your script, we can run it and test to see if it works. Here is an example output, showing what it should look like when it is working:
WHAT IS YOUR COURSE ID?
➜ COURSE ID: 4788
➜ COURSE NAME: Kent's Sandbox
➜ ENROLLED: ["designer"]
Notice the simple output and how jsawk can easily parse our the information that is elegantly called with resty. We provide the argument "ID" and then pass this along our script and then return the output of that GET request ( which gives us a json document ). What would happen if you got rid of that fancy jsawk stuff? You get a mess of json that is very well documented. Picking out the appropriate fields is easy enough to extract relevant information.
As shown for "ENROLLED", jsawk can parse our information down a the JSON document structure very easily. Below is a sample JSON response for enrollments to show how the command works on a more visual level by piping the output of the GET response using " | python -m json.tool" ...
➜ ENROLLED: {
"account_id": 1,
"apply_assignment_group_weights": false,
"calendar": {
"ics": "https://your.domain.edu/feeds/calendars/course_xxxxxxxxxxxxxxxxxxxxxx.ics"
},
"course_code": "STUBOX",
"default_view": "syllabus",
"end_at": null,
"enrollment_term_id": 68,
"enrollments": [
{
"enrollment_state": "active",
"role": "DesignerEnrollment",
"role_id": 53,
"type": "designer"
}
],
"grading_standard_id": null,
"hide_final_grades": false,
"id": 4788,
"is_public": false,
"is_public_to_auth_users": false,
"name": "Kent's Sandbox",
"public_syllabus": false,
"restrict_enrollments_to_course_dates": false,
"start_at": null,
"storage_quota_mb": 1023,
"workflow_state": "unpublished"
}
jsawk 'return this.enrollments' | jsawk 'return this.type' will search for the "enrollments" field and then parse out the information from the "type" field for the enrollments section. Very straightforward, though it can be a little confusing from time to time. I recommend using the API documentation, as it is fantastic for figuring out things without having to run JSON tests and search for stuff "by hand".
So far I haven't covered the idea of creating a menu for users to do things on the command line, and it probably is most useful in situations like these where you may be constantly doing something to a course and want a tool to remain active and do things as you're working on the course ( but with an option to quit still available ).
Below is an example of a menu in a BASH script that will perform a task based on user input.
PS3='What do?: '
options=("${BOLD}CREATE${NORM}" "${BOLD}QUIT${NORM}")
select opt in "${options[@]}"
do
case $opt in
"CREATE")
echo "Ok!"
;;
"QUIT")
echo "Bye!"
# add a break to get out of options
break
;;
*) echo invalid option;;
esac
done
I think it's time for it to be all put together, and we will have a script that will create modules ( one at a time ) to show an example workflow.
#!/usr/bin/env bash
# CANBASH | Canvas API + BASH Scripting
# Using RESTY + JSAWK
# RESTY : https://github.com/micha/resty
# JSAWK : https://github.com/micha/jsawk
# clear screen
clear
# formatting stuff
BOLD=$(tput bold)
NORM=$(tput sgr0)
UNDER=$(tput smul)
UNDERstop=$(tput rmul)
NONE='\033[00m'
# display CANBASH banner, because it's not like you can stop me
echo " ${BOLD}
__ __ __
/ /\ |\ ||__) /\ (_ |__|
\__/--\| \||__)/--\__)| | ${NORM}v2 - w/ ${UNDER}RESTY${UNDERstop} & ${UNDER}JSAWK${UNDERstop}
...give it a rest(y) already"
# prompt the user for input
echo
echo "WHAT COURSE WOULD YOU LIKE TO ADD MODULES TO?"
read -p "➜ ${BOLD}COURSE ID${NORM}: " ID
# IMPORTANT STUFF HERE ###########################################################################
#
# Alright, so here is the real part that makes things easy for us: RESTY
# It is a A tiny command line REST interface for bash and zsh written by Micha Niskin
# Read more about it here ---> http://github.com/micha/resty
# You need to have an API token for this to work too, and a course ID number for a course you have have
# appropriate permissions for to be able to add modules to a course.
# your domain goes here -- format : https://your.domain.edu/api* -- Don't forget the api part!
. /path/to/file/resty -W 'https://your.domain.edu/api*' \
-H 'Authorization: Bearer XXXX~XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
#
########################################################################## NO MORE IMPORTANT STUFF #
# it really isn't kosher to indent here, but I do it anyway.
# List course details.
echo "➜ COURSE NAME: $(GET /v1/courses/${ID} | jsawk 'return this.name')"
# List course status.
echo "➜ PUBLIC?: ${BOLD}$(GET /v1/courses/${ID} | jsawk 'return this.is_public_to_auth_users')${NORM} "
# Storage limit.
echo "➜ STORAGE: $(GET /v1/courses/${ID} | jsawk 'return this.storage_quota_mb') mb"
# Enrollment.
echo "➜ ENROLLED: $(GET /v1/courses/${ID} | jsawk 'return this.enrollments' | jsawk 'return this.type')"
# List modules.
echo "➜ ${BOLD}CURRENT MODS${NORM}: $(GET /v1/courses/${ID}/modules | jsawk 'return this.name' | python -m json.tool | sed 's|["{},]||g' | sed 's/\[//g' | sed 's/\]//g')"
echo "➜ ${BOLD}MOD IDs${NORM}: $(GET /v1/courses/${ID}/modules | jsawk 'return this.id' | python -m json.tool | sed 's|["{},]||g' | sed 's/\[//g' | sed 's/\]//g' )"
echo
echo "${BOLD}⤱${NORM} COURSE MODULES"
# list modules with ....
# CRAZY SED PARTY
# Output is important, so let's learn how to output to simple text files
# create report file
touch modreport.txt
# do other stuff
GET /v1/courses/${ID}/modules | jsawk 'return this.name' | python -m json.tool | sed 's/\]//g; s/\[//g; s|["{},]||g; s/^ *//; s/ *$//; /^$/d; /^\s*$/d' > mods.txt
GET /v1/courses/${ID}/modules | jsawk 'return this.id' | tr ',' '\n' | sed 's/\[//g' | sed 's/\]//g'> ids.txt
paste mods.txt ids.txt > modreport.txt
awk '{printf("➜ MODULE %2d : %s\n", NR,$0)}' < modreport.txt
# leave mod report, otherwise all "modreport.txt to the line below with other ".txt" files...
rm mods.txt ids.txt
echo
# let the fun begin
echo "[ ${BOLD}M${NORM} ] THE ${BOLD}MODULATATION STATION${NORM} FOR ${BOLD}EDUMACATION${NORM} "
echo
# show users options, and then do thing
PS3='What would you like to do ?: '
options=("${BOLD}CREATE${NORM}" "${BOLD}QUIT${NORM}")
select opt in "${options[@]}"
do
case $opt in
"${BOLD}CREATE${NORM}")
echo
echo "${BOLD}+${NORM} ${BOLD}CREATE MODULE${NORM}!"
read -p "${BOLD}MODULE NAME?: ${NORM} " cName
#echo -e '\t' "NAME?" ; read -p "${BOLD}>>${NORM} " cName
echo
echo -e '\t' "Creating ${BOLD}$cName${NORM} with ID of... "
echo -e '\t' "ID: $(POST /v1/courses/${ID}/modules -d "module[name]=$cName" | jsawk 'return this.id')"
echo
;;
"${BOLD}QUIT${NORM}")
clear
echo "Bye!"
exit
;;
*) echo invalid option;;
esac
done
And that's really about all there is to it. For formatting purposes I find these sed commands useful when combined with the python json tool format:
command/jsawk return stuff | python -m json.tool | sed 's/\]//g; s/\[//g; s|["{},]||g; s/^ *//;
# `s/^ *//` => left trim
# `s/ *$//` => right trim
# `/^$/d` => remove empty line
# `/^\s*$/d` => delete lines which may contain white space
# s/\]//g => I hate right brackets
# s/\[//g => I hate left brackets
# s|["{},]||g => I hate json stuff left behind
One note: modules will be saved to a text file called "modreport.txt" and can be deleted easily in the script if you choose that option. I like having documentation myself, and you can store other information into reports like these very easily by piping/appending information to documents with proper formatting.
Until next time, bye!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
