[ARCHIVED] SIS Import Malformed CSV (Hidden Characters?)

[Milty]

Hello Everyone,

I have a csv file that is extracted from our SIS (Infinite Campus) that triggers the "malformed csv" error message when trying to import with the SIS Import feature in Canvas.

I have used the documentation below to make sure that the csv file is actually formatted correctly.

https://community.canvaslms.com/t5/Admin-Guide/How-do-I-format-CSV-text-files-for-uploading-SIS-data...

https://asuprepaz.test.instructure.com/doc/api/file.sis_csv.html

Here is where it gets interesting. If I open the csv file and make no changes but save it using excel. The file size decreases by about 80 kb. Going from 580 kb to 500kb. The file can then be imported without triggering the "malformed csv" error. This makes me think there may be some hidden formatting characters that I cannot see.

Has anyone else encountered this issue or have any possible solutions?

[mzimmerman]

Hi Milty,

I've definitely seen this.  My understanding is that there are at least a couple of different underlying formats ("CSV" vs. "CSV UFT-8"), and depending on which platform the file was originally created on, you can end up with the wrong one.  Unfortunately, I always forget which is the right one 😂

Mike

[Milty]

Ya, originally the extract was set to use ASCII character set. I changed it to UTF-8 but still no luck. 

I also have 5 other extracts with the same settings that are all working fine. I'm a bit at a loss really. I even made a new extract instance from scratch just in case I was overlooking something weird or the settings for the instance got corrupted or something.

[Milty · Accepted solution]

It turns out some of the data that was being imported had accented characters. So while the settings were all correct the data in the .csv file or the source needed to be cleaned better than it had been.