RCE breaking LaTeX expression entered into a page if you view the page in HTML mode and other misbehavior

[maguire]

If one enters the equation that @James used in https://community.canvaslms.com/t5/Canvas-Question-Forum/What-is-latest-Update-on-LaTeX-accessibilit...

Directly into RCE with

\ (\small{(x+1)^{10}=x^{10}+20x^9+180x^8+960x^7+3360x^6+8064x^5+13440x^4+15360x^3+11520x^2+5120x+1024}\ )

rather than using the RCE equation editor. [The space after the \ is to avoid the expression being converted in this message.]

When you save the page - it will now display properly with MathJax.

However, if you edit the page and go to RCE's HTML view you will see

 <p>(x+1)10=x10+20x9+180x8+960x7+3360x6+8064x5+13440x4+15360x3+11520x2+5120x+1024\small{(x+1)^{10}=x^{10}+20x^9+180x^8+960x^7+3360x^6+8064x^5+13440x^4+15360x^3+11520x^2+5120x+1024}</p>

If you now go back to the regular view, you see the text and the LaTeX - but the \( and \) have been removed! Definitely not the expected behavior. 

This seems to be a problem due to RCE misbehavior.

Additionally, if you enter in HTML mode using RCE unicode strings such as the following:

<p><span>&#x1D49C;&#xFE00;</span> chancery style; # MATHEMATICAL SCRIPT CAPITAL A &#x1D49C;</p>
<p><span>&#x1D49C;&#xFE01;</span> roundhand style; # MATHEMATICAL SCRIPT CAPITAL A &#x1D49C;</p>
<p><span>&#x2131;&#xFE00;</span> chancery style; # MATHEMATICAL SCRIPT CAPITAL F &#x2131;</p>
<p><span>&#x2131;&#xFE01;</span> roundhand style; # MATHEMATICAL SCRIPT CAPITAL F &#x2131;</p>

When you exit HTML mode, RCE removes the variation selectors and just puts the letter in. However, you now do not have the variant of the character that you expected. If you now go to HTML mode, you see the character and not the &#x1D49C; or &#x2131; and the variation selectors are all gone.

Being unable to specify the variation leads to very peculiar behaviors where the same character will be rendered in different ways in different places in the content! This is because the browser is trying to guess which variant to display. In LaTeX notation, the character you would expect from \mathcal{L} can become the character generated by \mathscr{L].

Has anyone else seen these (mis-)behaviors?

[chriscas]

Hi @maguire,

I'm not an expert in this area at all, but I'm going to tag @James, who I know teacher math and is quite the wizard (in my opinion) with mathematical display options in Canvas.  I can't guarantee that he'll be able to reply, since we're all just here contributing things to the community in our free time, but if he does see this and has time available, I'm sure he'll provide you with a great answer!

-Chris

[maguire]

Locally, the campus Canvas administrators added a note to their page about putting equations into Canvas pages, saying:

Note! LaTeX added directly in the text can be ruined if you switch to the HTML view. If you have to switch between normal text editing and HTML editing, we recommend you use the equation editor instead.

They had also put a request into Instructure about the problem, but as far as I know, have not heard that the problem is being addressed.

That going into an out of editing in HTML mode leads to irreversible damage would seem to be a design flaw.

With respect to the variants: My suspicion is that there is an error in the class: class Sanitize; module Transformers; class CleanElement

in the file canvas_sanitize.rb where REGEX_DATA_ATTR is removing things that it should not; hence, it is losing the variant information. 

Personally, I would prefer to have the editor invoke an external editor - such as emacs and not do the sanitizing that RCE tries to do. I understand that it is trying to enforce accessibility and other constraints - but I find that it often gets in the way of instructors' efforts to present information as they wish and need to for their students. I also understand that the local administrators do not want to have to install custom javascript, stylesheets, etc. - as they may encounter problems and this adds to the difficulty of supporting these custom changes. However, having a system where the symbols shown change depending on the browser's guess of the context does not make sense. The inability to ensure that the symbols shown are the expected symbols is quite important.

Chip