[webkit-dev] MathML layout refactor proposal

[Frédéric WANG]

Hi Dave,

Thank you for your message. It's good to see that you are still
interested in what's happening in WebKit MathML. I believe we agree with
the goals you state, however most of your claims on flexbox VS a
separate renderer class seemed to be assumptions you made 3 years ago.
People (including me) have tried to maintain the MathML code after you
left and these assumptions turned out to be at least inaccurate. With
Alejandro's approach, I believe that in the upcoming months we can at
the same time get a good MathML implementation and address all of
reviewer's concerns on your original code. At the moment, I do not think
there is an alternative proposal with these features, and even less
someone willing to work on any other approach...

More replies inline.

Le 23/12/2015 08:11, dbarton at mathscribe.com a écrit :
> 1. Ideally, we want most webkit developers to not even have to think about MathML. If you make some change to the rendering and layout code and it doesn't break non-MathML layout features, including flexbox, then ideally that should almost always mean that MathML layout is still ok.
However when a legitimate change elsewhere does break something in
MathML (Alejandro mentioned one bug but I have seen several of them) it
becomes a burden for the non-MathML developers. This is especially true
because it's hard to understand the code given the complexity of the
current flexbox-based implementation and the vagueness of the MathML
spec (which we are trying to solve by using
http://www.mathml-association.org/MathMLinHTML5 and
http://tests.mathml-association.org/).

Using an independent renderer class allows to isolate the MathML
implementation and reduces the risk of breaking the MathML code while
changing another part of the code. This has been confirmed with the
experience in Gecko (actually, such an issue only happened twice in
Gecko in the past decade and that was due to changes in graphics and
font modules, not layout).
> 2. Conversely, it would be ideal if the MathML layout code didn't have to worry about a lot of non-math things - floats, columns, margin collapsing, internal RenderBlock voodoo, etc. We just want good fast MathML layout, that integrates well with HTML, CSS, and JavaScript.
By using our own renderer class we can actually ignore CSS properties
that we consider irrelevant (see
https://github.com/MathML/MathMLinHTML5/issues/9). As Alejandro said in
his initial message, we now only care about math features and not about
the whole flexbox rules. Hence that's less code executed and we hope it
would actually improve performance.
> 3. MathML must introduce few or no bugs, and especially no security bugs.
With a separate renderer class, the layout code essentially becomes
arithmetic computations (of box coordinates and size). This is more
reliable than manipulation of anonymous renderers and style that need to
be kept updated with respect to dynamic tree/style changes. See for
example the current implementation of mmultiscripts which is just crazy
(http://trac.webkit.org/browser/trunk/Source/WebCore/rendering/mathml/RenderMathMLScripts.cpp).
If I understand correctly, one of the improvements you obtained by using
flexboxes was that you removed some of the (potentially unsafe)
manipulations of margin / border style that used to be necessary for box
alignments. With the new approach, we go even farther and expect to
remove all the anonymous renderers and style, so that these are no
longer a problem.
> At the end of 2012, the webkit MathML code was around 2,500 lines (admittedly it didn't implement all of MathML). In comparison, the MathPlayer plugin for Internet Explorer, MathML implementation inside Gecko, and the MathJax javascript library, were each about 20,000-40,000+ lines of code. Another comparison is that the webkit rendering/svg source code folder then contained about 17,700 lines. If we try to implement MathML from scratch, sort of like SVG, then I think it will take many developer-years to do, which frankly may never happen.
I'm not sure the comparison is really relevant, given the limited MathML
implementations in WebKit and the fact that the coding style are very
different (e.g. MathJax is written in Javascript and has various hacks
to workaround CSS bugs from old browsers, Gecko has a 80 characters
limit for lines and has fallback code when a MATH font is not
available). Instead, let's compare the two approaches in WebKit. The
flexbox-based implementation is really naive and just aligns the edges
of base and scripts (see attachment integrals.png) ; implementing
mmultiscripts that way was overly complex as I said above. As a
comparison, in only a single day, Alejandro and I were able to get an
implementation of msub/msup/msubsup/mmultiscripts using the TeX rules
and a decade of parameters from the OpenType MATH table (see
http://www.mathml-association.org/MathMLinHTML5/S3.html#SS4.SSS1).
Moreover, RenderMathMLScripts.[h|cpp] went down from 631 lines to 454
lines i.e. a ratio of 72%.
> MathML may not absolutely need all these, but several (like changing the order for mroot) are very useful, and outlawing floats and columns and margin collapsing is great for avoiding bugs or unintended interactions with other code. So again by using flexbox, I think MathML can be thousands of lines instead of tens of thousands of lines, and will need a lot less new tricky lines to avoid buggy interactions with existing layout features and voodoo.
AFAIK, neither Gecko nor MathJax rely on flexboxes but just absolutely
place the math boxes (as we want to do in the new approach) and they are
able to get good math rendering. With hundred of lines, you can do a
pure CSS implementation (cf
https://github.com/fred-wang/mathml.css/blob/master/mathml.css) and
that's fine as long as you only expect low quality and limited support.
However, users want high-quality rendering similar to what is done in
TeX or Microsoft Word and so we need more flexibility to position math
boxes. Like for mmultiscripts, you can probably always add more and more
nested anonymous boxes but then the code becomes even more complex and
we spend too much time maintaining the integrity of the render tree
hierarchy instead of just focusing on arithmetic computations and simple
rules.

> Finally, I'd like to say that I think it's great that you guys are asking for feedback on this. There are obviously webkit layout and rendering experts that know a lot more than I do, and I urge you very strongly to try to get one of them to help you with this design decision. Otherwise someone else in a few years may delete your code, as has happened to me. :)
As Alejandro said, we are going to submit patches for review soon.
Additionally, we are really interested in hearing any feedback from
WebKit experts on the following implementation note and tests:

http://www.mathml-association.org/MathMLinHTML5/
http://tests.mathml-association.org/

Finally, I believe you are seeing the glass half empty if you reduce our
work to "deleting your code". I would rather say that the flexbox
implementation was an opportunity for developers/users to experiment
MathML in WebKit, improve our test suite, get feedback and discuss the
pros and cons of that approach. In particular, thanks to your work we
have a good idea of what would be acceptable by the Google reviewers in
the future. For now, we are just rewriting the MathML layout to address
the feedback from the WebKit community and get something better! Such a
refactoring happens everytime in software development, so there are no
reasons to worry...

-- 
Frédéric Wang

-------------- next part --------------
A non-text attachment was scrubbed...
Name: integrals.png
Type: image/png
Size: 16811 bytes
Desc: not available
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20151229/22ccb271/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <https://lists.webkit.org/pipermail/webkit-dev/attachments/20151229/22ccb271/attachment.sig>