[webkit-dev] Request for Position: COLR v1 Vector Color Fonts

Mon May 3 07:09:41 PDT 2021

Hi Myles,

Thank you for responding to our request for a position statement. I
conclude we both want to bring adoption to a successful color vector format
to the web. That’s why I’d like to respond to your points in detail,
highlight benefits of COLRv1 and also point out where we disagree with
assumptions or arguments in your list.

In short, on the basis of data we’re presenting below, we are convinced
that COLRv1

   -

   is easy to integrate with fewer code and dependencies than SVG. (Given
   the current level of support for CSS, SVG, OT-SVG in WebKit, CoreText,
   CoreGraphics which all serve as foundations, I think you could have a
   working COLRv1 implementation in two weeks to a month.)
   -

   shifts complexity away from the client to a font production pipeline
   (assisted by publicly available tools we offer, nanoemoji
   <https://github.com/googlefonts/nanoemoji/>)
   -

   ultimately provides significantly better performance at a much smaller
   font size, see measurements below.

COLRv1 seamlessly integrates with existing font concepts, does support
variations and does not reinvent contour/path definitions. It’s also easily
extensible should support for bitmaps or filtering be required.

With this in mind, in particular the relatively low complexity of
implementing the format and the extremely low usage of OT-SVG on the web
(see below), we invite you to reconsider your position.

Here’s data we collected based on our current work in progress in Chrome
<https://github.com/googlefonts/colr-gradients-spec/#chromium-skia-freetype-support>
.

*Performance*

In a comparison between cold cache SVG and COLRv1 glyph drawing (Benchmark
code <https://skia-review.googlesource.com/c/skia/+/395616>), drawing an
SVG glyph from the Noto Color Emoji set is 20-45% slower on average than
drawing the glyph from COLRv1. This performance difference is relevant for
initial page drawing and font size changes, i.e. in all situations where a
fresh glyph rasterization is requested. Page redraws or repainting a same
glyph would usually be covered by a form of texture caching.

Performance tracing shows that COLRv1 succeeds here due to SVG parsing
overhead, while the drawing execution is roughly in the same ballpark.
Using SVGZ would add to the parsing overhead. In addition, the comparison
performed favors OT-SVG as it does not cover additional steps required in
OT-SVG such as finding the matching SVG document for a glyph in the SVG
table, and finding the glyph in the XML tree.
*Font Size*

For the full emoji set of Noto Color Emoji, the smallest SFNT size for
COLRv1 (cff2 contours) is ~3.97MB, the smallest SVGZ size is ~8.91MB (after
picosvg SVG flattening / optimisation). After WOFF2 compression, the
smallest COLRv1 (glyf contours) is ~1.94 MB and the smallest SVG is ~5.66MB
(uncompressed SVG woff2-compressed).

So, in this example glyph set:

   -

   WOFF2 compressed: COLRv1 is roughly a third (34.3%) of the size of
   OT-SVG,
   -

   Non-WOFF2-compressed fonts: COLRv1 is less than half (44.5%) of the size
   of OT-SVG.

*Complexity / Security*

   -

   The total LoC of C, C++ code and headers for COLRv1 glyph rendering
   functionality in Skia and FreeType is ~1257 lines of code. Details in
   this analysis.
   <https://docs.google.com/spreadsheets/d/1W4aQWXb1rlM67BoljWlOECN81llkhks4GVmWD19MqNc/edit#gid=0>
   For comparison, Skia’s SVG module has 6168 LoC - that’s almost 5 times as
   much, and the LoC count does not even include the XML parser dependency. A
   big portion of SVG code is taken up by custom XML attribute parsers for SVG
   on top of the XML library, for example SVG’s path definition.
   -

   The binary size impact of a COLRv1 implementation is minimal as shown by
   data from Chromium
   <https://docs.google.com/spreadsheets/d/16vizzDjHqG1oiVN0b5Tuox1Nu0RmfeKuRHM4WW-pAOw/edit?resourcekey=0-bH-kT18E2iW6gKmI8wecpQ#gid=0>.
   Chromium’s COLRv1 implementation in FreeType + Skia adds only 8-12kb binary
   size on a Chrome Android APK build or on a Windows release build.
   For comparison, the threshold where our Android release engineering’s
   bot warns about a significant binary size increase is 20kb. libxml2 on its
   own compiles to roughly 30kb in a release configuration for Android.

*Real world OT-SVG usage*

Data from the httparchive
<https://httparchive.org/faq#how-do-i-use-bigquery-to-write-custom-queries-over-the-data>
set shows that out of 25.8 million desktop requests to resources that
contain additional font metadata in the requests_desktop table, only 786
requests, or 0.00305% of them are fonts that contain an SVG table. Among
those, the main ones are Adobe’s Ten Mincho and Source Code Pro, which are
not mainly color fonts but contour fonts that contain a set of SVG glyphs
as a feature. Without those, the usage of SVG fonts is even lower, 95
requests, or 0.00037%.

On Wed, Mar 31, 2021 at 2:00 AM Myles C. Maxfield <mmaxfield at apple.com>
wrote:

> Hi Dominik!
>
> [...]
>
> At a high level, here is a list of things we like about the proposal:
>
>    1. Chrome aims to ship support for advanced vector-based color fonts
>
>
> Inversely, here is a list of things we don’t like about the proposal:
>
1.It re-invents the wheel. This new format is as expressive and powerful as
> any general-purpose 2D graphics serialization format. There are many, many
> existing serialization formats for general-purpose 2D graphics.
>

It deliberately shifts complexity away from the client to the preprocessing
stage, thus improving performance and security. It provides 2D graphics
primitives specifically integrated with existing and not reinventing font
format concepts.
It uses existing contour definitions from the `glyf` or `CFF ` tables
instead of defining a new path format and secondly it natively supports
variations if you need them. In several iterations we have made sure that
the format is as compact as possible, for example when defining Paint
formats that save space when not using variations.
This compact and efficient encoding brings performance advantages and font
file size advantages, as you can see from our results at the beginning.

2. It doesn’t exist yet (outside of a development configuration of Chrome).
> OT-SVG, which is just as expressive*, exists and has shipping
> implementations in DirectWrite, Core Text, Firefox, and many (most? all?)
> of Adobe creation apps. Many OT-SVG fonts already exist.

While shipping implementations and experimental fonts exist, usage on the
web is extremely low. See httparchive data at the beginning.

> 3. Because this proposal doesn’t exist yet outside of Chrome, there is no
> ecosystem in existing authoring tools. Conversely, many design authoring
> tools already export SVG.
>

The process of creating an SVG font involves processing steps to turn a
source drawing into a font glyph: it needs to be ensured that the output
SVG conforms to the SVG-native <https://svgwg.org/specs/svg-native/> subset
(which may require automated or manual changes to the authored source
images). Glyph mappings need to be defined, an appropriate SVG document
split needs to be found (which is a process full of pitfalls due to
application limits and compatibility issues, see this list of problematic
examples
<https://github.com/googlefonts/nanoemoji/issues?q=label%3A%22svg+compat%22+>).
Many such undocumented limit exist in Safari’s OT-SVG implementation,
making it hard to make a compatible font.

Which current pipelines and tools exist that produce performant and
interoperable OT-SVG fonts?

On their SVG type tools page, Adobe mentions tools
<https://github.com/adobe-type-tools/opentype-svg> such as SVG Cleaner, SVG
Optimizer, Scour etc. and the stages are described to go from source images
to SVG fonts. Due to the way SVG-Native is spec’ed, OT-SVG makes it hard to
produce a performant and interoperable font.

For COLRv1, the format has tighter definitions and thus better chances of
interoperability than SVG-Native.

For a font production pipeline, we’re providing a freely available open
source tool nanoemoji <https://github.com/googlefonts/nanoemoji/> to
process SVG sources into COLRv1 fonts, including optimizations to flatten
and reduce the SVG complexity, find repetitive contours etc.

An SVG authoring ecosystem exists, we’re providing an open source
implementation of a tool that converts and optimizes SVG into COLRv1 fonts.

> 4. Supporting both OT-SVG and this new proposal is twice (-ish) the
> maintenance burden, for a format that isn’t any more expressive* than the
> format we support already.
>

COLRv1 is not hard to integrate or maintain. I would estimate you could
have a working COLRv1 implementation in two weeks to a month. Maintenance
of COLRv1 should be way lower than SVG if the lines of code metrics at the
beginning is any indication, plus you gain support for variations.

> 5. Supporting both OT-SVG and this new proposal increases our binary
> size. We expect the additional binary size increase to be roughly
> equivalent to the binary size increase we observed after implementing
> OT-SVG. (OT-SVG does involve an XML parser, but WebKit already links with
> an XML parser, so we expect the size of this new proposal to be roughly
> equal to the size *increase* we saw after implementing OT-SVG. This
> proposal would require its own novel parsing / overflow detecting /
> interpretation code.)

Looking at the LoC comparison, and font size advantage presented above, and
the specified format definition itself, I am convinced a COLRv1
implementation has lower complexity and size than an SVG implementation.

Font formats are not only implemented in large browser codebases. Embedded
devices save the dependency on a large XML parser for parsing fonts. We
deliberately aimed for supporting 2D graphics constructs that are widely
available. We did not add new 2D graphics primitives when implementing
COLRv1 - the only new code that’s new is the font parsing and connecting
the parsed results to the 2D graphics library.

Given the low usage from the measurement results of httparchive, we may
together consider the possibility to phase out OT-SVG in favor of COLRv1
and reduce the maintenance cost of maintaining two formats. Where do you
see significant established OT-SVG usage on the web at the moment?

> 6. Supporting both OT-SVG and this new proposal roughly doubles the
> surface area for security attacks for vector-based color fonts.
> 7. Even considering an engine that only supports this proposal and not
> SVG, we haven’t seen any evidence that avoiding XML will reduce security
> bugs compared to a novel binary format. Historically, in WebKit, we’ve
> observed that opaque binary formats (like image formats) have plenty of
> their own security bugs.
>

In these two points I don’t find any arguments against COLRv1 itself. From
your perspective, you may ask why would you want to add this functionality
and increase your attack surface. (I list advantages above). In contrast,
we ask ourselves: which efficient and fast format do we want to add at all.
Of course, any added font or image format needs to be made secure.

COLR v1 is compact and both simple and efficient to parse and validate. See
section “5.7.11.1.9 Color glyphs as a directed acyclic graph” which lists
the criteria for well-formedness.
If you’d choose to base an implementation of COLRv1 on FreeType its COLRv1
parsing code is already being fuzzed
<https://github.com/freetype/freetype2-testing/blob/master/fuzzing/src/visitors/facevisitor-colrv1.cpp>
as part of oss-fuzz <https://github.com/google/oss-fuzz>.

> 8. The spec has over 2,500 lines, and the images/ directory of the spec
> has 77 figures, and there is only a single implementation of this proposal.
> It’s complicated enough that we’re not confident that it can be
> implemented interoperably. We’re worried the behavior of the drawing
> operations may be specific to Skia, and difficult/impossible to implement
> on Core Graphics. For example, at first glance, we are not sure that the
> radial gradients in this proposal can be implemented on Core Graphics. As
> far as we’re aware, this proposal has not undergone a rigorous
> standardization process from many independent stakeholders.
>
We deliberately put effort into a specification that is easy to support
across platforms. We investigated which graphical primitives are available
in Direct2D, the Cairo graphics library, CoreGraphics and Skia - we also
looked at what web browsers support and what’s specified in HTML, CSS and
SVG.

A second COLRv1 implementation called BlackRenderer
<https://github.com/BlackFoundryCom/black-renderer> with support for
multiple backends, including CoreGraphics and Cairo is already underway.

The specification has been reviewed and received contributions by several
industry stakeholders. Two examples: Here’s Peter Constable’s response on
webkit-dev
<https://lists.webkit.org/pipermail/webkit-dev/2021-April/031789.html> and
here’s what Chris Liley (W3C) had to say on the mpeg-otspec ml
<https://lists.aau.at/pipermail/mpeg-otspec/2021-January/002724.html>:

“Combining the graphical expressiveness of SVG with the typographic
expressiveness of font variations (and of course using vectors not rasters)
will make COLRv1 the clear leader among the assorted chromatic font
options.“

Re your concern regarding radial gradients:

COLRv1 radial gradients are conceptually very close to HTML Canvas radial
gradients and they are supported in Safari, see this codepen
https://codepen.io/drott_chrome/pen/zYNZGZV or this separate document with
examples
<https://docs.google.com/document/d/1NIKwcubEYkla_k6rRHypDg5slaAGp0zZSYet23luLu8/edit?usp=sharing>.
Thus the building blocks for supporting COLRv1 radial gradients are there.
The missing extend modes can most likely be implemented on top of the
existing primitives. This work would at the same time help fix existing
issues with lack of spreadMethod support for SVG radial gradients in WebKit
(Webkit Issue #223083 <https://bugs.webkit.org/show_bug.cgi?id=223083>) and
different spreadMethod issues in Safari’s OT-SVG
<https://github.com/googlefonts/nanoemoji/issues/281> implementation.
The quick overview of C++ structures
<https://github.com/googlefonts/colr-gradients-spec#c-structures> and the
pseudocode <https://github.com/googlefonts/colr-gradients-spec#pseudocode>
listed as part of the spec repository may further help gauge the overall
complexity. Again, I believe an implementation would not take you very long.

9. Embedding raster image data inside color font tables is common today,
> but this new proposal has no affordances for allowing it, despite its
> vector facilities being as expressive as any general-purpose 2d graphics
> serialization format. Therefore, it doesn’t actually improve upon the color
> font table fragmentation situation, which is widely regarded as one of the
> biggest drawbacks to color fonts today.
>

I’ll not go into details whether embedding bitmaps in OT-SVG fonts makes
sense over choosing a bitmap font format such as CBDT/CBLC or SBIX.

Cross-browser supported bitmap color font formats already exist (SBIX for
example), but it’s not our primary focus with COLRv1 to embed bitmaps as it
is not on the critical path of those exact scalability and rendering
quality issues we want to address.
However, taking on your feedback, we’re discussing embedded and reusing
bitmaps from existing bitmap formats in this github issue
<https://github.com/googlefonts/colr-gradients-spec/issues/272> as a
possible future extension to COLRv1, possibly incorporating existing bitmap
font table definitions.

Given these two lists, Apple’s WebKit team and Core Text team are negative
> on this proposal.
>
> Thanks,
> Myles
>
> * Except for font variations, which A) can be added to OT-SVG easily, and
> B) probably aren't a particularly compelling feature in conjunction with
> color font technology, at least now.
>
>
I have not seen an approach that defines an “easy” definition of applying
OT variations to SVG paths, let alone in a compact way in terms of font
file size. From what we hear from font designers and reviewers of the
specification (see also Chris Liley’s statement above), we certainly do not
want to offer a vector font format that lacks variations as exciting
variable fonts are being used and designed at the moment (for example a
full CJK variable font) and we would not want to lose this momentum when it
comes to color fonts.

With these additional points in mind, and the additional data we shared on
font size and performance advantages, we kindly invite you to reconsider
your position. We’d be very happy if you’d consider experimenting with
implementing COLRv1 and evaluate it from that perspective.

Dominik

> On Mar 23, 2021, at 4:23 AM, Dominik Röttsches <drott at chromium.org> wrote:
>
> Hi,
>
> This is a request for WebKit's position on COLR v1 vector color fonts.
>
> Spec:
>
> https://github.com/googlefonts/colr-gradients-spec/blob/main/OFF_AMD2_WD.md
> *Proposed changes to ISO/IEC 14496-22 (Amendment 2)*
>
> Chromium Bug:
> https://crbug.com/1170733
>
> Summary:
>
> COLR v1 fonts are a new vector color font format that provides gradients,
> transforms and compositions for font glyph drawing enabling an expressive,
> very compact glyph definition format for OFF/OpenType fonts. This format
> provides size and rendering fidelity (scaling) advantages over bitmap fonts
> such as SBIX, CBDT/CBLC fonts for glyph designs that lend themselves well
> to being encoded as vector art.
>
> Thank you in advance for your position statement on this topic,
>
> Dominik
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-dev/attachments/20210503/4c3e4443/attachment.htm>