Reviving SVG client-side rendering task

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Reviving SVG client-side rendering task

Brion Vibber-4
For the last decade we've supported uploading SVG vector images to
MediaWiki, but we serve them as rasterized PNGs to browsers. Recently,
display resolutions are going up and up, but so is concern about
low-bandwidth mobile users.

This means we'd like sharper icons and diagrams on high-density phone
displays, but are leery of adding extra srcset entries with 3x or 4x
size PNGs which could become very large. (In fact currently MobileFrontend
strips even the 1.5x and 2x renderings we have now, making diagrams very
blurry on many mobile devices. See https://phabricator.wikimedia.org/T133496 -
fix in works.)


Here's the base bug for SVG client side rendering:
https://phabricator.wikimedia.org/T5593
I've turned it into an "epic" story tracking task and hung some blocking
tasks off it; see those for more details.

TL;DR stop reading here. ;)


One of the basic problems in the past was reliably showing them natively in
an <img>, with the same behavior as before, without using JavaScript hacks
or breaking the hamlet caching layer. This is neatly resolved for current
browsers by using the "srcset" attribute -- the same one we use to specify
higher-resolution rasterizations. If instead of PNGs at 1.5x and 2x
density, we specify an SVG at 1x, the SVG will be loaded instead of the
default PNG.

Since all srcset-supporting browsers allow SVG in <img> this should "just
work", and will be more compatible than using the experimental <picture>
element or the classic <object> which deals with events differently. Older
browsers will still see the PNG, and we can tweak the jquery.hidpi srcset
polyfill to test for SVG support to avoid breaking on some older browsers.

This should let us start testing client-side SVG via a beta feature (with
parser cache split on the user pref) at which point we can gather more
real-world feedback on performance and compatibility issues.


Rendering consistency across browser engines is a concern. Supposedly
modern browsers are more consistent than librsvg but we haven't done a
compatibility survey to confirm this or identify problematic constructs.
This is probably worth doing.


Performance is a big question. While clean simple SVGs are often nice and
small and efficient, it's also easy to make a HUGEly detailed SVG that is
much larger than the rasterized PNGs. Or a fairly simple small file may
still render slowly due to use of filters.

So we probably want to provide good tools for our editors and image authors
to help optimize their files. Show the renderings and the bandwidth balance
versus rasterization; maybe provide in-wiki implementation of svgo or other
lossy optimizer tools. Warn about things that are large or render slowly.
Maybe provide a switch to run particular files through rasterization always.

And we'll almost certainly want to strip comments and white space to save
bandwidth on page views, while retaining them all in the source file for
download and reediting.


Feature parity also needs more work. Localized text in SVGs is supported
with our server side rendering but this won't be reliable in the client;
which means we'll want to perform a server side transformation that creates
per-language "thumbnail" SVGs. Fonts for internationalized text are a big
deal, and may require similar transformations if we want to serve them...
Which may mean additional complications and bandwidth usage.


And then there are long term goals of taking more advantage of SVGs dynamic
nature -- making things animated or interactive. That's a much bigger
question and has implementation and security issues!

-- brion
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Brad Jorsch (Anomie)
On Thu, May 5, 2016 at 9:49 AM, Brion Vibber <[hidden email]> wrote:

> Rendering consistency across browser engines is a concern. Supposedly
> modern browsers are more consistent than librsvg but we haven't done a
> compatibility survey to confirm this or identify problematic constructs.
> This is probably worth doing.
>

There's also the font issue: with the current rasterizing, only certain
free fonts are available and we know what it falls back to. Serve the SVG
to the client and those free fonts may well be missing (falling back to
who-knows-what), while other fonts with different metrics that our
rasterizer ignores might be present on the client.


> And we'll almost certainly want to strip comments and white space to save
> bandwidth on page views, while retaining them all in the source file for
> download and reediting.
>

There's other stuff that could be stripped, too. For example, Inkscape's
default "Inkscape SVG" format saves a lot of extra data under its own
"inkscape" and "sodipodi" namespaces and some style rules with a
'-inkscape-' vendor prefix that we could probably kill.

Inkscape also likes to put 'id' attributes on absolutely everything (e.g.
rect3342, tspan8263), which we could strip if we can somehow determine
they're not used somewhere else (including <style> or external reference of
some sort). It also likes to be redundant with inline CSS (e.g. reiterating
defaults, including all stroke-* properties despite stroke:none meaning
they're unused), but that's even more problematic to determine whether it's
safe to strip.

Some of the stuff in the svg:metadata node could probably also be killed,
although we might want to preserve some pieces (e.g. license, author) even
in minimized form and/or add some of our own (e.g. URL of the file
description page where the original can be downloaded).


--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Brion Vibber-4
On Thursday, May 5, 2016, Brad Jorsch (Anomie) <[hidden email]>
wrote:

> On Thu, May 5, 2016 at 9:49 AM, Brion Vibber <[hidden email]
> <javascript:;>> wrote:
> There's also the font issue: with the current rasterizing, only certain
> free fonts are available and we know what it falls back to. Serve the SVG
> to the client and those free fonts may well be missing (falling back to
> who-knows-what), while other fonts with different metrics that our
> rasterizer ignores might be present on the client.


Yep... It's worth experimenting with injecting fonts into the
"thumbnail" SVG output. That'll increase total download size again, but
common fonts should be shared between files, making them neatly cacheable,
and if there are no relevant characters we have no need to inject them.

I think all the CSS @font stuff works in SVG as it does in HTML, so if we
have known client-side-safe fonts we can reference them as we do in
HTML land... For common cases where text is in the language of the
surrounding site, that means we may have loaded the same font already if
one was needed.


Making SVG safe for fallback fonts can indeed be tricky if the metrics
don't match, though for simple labels it's usually a matter of setting
reasonable bounding boxes and making use of the alignment specifications,
rather than using the tightest possible bounding box and aligning things
manually by x/y.

For extreme cases where exact glyph shapes and metrics matter because the
text is mixed with graphical elements directly, files should probably
use paths instead of text (or at least embed a specific SVG font made of
specific paths). That can be something that good authoring tools can help
with.

-- brion

>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Chris Steipp
In reply to this post by Brion Vibber-4
On Thu, May 5, 2016 at 6:49 AM, Brion Vibber <[hidden email]> wrote:

>
> And then there are long term goals of taking more advantage of SVGs dynamic
> nature -- making things animated or interactive. That's a much bigger
> question and has implementation and security issues!


Sorry for the late response (and if this is covered in one of the linked
bugs), but getting back to this-- are you envisioning SVG's inlined into
the html? Or would they be served from a domain like upload.wm.o, like we
currently use for uploaded images?
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Brion Vibber-4
On Wed, May 11, 2016 at 11:10 AM, Chris Steipp <[hidden email]>
wrote:

> On Thu, May 5, 2016 at 6:49 AM, Brion Vibber <[hidden email]>
> wrote:
>
> >
> > And then there are long term goals of taking more advantage of SVGs
> dynamic
> > nature -- making things animated or interactive. That's a much bigger
> > question and has implementation and security issues!
>
>
> Sorry for the late response (and if this is covered in one of the linked
> bugs), but getting back to this-- are you envisioning SVG's inlined into
> the html?


Maybe someday that might be interesting for interactive bits, but not in
the short term.


> Or would they be served from a domain like upload.wm.o, like we
> currently use for uploaded images?
>

Yes for now.

-- brion
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Jon Robson-2
Thanks for starting this conversation Brion!

On mobile we are constantly tackling the trade offs between data
shipped (cost for end user) and quality. Hopefully we'll have a better
solution for this by the end of the quarter which will allow us to
reconsider srcset usage. That said relying on srcset to render SVGs
well seems like a hack to me and thus I'd rather we fixed the problem
at the source.

A few thoughts
* The ResourceLoaderImage module is being used widely to generate SVG
icons with png fallbacks. I'd be interested in seeing if we can use
this in some way for optimising SVGs and removing meta data.
* I wonder if there is an opportunity here to use BetaFeatures and
mobile beta to get a sense of performance impact of these changes?
This would allow people to opt in and for us to crowdsource feedback
in a safe enviroment.

Jon


On Wed, May 11, 2016 at 11:12 AM, Brion Vibber <[hidden email]> wrote:

> On Wed, May 11, 2016 at 11:10 AM, Chris Steipp <[hidden email]>
> wrote:
>
>> On Thu, May 5, 2016 at 6:49 AM, Brion Vibber <[hidden email]>
>> wrote:
>>
>> >
>> > And then there are long term goals of taking more advantage of SVGs
>> dynamic
>> > nature -- making things animated or interactive. That's a much bigger
>> > question and has implementation and security issues!
>>
>>
>> Sorry for the late response (and if this is covered in one of the linked
>> bugs), but getting back to this-- are you envisioning SVG's inlined into
>> the html?
>
>
> Maybe someday that might be interesting for interactive bits, but not in
> the short term.
>
>
>> Or would they be served from a domain like upload.wm.o, like we
>> currently use for uploaded images?
>>
>
> Yes for now.
>
> -- brion
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Brion Vibber-4
On Fri, May 13, 2016 at 1:28 PM, Jon Robson <[hidden email]> wrote:

> Thanks for starting this conversation Brion!
>
> On mobile we are constantly tackling the trade offs between data
> shipped (cost for end user) and quality. Hopefully we'll have a better
> solution for this by the end of the quarter which will allow us to
> reconsider srcset usage.


Note that srcset usage for raster images is a different issue from native
SVG, in that the higher-resolution renderings *do* require more bandwidth
than the lower-resolution images -- especially for photos.


> That said relying on srcset to render SVGs
> well seems like a hack to me and thus I'd rather we fixed the problem
> at the source.
>

The problem is the intersection of:
* some older browsers do not support SVG in <img>
* <img> provides no system for automatic fallback based on the browser's
format support
* <picture> does provide that, but the spec is not finalized, browser
support is not finalized, and it may have implications for scripting and
styling by introducing a new wrapper element

Using <img> with PNG in the src and a "1x" SVG in the srcset resolves the
first case by feeding a compatible rasterization to old browsers, solves
the second problem by using srcset support as a proxy check for SVG support
(native srcset support implies SVG support in all known cases), and avoids
the third problem by not using a new, different element for images.

We could jump straight to <picture>, but it'll have bigger QA implications
in that it may affect styling and JS gadgets that look for <img> elements
specifically.

<picture> would be nice for other things though -- future support of WebP
for alternate, more bandwidth-efficient thumbnails, or if we start doing
fancier types of responsive images based on "art direction" usages where a
narrow column gets a different image than a wide window.



> A few thoughts
> * The ResourceLoaderImage module is being used widely to generate SVG
> icons with png fallbacks. I'd be interested in seeing if we can use
> this in some way for optimising SVGs and removing meta data.
>

Definitely yes! Just as we minify CSS and JS source, we should make sure
the SVG icons are efficiently packed as well. (There's usually good
opportunity to do that at commit time, but it's nice to be able to include
comments and such in the source.)



> * I wonder if there is an opportunity here to use BetaFeatures and
> mobile beta to get a sense of performance impact of these changes?
> This would allow people to opt in and for us to crowdsource feedback
> in a safe enviroment.
>

Yes, I am suggesting exactly that. :)

-- brion
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Bartosz Dziewoński
In reply to this post by Jon Robson-2
On 2016-05-13 22:28, Jon Robson wrote:
> The ResourceLoaderImage module is being used widely to generate SVG
> icons with png fallbacks. I'd be interested in seeing if we can use
> this in some way for optimising SVGs and removing meta data.

I don't know what you have in mind, but please remember that
ResourceLoaderImage was not written with security in mind. It has a very
simplified version of our usual SVG rendering code, and it assumes that
any SVG files passed to it is trusted. We traded some caution for some
performance. Giving it user-controlled data is going to result in
security vulnerabilities (at the very least some denial of service ones).

--
Bartosz Dziewoński

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

Gabriel Wicke-3
Another option might be to piggy-back on the current work towards
lazy-loaded images [1]. Since this is using JS, it could take into
account network performance & screen resolutions, in addition to
browser capabilities. Designing this to degrade gracefully without JS
might be a bit tricky, though.

Gabriel

[1]: https://phabricator.wikimedia.org/T124390

On Fri, May 13, 2016 at 3:29 PM, Bartosz Dziewoński <[hidden email]> wrote:

> On 2016-05-13 22:28, Jon Robson wrote:
>>
>> The ResourceLoaderImage module is being used widely to generate SVG
>> icons with png fallbacks. I'd be interested in seeing if we can use
>> this in some way for optimising SVGs and removing meta data.
>
>
> I don't know what you have in mind, but please remember that
> ResourceLoaderImage was not written with security in mind. It has a very
> simplified version of our usual SVG rendering code, and it assumes that any
> SVG files passed to it is trusted. We traded some caution for some
> performance. Giving it user-controlled data is going to result in security
> vulnerabilities (at the very least some denial of service ones).
>
> --
> Bartosz Dziewoński
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



--
Gabriel Wicke
Principal Engineer, Wikimedia Foundation

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Reviving SVG client-side rendering task

C. Scott Ananian
I'll note that wikipedia also currently uses the SVG systemLanguage option
in a non-standard way which isn't supported by browsers:
https://phabricator.wikimedia.org/T60920

So SVGs which use this feature would have to be blacklisted somehow and
always rendered to PNG.
 --scott

On Fri, May 13, 2016 at 7:58 PM, Gabriel Wicke <[hidden email]> wrote:

> Another option might be to piggy-back on the current work towards
> lazy-loaded images [1]. Since this is using JS, it could take into
> account network performance & screen resolutions, in addition to
> browser capabilities. Designing this to degrade gracefully without JS
> might be a bit tricky, though.
>
> Gabriel
>
> [1]: https://phabricator.wikimedia.org/T124390
>
> On Fri, May 13, 2016 at 3:29 PM, Bartosz Dziewoński <[hidden email]>
> wrote:
> > On 2016-05-13 22:28, Jon Robson wrote:
> >>
> >> The ResourceLoaderImage module is being used widely to generate SVG
> >> icons with png fallbacks. I'd be interested in seeing if we can use
> >> this in some way for optimising SVGs and removing meta data.
> >
> >
> > I don't know what you have in mind, but please remember that
> > ResourceLoaderImage was not written with security in mind. It has a very
> > simplified version of our usual SVG rendering code, and it assumes that
> any
> > SVG files passed to it is trusted. We traded some caution for some
> > performance. Giving it user-controlled data is going to result in
> security
> > vulnerabilities (at the very least some denial of service ones).
> >
> > --
> > Bartosz Dziewoński
> >
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
> --
> Gabriel Wicke
> Principal Engineer, Wikimedia Foundation
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l