New RFC: remove mbstring fallbacks

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

New RFC: remove mbstring fallbacks

Max Semenik
Hey, I have a new topic I'd like to discuss. It's about mbstring and
whether do we really need to support running without it.

The RFC is at https://gerrit.wikimedia.org/r/#/c/267309/

Here's a copy:

MediaWiki currently relies heavily on Unicode support to provide support
for 300+ languages yet does not require the mbstring PHP extension to
function. Instead, we create PHP-only fallbacks if a native support is not
available. This creates a few problems:
* These fallbacks are extremely slow. The script in P2734
<https://phabricator.wikimedia.org/P2734> demonstrates that fallbacks are
roughly order of magnitude slower on PHP 5.6. In extreme cases, it can be
100+ times slower, per comment in Fallback.php).
* These fallbacks cover only a few functions. If there's no fallback,
either ad-hoc solutions are used in places, or, like in SwiftFileBackend,
we just say "mbstring is required".
* This also means that extensions can't expect any consistent Unicode
support.
* Won't somebody please think of the children!

Now that we've dramatically increased PHP requirements, we've already cut
off a lot of crappy environments so this change will likely not affect too
many users.

OS support:
* On Debian-based systems, a simple apt-get install php5 gives you mbstring
by default.
* On RPM-based, a separate package is required
* On Windows, people tend to use *AMP all-in-one packages that have
mbstring.


Current mbstring usage in core (excluding fallbacks themselves):

mediawiki/includes$ grep -orEh '\bmb_\w+' . | sort | uniq -c
   7 mb_check_encoding
   6 mb_convert_encoding
  12 mb_strlen
   4 mb_substr


Some time ago, I committed https://gerrit.wikimedia.org/r/#/c/267309/ to
start a discussion, but it went largely unnoticed so I'd like to start a
formal RFC.

--
Best regards,
Max Semenik ([[User:MaxSem]])
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: New RFC: remove mbstring fallbacks

Legoktm
Hi,

On 03/09/2016 06:47 PM, Max Semenik wrote:
> Hey, I have a new topic I'd like to discuss. It's about mbstring and
> whether do we really need to support running without it.
>
> The RFC is at https://gerrit.wikimedia.org/r/#/c/267309/

I think you meant <https://phabricator.wikimedia.org/T129435> :)

-- Legoktm

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: New RFC: remove mbstring fallbacks

Gabriel Wicke-3
So far, no serious concerns have been expressed on the task [1].

Users of Debian-based distributions or Windows should have mbstring
enabled by default, so should not be affected. However, users of
RPM-based distributions would potentially need to install an extra
package.

So, I would like to hear from users on RPM-based environments. Do you
already have mbstring installed, and if not, would it be an issue to
install it?

Thanks,

Gabriel

[1]: https://phabricator.wikimedia.org/T129435

On Thu, Mar 10, 2016 at 2:27 AM, Legoktm <[hidden email]> wrote:

> Hi,
>
> On 03/09/2016 06:47 PM, Max Semenik wrote:
>> Hey, I have a new topic I'd like to discuss. It's about mbstring and
>> whether do we really need to support running without it.
>>
>> The RFC is at https://gerrit.wikimedia.org/r/#/c/267309/
>
> I think you meant <https://phabricator.wikimedia.org/T129435> :)
>
> -- Legoktm
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



--
Gabriel Wicke
Principal Engineer, Wikimedia Foundation

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: New RFC: remove mbstring fallbacks

Brad Jorsch (Anomie)
On Thu, Mar 17, 2016 at 5:54 PM, Gabriel Wicke <[hidden email]> wrote:

> Users of Debian-based distributions or Windows should have mbstring
> enabled by default, so should not be affected.


I note that the current packages in Debian unstable for PHP7 have a
separate mbstring package for some reason.

Also, as a side note, once I got around to figuring out everything that
needed installation[1] my local dev environment hasn't had much issue with
PHP7. The biggest problem is that luasandbox needs updating.

[1]: IIRC that's php-curl, php-intl, php-json, php-mbstring, php-mysql,
php-sqlite3, php-tidy, php-xml, php-redis, and php-xdebug, in addition to
the usual basic packages.


--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l