Visual Editor and Parsoid New Pages in Wikitext?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Visual Editor and Parsoid New Pages in Wikitext?

Derric Atzrott
If I install the Visual Editor mediawiki extension on the Wiki that I manage
here at my work and also setup Parsoid, will new pages be saved with Wikitext
still?

 

The documentation for the extension's installation mentions that after you
install Visual Editor you won't be able to edit Wikitext pages, but you will be
able to create new pages.  It also mentions that with Parsoid you will be able
to edit Wikitext pages again.  This leaves unclear whether or not having Parsoid
installed makes Visual Editor save new pages as Wikitext as well, or just
previously existing pages.

 

I'm thinking about letting folks try Visual Editor here, but I need to be able
to work with Wikitext still as there are definitely some unusual things I do in
Wikitext from time to time.

 

Thank you,

Derric Atzrott

Computer Specialist

Alizee Pathology

 

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Gabriel Wicke-3
On 02/14/2014 11:32 AM, Derric Atzrott wrote:
> If I install the Visual Editor mediawiki extension on the Wiki that I manage
> here at my work and also setup Parsoid, will new pages be saved with Wikitext
> still?

Yes. All content is currently stored as wikitext, so you are naturally
able to use the wikitext editor to create pages or edit existing pages.

In the longer term we are working on the ability to store HTML instead
for new wikis, in which case it might become possible to run without
Parsoid if you don't need a wikitext editor front-end. This is not going
to happen over night and will just be an option, so no reason to worry.

> The documentation for the extension's installation mentions

Do you have a link to this documentation?

Thanks,

Gabriel

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Derric Atzrott
>> If I install the Visual Editor mediawiki extension on the Wiki that I manage
>> here at my work and also setup Parsoid, will new pages be saved with Wikitext
>> still?
>
>Yes. All content is currently stored as wikitext, so you are naturally
>able to use the wikitext editor to create pages or edit existing pages.
>
>In the longer term we are working on the ability to store HTML instead
>for new wikis, in which case it might become possible to run without
>Parsoid if you don't need a wikitext editor front-end. This is not going
>to happen over night and will just be an option, so no reason to worry.

So right now, assuming I don't have Parsoid, Visual Editor creates pages in Wikitext, it just can't edit previously existing pages?  I would assume that this also means that without Parsoid if I edit a page in the regular Wikitext editor, I won't be able to use Visual Editor with it anymore.

>> The documentation for the extension's installation mentions
>
>Do you have a link to this documentation?

I was reading this page: https://www.mediawiki.org/wiki/Extension:VisualEditor

Thank you,
Derric Atzrott


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Gabriel Wicke-3
On 02/14/2014 12:47 PM, Derric Atzrott wrote:

>>> If I install the Visual Editor mediawiki extension on the Wiki
>>> that I manage here at my work and also setup Parsoid, will new
>>> pages be saved with Wikitext still?
>>
>> Yes. All content is currently stored as wikitext, so you are
>> naturally able to use the wikitext editor to create pages or edit
>> existing pages.
>>
>> In the longer term we are working on the ability to store HTML
>> instead for new wikis, in which case it might become possible to
>> run without Parsoid if you don't need a wikitext editor front-end.
>> This is not going to happen over night and will just be an option,
>> so no reason to worry.
>
> So right now, assuming I don't have Parsoid, Visual Editor creates
> pages in Wikitext, it just can't edit previously existing pages?  I
> would assume that this also means that without Parsoid if I edit a
> page in the regular Wikitext editor, I won't be able to use Visual
> Editor with it anymore.

VisualEditor is an HTML editor and doesn't know about wikitext. All
conversions between wikitext and HTML are done by Parsoid. You need
Parsoid if you want to use VisualEditor on current wikis.

Gabriel

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Lars Aronsson
On 02/14/2014 10:39 PM, Gabriel Wicke wrote:
> VisualEditor is an HTML editor and doesn't know about wikitext.

That single sentence explains so much.
If you would have told me two years ago,
I could have said "wrong path, this will fail".
(Of course, many would have protested and
refused to listen to sound advice, but now
we know the outcome.)

What do you predict we will be using five
years from now, in 2019? Plain old wikitext,
VisualEditor, or some other path?


--
   Lars Aronsson ([hidden email])
   Aronsson Datateknik - http://aronsson.se



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

James Forrester-4
On 14 February 2014 15:40, Lars Aronsson <[hidden email]> wrote:

> On 02/14/2014 10:39 PM, Gabriel Wicke wrote:
>
>> VisualEditor is an HTML editor and doesn't know about wikitext.
>>
>
> That single sentence explains so much.
> If you would have told me two years ago,
> I could have said "wrong path, this will fail".
> (Of course, many would have protested and
> refused to listen to sound advice, but now
> we know the outcome.)
>

​Thanks for your timely input. :-)

We pointed out some of the (dozens of) reasons why a client-side
implementation of the parser wasn't a workable solution when we made the
choice to take this route; sorry you didn't notice that at the time.


What do you predict we will be using five
> years from now, in 2019? Plain old wikitext,
> VisualEditor, or some other path?


​For new out-of-the-box MediaWiki installs? Almost certainly HTML.​

​For existing legacy MediaWiki sites with sane quantities of extensions?
Probably HTML for most.​

For WMF's currently-hosted sites? Probably wikitext, but it's possible that
we'll have switched.

J.
--
James D. Forrester
Product Manager, VisualEditor
Wikimedia Foundation, Inc.

[hidden email] | @jdforrester
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Matthew Flaschen-2
In reply to this post by Lars Aronsson
On 02/14/2014 06:40 PM, Lars Aronsson wrote:
> On 02/14/2014 10:39 PM, Gabriel Wicke wrote:
>> VisualEditor is an HTML editor and doesn't know about wikitext.
>
> That single sentence explains so much.

I think Gabriel is simplifying a little.  VisualEditor is an editor that
understands, edits, and preserves semantically rich, augmented, HTML 5.

If you look at the HTML source code of
http://parsoid.wmflabs.org/enwiki/Earth , you can see that while it is
HTML, it is carefully augmented with additional information.

Among other things, that allows VisualEditor to handle
MediaWiki-specific features, such as templates.

After the user is done editing with VisualEditor, Parsoid then turns
that augmented HTML back into wikitext, generally without dirty diffs.

> If you would have told me two years ago,
> I could have said "wrong path, this will fail".
> (Of course, many would have protested and
> refused to listen to sound advice, but now
> we know the outcome.)

If you're saying that VisualEditor has failed, I disagree.  I don't
judge software quality by whether it's opt-in or opt-out (this is a
question of configuration, and will probably change over time), but on
its own merits.  VisualEditor has already fulfilled many requirements,
and is still improving.

Matt Flaschen

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Daniel Kinzler
In reply to this post by Gabriel Wicke-3
Am 14.02.2014 22:39, schrieb Gabriel Wicke:
> VisualEditor is an HTML editor and doesn't know about wikitext. All
> conversions between wikitext and HTML are done by Parsoid. You need
> Parsoid if you want to use VisualEditor on current wikis.

Implementing a HTML content type in mediawiki would be pretty trivial. That way,
a page could "natively" contain HTML, with no need of conversion. Anyone up to
doing it?...

-- daniel


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Gabriel Wicke-3


On February 15, 2014 12:05:49 PM PST, Daniel Kinzler <[hidden email]> wrote:
>Implementing a HTML content type in mediawiki would be pretty trivial.
>That way,
>a page could "natively" contain HTML, with no need of conversion.
>Anyone up to
>doing it?...

We are working towards this, but actual HTML-only MediaWiki is not as trivial as you make it sound. In the Parsoid roadmap we are discussing some of the issues involved. We might be able to use Parsoid HTML for page views (for both anonymous and logged-in views) before we have proper support for HTML-only wikis.

Gabriel

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

David Gerard-2
In reply to this post by Daniel Kinzler
On 15 February 2014 20:05, Daniel Kinzler <[hidden email]> wrote:
> Am 14.02.2014 22:39, schrieb Gabriel Wicke:

>> VisualEditor is an HTML editor and doesn't know about wikitext. All
>> conversions between wikitext and HTML are done by Parsoid. You need
>> Parsoid if you want to use VisualEditor on current wikis.

> Implementing a HTML content type in mediawiki would be pretty trivial. That way,
> a page could "natively" contain HTML, with no need of conversion. Anyone up to
> doing it?...


There are extensions that allow raw HTML widgets, just putting them
through unchecked. The hard part will be checking. Note that the
rawness of the somewhat-filtered HTML is a part of WordPress's not so
great security story (though they've had a lot less "update now!" in
the past year). So, may not involve much less parsing.


- d.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

David Gerard-2
In reply to this post by Lars Aronsson
On 14 February 2014 23:40, Lars Aronsson <[hidden email]> wrote:

> What do you predict we will be using five
> years from now, in 2019? Plain old wikitext,
> VisualEditor, or some other path?


I would hope Wikitext had been stabbed through its black and twisted
little heart, or at least as much as one can reasonably do that with
~5 billion words of legacy content. Horrible, horrible thing.


- d.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

David Gerard-2
In reply to this post by Matthew Flaschen-2
On 15 February 2014 08:06, Matthew Flaschen <[hidden email]> wrote:

> If you look at the HTML source code of
> http://parsoid.wmflabs.org/enwiki/Earth , you can see that while it is HTML,
> it is carefully augmented with additional information.


Currently giving "Error: EROFS, read-only file system" :-)


- d.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

addshorewiki
That is due to Wikimedia Labs currently having a small NFS issue.

Addshore


On 16 February 2014 10:34, David Gerard <[hidden email]> wrote:

> On 15 February 2014 08:06, Matthew Flaschen <[hidden email]>
> wrote:
>
> > If you look at the HTML source code of
> > http://parsoid.wmflabs.org/enwiki/Earth , you can see that while it is
> HTML,
> > it is carefully augmented with additional information.
>
>
> Currently giving "Error: EROFS, read-only file system" :-)
>
>
> - d.
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Gabriel Wicke-3
In reply to this post by David Gerard-2
On 02/16/2014 01:32 AM, David Gerard wrote:
> There are extensions that allow raw HTML widgets, just putting them
> through unchecked. The hard part will be checking. Note that the
> rawness of the somewhat-filtered HTML is a part of WordPress's not so
> great security story (though they've had a lot less "update now!" in
> the past year). So, may not involve much less parsing.

The difference is that you can run the sanitizer on save, and then only
need to re-run it when a bug in it was fixed (which can happen in a
background job rather than on view). We will maintain a sanitization
level in storage to track the degree to which the HTML is sanitized.

Sanitization is also the last part of parsing from wikitext to HTML. It
is one of the cheapest parts of the parsing process, so running just
that on a DOM is much cheaper than parsing from scratch.

Gabriel

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Derric Atzrott
In reply to this post by Gabriel Wicke-3
>>> In the longer term we are working on the ability to store HTML
>>> instead for new wikis, in which case it might become possible to
>>> run without Parsoid if you don't need a wikitext editor front-end.
>>> This is not going to happen over night and will just be an option,
>>> so no reason to worry.
>>
>> So right now, assuming I don't have Parsoid, Visual Editor creates
>> pages in Wikitext, it just can't edit previously existing pages?  I
>> would assume that this also means that without Parsoid if I edit a
>> page in the regular Wikitext editor, I won't be able to use Visual
>> Editor with it anymore.
>
>VisualEditor is an HTML editor and doesn't know about wikitext. All
>conversions between wikitext and HTML are done by Parsoid. You need
>Parsoid if you want to use VisualEditor on current wikis.

This is what I had originally thought, but then somehow along the way I managed to get confused.  Thank you for clarifying.

Thank you,
Derric Atzrott


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Visual Editor and Parsoid New Pages in Wikitext?

Daniel Kinzler
In reply to this post by David Gerard-2
Am 16.02.2014 10:32, schrieb David Gerard:
> There are extensions that allow raw HTML widgets, just putting them
> through unchecked.

I know, I wrote one :) But that's not the point. The point is maintaining
editable content as HTML instead of Wikitext.

> The hard part will be checking.

Wikitext already allows a wide range of HTML tags, and we have a pretty good
sanitizer for that. Adding support for a few additional structures (like links
and images) and the extra data embedded by/for parsoid should not be a lot of work.

> Note that the
> rawness of the somewhat-filtered HTML is a part of WordPress's not so
> great security story (though they've had a lot less "update now!" in
> the past year). So, may not involve much less parsing.

I think it would, since it doesn't add much to the sanitizer we use now, but
reducing the amount of parsing wasn't the point. The point was avoiding
conversion, which is potentially lossy and confusing, and essentially pointless.

If we edit using an HTML ewditor, why not store HTML, make (structural) HTML
diffs, etc? It just seems a lot more streight forward.

-- daniel


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l