Editing JSON in MediaWiki

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Editing JSON in MediaWiki

Denny Vrandečić-2
Here's my requirement:
- a wiki page is one JSON document
- when editing, the user edits the JSON directly
- when viewing, I have a viewer that turns the JSON into wikitext, and that
wikitext gets rendered as wikitext and turned into HTML by MediaWiki

I have several options, including:

1) hook for a tag like <json>, and write an extension that parses the
content between the tags and turns it into wikitext (not ideal, as I don't
use any of the existing support for JSON stuff, and also I could have
several such tags per page, which does not fit with my requirements)

2) I found the JsonConfig extension by yurik. This allows me to do almost
all of the things above - but it returns HTML directly, not wikitext. It
doesn't seem trivial to be able to return wikitext instead of HTML, but
hopefully I am wrong? Also, this ties in nicely with the Code Editor.

3) there is actually a JsonContentHandler in core. But looking through it
it seems that this suffers from the same limitations - I can return HTML,
but not wikitext.

3 seems to have the advantage to be more actively worked on that 2 (which
is not based on 3, probably because it is older than 3). So future goodies
like a Json Schema validator will probably go to 3, but not to 2, so I
should probably go to 3.

Writing this down, one solution could be to create the wikitext, and then
call the wikitext parser manually and have it create HTML?

I have already developed the extension in 1, and then fully rewritten it in
2.  Before I go and rewrite it again in 3, I wanted to ask whether I am
doing it right, or if should do it completely differently, and also if
there are examples of stuff developed in 3, i.e. of extensions or features
using the JsonContent class.

Example:
I have a JSON document
{ "username": "Denny" }
which gets turned into wikitext
''Hello, [[User:Denny|Denny]]!''
which then gets turned into the right HTML and displayed to the user, e.g.
<i>Hello, <a href="...">Denny</a>!</i>

Cheers,
Denny
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

James Hare-4
Why, exactly, do you want a wikitext intermediary between your JSON and
your HTML? The value of wikitext is that it’s a syntax that is easier to
edit than HTML. But if it’s not the native format of your data, nor can
browsers render it directly, what’s the point of having it?

The CollaborationKit extension [0] has two content models,
CollaborationHubContent and CollaborationListContent, and they have two
different strategies for parsing. CollaborationHubContent takes validated
JSON and puts out raw HTML; this is the most straightforward to work with.
CollaborationListContent instead puts out wikitext that is fed into the
parser, since it is expected that lists can be transcluded onto other
pages, and this means using wikitext (as transcluded HTML is not an
option). However, this creates a lot of limitations, including parser
restrictions that make sense in the context of arbitrary wikitext parsing
but not when the markup is provided directly by an extension. The long term
plan is for CollaborationListContent to put out HTML, since it’s more
straightforward than using a wikitext intermediary that the user does not
see anyway.

[0] https://www.mediawiki.org/wiki/Extension:CollaborationKit

On April 8, 2017 at 11:23:38 PM, Denny Vrandečić ([hidden email])
wrote:

Here's my requirement:
- a wiki page is one JSON document
- when editing, the user edits the JSON directly
- when viewing, I have a viewer that turns the JSON into wikitext, and that
wikitext gets rendered as wikitext and turned into HTML by MediaWiki

I have several options, including:

1) hook for a tag like <json>, and write an extension that parses the
content between the tags and turns it into wikitext (not ideal, as I don't
use any of the existing support for JSON stuff, and also I could have
several such tags per page, which does not fit with my requirements)

2) I found the JsonConfig extension by yurik. This allows me to do almost
all of the things above - but it returns HTML directly, not wikitext. It
doesn't seem trivial to be able to return wikitext instead of HTML, but
hopefully I am wrong? Also, this ties in nicely with the Code Editor.

3) there is actually a JsonContentHandler in core. But looking through it
it seems that this suffers from the same limitations - I can return HTML,
but not wikitext.

3 seems to have the advantage to be more actively worked on that 2 (which
is not based on 3, probably because it is older than 3). So future goodies
like a Json Schema validator will probably go to 3, but not to 2, so I
should probably go to 3.

Writing this down, one solution could be to create the wikitext, and then
call the wikitext parser manually and have it create HTML?

I have already developed the extension in 1, and then fully rewritten it in
2. Before I go and rewrite it again in 3, I wanted to ask whether I am
doing it right, or if should do it completely differently, and also if
there are examples of stuff developed in 3, i.e. of extensions or features
using the JsonContent class.

Example:
I have a JSON document
{ "username": "Denny" }
which gets turned into wikitext
''Hello, [[User:Denny|Denny]]!''
which then gets turned into the right HTML and displayed to the user, e.g.
<i>Hello, <a href="...">Denny</a>!</i>

Cheers,
Denny
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Gergo Tisza
In reply to this post by Denny Vrandečić-2
You probably want to subclass JsonContentHandler and add wikitext transform
and whatever else you need. For schemas, have a look
at JsonSchemaContentHandler in EventLogging.
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Daniel Kinzler-2
In reply to this post by Denny Vrandečić-2
Am 09.04.2017 um 08:23 schrieb Denny Vrandečić:
> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki

Quick thoughts off the top of my head:

Generating wikitext from some other thing is what Scribunto does. Instead of
using the Lua handler, you would make a handler for Json, with whetever rules
you like for generating wikitext.

I have ne4ver looked at customizing Scribunto, but this seems to right place to
plug this in.

The alternative is to try to have some kind of "cascading" ContentHandler, that
generates a WikiTextContent object first, and then turns that into HTML.


--
Daniel Kinzler
Senior Software Developer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

zppix e
In reply to this post by Gergo Tisza
That seems like a performance hog in the making in my opinion to not mention, mostly unneeded as far as I can tell.

Sent from my iPhone

> On Apr 9, 2017, at 6:11 AM, Gergo Tisza <[hidden email]> wrote:
>
> You probably want to subclass JsonContentHandler and add wikitext transform
> and whatever else you need. For schemas, have a look
> at JsonSchemaContentHandler in EventLogging.
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Isarra Yos
In reply to this post by James Hare-4
Sounds like what they want is what collaborationlistcontent/handler
does, specifically, at least for now.

On 09/04/17 06:30, James Hare wrote:

> Why, exactly, do you want a wikitext intermediary between your JSON and
> your HTML? The value of wikitext is that it’s a syntax that is easier to
> edit than HTML. But if it’s not the native format of your data, nor can
> browsers render it directly, what’s the point of having it?
>
> The CollaborationKit extension [0] has two content models,
> CollaborationHubContent and CollaborationListContent, and they have two
> different strategies for parsing. CollaborationHubContent takes validated
> JSON and puts out raw HTML; this is the most straightforward to work with.
> CollaborationListContent instead puts out wikitext that is fed into the
> parser, since it is expected that lists can be transcluded onto other
> pages, and this means using wikitext (as transcluded HTML is not an
> option). However, this creates a lot of limitations, including parser
> restrictions that make sense in the context of arbitrary wikitext parsing
> but not when the markup is provided directly by an extension. The long term
> plan is for CollaborationListContent to put out HTML, since it’s more
> straightforward than using a wikitext intermediary that the user does not
> see anyway.
>
> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
>
> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić ([hidden email])
> wrote:
>
> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> I have several options, including:
>
> 1) hook for a tag like <json>, and write an extension that parses the
> content between the tags and turns it into wikitext (not ideal, as I don't
> use any of the existing support for JSON stuff, and also I could have
> several such tags per page, which does not fit with my requirements)
>
> 2) I found the JsonConfig extension by yurik. This allows me to do almost
> all of the things above - but it returns HTML directly, not wikitext. It
> doesn't seem trivial to be able to return wikitext instead of HTML, but
> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>
> 3) there is actually a JsonContentHandler in core. But looking through it
> it seems that this suffers from the same limitations - I can return HTML,
> but not wikitext.
>
> 3 seems to have the advantage to be more actively worked on that 2 (which
> is not based on 3, probably because it is older than 3). So future goodies
> like a Json Schema validator will probably go to 3, but not to 2, so I
> should probably go to 3.
>
> Writing this down, one solution could be to create the wikitext, and then
> call the wikitext parser manually and have it create HTML?
>
> I have already developed the extension in 1, and then fully rewritten it in
> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
> doing it right, or if should do it completely differently, and also if
> there are examples of stuff developed in 3, i.e. of extensions or features
> using the JsonContent class.
>
> Example:
> I have a JSON document
> { "username": "Denny" }
> which gets turned into wikitext
> ''Hello, [[User:Denny|Denny]]!''
> which then gets turned into the right HTML and displayed to the user, e.g.
> <i>Hello, <a href="...">Denny</a>!</i>
>
> Cheers,
> Denny
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Thomas PT
An other maybe relevant example : ProofreadPage has a content model for proofreading pages with multiple areas of Wikitext and use some for transclusion and others for rendering:

* https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContent.php
* https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContentHandler.php

Thomas

> Le 9 avr. 2017 à 18:44, Isarra Yos <[hidden email]> a écrit :
>
> Sounds like what they want is what collaborationlistcontent/handler does, specifically, at least for now.
>
> On 09/04/17 06:30, James Hare wrote:
>> Why, exactly, do you want a wikitext intermediary between your JSON and
>> your HTML? The value of wikitext is that it’s a syntax that is easier to
>> edit than HTML. But if it’s not the native format of your data, nor can
>> browsers render it directly, what’s the point of having it?
>>
>> The CollaborationKit extension [0] has two content models,
>> CollaborationHubContent and CollaborationListContent, and they have two
>> different strategies for parsing. CollaborationHubContent takes validated
>> JSON and puts out raw HTML; this is the most straightforward to work with.
>> CollaborationListContent instead puts out wikitext that is fed into the
>> parser, since it is expected that lists can be transcluded onto other
>> pages, and this means using wikitext (as transcluded HTML is not an
>> option). However, this creates a lot of limitations, including parser
>> restrictions that make sense in the context of arbitrary wikitext parsing
>> but not when the markup is provided directly by an extension. The long term
>> plan is for CollaborationListContent to put out HTML, since it’s more
>> straightforward than using a wikitext intermediary that the user does not
>> see anyway.
>>
>> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
>>
>> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić ([hidden email])
>> wrote:
>>
>> Here's my requirement:
>> - a wiki page is one JSON document
>> - when editing, the user edits the JSON directly
>> - when viewing, I have a viewer that turns the JSON into wikitext, and that
>> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>>
>> I have several options, including:
>>
>> 1) hook for a tag like <json>, and write an extension that parses the
>> content between the tags and turns it into wikitext (not ideal, as I don't
>> use any of the existing support for JSON stuff, and also I could have
>> several such tags per page, which does not fit with my requirements)
>>
>> 2) I found the JsonConfig extension by yurik. This allows me to do almost
>> all of the things above - but it returns HTML directly, not wikitext. It
>> doesn't seem trivial to be able to return wikitext instead of HTML, but
>> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>>
>> 3) there is actually a JsonContentHandler in core. But looking through it
>> it seems that this suffers from the same limitations - I can return HTML,
>> but not wikitext.
>>
>> 3 seems to have the advantage to be more actively worked on that 2 (which
>> is not based on 3, probably because it is older than 3). So future goodies
>> like a Json Schema validator will probably go to 3, but not to 2, so I
>> should probably go to 3.
>>
>> Writing this down, one solution could be to create the wikitext, and then
>> call the wikitext parser manually and have it create HTML?
>>
>> I have already developed the extension in 1, and then fully rewritten it in
>> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
>> doing it right, or if should do it completely differently, and also if
>> there are examples of stuff developed in 3, i.e. of extensions or features
>> using the JsonContent class.
>>
>> Example:
>> I have a JSON document
>> { "username": "Denny" }
>> which gets turned into wikitext
>> ''Hello, [[User:Denny|Denny]]!''
>> which then gets turned into the right HTML and displayed to the user, e.g.
>> <i>Hello, <a href="...">Denny</a>!</i>
>>
>> Cheers,
>> Denny
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

signature.asc (465 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Denny Vrandečić-2
In reply to this post by James Hare-4
On Sat, Apr 8, 2017 at 11:30 PM James Hare <[hidden email]> wrote:

> Why, exactly, do you want a wikitext intermediary between your JSON and
> your HTML? The value of wikitext is that it’s a syntax that is easier to
> edit than HTML. But if it’s not the native format of your data, nor can
> browsers render it directly, what’s the point of having it?
>

Ah, good question indeed. The reason is that users would be actually
putting fragments of wikitext into the JSON structure, and then the JSON
structure gets assembled into wikitext. Not only would I prefer to have the
users work with fragments of wikitext than fragments of HTML, but some
things are almost impossible with HTML - e.g. making internal links red or
blue depending on the existence of the article, etc.


>
> The CollaborationKit extension [0] has two content models,
> CollaborationHubContent and CollaborationListContent, and they have two
> different strategies for parsing. CollaborationHubContent takes validated
> JSON and puts out raw HTML; this is the most straightforward to work with.
> CollaborationListContent instead puts out wikitext that is fed into the
> parser, since it is expected that lists can be transcluded onto other
> pages, and this means using wikitext (as transcluded HTML is not an
> option). However, this creates a lot of limitations, including parser
> restrictions that make sense in the context of arbitrary wikitext parsing
> but not when the markup is provided directly by an extension. The long term
> plan is for CollaborationListContent to put out HTML, since it’s more
> straightforward than using a wikitext intermediary that the user does not
> see anyway.
>

Thanks, that is super helpful to know! And thanks for the rationale in
particular.


>
> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
>
> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić ([hidden email])
> wrote:
>
> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> I have several options, including:
>
> 1) hook for a tag like <json>, and write an extension that parses the
> content between the tags and turns it into wikitext (not ideal, as I don't
> use any of the existing support for JSON stuff, and also I could have
> several such tags per page, which does not fit with my requirements)
>
> 2) I found the JsonConfig extension by yurik. This allows me to do almost
> all of the things above - but it returns HTML directly, not wikitext. It
> doesn't seem trivial to be able to return wikitext instead of HTML, but
> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>
> 3) there is actually a JsonContentHandler in core. But looking through it
> it seems that this suffers from the same limitations - I can return HTML,
> but not wikitext.
>
> 3 seems to have the advantage to be more actively worked on that 2 (which
> is not based on 3, probably because it is older than 3). So future goodies
> like a Json Schema validator will probably go to 3, but not to 2, so I
> should probably go to 3.
>
> Writing this down, one solution could be to create the wikitext, and then
> call the wikitext parser manually and have it create HTML?
>
> I have already developed the extension in 1, and then fully rewritten it in
> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
> doing it right, or if should do it completely differently, and also if
> there are examples of stuff developed in 3, i.e. of extensions or features
> using the JsonContent class.
>
> Example:
> I have a JSON document
> { "username": "Denny" }
> which gets turned into wikitext
> ''Hello, [[User:Denny|Denny]]!''
> which then gets turned into the right HTML and displayed to the user, e.g.
> <i>Hello, <a href="...">Denny</a>!</i>
>
> Cheers,
> Denny
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Denny Vrandečić-2
In reply to this post by Gergo Tisza
On Sun, Apr 9, 2017 at 4:11 AM Gergo Tisza <[hidden email]> wrote:

> You probably want to subclass JsonContentHandler and add wikitext transform
>

What does it mean to add wikitext transform?


> and whatever else you need. For schemas, have a look
> at JsonSchemaContentHandler in EventLogging.
>

Thanks, that looks very helpful! I am surprised this is not part of Core,
or a stand-alone component, but it is in EventLogging. Thanks!




> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Denny Vrandečić-2
In reply to this post by Daniel Kinzler-2
On Sun, Apr 9, 2017 at 8:38 AM Daniel Kinzler <[hidden email]>
wrote:

> Am 09.04.2017 um 08:23 schrieb Denny Vrandečić:
> > Here's my requirement:
> > - a wiki page is one JSON document
> > - when editing, the user edits the JSON directly
> > - when viewing, I have a viewer that turns the JSON into wikitext, and
> that
> > wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> Quick thoughts off the top of my head:
>
> Generating wikitext from some other thing is what Scribunto does. Instead
> of
> using the Lua handler, you would make a handler for Json, with whetever
> rules
> you like for generating wikitext.
>

Thanks, I will take a look into how Scribunto does that.

>
> I have ne4ver looked at customizing Scribunto, but this seems to right
> place to
> plug this in.
>
> The alternative is to try to have some kind of "cascading" ContentHandler,
> that
> generates a WikiTextContent object first, and then turns that into HTML.
>
>
Not sure what you mean here - just create wikitext and then run it through
the parser?


>
> --
> Daniel Kinzler
> Senior Software Developer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Denny Vrandečić-2
In reply to this post by Thomas PT
Thanks Tpt, that sounds relevant! I will take a look into the code. More
code examples are really useful to figure this all out :)

On Sun, Apr 9, 2017 at 9:53 AM Thomas PT <[hidden email]> wrote:

> An other maybe relevant example : ProofreadPage has a content model for
> proofreading pages with multiple areas of Wikitext and use some for
> transclusion and others for rendering:
>
> *
> https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContent.php
> *
> https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContentHandler.php
>
> Thomas
>
> > Le 9 avr. 2017 à 18:44, Isarra Yos <[hidden email]> a écrit :
> >
> > Sounds like what they want is what collaborationlistcontent/handler
> does, specifically, at least for now.
> >
> > On 09/04/17 06:30, James Hare wrote:
> >> Why, exactly, do you want a wikitext intermediary between your JSON and
> >> your HTML? The value of wikitext is that it’s a syntax that is easier to
> >> edit than HTML. But if it’s not the native format of your data, nor can
> >> browsers render it directly, what’s the point of having it?
> >>
> >> The CollaborationKit extension [0] has two content models,
> >> CollaborationHubContent and CollaborationListContent, and they have two
> >> different strategies for parsing. CollaborationHubContent takes
> validated
> >> JSON and puts out raw HTML; this is the most straightforward to work
> with.
> >> CollaborationListContent instead puts out wikitext that is fed into the
> >> parser, since it is expected that lists can be transcluded onto other
> >> pages, and this means using wikitext (as transcluded HTML is not an
> >> option). However, this creates a lot of limitations, including parser
> >> restrictions that make sense in the context of arbitrary wikitext
> parsing
> >> but not when the markup is provided directly by an extension. The long
> term
> >> plan is for CollaborationListContent to put out HTML, since it’s more
> >> straightforward than using a wikitext intermediary that the user does
> not
> >> see anyway.
> >>
> >> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
> >>
> >> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić ([hidden email])
> >> wrote:
> >>
> >> Here's my requirement:
> >> - a wiki page is one JSON document
> >> - when editing, the user edits the JSON directly
> >> - when viewing, I have a viewer that turns the JSON into wikitext, and
> that
> >> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
> >>
> >> I have several options, including:
> >>
> >> 1) hook for a tag like <json>, and write an extension that parses the
> >> content between the tags and turns it into wikitext (not ideal, as I
> don't
> >> use any of the existing support for JSON stuff, and also I could have
> >> several such tags per page, which does not fit with my requirements)
> >>
> >> 2) I found the JsonConfig extension by yurik. This allows me to do
> almost
> >> all of the things above - but it returns HTML directly, not wikitext. It
> >> doesn't seem trivial to be able to return wikitext instead of HTML, but
> >> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
> >>
> >> 3) there is actually a JsonContentHandler in core. But looking through
> it
> >> it seems that this suffers from the same limitations - I can return
> HTML,
> >> but not wikitext.
> >>
> >> 3 seems to have the advantage to be more actively worked on that 2
> (which
> >> is not based on 3, probably because it is older than 3). So future
> goodies
> >> like a Json Schema validator will probably go to 3, but not to 2, so I
> >> should probably go to 3.
> >>
> >> Writing this down, one solution could be to create the wikitext, and
> then
> >> call the wikitext parser manually and have it create HTML?
> >>
> >> I have already developed the extension in 1, and then fully rewritten
> it in
> >> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
> >> doing it right, or if should do it completely differently, and also if
> >> there are examples of stuff developed in 3, i.e. of extensions or
> features
> >> using the JsonContent class.
> >>
> >> Example:
> >> I have a JSON document
> >> { "username": "Denny" }
> >> which gets turned into wikitext
> >> ''Hello, [[User:Denny|Denny]]!''
> >> which then gets turned into the right HTML and displayed to the user,
> e.g.
> >> <i>Hello, <a href="...">Denny</a>!</i>
> >>
> >> Cheers,
> >> Denny
> >> _______________________________________________
> >> Wikitech-l mailing list
> >> [hidden email]
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >> _______________________________________________
> >> Wikitech-l mailing list
> >> [hidden email]
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> >
> >
> > _______________________________________________
> > Wikitech-l mailing list
> > [hidden email]
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Brad Jorsch (Anomie)
In reply to this post by Denny Vrandečić-2
On Sun, Apr 9, 2017 at 11:38 AM, Daniel Kinzler <[hidden email]
> wrote:

> Generating wikitext from some other thing is what Scribunto does.


Not really. What Scribunto does is let you run a program to generate
wikitext.

If you wanted to write code that took some JSON and turned it into wikitext
without going through all the trouble of writing an extension and getting
it deployed, you might write that code in Lua as a Scribunto module.


On Mon, Apr 10, 2017 at 12:17 AM, Denny Vrandečić <[hidden email]>
wrote:

> On Sat, Apr 8, 2017 at 11:30 PM James Hare <[hidden email]> wrote:
>
> > Why, exactly, do you want a wikitext intermediary between your JSON and
> > your HTML? The value of wikitext is that it’s a syntax that is easier to
> > edit than HTML. But if it’s not the native format of your data, nor can
> > browsers render it directly, what’s the point of having it?
>
> Ah, good question indeed. The reason is that users would be actually
> putting fragments of wikitext into the JSON structure, and then the JSON
> structure gets assembled into wikitext. Not only would I prefer to have the
> users work with fragments of wikitext than fragments of HTML, but some
> things are almost impossible with HTML - e.g. making internal links red or
> blue depending on the existence of the article, etc.
>

What you probably want to do then is to extend JsonContent and
JsonContentHandler. In the fillParserOutput() method, you'd convert the
JSON to wikitext and then pass that wikitext to the Parser; for the latter
step you could look at how WikitextContent does it.

You might also look at implementing Content::getWikitextForTransclusion()
to let people transclude the resulting wikitext.


--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Denny Vrandečić-2
Thanks Brad, that's perfect! I'll proceed as suggested.

Thanks list, that was really helpful and saved me plenty of trial and
errors!

On Mon, Apr 10, 2017 at 7:07 AM Brad Jorsch (Anomie) <[hidden email]>
wrote:

> On Sun, Apr 9, 2017 at 11:38 AM, Daniel Kinzler <
> [hidden email]
> > wrote:
>
> > Generating wikitext from some other thing is what Scribunto does.
>
>
> Not really. What Scribunto does is let you run a program to generate
> wikitext.
>
> If you wanted to write code that took some JSON and turned it into wikitext
> without going through all the trouble of writing an extension and getting
> it deployed, you might write that code in Lua as a Scribunto module.
>
>
> On Mon, Apr 10, 2017 at 12:17 AM, Denny Vrandečić <[hidden email]>
> wrote:
>
> > On Sat, Apr 8, 2017 at 11:30 PM James Hare <[hidden email]> wrote:
> >
> > > Why, exactly, do you want a wikitext intermediary between your JSON and
> > > your HTML? The value of wikitext is that it’s a syntax that is easier
> to
> > > edit than HTML. But if it’s not the native format of your data, nor can
> > > browsers render it directly, what’s the point of having it?
> >
> > Ah, good question indeed. The reason is that users would be actually
> > putting fragments of wikitext into the JSON structure, and then the JSON
> > structure gets assembled into wikitext. Not only would I prefer to have
> the
> > users work with fragments of wikitext than fragments of HTML, but some
> > things are almost impossible with HTML - e.g. making internal links red
> or
> > blue depending on the existence of the article, etc.
> >
>
> What you probably want to do then is to extend JsonContent and
> JsonContentHandler. In the fillParserOutput() method, you'd convert the
> JSON to wikitext and then pass that wikitext to the Parser; for the latter
> step you could look at how WikitextContent does it.
>
> You might also look at implementing Content::getWikitextForTransclusion()
> to let people transclude the resulting wikitext.
>
>
> --
> Brad Jorsch (Anomie)
> Senior Software Engineer
> Wikimedia Foundation
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Daniel Kinzler-2
In reply to this post by Denny Vrandečić-2
Am 10.04.2017 um 06:17 schrieb Denny Vrandečić:
> Ah, good question indeed. The reason is that users would be actually
> putting fragments of wikitext into the JSON structure, and then the JSON
> structure gets assembled into wikitext. Not only would I prefer to have the
> users work with fragments of wikitext than fragments of HTML, but some
> things are almost impossible with HTML - e.g. making internal links red or
> blue depending on the existence of the article, etc.

Why JSON? JSON is an absolute pain to edit by hand. HTML/XML, as annoying as it
is, is still much better for that than JSON is. And Wikitext is designed to mix
well with HTML.

--
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

C. Scott Ananian
In reply to this post by Denny Vrandečić-2
More related functionality: Parsoid supports JSON as a contenttype, and
emits a special type of table (the same as the HTML table generated by the
HTML handler for JSON in mediawiki).  You can edit this in VE, although w/o
any special support, and Parsoid will serialize it back to JSON.

This could be turned into a very pleasant type-aware editor for JSON in VE
pretty easily.
 --scott

On Sun, Apr 9, 2017 at 2:23 AM, Denny Vrandečić <[hidden email]> wrote:

> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> I have several options, including:
>
> 1) hook for a tag like <json>, and write an extension that parses the
> content between the tags and turns it into wikitext (not ideal, as I don't
> use any of the existing support for JSON stuff, and also I could have
> several such tags per page, which does not fit with my requirements)
>
> 2) I found the JsonConfig extension by yurik. This allows me to do almost
> all of the things above - but it returns HTML directly, not wikitext. It
> doesn't seem trivial to be able to return wikitext instead of HTML, but
> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>
> 3) there is actually a JsonContentHandler in core. But looking through it
> it seems that this suffers from the same limitations - I can return HTML,
> but not wikitext.
>
> 3 seems to have the advantage to be more actively worked on that 2 (which
> is not based on 3, probably because it is older than 3). So future goodies
> like a Json Schema validator will probably go to 3, but not to 2, so I
> should probably go to 3.
>
> Writing this down, one solution could be to create the wikitext, and then
> call the wikitext parser manually and have it create HTML?
>
> I have already developed the extension in 1, and then fully rewritten it in
> 2.  Before I go and rewrite it again in 3, I wanted to ask whether I am
> doing it right, or if should do it completely differently, and also if
> there are examples of stuff developed in 3, i.e. of extensions or features
> using the JsonContent class.
>
> Example:
> I have a JSON document
> { "username": "Denny" }
> which gets turned into wikitext
> ''Hello, [[User:Denny|Denny]]!''
> which then gets turned into the right HTML and displayed to the user, e.g.
> <i>Hello, <a href="...">Denny</a>!</i>
>
> Cheers,
> Denny
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Editing JSON with VisualEditor (was: Editing JSON in MediaWiki)

James Hare-4
On April 10, 2017 at 12:15:34 PM, C. Scott Ananian ([hidden email])
wrote:

More related functionality: Parsoid supports JSON as a contenttype, and
emits a special type of table (the same as the HTML table generated by the
HTML handler for JSON in mediawiki). You can edit this in VE, although w/o
any special support, and Parsoid will serialize it back to JSON.

This could be turned into a very pleasant type-aware editor for JSON in VE
pretty easily.
--scott


This is actually very interesting.

So, take a CollaborationHubContent page:
https://wpx.wmflabs.org/w/index.php/WPX:WikiProject_Dogs

At some point we would like to create a VisualEditor-type editing
interface. Note that behind the scenes, that page is JSON. So in this
“VisualEditor” you would edit fields and such and it would appear you are
editing the page as though it were a regular document, but really you are
just changing values in a JSON blob.

Since Parsoid supports JSON as a content type, would it thus be trivial to
create an extension for VisualEditor to support editing this content type
and others like it?


Thanks,
James Hare
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON with VisualEditor (was: Editing JSON in MediaWiki)

C. Scott Ananian
I *believe* so.  Both Parsoid and Visual Editor have extension interfaces.
However, they haven't really been used by folks outside WMF yet.  I'm eager
to have that change.

We're thinking about proposing a "Workshop" session at wikimania for "How
to write extensions for Parsoid and Visual Editor".  But the area's only
half-baked, it might be more appropriate to do it as a hackathon session
instead.  Thoughts?
  --scott

On Mon, Apr 10, 2017 at 3:20 PM, James Hare <[hidden email]> wrote:

> On April 10, 2017 at 12:15:34 PM, C. Scott Ananian ([hidden email])
> wrote:
>
> More related functionality: Parsoid supports JSON as a contenttype, and
> emits a special type of table (the same as the HTML table generated by the
> HTML handler for JSON in mediawiki). You can edit this in VE, although w/o
> any special support, and Parsoid will serialize it back to JSON.
>
> This could be turned into a very pleasant type-aware editor for JSON in VE
> pretty easily.
> --scott
>
>
> This is actually very interesting.
>
> So, take a CollaborationHubContent page: https://wpx.wmflabs.org/
> w/index.php/WPX:WikiProject_Dogs
>
> At some point we would like to create a VisualEditor-type editing
> interface. Note that behind the scenes, that page is JSON. So in this
> “VisualEditor” you would edit fields and such and it would appear you are
> editing the page as though it were a regular document, but really you are
> just changing values in a JSON blob.
>
> Since Parsoid supports JSON as a content type, would it thus be trivial to
> create an extension for VisualEditor to support editing this content type
> and others like it?
>
>
> Thanks,
> James Hare
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON with VisualEditor (was: Editing JSON in MediaWiki)

James Hare-4
This upcoming hackathon might be too soon, considering we have more urgent
priorities that need to be addressed first (like having it not be broken on
the Beta Cluster). Beyond that, it’s something that can be considered, but
how many people are actually interested in this?


On April 10, 2017 at 12:26:26 PM, C. Scott Ananian ([hidden email])
wrote:

I *believe* so.  Both Parsoid and Visual Editor have extension interfaces.
However, they haven't really been used by folks outside WMF yet.  I'm eager
to have that change.

We're thinking about proposing a "Workshop" session at wikimania for "How
to write extensions for Parsoid and Visual Editor".  But the area's only
half-baked, it might be more appropriate to do it as a hackathon session
instead.  Thoughts?
  --scott

On Mon, Apr 10, 2017 at 3:20 PM, James Hare <[hidden email]> wrote:

> On April 10, 2017 at 12:15:34 PM, C. Scott Ananian ([hidden email])
> wrote:
>
> More related functionality: Parsoid supports JSON as a contenttype, and
> emits a special type of table (the same as the HTML table generated by the
> HTML handler for JSON in mediawiki). You can edit this in VE, although w/o
> any special support, and Parsoid will serialize it back to JSON.
>
> This could be turned into a very pleasant type-aware editor for JSON in VE
> pretty easily.
> --scott
>
>
> This is actually very interesting.
>
> So, take a CollaborationHubContent page: https://wpx.wmflabs.org/
> w/index.php/WPX:WikiProject_Dogs
>
> At some point we would like to create a VisualEditor-type editing
> interface. Note that behind the scenes, that page is JSON. So in this
> “VisualEditor” you would edit fields and such and it would appear you are
> editing the page as though it were a regular document, but really you are
> just changing values in a JSON blob.
>
> Since Parsoid supports JSON as a content type, would it thus be trivial to
> create an extension for VisualEditor to support editing this content type
> and others like it?
>
>
> Thanks,
> James Hare
>



--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON with VisualEditor (was: Editing JSON in MediaWiki)

C. Scott Ananian
I was thinking of the pre-WIkimania hackathon.

There are a *lot* of extensions installed on WMF wikis.  Ideally we'd start
getting extension maintainers to take over the Parsoid/VE support for their
extensions.
 --scott

On Mon, Apr 10, 2017 at 3:37 PM, James Hare <[hidden email]> wrote:

> This upcoming hackathon might be too soon, considering we have more urgent
> priorities that need to be addressed first (like having it not be broken on
> the Beta Cluster). Beyond that, it’s something that can be considered, but
> how many people are actually interested in this?
>
>
> On April 10, 2017 at 12:26:26 PM, C. Scott Ananian ([hidden email])
> wrote:
>
> I *believe* so.  Both Parsoid and Visual Editor have extension
> interfaces.  However, they haven't really been used by folks outside WMF
> yet.  I'm eager to have that change.
>
> We're thinking about proposing a "Workshop" session at wikimania for "How
> to write extensions for Parsoid and Visual Editor".  But the area's only
> half-baked, it might be more appropriate to do it as a hackathon session
> instead.  Thoughts?
>   --scott
>
> On Mon, Apr 10, 2017 at 3:20 PM, James Hare <[hidden email]> wrote:
>
>> On April 10, 2017 at 12:15:34 PM, C. Scott Ananian (
>> [hidden email]) wrote:
>>
>> More related functionality: Parsoid supports JSON as a contenttype, and
>> emits a special type of table (the same as the HTML table generated by the
>> HTML handler for JSON in mediawiki). You can edit this in VE, although w/o
>> any special support, and Parsoid will serialize it back to JSON.
>>
>> This could be turned into a very pleasant type-aware editor for JSON in VE
>> pretty easily.
>> --scott
>>
>>
>> This is actually very interesting.
>>
>> So, take a CollaborationHubContent page: https://wpx.wmflabs.org/
>> w/index.php/WPX:WikiProject_Dogs
>>
>> At some point we would like to create a VisualEditor-type editing
>> interface. Note that behind the scenes, that page is JSON. So in this
>> “VisualEditor” you would edit fields and such and it would appear you are
>> editing the page as though it were a regular document, but really you are
>> just changing values in a JSON blob.
>>
>> Since Parsoid supports JSON as a content type, would it thus be trivial
>> to create an extension for VisualEditor to support editing this content
>> type and others like it?
>>
>>
>> Thanks,
>> James Hare
>>
>
>
>
> --
> (http://cscott.net)
>
>


--
(http://cscott.net)
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Editing JSON in MediaWiki

Denny Vrandečić-2
In reply to this post by Daniel Kinzler-2
Ah, just an internal data structure. In the end, the UI will be form-based
anyway. But in these forms, the user will be able to enter some wikitext
fragments. Yucky, I know.

I prefer JSON over XML only because it is the Zeitgeist, and I expect the
tool support for JSON to grow whereas I don't see similar activity around
XML in the MediaWiki development community.

Or, put differently, the same reason Wikibase is using JSON.

On Mon, Apr 10, 2017 at 11:06 AM Daniel Kinzler <[hidden email]>
wrote:

> Am 10.04.2017 um 06:17 schrieb Denny Vrandečić:
> > Ah, good question indeed. The reason is that users would be actually
> > putting fragments of wikitext into the JSON structure, and then the JSON
> > structure gets assembled into wikitext. Not only would I prefer to have
> the
> > users work with fragments of wikitext than fragments of HTML, but some
> > things are almost impossible with HTML - e.g. making internal links red
> or
> > blue depending on the existence of the article, etc.
>
> Why JSON? JSON is an absolute pain to edit by hand. HTML/XML, as annoying
> as it
> is, is still much better for that than JSON is. And Wikitext is designed
> to mix
> well with HTML.
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
12