Question on external data extension

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Question on external data extension

Jeremi Plazas
Hi y’all,

Wondering if anyone knows if it’s possible to use #get_web_data function
from the ExternalData extension
<https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
includes api.php?action=ask...&format=json as explained here:
https://www.semantic-mediawiki.org/wiki/Ask_API ?

Basically we’re trying to query some data from a private wiki onto a public
wiki.

Thanks,

<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Yaron Koren-2
Hi Jeremi,

If the server on which the public wiki resides can access the private wiki,
then yes; otherwise, no.

-Yaron

On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]> wrote:

> Hi y’all,
>
> Wondering if anyone knows if it’s possible to use #get_web_data function
> from the ExternalData extension
> <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
> includes api.php?action=ask...&format=json as explained here:
> https://www.semantic-mediawiki.org/wiki/Ask_API ?
>
> Basically we’re trying to query some data from a private wiki onto a public
> wiki.
>
> Thanks,
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
Would you kindly steer my noob mind in the direction of how one might make
the public server access the private one? Can the server just be allowed to
access the private wiki?

Thanks so much (again sorry for my n00bness.)

<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 20, 2016 at 3:27:22 PM, Yaron Koren ([hidden email]) wrote:

Hi Jeremi,

If the server on which the public wiki resides can access the private wiki,
then yes; otherwise, no.

-Yaron

On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]> wrote:

> Hi y’all,
>
> Wondering if anyone knows if it’s possible to use #get_web_data function
> from the ExternalData extension
> <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
> includes api.php?action=ask...&format=json as explained here:
> https://www.semantic-mediawiki.org/wiki/Ask_API ?
>
> Basically we’re trying to query some data from a private wiki onto a public
> wiki.
>
> Thanks,
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Yaron Koren-2
That's a systems question; it totally depends on how the private wiki is
structured. If it's a MediaWiki instance running on someone's laptop,
that's totally different from, say, a wiki on a public server that has
password protection on it.

On Tue, Dec 20, 2016 at 5:29 PM, Jeremi Plazas <[hidden email]> wrote:

> Would you kindly steer my noob mind in the direction of how one might make
> the public server access the private one? Can the server just be allowed to
> access the private wiki?
>
> Thanks so much (again sorry for my n00bness.)
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 20, 2016 at 3:27:22 PM, Yaron Koren ([hidden email])
> wrote:
>
> Hi Jeremi,
>
> If the server on which the public wiki resides can access the private wiki,
> then yes; otherwise, no.
>
> -Yaron
>
> On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]> wrote:
>
> > Hi y’all,
> >
> > Wondering if anyone knows if it’s possible to use #get_web_data function
> > from the ExternalData extension
> > <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
> > includes api.php?action=ask...&format=json as explained here:
> > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> >
> > Basically we’re trying to query some data from a private wiki onto a
> public
> > wiki.
> >
> > Thanks,
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
Ok, that makes sense. The private wiki is just private in the sense that
you need a login to view pages. There’s no server password (anymore). The
server is public, but the wiki requires people to be logged in to navigate
it.


<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 20, 2016 at 3:33:09 PM, Yaron Koren ([hidden email]) wrote:

That's a systems question; it totally depends on how the private wiki is
structured. If it's a MediaWiki instance running on someone's laptop,
that's totally different from, say, a wiki on a public server that has
password protection on it.

On Tue, Dec 20, 2016 at 5:29 PM, Jeremi Plazas <[hidden email]> wrote:

> Would you kindly steer my noob mind in the direction of how one might make
> the public server access the private one? Can the server just be allowed to
> access the private wiki?
>
> Thanks so much (again sorry for my n00bness.)
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 20, 2016 at 3:27:22 PM, Yaron Koren ([hidden email])
> wrote:
>
> Hi Jeremi,
>
> If the server on which the public wiki resides can access the private wiki,
> then yes; otherwise, no.
>
> -Yaron
>
> On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]> wrote:
>
> > Hi y’all,
> >
> > Wondering if anyone knows if it’s possible to use #get_web_data function
> > from the ExternalData extension
> > <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
> > includes api.php?action=ask...&format=json as explained here:
> > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> >
> > Basically we’re trying to query some data from a private wiki onto a
> public
> > wiki.
> >
> > Thanks,
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
To recap, when I’m logged into the private wiki, if I point my browser to
https://research.tsadra.org/api.php?action=ask&query=%5B%5BAsian%20Traditions%20of%20Meditation%5D%5D%7C%3Fcitation%7C%3Fcopyright&format=json
It does get me the following:

{"query":{"printrequests":[{"label":"","typeid":"_wpg","mode":2},{"label":"Citation","typeid":"_txt","mode":1,"format":""},{"label":"Copyright","typeid":"_txt","mode":1,"format":""}],"results":{"Asian
Traditions of Meditation":{"printouts":{"Citation":["Eifring, Halvor,
ed. Asian Traditions of Meditation. Honolulu, University of Hawai'i
Press, 2016."],"Copyright":["\u00a9 2016 University of Hawai'i
Press"]},"fulltext":"Asian Traditions of
Meditation","fullurl":"https://research.tsadra.org/index.php/Asian_Traditions_of_Meditation","namespace":0,"exists":""}},"serializer":"SMW\\Serializers\\QueryResultSerializer","version":0.5,"meta":{"hash":"b31f2f12c4f0528e129901370f0a1715","count":1,"offset":0}}}


But having that URL inside the #get_web_data function specifying JSON
format, still doesn’t work. Is this format not understood by #get_web_data ?


<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 20, 2016 at 3:33:09 PM, Yaron Koren ([hidden email]) wrote:

That's a systems question; it totally depends on how the private wiki is
structured. If it's a MediaWiki instance running on someone's laptop,
that's totally different from, say, a wiki on a public server that has
password protection on it.

On Tue, Dec 20, 2016 at 5:29 PM, Jeremi Plazas <[hidden email]> wrote:

> Would you kindly steer my noob mind in the direction of how one might make
> the public server access the private one? Can the server just be allowed to
> access the private wiki?
>
> Thanks so much (again sorry for my n00bness.)
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 20, 2016 at 3:27:22 PM, Yaron Koren ([hidden email])
> wrote:
>
> Hi Jeremi,
>
> If the server on which the public wiki resides can access the private wiki,
> then yes; otherwise, no.
>
> -Yaron
>
> On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]> wrote:
>
> > Hi y’all,
> >
> > Wondering if anyone knows if it’s possible to use #get_web_data function
> > from the ExternalData extension
> > <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
> > includes api.php?action=ask...&format=json as explained here:
> > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> >
> > Basically we’re trying to query some data from a private wiki onto a
> public
> > wiki.
> >
> > Thanks,
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Yaron Koren-2
Well, the issue is either that the server can't access the private wiki, or
that the #get_web_data call isn't parsing the JSON correctly, or (my guess)
both. You haven't specified your #get_web_data call, so I can't say for
sure, but the JSON output produced by SMW is rather complex, so I wouldn't
be surprised if retrieving data from it is a challenge. You'd probably be
better off using the CSV format. Which doesn't take away from the problem
of getting data from a password-protected wiki, though. I don't know how
best to get around that. Though I have to ask: if all of this wiki's
semantic data is being made public, why not just make the wiki itself
public?

On Tue, Dec 20, 2016 at 5:40 PM, Jeremi Plazas <[hidden email]> wrote:

> To recap, when I’m logged into the private wiki, if I point my browser to
> <a href="https://research.tsadra.org/api.php?action=ask&query=%5B%">https://research.tsadra.org/api.php?action=ask&query=%5B%
> 5BAsian%20Traditions%20of%20Meditation%5D%5D%7C%3Fcitation%7C%3Fcopyright&
> format=json
> It does get me the following:
>
> {"query":{"printrequests":[{"label":"","typeid":"_wpg","
> mode":2},{"label":"Citation","typeid":"_txt","mode":1,"
> format":""},{"label":"Copyright","typeid":"_txt","mode":1,"format":""}],"
> results":{"Asian
> Traditions of Meditation":{"printouts":{"Citation":["Eifring, Halvor,
> ed. Asian Traditions of Meditation. Honolulu, University of Hawai'i
> Press, 2016."],"Copyright":["\u00a9 2016 University of Hawai'i
> Press"]},"fulltext":"Asian Traditions of
> Meditation","fullurl":"https://research.tsadra.org/index.
> php/Asian_Traditions_of_Meditation","namespace":0,"
> exists":""}},"serializer":"SMW\\Serializers\\QueryResultSerializer","
> version":0.5,"meta":{"hash":"b31f2f12c4f0528e129901370f0a17
> 15","count":1,"offset":0}}}
>
>
> But having that URL inside the #get_web_data function specifying JSON
> format, still doesn’t work. Is this format not understood by #get_web_data
> ?
>
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 20, 2016 at 3:33:09 PM, Yaron Koren ([hidden email])
> wrote:
>
> That's a systems question; it totally depends on how the private wiki is
> structured. If it's a MediaWiki instance running on someone's laptop,
> that's totally different from, say, a wiki on a public server that has
> password protection on it.
>
> On Tue, Dec 20, 2016 at 5:29 PM, Jeremi Plazas <[hidden email]> wrote:
>
> > Would you kindly steer my noob mind in the direction of how one might
> make
> > the public server access the private one? Can the server just be allowed
> to
> > access the private wiki?
> >
> > Thanks so much (again sorry for my n00bness.)
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> >
> > On December 20, 2016 at 3:27:22 PM, Yaron Koren ([hidden email])
> > wrote:
> >
> > Hi Jeremi,
> >
> > If the server on which the public wiki resides can access the private
> wiki,
> > then yes; otherwise, no.
> >
> > -Yaron
> >
> > On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]>
> wrote:
> >
> > > Hi y’all,
> > >
> > > Wondering if anyone knows if it’s possible to use #get_web_data
> function
> > > from the ExternalData extension
> > > <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL
> that
> > > includes api.php?action=ask...&format=json as explained here:
> > > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> > >
> > > Basically we’re trying to query some data from a private wiki onto a
> > public
> > > wiki.
> > >
> > > Thanks,
> > >
> > > <http://www.tsadra.org>
> > >
> > > Jeremi Plazas
> > > [hidden email]
> > >
> > > Assistant Director of Research in Boulder, CO
> > > www.tsadra.org // www.translationandtransmission.org
> > > [image: Tsadra Foundation] <http://www.tsadra.org>
> > >
> > > Donate
> > > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > > hosted_button_id=JAD5XXS9ZXBA8>
> > > ------------------------------------------------------------
> > > ------------------
> > > Developer Access Program for Intel Xeon Phi Processors
> > > Access to Intel Xeon Phi processor-based developer platforms.
> > > With one year of Intel Parallel Studio XE.
> > > Training and support from Colfax.
> > > Order your platform today.http://sdm.link/intel
> > > _______________________________________________
> > > Semediawiki-user mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> > >
> >
> >
> >
> > --
> > WikiWorks · MediaWiki Consulting · http://wikiworks.com
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

lbillett
In reply to this post by Jeremi Plazas
Jeremi Plazas wrote
Wondering if anyone knows if it’s possible to use #get_web_data function
from the ExternalData extension
<https://www.mediawiki.org/wiki/Extension:External_Data> with a URL that
includes api.php?action=ask...&format=json as explained here:
https://www.semantic-mediawiki.org/wiki/Ask_API ?
While not specifically related to Extension:External Data, this discussion [1] a little while back might be applicable. Not sure what provision exists (or is needed) to query a private wiki.

[1] http://wikimedia.7.x6.nabble.com/Query-Properties-from-Another-Wiki-td5055986.html
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
In reply to this post by Yaron Koren-2
Hey Yaron,

Thanks for the followup, a couple of things. We don’t want ALL the
properties on the private wiki to be public, that’s kind of why we’re doing
this. Our private wiki is a repository of books, including full content in
some cases. We only want some of the citation information for each book to
be represented on the public wiki. Hence the trouble we’re trying to go
through.

Also, the API doesn’t seem to provide a CSV format. The #get_web_data call
does have that option but the API or Ask API for that matter, don’t.
https://www.mediawiki.org/w/api.php
https://www.semantic-mediawiki.org/w/api.php
The formats only seem to include: json, jsonfm, none, php, phpfm, rawfm,
xml, xmlfm. No CSV.

I did encounter another way on
https://www.mediawiki.org/wiki/Extension:External_Data#Getting_data_from_a_non-API_text_file

*For wiki pages that have additional information, the External Data
extension provides a way to create an API of your own, at least for CSV
data. To get this working, first place the data you want accessed in its
own wiki page, in CSV format, with the headers as the top row of data (see
here for an example). Then, the special page 'GetData' will provide an
"instant API" for accessing either certain rows of that data, or the entire
table. By adding "field-name=value" to the URL, you can limit the set of
rows returned.*

*A URL for the 'GetData' page can then be used in a call to #get_web_data,
just as any other data URL would be; the data will be returned as a CSV
file with a header row, so the 'format' parameter of #get_web_data should
be set to 'CSV with header'. See here for an example of such data being
retrieved and displayed using #get_web_data and #for_external_table. In
this way, you can use any table-based data within your wiki without the
need for custom programming.*

But i’m not sure if that would solve our problems. I would have to create a
csv page for each book. Just not practical.

<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 20, 2016 at 7:18:11 PM, Yaron Koren ([hidden email]) wrote:

Well, the issue is either that the server can't access the private wiki, or
that the #get_web_data call isn't parsing the JSON correctly, or (my guess)
both. You haven't specified your #get_web_data call, so I can't say for
sure, but the JSON output produced by SMW is rather complex, so I wouldn't
be surprised if retrieving data from it is a challenge. You'd probably be
better off using the CSV format. Which doesn't take away from the problem
of getting data from a password-protected wiki, though. I don't know how
best to get around that. Though I have to ask: if all of this wiki's
semantic data is being made public, why not just make the wiki itself
public?

On Tue, Dec 20, 2016 at 5:40 PM, Jeremi Plazas <[hidden email]> wrote:

> To recap, when I’m logged into the private wiki, if I point my browser to
> <a href="https://research.tsadra.org/api.php?action=ask&query=%5B%">https://research.tsadra.org/api.php?action=ask&query=%5B%
> 5BAsian%20Traditions%20of%20Meditation%5D%5D%7C%3Fcitation%7C%3Fcopyright&
> format=json
> It does get me the following:
>
> {"query":{"printrequests":[{"label":"","typeid":"_wpg","
> mode":2},{"label":"Citation","typeid":"_txt","mode":1,"
> format":""},{"label":"Copyright","typeid":"_txt","mode":1,"format":""}],"
> results":{"Asian
> Traditions of Meditation":{"printouts":{"Citation":["Eifring, Halvor,
> ed. Asian Traditions of Meditation. Honolulu, University of Hawai'i
> Press, 2016."],"Copyright":["\u00a9 2016 University of Hawai'i
> Press"]},"fulltext":"Asian Traditions of
> Meditation","fullurl":"https://research.tsadra.org/index.
> php/Asian_Traditions_of_Meditation","namespace":0,"
> exists":""}},"serializer":"SMW\\Serializers\\QueryResultSerializer","
> version":0.5,"meta":{"hash":"b31f2f12c4f0528e129901370f0a17
> 15","count":1,"offset":0}}}
>
>
> But having that URL inside the #get_web_data function specifying JSON
> format, still doesn’t work. Is this format not understood by #get_web_data
> ?
>
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 20, 2016 at 3:33:09 PM, Yaron Koren ([hidden email])
> wrote:
>
> That's a systems question; it totally depends on how the private wiki is
> structured. If it's a MediaWiki instance running on someone's laptop,
> that's totally different from, say, a wiki on a public server that has
> password protection on it.
>
> On Tue, Dec 20, 2016 at 5:29 PM, Jeremi Plazas <[hidden email]> wrote:
>
> > Would you kindly steer my noob mind in the direction of how one might
> make
> > the public server access the private one? Can the server just be allowed
> to
> > access the private wiki?
> >
> > Thanks so much (again sorry for my n00bness.)
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> >
> > On December 20, 2016 at 3:27:22 PM, Yaron Koren ([hidden email])
> > wrote:
> >
> > Hi Jeremi,
> >
> > If the server on which the public wiki resides can access the private
> wiki,
> > then yes; otherwise, no.
> >
> > -Yaron
> >
> > On Tue, Dec 20, 2016 at 5:05 PM, Jeremi Plazas <[hidden email]>
> wrote:
> >
> > > Hi y’all,
> > >
> > > Wondering if anyone knows if it’s possible to use #get_web_data
> function
> > > from the ExternalData extension
> > > <https://www.mediawiki.org/wiki/Extension:External_Data> with a URL
> that
> > > includes api.php?action=ask...&format=json as explained here:
> > > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> > >
> > > Basically we’re trying to query some data from a private wiki onto a
> > public
> > > wiki.
> > >
> > > Thanks,
> > >
> > > <http://www.tsadra.org>
> > >
> > > Jeremi Plazas
> > > [hidden email]
> > >
> > > Assistant Director of Research in Boulder, CO
> > > www.tsadra.org // www.translationandtransmission.org
> > > [image: Tsadra Foundation] <http://www.tsadra.org>
> > >
> > > Donate
> > > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > > hosted_button_id=JAD5XXS9ZXBA8>
> > > ------------------------------------------------------------
> > > ------------------
> > > Developer Access Program for Intel Xeon Phi Processors
> > > Access to Intel Xeon Phi processor-based developer platforms.
> > > With one year of Intel Parallel Studio XE.
> > > Training and support from Colfax.
> > > Order your platform today.http://sdm.link/intel
> > > _______________________________________________
> > > Semediawiki-user mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> > >
> >
> >
> >
> > --
> > WikiWorks · MediaWiki Consulting · http://wikiworks.com
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
In reply to this post by lbillett
Thanks for this suggestion, I will look into the SemanticExternalQueryLookup
<https://github.com/SemanticMediaWiki/SemanticExternalQueryLookup>
 extension.

<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 21, 2016 at 5:28:39 AM, lbillett ([hidden email]) wrote:

Jeremi Plazas wrote
> Wondering if anyone knows if it’s possible to use #get_web_data function
> from the ExternalData extension
> &lt;https://www.mediawiki.org/wiki/Extension:External_Data&gt; with a URL
> that
> includes api.php?action=ask...&format=json as explained here:
> https://www.semantic-mediawiki.org/wiki/Ask_API ?

While not specifically related to Extension:External Data, this discussion
[1] a little while back might be applicable. Not sure what provision exists
(or is needed) to query a private wiki.

[1]
http://wikimedia.7.x6.nabble.com/Query-Properties-from-Another-Wiki-td5055986.html



--
View this message in context:
http://wikimedia.7.x6.nabble.com/Question-on-external-data-extension-tp5070246p5070285.html
Sent from the Semantic Mediawiki - User mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Yaron Koren-2
Hi,

It should be noted that, whether you're using External Data or
SemanticExternalQueryLookup, I'm pretty sure that users will be able to
access all SMW data on that private wiki, if they can access any of it.
There's "security through obscurity" there, in that users won't know the
names of all the categories and properties on the private wiki, plus most
users won't know how to create their own external query (or have any desire
to), but in theory, all the information will be accessible.

As for CSV output - I forgot that the API doesn't include a CSV option. You
can still do it, though, using Special:Ask and the "csv" format. Come to
think of it, the "json" and "xml" formats of Special:Ask may have a simpler
output than their API equivalents.

-Yaron

On Wed, Dec 21, 2016 at 10:46 AM, Jeremi Plazas <[hidden email]> wrote:

> Thanks for this suggestion, I will look into the
> SemanticExternalQueryLookup
> <https://github.com/SemanticMediaWiki/SemanticExternalQueryLookup>
>  extension.
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 21, 2016 at 5:28:39 AM, lbillett ([hidden email]) wrote:
>
> Jeremi Plazas wrote
> > Wondering if anyone knows if it’s possible to use #get_web_data function
> > from the ExternalData extension
> > &lt;https://www.mediawiki.org/wiki/Extension:External_Data&gt; with a
> URL
> > that
> > includes api.php?action=ask...&format=json as explained here:
> > https://www.semantic-mediawiki.org/wiki/Ask_API ?
>
> While not specifically related to Extension:External Data, this discussion
> [1] a little while back might be applicable. Not sure what provision exists
> (or is needed) to query a private wiki.
>
> [1]
> http://wikimedia.7.x6.nabble.com/Query-Properties-from-
> Another-Wiki-td5055986.html
>
>
>
> --
> View this message in context:
> http://wikimedia.7.x6.nabble.com/Question-on-external-data-
> extension-tp5070246p5070285.html
> Sent from the Semantic Mediawiki - User mailing list archive at Nabble.com.
>
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
Right, yes, I imagined that would be true about a cross-wiki query
technically opening it all up. But I think we’re comfortable enough with
the level of “obscurity” it provides.

Also, Special:Ask is what we were using before with a proxy redirect for
getting through the server firewall. The private wiki is now accessible
publicly (no firewall anymore). But the wiki remains private with a login
required. So I thought the following call would work (when logged in):

{{#get_web_data:url=
https://research.tsadra.org/index.php?title=Special%3AAsk&q=[[{{urlencode:{{PAGENAME}}}}]]&po=translatorlibwiki%0Aauthorlibwiki%0Acontributorslibwiki%0Afulltitle%0Apublisherlibwiki%0Aclassification%0Asubclasslibwiki%0Acitation%0Aplace%0Apubdate%0Aisbn%0Atotalpages%0Alibrarytype%0Alibrarylocation%0Asectionarea%0Apagecreationdate%0Asourcetitle%0Asourceauthorlibwiki%0Acitationofsourcetexts%0Afrontcoverimg%0Abackcoverimg%0Aspineimg%0Arelatedpages%0Aeditorlibwiki%0Aserieslibwiki%0Anumberinserieslibwiki%0Aindividuals%0Aistranslation%0Avolumenumber%0Atotalvolumes%0Apubdatenote%0Atranslatedfrom%0Anumberofcopies%0Alastinventorydate%0Amissing%0Aotheredition%0A&*p[format]=csv&p[sep]=,|format=csv
with header|data=*
classification=classification,subclasslibwiki=subclasslibwiki,authorlibwiki=authorlibwiki,contributorslibwiki=contributorslibwiki,fulltitle=fulltitle,publisherlibwiki=publisherlibwiki,citation=citation,place=place,pubdate=pubdate,isbn=isbn,contributors=contributors,totalpages=totalpages,librarytype=librarytype,librarylocation=librarylocation,sectionarea=sectionarea,pagecreationdate=pagecreationdate,sourcetitle=sourcetitle,sourceauthorlibwiki=sourceauthorlibwiki,citationofsourcetexts=citationofsourcetexts,frontcoverimg=frontcoverimg,frontcoverimglibwiki=frontcoverimglibwiki,backcoverimg=backcoverimg,backcoverimglibwiki=backcoverimglibwiki,spineimg=spineimg,translatorlibwiki=translatorlibwiki,relatedpages=relatedpages,editorlibwiki=editorlibwiki,serieslibwiki=serieslibwiki,numberinserieslibwiki=numberinserieslibwiki,individuals=individuals,istranslation=istranslation,volumenumber=volumenumber,totalvolumes=totalvolumes,pubdatenote=pubdatenote,translatedfrom=translatedfrom,numberofcopies=numberofcopies,lastinventorydate=lastinventorydate,missing=missing,otheredition=otheredition}}

But somehow this isn’t doing it. When I take the URL itself and paste it
into my browser, it works and downloads a CSV file (as long as I’m logged
in). But inside the #get_web_data it doesn’t work.

Any clues?


<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 21, 2016 at 9:50:02 AM, Yaron Koren ([hidden email]) wrote:

Hi,

It should be noted that, whether you're using External Data or
SemanticExternalQueryLookup, I'm pretty sure that users will be able to
access all SMW data on that private wiki, if they can access any of it.
There's "security through obscurity" there, in that users won't know the
names of all the categories and properties on the private wiki, plus most
users won't know how to create their own external query (or have any desire
to), but in theory, all the information will be accessible.

As for CSV output - I forgot that the API doesn't include a CSV option. You
can still do it, though, using Special:Ask and the "csv" format. Come to
think of it, the "json" and "xml" formats of Special:Ask may have a simpler
output than their API equivalents.

-Yaron

On Wed, Dec 21, 2016 at 10:46 AM, Jeremi Plazas <[hidden email]> wrote:

> Thanks for this suggestion, I will look into the
> SemanticExternalQueryLookup
> <https://github.com/SemanticMediaWiki/SemanticExternalQueryLookup>
>  extension.
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 21, 2016 at 5:28:39 AM, lbillett ([hidden email]) wrote:
>
> Jeremi Plazas wrote
> > Wondering if anyone knows if it’s possible to use #get_web_data function
> > from the ExternalData extension
> > &lt;https://www.mediawiki.org/wiki/Extension:External_Data&gt; with a
> URL
> > that
> > includes api.php?action=ask...&format=json as explained here:
> > https://www.semantic-mediawiki.org/wiki/Ask_API ?
>
> While not specifically related to Extension:External Data, this discussion
> [1] a little while back might be applicable. Not sure what provision exists
> (or is needed) to query a private wiki.
>
> [1]
> http://wikimedia.7.x6.nabble.com/Query-Properties-from-
> Another-Wiki-td5055986.html
>
>
>
> --
> View this message in context:
> http://wikimedia.7.x6.nabble.com/Question-on-external-data-
> extension-tp5070246p5070285.html
> Sent from the Semantic Mediawiki - User mailing list archive at Nabble.com.
>
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Yaron Koren-2
Presumably, the issue is that External Data (or SEQL, if you switched to
that) can't access the wiki. If you're using Special:Ask as the data
source, adding this to LocalSettings.php on the private wiki may fix the
problem:

$wgWhitelistRead = array( 'Special:Ask' );

On Wed, Dec 21, 2016 at 12:03 PM, Jeremi Plazas <[hidden email]> wrote:

> Right, yes, I imagined that would be true about a cross-wiki query
> technically opening it all up. But I think we’re comfortable enough with
> the level of “obscurity” it provides.
>
> Also, Special:Ask is what we were using before with a proxy redirect for
> getting through the server firewall. The private wiki is now accessible
> publicly (no firewall anymore). But the wiki remains private with a login
> required. So I thought the following call would work (when logged in):
>
> {{#get_web_data:url=
> https://research.tsadra.org/index.php?title=Special%3AAsk&
> q=[[{{urlencode:{{PAGENAME}}}}]]&po=translatorlibwiki%0Aauthorlibwiki%
> 0Acontributorslibwiki%0Afulltitle%0Apublisherlibwiki%0Aclassification%
> 0Asubclasslibwiki%0Acitation%0Aplace%0Apubdate%0Aisbn%
> 0Atotalpages%0Alibrarytype%0Alibrarylocation%0Asectionarea%
> 0Apagecreationdate%0Asourcetitle%0Asourceauthorlibwiki%
> 0Acitationofsourcetexts%0Afrontcoverimg%0Abackcoverimg%0Aspineimg%
> 0Arelatedpages%0Aeditorlibwiki%0Aserieslibwiki%0Anumberinserieslibwiki%
> 0Aindividuals%0Aistranslation%0Avolumenumber%0Atotalvolumes%0Apubdatenote%
> 0Atranslatedfrom%0Anumberofcopies%0Alastinventorydate%0Amissing%
> 0Aotheredition%0A&*p[format]=csv&p[sep]=,|format=csv
> with header|data=*
> classification=classification,subclasslibwiki=
> subclasslibwiki,authorlibwiki=authorlibwiki,contributorslibwiki=
> contributorslibwiki,fulltitle=fulltitle,publisherlibwiki=
> publisherlibwiki,citation=citation,place=place,pubdate=pubdate,isbn=isbn,
> contributors=contributors,totalpages=totalpages,librarytype=librarytype,
> librarylocation=librarylocation,sectionarea=sectionarea,pagecreationdate=
> pagecreationdate,sourcetitle=sourcetitle,sourceauthorlibwiki=
> sourceauthorlibwiki,citationofsourcetexts=citationofsourcetexts,
> frontcoverimg=frontcoverimg,frontcoverimglibwiki=frontcoverimglibwiki,
> backcoverimg=backcoverimg,backcoverimglibwiki=
> backcoverimglibwiki,spineimg=spineimg,translatorlibwiki=translatorlibwiki,
> relatedpages=relatedpages,editorlibwiki=editorlibwiki,
> serieslibwiki=serieslibwiki,numberinserieslibwiki=numberinserieslibwiki,
> individuals=individuals,istranslation=istranslation,
> volumenumber=volumenumber,totalvolumes=totalvolumes,
> pubdatenote=pubdatenote,translatedfrom=translatedfrom,
> numberofcopies=numberofcopies,lastinventorydate=lastinventorydate,missing=
> missing,otheredition=otheredition}}
>
> But somehow this isn’t doing it. When I take the URL itself and paste it
> into my browser, it works and downloads a CSV file (as long as I’m logged
> in). But inside the #get_web_data it doesn’t work.
>
> Any clues?
>
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 21, 2016 at 9:50:02 AM, Yaron Koren ([hidden email])
> wrote:
>
> Hi,
>
> It should be noted that, whether you're using External Data or
> SemanticExternalQueryLookup, I'm pretty sure that users will be able to
> access all SMW data on that private wiki, if they can access any of it.
> There's "security through obscurity" there, in that users won't know the
> names of all the categories and properties on the private wiki, plus most
> users won't know how to create their own external query (or have any desire
> to), but in theory, all the information will be accessible.
>
> As for CSV output - I forgot that the API doesn't include a CSV option. You
> can still do it, though, using Special:Ask and the "csv" format. Come to
> think of it, the "json" and "xml" formats of Special:Ask may have a simpler
> output than their API equivalents.
>
> -Yaron
>
> On Wed, Dec 21, 2016 at 10:46 AM, Jeremi Plazas <[hidden email]> wrote:
>
> > Thanks for this suggestion, I will look into the
> > SemanticExternalQueryLookup
> > <https://github.com/SemanticMediaWiki/SemanticExternalQueryLookup>
> >  extension.
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> >
> > On December 21, 2016 at 5:28:39 AM, lbillett ([hidden email])
> wrote:
> >
> > Jeremi Plazas wrote
> > > Wondering if anyone knows if it’s possible to use #get_web_data
> function
> > > from the ExternalData extension
> > > &lt;https://www.mediawiki.org/wiki/Extension:External_Data&gt; with a
> > URL
> > > that
> > > includes api.php?action=ask...&format=json as explained here:
> > > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> >
> > While not specifically related to Extension:External Data, this
> discussion
> > [1] a little while back might be applicable. Not sure what provision
> exists
> > (or is needed) to query a private wiki.
> >
> > [1]
> > http://wikimedia.7.x6.nabble.com/Query-Properties-from-
> > Another-Wiki-td5055986.html
> >
> >
> >
> > --
> > View this message in context:
> > http://wikimedia.7.x6.nabble.com/Question-on-external-data-
> > extension-tp5070246p5070285.html
> > Sent from the Semantic Mediawiki - User mailing list archive at
> Nabble.com.
> >
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: Question on external data extension

Jeremi Plazas
So thanks Yaron,

$wgWhitelistRead = array( 'Special:Ask' );

does work with a direct request:
{{#get_web_data:url=
https://research.tsadra.org/index.php?title=Special%3AAsk&q=

However, for everyone’s information, it does NOT work the SEQL. The API
apparently doesn’t allow access to whitelisted pages, but requires a full
login.

Why do I get the readapidenied error?
<https://www.mediawiki.org/wiki/API:FAQ#do_I_get_the_readapidenied_error.3F>
The wiki you're querying contains private content and requires users to log
in in order to be able to read all pages. This means that a client needs to
be logged in to query any information at all through the API. See the quick
start guide for more information. *It's not currently possible to query the
contents of whitelisted pages without logging in*, even though they're
available in the regular user interface.


Cheers,

<http://www.tsadra.org>

Jeremi Plazas
[hidden email]

Assistant Director of Research in Boulder, CO
www.tsadra.org // www.translationandtransmission.org
[image: Tsadra Foundation] <http://www.tsadra.org>

Donate
<https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=JAD5XXS9ZXBA8>

On December 21, 2016 at 10:47:42 AM, Yaron Koren ([hidden email])
wrote:

Presumably, the issue is that External Data (or SEQL, if you switched to
that) can't access the wiki. If you're using Special:Ask as the data
source, adding this to LocalSettings.php on the private wiki may fix the
problem:

$wgWhitelistRead = array( 'Special:Ask' );

On Wed, Dec 21, 2016 at 12:03 PM, Jeremi Plazas <[hidden email]> wrote:

> Right, yes, I imagined that would be true about a cross-wiki query
> technically opening it all up. But I think we’re comfortable enough with
> the level of “obscurity” it provides.
>
> Also, Special:Ask is what we were using before with a proxy redirect for
> getting through the server firewall. The private wiki is now accessible
> publicly (no firewall anymore). But the wiki remains private with a login
> required. So I thought the following call would work (when logged in):
>
> {{#get_web_data:url=
> https://research.tsadra.org/index.php?title=Special%3AAsk&
> q=[[{{urlencode:{{PAGENAME}}}}]]&po=translatorlibwiki%0Aauthorlibwiki%
> 0Acontributorslibwiki%0Afulltitle%0Apublisherlibwiki%0Aclassification%
> 0Asubclasslibwiki%0Acitation%0Aplace%0Apubdate%0Aisbn%
> 0Atotalpages%0Alibrarytype%0Alibrarylocation%0Asectionarea%
> 0Apagecreationdate%0Asourcetitle%0Asourceauthorlibwiki%
> 0Acitationofsourcetexts%0Afrontcoverimg%0Abackcoverimg%0Aspineimg%
> 0Arelatedpages%0Aeditorlibwiki%0Aserieslibwiki%0Anumberinserieslibwiki%
> 0Aindividuals%0Aistranslation%0Avolumenumber%0Atotalvolumes%0Apubdatenote%
> 0Atranslatedfrom%0Anumberofcopies%0Alastinventorydate%0Amissing%
> 0Aotheredition%0A&*p[format]=csv&p[sep]=,|format=csv
> with header|data=*
> classification=classification,subclasslibwiki=
> subclasslibwiki,authorlibwiki=authorlibwiki,contributorslibwiki=
> contributorslibwiki,fulltitle=fulltitle,publisherlibwiki=
> publisherlibwiki,citation=citation,place=place,pubdate=pubdate,isbn=isbn,
> contributors=contributors,totalpages=totalpages,librarytype=librarytype,
> librarylocation=librarylocation,sectionarea=sectionarea,pagecreationdate=
> pagecreationdate,sourcetitle=sourcetitle,sourceauthorlibwiki=
> sourceauthorlibwiki,citationofsourcetexts=citationofsourcetexts,
> frontcoverimg=frontcoverimg,frontcoverimglibwiki=frontcoverimglibwiki,
> backcoverimg=backcoverimg,backcoverimglibwiki=
> backcoverimglibwiki,spineimg=spineimg,translatorlibwiki=translatorlibwiki,
> relatedpages=relatedpages,editorlibwiki=editorlibwiki,
> serieslibwiki=serieslibwiki,numberinserieslibwiki=numberinserieslibwiki,
> individuals=individuals,istranslation=istranslation,
> volumenumber=volumenumber,totalvolumes=totalvolumes,
> pubdatenote=pubdatenote,translatedfrom=translatedfrom,
> numberofcopies=numberofcopies,lastinventorydate=lastinventorydate,missing=
> missing,otheredition=otheredition}}
>
> But somehow this isn’t doing it. When I take the URL itself and paste it
> into my browser, it works and downloads a CSV file (as long as I’m logged
> in). But inside the #get_web_data it doesn’t work.
>
> Any clues?
>
>
> <http://www.tsadra.org>
>
> Jeremi Plazas
> [hidden email]
>
> Assistant Director of Research in Boulder, CO
> www.tsadra.org // www.translationandtransmission.org
> [image: Tsadra Foundation] <http://www.tsadra.org>
>
> Donate
> <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> hosted_button_id=JAD5XXS9ZXBA8>
>
> On December 21, 2016 at 9:50:02 AM, Yaron Koren ([hidden email])
> wrote:
>
> Hi,
>
> It should be noted that, whether you're using External Data or
> SemanticExternalQueryLookup, I'm pretty sure that users will be able to
> access all SMW data on that private wiki, if they can access any of it.
> There's "security through obscurity" there, in that users won't know the
> names of all the categories and properties on the private wiki, plus most
> users won't know how to create their own external query (or have any desire
> to), but in theory, all the information will be accessible.
>
> As for CSV output - I forgot that the API doesn't include a CSV option. You
> can still do it, though, using Special:Ask and the "csv" format. Come to
> think of it, the "json" and "xml" formats of Special:Ask may have a simpler
> output than their API equivalents.
>
> -Yaron
>
> On Wed, Dec 21, 2016 at 10:46 AM, Jeremi Plazas <[hidden email]> wrote:
>
> > Thanks for this suggestion, I will look into the
> > SemanticExternalQueryLookup
> > <https://github.com/SemanticMediaWiki/SemanticExternalQueryLookup>
> >  extension.
> >
> > <http://www.tsadra.org>
> >
> > Jeremi Plazas
> > [hidden email]
> >
> > Assistant Director of Research in Boulder, CO
> > www.tsadra.org // www.translationandtransmission.org
> > [image: Tsadra Foundation] <http://www.tsadra.org>
> >
> > Donate
> > <https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&
> > hosted_button_id=JAD5XXS9ZXBA8>
> >
> > On December 21, 2016 at 5:28:39 AM, lbillett ([hidden email])
> wrote:
> >
> > Jeremi Plazas wrote
> > > Wondering if anyone knows if it’s possible to use #get_web_data
> function
> > > from the ExternalData extension
> > > &lt;https://www.mediawiki.org/wiki/Extension:External_Data&gt; with a
> > URL
> > > that
> > > includes api.php?action=ask...&format=json as explained here:
> > > https://www.semantic-mediawiki.org/wiki/Ask_API ?
> >
> > While not specifically related to Extension:External Data, this
> discussion
> > [1] a little while back might be applicable. Not sure what provision
> exists
> > (or is needed) to query a private wiki.
> >
> > [1]
> > http://wikimedia.7.x6.nabble.com/Query-Properties-from-
> > Another-Wiki-td5055986.html
> >
> >
> >
> > --
> > View this message in context:
> > http://wikimedia.7.x6.nabble.com/Question-on-external-data-
> > extension-tp5070246p5070285.html
> > Sent from the Semantic Mediawiki - User mailing list archive at
> Nabble.com.
> >
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> > ------------------------------------------------------------
> > ------------------
> > Developer Access Program for Intel Xeon Phi Processors
> > Access to Intel Xeon Phi processor-based developer platforms.
> > With one year of Intel Parallel Studio XE.
> > Training and support from Colfax.
> > Order your platform today.http://sdm.link/intel
> > _______________________________________________
> > Semediawiki-user mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/semediawiki-user
> >
>
>
>
> --
> WikiWorks · MediaWiki Consulting · http://wikiworks.com
> ------------------------------------------------------------
> ------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/intel
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user