Semantic MediaWiki and Wikidata ContentHandler

[James HK]

Hi,

==WD ContentHandler ==
Now that the Wikidata project is introducing a new ContentHandler [1],
we where curious if in near future SMW would make use of those methods
to store its own objects in the same way and reduces its reliance on
non-mediawiki tables/methods.

As Wikidata and SMW share the same bottlenecks in terms of
performance, caching, scalability it might prove beneficial for SMW to
use similar methods to store and fetch dataobjects from a mw-database
while at the same time reduces its reliance on SQL specific code and
lift the ban on other supported database systems(Oracle,Sqlite etc.).

The reason why we are asking is that we hope that some of the work
that has be done in SMW is reflected in Wikidata as visa versa to
ensure that both developments share some similarities to the extend
that their "plug and playable"  (meaning causing the least
interruptions when installed together and make it easier for code
reviewers to understand the inner workings which increases stability).

==SMW and WD ==
Furthermore, with the Wikidata project on the horizon, we would be
interested to see how SMW positioned and distinguished itself from WD
in light of recent developments. (A simple table could help to explain
those differences fairly quick while at the same time allows some
directional guidance for the future of SMW.)

==Validator==
The Validator provides validation methods for datatypes (and parameter
definitions etc.) and WD certainly needs similar methods, is it
planned that those methods (or part of it) are being moved into the
core?

[1] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki%2Fcore.git;a=commit;h=337f65c03b59eaddc038bc1a0fb150dc97e03154

Cheers,

mwjames

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[Markus Krötzsch-2]

Hi James, hi all,

the relationship and synergies of SMW and Wikidata are of course quite
important to us, so I am taking the time to write a slightly longer
email that I also send to the user list.

At the moment, Wikidata is still under development, so it is a bit early
to make concrete plans on how to align the project.
Here are some things that I can already answer now:


(1) What is the software in "Wikidata"?

The software underlying Wikidata is called Wikibase. It consists of two
extensions: Repository (where data is stored; this will run at
wikidata.org) and Client (that pulls data into other MediaWiki sites;
this will run at wikipedia.org). Both extensions will be installable on
MediaWiki sites to get Wikidata-like features there.


(2) What are the main differences between SMW and Wikidata?

Wikidata is being built with SMW in mind, and the experiences with SMW
are a valuable input for its design. The main use case of Wikidata (a
centralised, multi-lingual site that serves as a data repository) is
different from that of SMW (a data-enhanced MediaWiki), and this leads
to a number of differences.

The data model of Wikidata is a bit more complex than that of SMW. For
example, every fact will also have an (optional) reference. Wikidata
will also use more complex datatypes (e.g., since many text values can
be translated). However, both Wikidata and SMW are still mostly based on
property-value assignments, and Wikidata will have a Property namespace.
Also, the more complex structures in Wikidata could be captured in SMW
using internal objects.

The user interface of Wikibase Repository will be based on input forms,
and thus quite different from SMW. The UI will be a bit like a
full-page, editable Factbox, but much more advanced, with a lot of
support for entering values.

It is not defined yet what kind of query language Wikidata will support
in Phase 3 (or thereafter). This needs to be designed first, but the
more complicated data model will also have an impact there.


(3) What is a ContentHandler and how could it help SMW?

ContentHandlers allow MediaWiki to manage textual content that is not
interpreted as wikitext. Wikidata uses this to store data in a
text-based format (JSON) without having to invent a MediaWiki syntax
extension. In the future, it might be possible to have "multi-part"
content for one page, so that one can store both Wikitext and other
structured data in a cleaner way.

SMW could use this to allow data to be stored directly for a page,
without having to specify the data in the wikitext. This could be a bit
like an "editable Factbox". MediaWiki could do something similar with
Categories.


(4) Will ContentHandlers reduce the dependence on custom SQL?

No. Content that is stored in textual form on wiki pages is not suitable
for answering queries quickly. To do this, it will always be necessary
to have custom tables and/or external storage engines that provide fast
access to the data. This is also the case for Wikidata.

Currently, the MW page is the *primary* storage for SMW data, while the
SQL tables are just *secondary* storage for faster lookup. The
ContentHandlers can improve primary storage handling, but won't affect
secondary storages and querying.


(5) What are the possible synergies and joint uses of Wikidata and SMW?

The details can only be worked out when Wikidata is a bit more stable
than it is now, but there are a few obvious scenarios:

* Wikibase Repository + SMW: data given in SMW annotations and data
edited in Wikidata end up in the same secondary data structures, and can
be retrieved through the same interfaces/queries. There are three main
options: SMW tables are used for Wikibase content (this would yield
query support via #ask), Wikibase tables are used for SMW content
(yields full integration with Wikibase data access features), or both.

* SMW as a Wikibase Repository: SMW could support the use of Wikibase
APIs and exports on top of its native data store, so that other sites
could fetch SMW data via the Wikidata Client.

* SMW as a Wikibase Client: SMW could also pull data from Wikibase
Repositories to use it locally and to display it in its interfaces.

* Exchanging code: Besides application-level integration, SMW and
Wikidata are also going to exchange lines of code. Initially, Wikidata
will exploit code from SMW (e.g., the calendar conversion code for times
and dates). Later, smaller pieces of Wikidata code might be used for
SMW. In many cases, this will mean that the code is adapted heavily; if
it can be used without much change, one would possibly create a "library
extension" (like Validator).


(6) Will Wikidata replace SMW in the long run?

No. If anything, the integration between both could become so good that
the distinction is no longer relevant and we are only talking about
different data authoring and data access interface extensions to
MediaWiki, which users can enable and disable at their leisure. This
will still take some time though.


(7) What is the development plan for SMW with respect to Wikidata?

The strategy is to wait until Wikidata has settled a bit before
approaching any integration. This will probably be another 10 or 11
months from now (note that data management only comes in at Phase 2 of
Wikidata, i.e., in 5 months). Exchanging some code might be possible
before that, but integrating Wikibase while it is a moving target would
not be feasible (there would not be a chance of keeping up with the
speed of Wikidata, esp. since half the SMW team is engaged in Wikidata
development right now).


I hope this answers most questions on that topic for now. Maybe it would
be nice to have a wikipage on semantic-mediawiki.org with these answers.

Best regards,

Markus


On 30/04/12 23:44, James HK wrote:

> Hi,
>
> ==WD ContentHandler ==
> Now that the Wikidata project is introducing a new ContentHandler [1],
> we where curious if in near future SMW would make use of those methods
> to store its own objects in the same way and reduces its reliance on
> non-mediawiki tables/methods.
>
> As Wikidata and SMW share the same bottlenecks in terms of
> performance, caching, scalability it might prove beneficial for SMW to
> use similar methods to store and fetch dataobjects from a mw-database
> while at the same time reduces its reliance on SQL specific code and
> lift the ban on other supported database systems(Oracle,Sqlite etc.).
>
> The reason why we are asking is that we hope that some of the work
> that has be done in SMW is reflected in Wikidata as visa versa to
> ensure that both developments share some similarities to the extend
> that their "plug and playable"  (meaning causing the least
> interruptions when installed together and make it easier for code
> reviewers to understand the inner workings which increases stability).
>
> ==SMW and WD ==
> Furthermore, with the Wikidata project on the horizon, we would be
> interested to see how SMW positioned and distinguished itself from WD
> in light of recent developments. (A simple table could help to explain
> those differences fairly quick while at the same time allows some
> directional guidance for the future of SMW.)
>
> ==Validator==
> The Validator provides validation methods for datatypes (and parameter
> definitions etc.) and WD certainly needs similar methods, is it
> planned that those methods (or part of it) are being moved into the
> core?
>
> [1] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki%2Fcore.git;a=commit;h=337f65c03b59eaddc038bc1a0fb150dc97e03154
>
> Cheers,
>
> mwjames
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Semediawiki-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[Denny Vrandecic-3]

Markus,

do you mind if we take this text as the starting point for a text detailing the relation between SMW and Wikidata here?

<https://meta.wikimedia.org/wiki/Wikidata/Notes/SMW_and_Wikidata>

Cheers,
Denny


On 1 May 2012, at 10:04, Markus Krötzsch wrote:

> Hi James, hi all,
>
> the relationship and synergies of SMW and Wikidata are of course quite
> important to us, so I am taking the time to write a slightly longer
> email that I also send to the user list.
>
> At the moment, Wikidata is still under development, so it is a bit early
> to make concrete plans on how to align the project.
> Here are some things that I can already answer now:
>
>
> (1) What is the software in "Wikidata"?
>
> The software underlying Wikidata is called Wikibase. It consists of two
> extensions: Repository (where data is stored; this will run at
> wikidata.org) and Client (that pulls data into other MediaWiki sites;
> this will run at wikipedia.org). Both extensions will be installable on
> MediaWiki sites to get Wikidata-like features there.
>
>
> (2) What are the main differences between SMW and Wikidata?
>
> Wikidata is being built with SMW in mind, and the experiences with SMW
> are a valuable input for its design. The main use case of Wikidata (a
> centralised, multi-lingual site that serves as a data repository) is
> different from that of SMW (a data-enhanced MediaWiki), and this leads
> to a number of differences.
>
> The data model of Wikidata is a bit more complex than that of SMW. For
> example, every fact will also have an (optional) reference. Wikidata
> will also use more complex datatypes (e.g., since many text values can
> be translated). However, both Wikidata and SMW are still mostly based on
> property-value assignments, and Wikidata will have a Property namespace.
> Also, the more complex structures in Wikidata could be captured in SMW
> using internal objects.
>
> The user interface of Wikibase Repository will be based on input forms,
> and thus quite different from SMW. The UI will be a bit like a
> full-page, editable Factbox, but much more advanced, with a lot of
> support for entering values.
>
> It is not defined yet what kind of query language Wikidata will support
> in Phase 3 (or thereafter). This needs to be designed first, but the
> more complicated data model will also have an impact there.
>
>
> (3) What is a ContentHandler and how could it help SMW?
>
> ContentHandlers allow MediaWiki to manage textual content that is not
> interpreted as wikitext. Wikidata uses this to store data in a
> text-based format (JSON) without having to invent a MediaWiki syntax
> extension. In the future, it might be possible to have "multi-part"
> content for one page, so that one can store both Wikitext and other
> structured data in a cleaner way.
>
> SMW could use this to allow data to be stored directly for a page,
> without having to specify the data in the wikitext. This could be a bit
> like an "editable Factbox". MediaWiki could do something similar with
> Categories.
>
>
> (4) Will ContentHandlers reduce the dependence on custom SQL?
>
> No. Content that is stored in textual form on wiki pages is not suitable
> for answering queries quickly. To do this, it will always be necessary
> to have custom tables and/or external storage engines that provide fast
> access to the data. This is also the case for Wikidata.
>
> Currently, the MW page is the *primary* storage for SMW data, while the
> SQL tables are just *secondary* storage for faster lookup. The
> ContentHandlers can improve primary storage handling, but won't affect
> secondary storages and querying.
>
>
> (5) What are the possible synergies and joint uses of Wikidata and SMW?
>
> The details can only be worked out when Wikidata is a bit more stable
> than it is now, but there are a few obvious scenarios:
>
> * Wikibase Repository + SMW: data given in SMW annotations and data
> edited in Wikidata end up in the same secondary data structures, and can
> be retrieved through the same interfaces/queries. There are three main
> options: SMW tables are used for Wikibase content (this would yield
> query support via #ask), Wikibase tables are used for SMW content
> (yields full integration with Wikibase data access features), or both.
>
> * SMW as a Wikibase Repository: SMW could support the use of Wikibase
> APIs and exports on top of its native data store, so that other sites
> could fetch SMW data via the Wikidata Client.
>
> * SMW as a Wikibase Client: SMW could also pull data from Wikibase
> Repositories to use it locally and to display it in its interfaces.
>
> * Exchanging code: Besides application-level integration, SMW and
> Wikidata are also going to exchange lines of code. Initially, Wikidata
> will exploit code from SMW (e.g., the calendar conversion code for times
> and dates). Later, smaller pieces of Wikidata code might be used for
> SMW. In many cases, this will mean that the code is adapted heavily; if
> it can be used without much change, one would possibly create a "library
> extension" (like Validator).
>
>
> (6) Will Wikidata replace SMW in the long run?
>
> No. If anything, the integration between both could become so good that
> the distinction is no longer relevant and we are only talking about
> different data authoring and data access interface extensions to
> MediaWiki, which users can enable and disable at their leisure. This
> will still take some time though.
>
>
> (7) What is the development plan for SMW with respect to Wikidata?
>
> The strategy is to wait until Wikidata has settled a bit before
> approaching any integration. This will probably be another 10 or 11
> months from now (note that data management only comes in at Phase 2 of
> Wikidata, i.e., in 5 months). Exchanging some code might be possible
> before that, but integrating Wikibase while it is a moving target would
> not be feasible (there would not be a chance of keeping up with the
> speed of Wikidata, esp. since half the SMW team is engaged in Wikidata
> development right now).
>
>
> I hope this answers most questions on that topic for now. Maybe it would
> be nice to have a wikipage on semantic-mediawiki.org with these answers.
>
> Best regards,
>
> Markus
>
>
> On 30/04/12 23:44, James HK wrote:
>> Hi,
>>
>> ==WD ContentHandler ==
>> Now that the Wikidata project is introducing a new ContentHandler [1],
>> we where curious if in near future SMW would make use of those methods
>> to store its own objects in the same way and reduces its reliance on
>> non-mediawiki tables/methods.
>>
>> As Wikidata and SMW share the same bottlenecks in terms of
>> performance, caching, scalability it might prove beneficial for SMW to
>> use similar methods to store and fetch dataobjects from a mw-database
>> while at the same time reduces its reliance on SQL specific code and
>> lift the ban on other supported database systems(Oracle,Sqlite etc.).
>>
>> The reason why we are asking is that we hope that some of the work
>> that has be done in SMW is reflected in Wikidata as visa versa to
>> ensure that both developments share some similarities to the extend
>> that their "plug and playable"  (meaning causing the least
>> interruptions when installed together and make it easier for code
>> reviewers to understand the inner workings which increases stability).
>>
>> ==SMW and WD ==
>> Furthermore, with the Wikidata project on the horizon, we would be
>> interested to see how SMW positioned and distinguished itself from WD
>> in light of recent developments. (A simple table could help to explain
>> those differences fairly quick while at the same time allows some
>> directional guidance for the future of SMW.)
>>
>> ==Validator==
>> The Validator provides validation methods for datatypes (and parameter
>> definitions etc.) and WD certainly needs similar methods, is it
>> planned that those methods (or part of it) are being moved into the
>> core?
>>
>> [1] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki%2Fcore.git;a=commit;h=337f65c03b59eaddc038bc1a0fb150dc97e03154
>>
>> Cheers,
>>
>> mwjames
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Semediawiki-devel mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Semediawiki-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[Markus Krötzsch-2]

On 01/05/12 11:45, Denny Vrandecic wrote:
> Markus,
>
> do you mind if we take this text as the starting point for a text detailing the relation between SMW and Wikidata here?

Not at all.

Markus

>
> <https://meta.wikimedia.org/wiki/Wikidata/Notes/SMW_and_Wikidata>
>
> Cheers,
> Denny
>
>
> On 1 May 2012, at 10:04, Markus Krötzsch wrote:
>
>> Hi James, hi all,
>>
>> the relationship and synergies of SMW and Wikidata are of course quite
>> important to us, so I am taking the time to write a slightly longer
>> email that I also send to the user list.
>>
>> At the moment, Wikidata is still under development, so it is a bit early
>> to make concrete plans on how to align the project.
>> Here are some things that I can already answer now:
>>
>>
>> (1) What is the software in "Wikidata"?
>>
>> The software underlying Wikidata is called Wikibase. It consists of two
>> extensions: Repository (where data is stored; this will run at
>> wikidata.org) and Client (that pulls data into other MediaWiki sites;
>> this will run at wikipedia.org). Both extensions will be installable on
>> MediaWiki sites to get Wikidata-like features there.
>>
>>
>> (2) What are the main differences between SMW and Wikidata?
>>
>> Wikidata is being built with SMW in mind, and the experiences with SMW
>> are a valuable input for its design. The main use case of Wikidata (a
>> centralised, multi-lingual site that serves as a data repository) is
>> different from that of SMW (a data-enhanced MediaWiki), and this leads
>> to a number of differences.
>>
>> The data model of Wikidata is a bit more complex than that of SMW. For
>> example, every fact will also have an (optional) reference. Wikidata
>> will also use more complex datatypes (e.g., since many text values can
>> be translated). However, both Wikidata and SMW are still mostly based on
>> property-value assignments, and Wikidata will have a Property namespace.
>> Also, the more complex structures in Wikidata could be captured in SMW
>> using internal objects.
>>
>> The user interface of Wikibase Repository will be based on input forms,
>> and thus quite different from SMW. The UI will be a bit like a
>> full-page, editable Factbox, but much more advanced, with a lot of
>> support for entering values.
>>
>> It is not defined yet what kind of query language Wikidata will support
>> in Phase 3 (or thereafter). This needs to be designed first, but the
>> more complicated data model will also have an impact there.
>>
>>
>> (3) What is a ContentHandler and how could it help SMW?
>>
>> ContentHandlers allow MediaWiki to manage textual content that is not
>> interpreted as wikitext. Wikidata uses this to store data in a
>> text-based format (JSON) without having to invent a MediaWiki syntax
>> extension. In the future, it might be possible to have "multi-part"
>> content for one page, so that one can store both Wikitext and other
>> structured data in a cleaner way.
>>
>> SMW could use this to allow data to be stored directly for a page,
>> without having to specify the data in the wikitext. This could be a bit
>> like an "editable Factbox". MediaWiki could do something similar with
>> Categories.
>>
>>
>> (4) Will ContentHandlers reduce the dependence on custom SQL?
>>
>> No. Content that is stored in textual form on wiki pages is not suitable
>> for answering queries quickly. To do this, it will always be necessary
>> to have custom tables and/or external storage engines that provide fast
>> access to the data. This is also the case for Wikidata.
>>
>> Currently, the MW page is the *primary* storage for SMW data, while the
>> SQL tables are just *secondary* storage for faster lookup. The
>> ContentHandlers can improve primary storage handling, but won't affect
>> secondary storages and querying.
>>
>>
>> (5) What are the possible synergies and joint uses of Wikidata and SMW?
>>
>> The details can only be worked out when Wikidata is a bit more stable
>> than it is now, but there are a few obvious scenarios:
>>
>> * Wikibase Repository + SMW: data given in SMW annotations and data
>> edited in Wikidata end up in the same secondary data structures, and can
>> be retrieved through the same interfaces/queries. There are three main
>> options: SMW tables are used for Wikibase content (this would yield
>> query support via #ask), Wikibase tables are used for SMW content
>> (yields full integration with Wikibase data access features), or both.
>>
>> * SMW as a Wikibase Repository: SMW could support the use of Wikibase
>> APIs and exports on top of its native data store, so that other sites
>> could fetch SMW data via the Wikidata Client.
>>
>> * SMW as a Wikibase Client: SMW could also pull data from Wikibase
>> Repositories to use it locally and to display it in its interfaces.
>>
>> * Exchanging code: Besides application-level integration, SMW and
>> Wikidata are also going to exchange lines of code. Initially, Wikidata
>> will exploit code from SMW (e.g., the calendar conversion code for times
>> and dates). Later, smaller pieces of Wikidata code might be used for
>> SMW. In many cases, this will mean that the code is adapted heavily; if
>> it can be used without much change, one would possibly create a "library
>> extension" (like Validator).
>>
>>
>> (6) Will Wikidata replace SMW in the long run?
>>
>> No. If anything, the integration between both could become so good that
>> the distinction is no longer relevant and we are only talking about
>> different data authoring and data access interface extensions to
>> MediaWiki, which users can enable and disable at their leisure. This
>> will still take some time though.
>>
>>
>> (7) What is the development plan for SMW with respect to Wikidata?
>>
>> The strategy is to wait until Wikidata has settled a bit before
>> approaching any integration. This will probably be another 10 or 11
>> months from now (note that data management only comes in at Phase 2 of
>> Wikidata, i.e., in 5 months). Exchanging some code might be possible
>> before that, but integrating Wikibase while it is a moving target would
>> not be feasible (there would not be a chance of keeping up with the
>> speed of Wikidata, esp. since half the SMW team is engaged in Wikidata
>> development right now).
>>
>>
>> I hope this answers most questions on that topic for now. Maybe it would
>> be nice to have a wikipage on semantic-mediawiki.org with these answers.
>>
>> Best regards,
>>
>> Markus
>>
>>
>> On 30/04/12 23:44, James HK wrote:
>>> Hi,
>>>
>>> ==WD ContentHandler ==
>>> Now that the Wikidata project is introducing a new ContentHandler [1],
>>> we where curious if in near future SMW would make use of those methods
>>> to store its own objects in the same way and reduces its reliance on
>>> non-mediawiki tables/methods.
>>>
>>> As Wikidata and SMW share the same bottlenecks in terms of
>>> performance, caching, scalability it might prove beneficial for SMW to
>>> use similar methods to store and fetch dataobjects from a mw-database
>>> while at the same time reduces its reliance on SQL specific code and
>>> lift the ban on other supported database systems(Oracle,Sqlite etc.).
>>>
>>> The reason why we are asking is that we hope that some of the work
>>> that has be done in SMW is reflected in Wikidata as visa versa to
>>> ensure that both developments share some similarities to the extend
>>> that their "plug and playable"  (meaning causing the least
>>> interruptions when installed together and make it easier for code
>>> reviewers to understand the inner workings which increases stability).
>>>
>>> ==SMW and WD ==
>>> Furthermore, with the Wikidata project on the horizon, we would be
>>> interested to see how SMW positioned and distinguished itself from WD
>>> in light of recent developments. (A simple table could help to explain
>>> those differences fairly quick while at the same time allows some
>>> directional guidance for the future of SMW.)
>>>
>>> ==Validator==
>>> The Validator provides validation methods for datatypes (and parameter
>>> definitions etc.) and WD certainly needs similar methods, is it
>>> planned that those methods (or part of it) are being moved into the
>>> core?
>>>
>>> [1] https://gerrit.wikimedia.org/r/gitweb?p=mediawiki%2Fcore.git;a=commit;h=337f65c03b59eaddc038bc1a0fb150dc97e03154
>>>
>>> Cheers,
>>>
>>> mwjames
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Semediawiki-devel mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Semediawiki-devel mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[Jeroen De Dauw-2]

Hey,

> The Validator provides validation methods for datatypes (and parameter
> definitions etc.) and WD certainly needs similar methods

The main things that Validator provides is a declarative parameter handling system. So far we do not need this anywhere in the WikiData software itself (next to the API, but this has it's own parameter definition mechanism). If we run into a place where we need this kind of parameter handling, we'll definitely look at Validator.

> is it planned that those methods (or part of it) are being moved into the core?

There are no plans to move this into core right now. If we do end up using it in Wikidata, that'll increase the incentive to at least have it in the default MediaWiki distribution tarball. I am however working on a rewrite of the extension making it better suited to do what's actually being done with it. Will have more news on this in a few weeks.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[zehetner]

Hi,

Executing a query with format=debug and a property of type record where at
least one field has a value e.g.

{{#ask: [[GOslim:membrane;?;?]] | format=debug}}

gives the error

Fatal error: Call to a member function findPropertyTypeID() on a
non-object in
/w/extensions/SemanticMediaWiki/includes/SMW_DataValueFactory.php  on line
130

Queries with other format values than 'debug' e.g. 'format=table' are
working ok and give the expected results.
Also {{#ask: [[GOslim:?;?;?]] | format=debug}} (no field values given)
returns the expected result.


It seems the function newDataItemValue is at some point called with the
variable $property having the value '1' causing
if ( !is_null( $property ) ) {
   $typeId = $property->findPropertyTypeID();
to throw the error.
Changing the if to if ((is_object($property)) && (!is_null($property)))
results in an empty white page instead of the error message.


I created such a record property on sandbox.semantic-mediawiki.org and see
there the same problem, except that with format=debug I don't get an error
message but always a white empty page.

Versions I use:
MediaWiki 1.18.1
Semantic MediaWiki (Version 1.8 alpha) (r115105)

Cheers,
Gu

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[Jeroen De Dauw-2]

Hey,

> Changing the if to if ((is_object($property)) && (!is_null($property))) results in an empty white page instead of the error message.

That would not fix the actual issue in any case, it would just hide it. I added some extra checks in the code to make the error appear before this point now. However, to find the real issue I need to know what the actual value of this variable is and from where it is coming. Can you put this code at the top of the function and get me the resulting output?

if ( !is_object( $property ) ) {
       var_dump( $property ); throw new MWException();
}

Bugzilla is a better place to report such issues, so next time you run into something like this, go here: https://bugzilla.wikimedia.org/ :)

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[zehetner]

Hi,
does anyone know if there is somewhere a description how record type
property values are stored in the new SMWSQLStore3 (or a general
description of the table structure)?

Thanks!
Gu

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[nischay nahata]

Hi,

A record type is stored in the form of a virtual page having many properties (called container).

So a page name is generated using some md4 and then all the property values are stored as if they belonged to this *virtual* page.

Which tables? they are distributed over all the property-values tables. To get to these values you have to first look into the wiki page table and find out the generated name of the wiki page and then use that to find values in other tables. Hope that makes sense :)

Somebody, please correct me if I am wrong here. :)

On Thu, Apr 18, 2013 at 5:28 PM, zehetner <[hidden email]> wrote:

Hi,
does anyone know if there is somewhere a description how record type
property values are stored in the new SMWSQLStore3 (or a general
description of the table structure)?

Thanks!
Gu

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

--
Cheers,

Nischay Nahata
nischayn22.in

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

[zehetner]

Hi Nischay

very helpful to get an idea where to start to look.
For my purpose I'm actually not interested to which page a property
belongs, just in all values of the single parts of a record property.

Thanks
Gu

On Fri, 19 Apr 2013 11:10:13 +0530, Nischay Nahata <[hidden email]>
wrote:
> Hi,
>
> A record type is stored in the form of a virtual page having many
> properties (called container).
> So a page name is generated using some md4 and then all the property
values
> are stored as if they belonged to this *virtual* page.
>
> Which tables? they are distributed over all the property-values tables.
To
> get to these values you have to first look into the wiki page table and
> find out the generated name of the wiki page and then use that to find
> values in other tables. Hope that makes sense :)
>
>
> Somebody, please correct me if I am wrong here. :)
>
>
> On Thu, Apr 18, 2013 at 5:28 PM, zehetner <[hidden email]>
wrote:

>
>> Hi,
>> does anyone know if there is somewhere a description how record type
>> property values are stored in the new SMWSQLStore3 (or a general
>> description of the table structure)?
>>
>> Thanks!
>> Gu
>>
>>
>>

------------------------------------------------------------------------------

>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for
>> building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> _______________________________________________
>> Semediawiki-devel mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>>

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel