External Data + Semantic Results Format?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

External Data + Semantic Results Format?

Ed Swing-2
I am developing a wiki where we will need to use external data in tandem with visualizations generated by the SRF. For a simple example, assume I have a wiki set up for monitoring population of several regions. I have a wiki page for each region in the wiki.

Now, assume I have a CSV (or any other form of supported data) file that contains population for all the regions over the time range, all as one large spreadsheet. On each region page, I want to display a graph of the population changes over time.

What would be the easiest way to do this? Are there any examples?

Thanks for any help.
------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
Now with support for Jelly Bean, Bluetooth, Mapview and more.
Get your Android app in front of a whole new audience.  Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: External Data + Semantic Results Format?

planetenxin
Hi Ad,

my approach would be to

1) Import ("semantify") the data from your external source using an ETL
tool like Pentaho Kettle via MediaWiki API.

2) Store the imported data in the wiki as either subobjects on your
population pages or as seperate pages with a property pointing to a
population (maybe in an additional namespace to avoid messing up your wiki)

3) Build the query with the result format you wish.

One of my customers had more or less the same requirement with price
information over time for a list of products with daily price updates
provided by an external data source.

/Alexander Gesinn


Am 12.02.2014 02:06, schrieb Ed Swing:

> I am developing a wiki where we will need to use external data in tandem with visualizations generated by the SRF. For a simple example, assume I have a wiki set up for monitoring population of several regions. I have a wiki page for each region in the wiki.
>
> Now, assume I have a CSV (or any other form of supported data) file that contains population for all the regions over the time range, all as one large spreadsheet. On each region page, I want to display a graph of the population changes over time.
>
> What would be the easiest way to do this? Are there any examples?
>
> Thanks for any help.
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>


--
________________________________________________
semantic::apps by gesinn.it
Business Applications with Semantic Mediawiki.
http://semantic-apps.com

------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
Now with support for Jelly Bean, Bluetooth, Mapview and more.
Get your Android app in front of a whole new audience.  Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user
Reply | Threaded
Open this post in threaded view
|

Re: External Data + Semantic Results Format?

jongfeli
Hello Ad,

A practical example on "Semantifying" your data. We use an extra database on the same MySQL instance that is used by our wiki to store the data we need in Semantic form. The MySQL instance is able to communicate with our SQL Server instance and visa versa. The data we need from our SQL Server is automatically updated on the MySQL instance. With the External Data extension we then assign values to the properties coming from this database like this for Property:Something :

{{#get_db_data:
db=Extradb
|from=TableOrView
|order by=Something
|data=Something=Something
}}
{{#for_external_table:<nowiki/>
* [[Something::{{{Something}}}]]
}}

Or to assign Allows value on a property:

{{#get_db_data:
db=Extradb
|from=TableOrView
|order by=Something
|data=Something=Something
}}
{{#for_external_table:<nowiki/>
* [[Allows value::{{{Something}}}]]
}}

Depending on the size of your data-set and how you want to store the data you can assign a lot of values to properties like this. We have no problem assigning thousands of values to properties or Allows value's. We also use the Data Transfer extension to call pages in specific categories via the MediaWiki API. The collected XML is then processed directly by our SQL Server and the altered date by the Wiki users is updated on the SQL Server database. Now it is a two way system, SQL data is updated on the wiki and data altered on the Wiki is updated on SQL Server. Mediawiki has become our UI on top of our SQL Database. This works as long as you are not in a hurry and your data is not very dynamic (does not change a lot). We update our SQL Server and the Semantic data on our wiki every night and for some properties, pages are created automatically.

There are probably a lot of scenarios on how to do this but this is one of them.

Best regards,

Felipe.
Reply | Threaded
Open this post in threaded view
|

Re: External Data + Semantic Results Format?

Yaron Koren-2
Hi Ed,

Here's a third option: get the CSV equivalent of that spreadsheet, and then
copy that into a wiki page. Then, in a separate page, store all the data
from that CSV semantically, using a combination of Special:GetData and
#store_external_table. The data will now all be stored as either subobjects
or internal objects, depending on what versions of SMW and ED you're using.
Either way, you can now use #ask queries to display graphical
representations of different subsets of that data.

By the way, if you modify that CSV, you'll have to also refresh the
semantic data. The easiest way to do that is probably to resave that 2nd
page (making a slight modification each time so that it will actually
resave).

-Yaron


On Wed, Feb 12, 2014 at 4:00 AM, jongfeli <[hidden email]> wrote:

> Hello Ad,
>
> A practical example on "Semantifying" your data. We use an extra database
> on
> the same MySQL instance that is used by our wiki to store the data we need
> in Semantic form. The MySQL instance is able to communicate with our SQL
> Server instance and visa versa. The data we need from our SQL Server is
> automatically updated on the MySQL instance. With the External Data
> extension we then assign values to the properties coming from this database
> like this for *Property:Something* :
>
> {{#get_db_data:
> db=Extradb
> |from=TableOrView
> |order by=Something
> |data=Something=Something
> }}
> {{#for_external_table:<nowiki/>
> * [[Something::{{{Something}}}]]
> }}
>
> Or to assign *Allows value* on a property:
>
> {{#get_db_data:
> db=Extradb
> |from=TableOrView
> |order by=Something
> |data=Something=Something
> }}
> {{#for_external_table:<nowiki/>
> * [[Allows value::{{{Something}}}]]
> }}
>
> Depending on the size of your data-set and how you want to store the data
> you can assign a lot of values to properties like this. We have no problem
> assigning thousands of values to properties or Allows value's. We also use
> the *Data Transfer* extension to call pages in specific categories via the
> MediaWiki API. The collected XML is then processed directly by our SQL
> Server and the altered date by the Wiki users is updated on the SQL Server
> database. Now it is a two way system, SQL data is updated on the wiki and
> data altered on the Wiki is updated on SQL Server. Mediawiki has become our
> UI on top of our SQL Database. This works as long as you are not in a hurry
> and your data is not very dynamic (does not change a lot). We update our
> SQL
> Server and the Semantic data on our wiki every night and for some
> properties, pages are created automatically.
>
> There are probably a lot of scenarios on how to do this but this is one of
> them.
>
> Best regards,
>
> Felipe.
>
>
>
> --
> View this message in context:
> http://wikimedia.7.x6.nabble.com/External-Data-Semantic-Results-Format-tp5021979p5021993.html
> Sent from the Semantic Mediawiki - User mailing list archive at Nabble.com.
>
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience.  Start now.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Semediawiki-user mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-user
>



--
WikiWorks · MediaWiki Consulting · http://wikiworks.com
------------------------------------------------------------------------------
Android apps run on BlackBerry 10
Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
Now with support for Jelly Bean, Bluetooth, Mapview and more.
Get your Android app in front of a whole new audience.  Start now.
http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
_______________________________________________
Semediawiki-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-user