Possible fix for automated refresh of templated semantic data

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible fix for automated refresh of templated semantic data

Yaron Koren
Hi,

I may have found a fix for the problem of pages not having their semantic data refreshed when templates they include get their own semantic markup changed. As you may know, this has been a fairly big problem for users, and constitutes (I think) most of the need for the "refreshData" script.

I discovered MediaWiki "jobs" recently, while working on my Replace Text extension. They're the MediaWiki equivalent of cron jobs, and it turns out that they are how automatic updating of categories happens - if the set of categories that a template includes is changed, a "job" is created for each page that uses that template, to update that page's category links when the server has time. (If you ever switch the category on a template, you can see the new category slowly getting filled up if you keep refreshing that category's page.)

Anyway, the code for this job can be found at /includes/RefreshLinksJob.php and /includes/LinksUpdate.php, and there's a hook there, 'LinksUpdateConstructed', that's called for every page that's refreshed when a template is saved. I added some code to use this hook to do semantic-data refreshing, and it works for me. In the SMW code, in /includes/SMW_GlobalFunctions.php, where the hooks are registered (around line 166), I added the following:

        $wgHooks['LinksUpdateConstructed'][] = 'smwfUpdateSemanticData';

Then, below that, among the other function declarations, I added this:

/**
 * Update the semantic data for a single page
 */
function smwfUpdateSemanticData($links_update) {
        $title = $links_update->mTitle;
        if ( smwfIsSemanticsProcessed($title->getNamespace()) ) {
                SMWFactbox::storeData($title, true);
        }
        return true;
}

There may be some performance or other issues that I'm missing; I'm curious to hear what other people have to say. If everything looks fine, please add this into the code - feel free to make any changes you want, of course.

-Yaron
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Reply | Threaded
Open this post in threaded view
|

Re: Possible fix for automated refresh of templated semantic data

Yaron Koren
Hi,

I have one change to the code from before - it was unfortunately saving the semantic data twice for pages that are newly-created. The new version of smwfUpdateSemanticData() is:

function smwfUpdateSemanticData($links_update) {
        $title = $links_update->mTitle;
        if ( smwfIsSemanticsProcessed($title->getNamespace()) && ! SMWFactbox::isNewArticle() ) {
                SMWFactbox::storeData(true);
        }
        return true;
}

On the bright side, I do have more confidence now that this is a correct solution in general.

-Yaron


On Tue, May 13, 2008 at 11:27 AM, Yaron Koren <[hidden email]> wrote:
Hi,

I may have found a fix for the problem of pages not having their semantic data refreshed when templates they include get their own semantic markup changed. As you may know, this has been a fairly big problem for users, and constitutes (I think) most of the need for the "refreshData" script.

I discovered MediaWiki "jobs" recently, while working on my Replace Text extension. They're the MediaWiki equivalent of cron jobs, and it turns out that they are how automatic updating of categories happens - if the set of categories that a template includes is changed, a "job" is created for each page that uses that template, to update that page's category links when the server has time. (If you ever switch the category on a template, you can see the new category slowly getting filled up if you keep refreshing that category's page.)

Anyway, the code for this job can be found at /includes/RefreshLinksJob.php and /includes/LinksUpdate.php, and there's a hook there, 'LinksUpdateConstructed', that's called for every page that's refreshed when a template is saved. I added some code to use this hook to do semantic-data refreshing, and it works for me. In the SMW code, in /includes/SMW_GlobalFunctions.php, where the hooks are registered (around line 166), I added the following:

        $wgHooks['LinksUpdateConstructed'][] = 'smwfUpdateSemanticData';

Then, below that, among the other function declarations, I added this:

/**
 * Update the semantic data for a single page
 */
function smwfUpdateSemanticData($links_update) {
        $title = $links_update->mTitle;
        if ( smwfIsSemanticsProcessed($title->getNamespace()) ) {
                SMWFactbox::storeData($title, true);
        }
        return true;
}

There may be some performance or other issues that I'm missing; I'm curious to hear what other people have to say. If everything looks fine, please add this into the code - feel free to make any changes you want, of course.

-Yaron


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Reply | Threaded
Open this post in threaded view
|

Re: Possible fix for automated refresh of templated semantic data

Denny Vrandecic
Hi Yaron,

I commited your patch to the SVN. Thanks, Yaron!

Furthermore, a second job was created for the case that a property's
type or a type is being changed. In that case also relevant job's are
created and added to the job queue, which will be triggered.

So, whenever your job queue is empty, your semantic data should be
consistent know. Otherwise it's a bug (please report!)

The second patch has been developed by Daniel Herzig. Thanks!

Best,
denny

Yaron Koren wrote:

> Hi,
>
> I have one change to the code from before - it was unfortunately saving
> the semantic data twice for pages that are newly-created. The new
> version of smwfUpdateSemanticData() is:
>
> function smwfUpdateSemanticData($links_update) {
>         $title = $links_update->mTitle;
>         if ( smwfIsSemanticsProcessed($title->getNamespace()) && !
> SMWFactbox::isNewArticle() ) {
>                 SMWFactbox::storeData(true);
>         }
>         return true;
> }
>
> On the bright side, I do have more confidence now that this is a correct
> solution in general.
>
> -Yaron
>
>
> On Tue, May 13, 2008 at 11:27 AM, Yaron Koren <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     I may have found a fix for the problem of pages not having their
>     semantic data refreshed when templates they include get their own
>     semantic markup changed. As you may know, this has been a fairly big
>     problem for users, and constitutes (I think) most of the need for
>     the "refreshData" script.
>
>     I discovered MediaWiki "jobs" recently, while working on my Replace
>     Text extension. They're the MediaWiki equivalent of cron jobs, and
>     it turns out that they are how automatic updating of categories
>     happens - if the set of categories that a template includes is
>     changed, a "job" is created for each page that uses that template,
>     to update that page's category links when the server has time. (If
>     you ever switch the category on a template, you can see the new
>     category slowly getting filled up if you keep refreshing that
>     category's page.)
>
>     Anyway, the code for this job can be found at
>     /includes/RefreshLinksJob.php and /includes/LinksUpdate.php, and
>     there's a hook there, 'LinksUpdateConstructed', that's called for
>     every page that's refreshed when a template is saved. I added some
>     code to use this hook to do semantic-data refreshing, and it works
>     for me. In the SMW code, in /includes/SMW_GlobalFunctions.php, where
>     the hooks are registered (around line 166), I added the following:
>
>             $wgHooks['LinksUpdateConstructed'][] = 'smwfUpdateSemanticData';
>
>     Then, below that, among the other function declarations, I added this:
>
>     /**
>      * Update the semantic data for a single page
>      */
>     function smwfUpdateSemanticData($links_update) {
>             $title = $links_update->mTitle;
>             if ( smwfIsSemanticsProcessed($title->getNamespace()) ) {
>                     SMWFactbox::storeData($title, true);
>             }
>             return true;
>     }
>
>     There may be some performance or other issues that I'm missing; I'm
>     curious to hear what other people have to say. If everything looks
>     fine, please add this into the code - feel free to make any changes
>     you want, of course.
>
>     -Yaron
>
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Semediawiki-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Semediawiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Reply | Threaded
Open this post in threaded view
|

Re: Possible fix for automated refresh of templated semantic data

planetenxin
Hi,

I'm having trouble with inverted annotation as mentioned here. Everything works fine but the inverse property is refreshed only after a 'NULL' edit of the article or the inverse property page.

Disable all kind of caching did not work.

Manually triggering SMW_refreshData.php also resolves the problem.

I assume that there's no job created telling MW to refresh the data. Can someone give me a hint?

My current config: MW 1.15.5, SMW 1.5.3, SF 1.9.1, Invert-property template, Invert-property/Separator template

/planetenxin