Problem with updateSearchIndexConfig.php --startover

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with updateSearchIndexConfig.php --startover

Aran Dunkley
Hello,

I have some wikis running elastic search and after some development
updates I find I need to rebuild the search indexes which I do as follows:

.../CirrusSearch/maintenance/forceSearchIndex.php --skipLinks --indexOnSkip
.../CirrusSearch/maintenance/forceSearchIndex.php --skipParse

But I found that this doesn't work for new wikis that don't have indexes
at all and so I put the following command before the other two:

.../CirrusSearch/maintenance/updateSearchIndexConfig.php --startOver

Later I found that this resulted in only the first namespace in each
wiki being included in any search results...

What is the proper command I should run to completely nuke and rebuild
all indexes on a wiki?

Thanks,
Aran

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Problem with updateSearchIndexConfig.php --startover

Erik Bernhardson
On Fri, Mar 31, 2017 at 8:25 AM, Aran <[hidden email]> wrote:

> Hello,
>
> I have some wikis running elastic search and after some development
> updates I find I need to rebuild the search indexes which I do as follows:
>
> .../CirrusSearch/maintenance/forceSearchIndex.php --skipLinks
> --indexOnSkip
> .../CirrusSearch/maintenance/forceSearchIndex.php --skipParse
>
> But I found that this doesn't work for new wikis that don't have indexes
> at all and so I put the following command before the other two:
>
> .../CirrusSearch/maintenance/updateSearchIndexConfig.php --startOver
>
>
The above three commands should be all you need, I use it all the time for
resetting development indices. The full order of commands should be:

updateSearchIndexConfig.php --startOver
forceSearchIndex.php --skipLinks --indexOnSkip
forceSearchIndex.php --skipParse

Later I found that this resulted in only the first namespace in each

> wiki being included in any search results...
>
> What is the proper command I should run to completely nuke and rebuild
> all indexes on a wiki?
>
> Thanks,
> Aran
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Problem with updateSearchIndexConfig.php --startover

Aran Dunkley
Thanks, that's good to know - I've narrowed my problem down a bit
further and found that the problem is with the second of those commands
not the first. For some reason forceSearchIndex --skipLinks
--indexOnSkip is only processing one namespace, it will process others
only if I explicitly specify them with the --namespace option....?!


On 31/03/17 13:02, Erik Bernhardson wrote:

> On Fri, Mar 31, 2017 at 8:25 AM, Aran <[hidden email]> wrote:
>
>> Hello,
>>
>> I have some wikis running elastic search and after some development
>> updates I find I need to rebuild the search indexes which I do as follows:
>>
>> .../CirrusSearch/maintenance/forceSearchIndex.php --skipLinks
>> --indexOnSkip
>> .../CirrusSearch/maintenance/forceSearchIndex.php --skipParse
>>
>> But I found that this doesn't work for new wikis that don't have indexes
>> at all and so I put the following command before the other two:
>>
>> .../CirrusSearch/maintenance/updateSearchIndexConfig.php --startOver
>>
>>
> The above three commands should be all you need, I use it all the time for
> resetting development indices. The full order of commands should be:
>
> updateSearchIndexConfig.php --startOver
> forceSearchIndex.php --skipLinks --indexOnSkip
> forceSearchIndex.php --skipParse
>
> Later I found that this resulted in only the first namespace in each
>> wiki being included in any search results...
>>
>> What is the proper command I should run to completely nuke and rebuild
>> all indexes on a wiki?
>>
>> Thanks,
>> Aran
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Problem with updateSearchIndexConfig.php --startover

Aran Dunkley
In reply to this post by Erik Bernhardson
For some reason forceSearchIndex.php --skipLinks --indexOnSkip is only
doing around 1000 pages and then stopping, if I set --fromId to the next
ID it does another 1000 or so, setting --limit with a high number does
not allow it to do more...?


On 31/03/17 13:02, Erik Bernhardson wrote:

> On Fri, Mar 31, 2017 at 8:25 AM, Aran <[hidden email]> wrote:
>
>> Hello,
>>
>> I have some wikis running elastic search and after some development
>> updates I find I need to rebuild the search indexes which I do as follows:
>>
>> .../CirrusSearch/maintenance/forceSearchIndex.php --skipLinks
>> --indexOnSkip
>> .../CirrusSearch/maintenance/forceSearchIndex.php --skipParse
>>
>> But I found that this doesn't work for new wikis that don't have indexes
>> at all and so I put the following command before the other two:
>>
>> .../CirrusSearch/maintenance/updateSearchIndexConfig.php --startOver
>>
>>
> The above three commands should be all you need, I use it all the time for
> resetting development indices. The full order of commands should be:
>
> updateSearchIndexConfig.php --startOver
> forceSearchIndex.php --skipLinks --indexOnSkip
> forceSearchIndex.php --skipParse
>
> Later I found that this resulted in only the first namespace in each
>> wiki being included in any search results...
>>
>> What is the proper command I should run to completely nuke and rebuild
>> all indexes on a wiki?
>>
>> Thanks,
>> Aran
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l