Wondering why the output isn't flushed a little sooner

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Wondering why the output isn't flushed a little sooner

Danny Joe Bauch
In my wiki installation (which is running on Windows/IIS6/MSSQL) I have
modified the finalCleanup method in Wiki.php to do this:

function finalCleanup ( &$deferredUpdates, &$output ) {
   wfProfileIn( __METHOD__ );
   $this->doUpdates( $deferredUpdates );
   $output->output();
   $this->doJobs();
   # Now commit any transactions, so that unreported errors after output()
don't roll back the whole thing
   $factory = wfGetLBFactory();
   $factory->shutdown();
   wfProfileOut( __METHOD__ );
}

rather than this:

function finalCleanup ( &$deferredUpdates, &$output ) {
   wfProfileIn( __METHOD__ );
   $this->doUpdates( $deferredUpdates );
   $this->doJobs();
   # Now commit any transactions, so that unreported errors after output()
don't roll back the whole thing
   $factory = wfGetLBFactory();
   $factory->shutdown();
   $output->output();
   wfProfileOut( __METHOD__ );
}

Basically, I am flushing the output before running anything from my job
queue. I frequently have a non-empty job queue, and have found that the
modification lets the browser see the page much faster than when I run some
jobs first. Is there any reason not to suggest this as a change to the code
base? It made quite a dramatic improvement on my wiki when I made the
change, although perhaps things are different with Linux/Apache/MySQL?
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wondering why the output isn't flushed a little sooner

Aryeh Gregor
On Wed, Jul 16, 2008 at 2:32 PM, DJ Bauch <[hidden email]> wrote:
> Basically, I am flushing the output before running anything from my job
> queue. I frequently have a non-empty job queue, and have found that the
> modification lets the browser see the page much faster than when I run some
> jobs first. Is there any reason not to suggest this as a change to the code
> base? It made quite a dramatic improvement on my wiki when I made the
> change, although perhaps things are different with Linux/Apache/MySQL?

Well, I don't know, but here's a guess, based on the comment there.
What happens if, in your setup, some error occurs during job queue
processing (say a deadlock, or a database server crash, etc.) and the
transaction is rolled back?  The page will already have been output,
so no error will be displayed.  The page can only safely be displayed
after all database activity is finished.

I don't see why the job queue needs to be using the same transactions
as the main page activity, if it is.  If there aren't other reasons
here I'm missing, your change would be fine if all transactions were
committed, then the page was output, then new transactions were
started for the job queue.  In your version of the code, I don't think
that's what's happening, since presumably the main transactions are
only committed at $factory->shutdown().

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Wondering why the output isn't flushed a little sooner

Brion Vibber-3
Simetrical wrote:

> On Wed, Jul 16, 2008 at 2:32 PM, DJ Bauch <[hidden email]> wrote:
>> Basically, I am flushing the output before running anything from my job
>> queue. I frequently have a non-empty job queue, and have found that the
>> modification lets the browser see the page much faster than when I run some
>> jobs first. Is there any reason not to suggest this as a change to the code
>> base? It made quite a dramatic improvement on my wiki when I made the
>> change, although perhaps things are different with Linux/Apache/MySQL?
>
> Well, I don't know, but here's a guess, based on the comment there.
> What happens if, in your setup, some error occurs during job queue
> processing (say a deadlock, or a database server crash, etc.) and the
> transaction is rolled back?  The page will already have been output,
> so no error will be displayed.  The page can only safely be displayed
> after all database activity is finished.

Job queue processing is meant to be transparent, so should *not* show an
error message to the end-user when processing unrelated background jobs.

A more general issue is to confirm whether the background processing
continues correctly if the connection gets aborted before it's complete
-- which is more likely if flushing output before this is done.

> I don't see why the job queue needs to be using the same transactions
> as the main page activity, if it is.  If there aren't other reasons
> here I'm missing, your change would be fine if all transactions were
> committed, then the page was output, then new transactions were
> started for the job queue.  In your version of the code, I don't think
> that's what's happening, since presumably the main transactions are
> only committed at $factory->shutdown().

Indeed, someone want to check what's going on in there? :)

-- brion

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l