protip: i/o and infinite loops in php

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

protip: i/o and infinite loops in php

River Tarnell-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

never do this:

do {
  $x = file_get_contents("http://www.example.com/");
} while ($x === false);

or this:

while (!feof($f)) {
  $x .= fgets($f);
}

why not: because if the i/o fails, your script will run forever, using
100% cpu, doing nothing.

instead, add error checking, and exit if there's a problem:

if (($x = file_get_contents("http://www.example.com") === false) {
  echo "shit happened\n";
  exit;
}

please make sure none of your scripts have this problem, because they
cause issues for all users on the toolserver.

     - river.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFHlvNLIXd7fCuc5vIRAjndAJ9Z+SFfe412VLIymbCZLYnDvqHXbQCeMUjo
FQOj5I2OpULhcqsVBJ0706M=
=xzww
-----END PGP SIGNATURE-----


_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Marc-Andre
River Tarnell wrote:
> never do this:
>
> [...]
>
> while (!feof($f)) {
>   $x .= fgets($f);
> }
>
That may indeed have been a problem with a script of mine; I presumed
libc semantics where if the stream had gotten invalid feof() would
return an error (and, thus, not 0).  At any rate, the script was
experimental and has since been disabled, but it's an important gotcha
to know of.

-- Marc


_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Daniel Schwen-2
In reply to this post by River Tarnell-2
> why not: because if the i/o fails, your script will run forever, using
> 100% cpu, doing nothing.
Mh, yeah, I had a similar problem with my scripts. No infinite loop,  but a
blocking write to a fifo, as my fifo listener process was killed off.
I wasted lots of memory with my stale php processes. Sorry about that!
I fixed it by adding checks for the fifo listener (using pid files) and
aborting the php script if the write would be blocking.
--
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Artur Fijałkowski
2008/1/23, Daniel Schwen <[hidden email]>:
> > why not: because if the i/o fails, your script will run forever, using
> > 100% cpu, doing nothing.
> Mh, yeah, I had a similar problem with my scripts. No infinite loop,  but a
> blocking write to a fifo, as my fifo listener process was killed off.
> I wasted lots of memory with my stale php processes. Sorry about that!
> I fixed it by adding checks for the fifo listener (using pid files) and
> aborting the php script if the write would be blocking.

Maybe using non-blocking I/O would help too ?:)

AJF/WarX

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Daniel Schwen-2
> > aborting the php script if the write would be blocking.
> Maybe using non-blocking I/O would help too ?:)

That would be the obvious choice. But unfortunately there seems to be no way
to realize that in PHP. And furthermore if the fifo has no listener there is
no point in writing the data. Having the data queued until the listener comes
up again would just create a huge backlog of work (map tiles to be rendered)
which no one would care about (since the requests for those tiles timed out
anyways).
--
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Bryan Tong Minh
On Jan 23, 2008 6:22 PM, Daniel Schwen <[hidden email]> wrote:

> > > aborting the php script if the write would be blocking.
> > Maybe using non-blocking I/O would help too ?:)
>
> That would be the obvious choice. But unfortunately there seems to be no way
> to realize that in PHP. And furthermore if the fifo has no listener there is
> no point in writing the data. Having the data queued until the listener comes
> up again would just create a huge backlog of work (map tiles to be rendered)
> which no one would care about (since the requests for those tiles timed out
> anyways).
> --
> [[en:User:Dschwen]]
> [[de:Benutzer:Dschwen]]
> [[commons:User:Dschwen]]
>
> _______________________________________________
>
> Toolserver-l mailing list
> [hidden email]
> http://lists.wikimedia.org/mailman/listinfo/toolserver-l
>
http://nl2.php.net/manual/fi/function.stream-select.php ?

Bryan

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Daniel Schwen-2
> > That would be the obvious choice. But unfortunately there seems to be no
> > way to realize that in PHP. And furthermore if the fifo has no listener
> http://nl2.php.net/manual/fi/function.stream-select.php ?

How do I get a stream object to pass to stream_select? fopen? fopen is already
blocking if my fifo listener is dead.
--
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Bryan Tong Minh
On Jan 23, 2008 6:53 PM, Daniel Schwen <[hidden email]> wrote:

> > > That would be the obvious choice. But unfortunately there seems to be no
> > > way to realize that in PHP. And furthermore if the fifo has no listener
> > http://nl2.php.net/manual/fi/function.stream-select.php ?
>
> How do I get a stream object to pass to stream_select? fopen? fopen is already
> blocking if my fifo listener is dead.
> --
>
> [[en:User:Dschwen]]
> [[de:Benutzer:Dschwen]]
> [[commons:User:Dschwen]]
>
> _______________________________________________
> Toolserver-l mailing list
> [hidden email]
> http://lists.wikimedia.org/mailman/listinfo/toolserver-l
>
Ah I assumed that the blocking occurred in fread. stream_select won't
work for fopen :(

Bryan

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l
Reply | Threaded
Open this post in threaded view
|

Re: protip: i/o and infinite loops in php

Kurt-36
Btw, there's a *very* beta (and not currently maintained) event extension on
pecl somewhere that supports epoll(), kqueue(), etc.  I haven't played too
much with it, because in cases where I need to use advanced polling engines I
probably don't need to be using PHP for whatever I need to do :P

Anyway, the easiest solution whenever you need to do something involving a
while() is almost *always* have a sleep() or usleep() call at the end of it.  
Even if it's only a miniscule stop it tends to make a big difference
depending on what you're doing. Another failsafe is turning a while() into a
for() loop with an arbitrary upper bounds (100000 for example) during
debugging. At least that way, when you're testing things out, if something
gets out of hand, the loop will break on its own.  From that point, it's easy
to simply convert the for() back to a while() once you know nothing bad will
happen.

Cheers.
-slakr (@en.wp)

On Wednesday 23 January 2008 11:56, Bryan Tong Minh wrote:

> On Jan 23, 2008 6:53 PM, Daniel Schwen <[hidden email]> wrote:
> > > > That would be the obvious choice. But unfortunately there seems to be
> > > > no way to realize that in PHP. And furthermore if the fifo has no
> > > > listener
> > >
> > > http://nl2.php.net/manual/fi/function.stream-select.php ?
> >
> > How do I get a stream object to pass to stream_select? fopen? fopen is
> > already blocking if my fifo listener is dead.
> > --
> >
> > [[en:User:Dschwen]]
> > [[de:Benutzer:Dschwen]]
> > [[commons:User:Dschwen]]
> >
> > _______________________________________________
> > Toolserver-l mailing list
> > [hidden email]
> > http://lists.wikimedia.org/mailman/listinfo/toolserver-l
>
> Ah I assumed that the blocking occurred in fread. stream_select won't
> work for fopen :(
>
> Bryan
>
> _______________________________________________
> Toolserver-l mailing list
> [hidden email]
> http://lists.wikimedia.org/mailman/listinfo/toolserver-l

_______________________________________________
Toolserver-l mailing list
[hidden email]
http://lists.wikimedia.org/mailman/listinfo/toolserver-l