Server reboots now through next week

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Server reboots now through next week

Greg Grossmeier-2
This past week there was an important security release for the Linux
kernel. As such, we will be updating and rebooting ALL of our machines
ASAP.

This may affect you.

ALL WMF services will experience some downtime of up to 10 or so minutes
(including Bugzilla, Gerrit, etc).

== SPECIAL CONSIDERATIONS ==

Some machines are OK for us to just reboot as needed but others are
being utilized by others for various tasks (scripts, cronjobs, etc).

If you have jobs running on any machine that you do not have puppetized
(ie: it won't just magically start up again after a reboot) you will
want to restart your jobs after the reboot.

There is, unfortunately, not set schedule of when any particular machine
will be rebooted, but Ops will be giving ~30 minutes notice in the
#wikimedia-operations IRC channel on Freenode. You can watch the public
Server Admin Log at <https://wikitech.wikimedia.org/wiki/Server_admin_log>
for the warnings and the reboot notice.

Sorry for the invonvenience,

Greg

--
| Greg Grossmeier            GPG: B2FA 27B1 F7EB D327 6B8E |
| identi.ca: @greg                A18D 1138 8E47 FAC8 1C7D |

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Server reboots now through next week

Petr Bena
hey, could you point me to that security patch? I am curious as I am
myself running bunch of linux boxes

On Fri, May 17, 2013 at 9:00 PM, Greg Grossmeier <[hidden email]> wrote:

> This past week there was an important security release for the Linux
> kernel. As such, we will be updating and rebooting ALL of our machines
> ASAP.
>
> This may affect you.
>
> ALL WMF services will experience some downtime of up to 10 or so minutes
> (including Bugzilla, Gerrit, etc).
>
> == SPECIAL CONSIDERATIONS ==
>
> Some machines are OK for us to just reboot as needed but others are
> being utilized by others for various tasks (scripts, cronjobs, etc).
>
> If you have jobs running on any machine that you do not have puppetized
> (ie: it won't just magically start up again after a reboot) you will
> want to restart your jobs after the reboot.
>
> There is, unfortunately, not set schedule of when any particular machine
> will be rebooted, but Ops will be giving ~30 minutes notice in the
> #wikimedia-operations IRC channel on Freenode. You can watch the public
> Server Admin Log at <https://wikitech.wikimedia.org/wiki/Server_admin_log>
> for the warnings and the reboot notice.
>
> Sorry for the invonvenience,
>
> Greg
>
> --
> | Greg Grossmeier            GPG: B2FA 27B1 F7EB D327 6B8E |
> | identi.ca: @greg                A18D 1138 8E47 FAC8 1C7D |
>
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Server reboots now through next week

Happy Melon-2
On 17 May 2013 23:26, Petr Bena <[hidden email]> wrote:

> hey, could you point me to that security patch? I am curious as I am
> myself running bunch of linux boxes
>

+1
_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Server reboots now through next week

Petr Bena
so far I found the problem with perf_events where exploit-containing
binary can elevate permissions of regular user to root. This is indeed
a big issue, but it seems to affect only systems with kernel newer
than 2.6.36 and only these where this feature is enabled. Also it
seems to me that only systems where untrusted users have shell access
are affected by this since it require local execution of exploit.

But thanks for information, despite it doesn't seem to require urgent
patch on systems with older kernel or any system where untrusted users
have no shell access (such as webservers) I will consider updating my
servers as well asap

On Sat, May 18, 2013 at 11:47 AM, Happy Melon
<[hidden email]> wrote:

> On 17 May 2013 23:26, Petr Bena <[hidden email]> wrote:
>
>> hey, could you point me to that security patch? I am curious as I am
>> myself running bunch of linux boxes
>>
>
> +1
> _______________________________________________
> Wikitech-l mailing list
> [hidden email]
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Reply | Threaded
Open this post in threaded view
|

Re: Server reboots now through next week

Petr Bena
More information:
http://www.h-online.com/open/news/item/Exploit-for-local-Linux-kernel-bug-in-circulation-Update-1863892.html

On Sat, May 18, 2013 at 3:18 PM, Petr Bena <[hidden email]> wrote:

> so far I found the problem with perf_events where exploit-containing
> binary can elevate permissions of regular user to root. This is indeed
> a big issue, but it seems to affect only systems with kernel newer
> than 2.6.36 and only these where this feature is enabled. Also it
> seems to me that only systems where untrusted users have shell access
> are affected by this since it require local execution of exploit.
>
> But thanks for information, despite it doesn't seem to require urgent
> patch on systems with older kernel or any system where untrusted users
> have no shell access (such as webservers) I will consider updating my
> servers as well asap
>
> On Sat, May 18, 2013 at 11:47 AM, Happy Melon
> <[hidden email]> wrote:
>> On 17 May 2013 23:26, Petr Bena <[hidden email]> wrote:
>>
>>> hey, could you point me to that security patch? I am curious as I am
>>> myself running bunch of linux boxes
>>>
>>
>> +1
>> _______________________________________________
>> Wikitech-l mailing list
>> [hidden email]
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
[hidden email]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l