GeoIP Performance Examined: Apache vs PHP

If there's one thing programmers love worrying about, it's performance. It's a pity, therefore, that while arguing about minuscule improvements and micro-benchmarks, many of them forget to put indexes on their queries… but that's a tale for another blog post.

Since we at WonderProxy are no different than the rest of the programming world (sadly, also with the index thing), one question that comes up quite often for us is: For maximum GeoIP performance, should I use the PHP extension or the Apache module?

We decided to figure that out.

GeoIP Options:

The Maxmind Legacy database is a popular solution for turning IP addresses into useful location information.
When using the Maxmind Legacy database (and PHP and Apache), there are actually three ways to integrate it into your application: Apache module, PHP Extension, PHP Library.

• Apache module:
  This approach adds a module to Apache that looks up GeoIP information in a local database for each request and makes location information available as server variables. PHP scripts and Apache configuration files such as .htaccess can then use the location information for fun and profit. You can obtain the Apache mod_geoip module from MaxMind.

• PHP extension:
  This approach adds an extension to PHP that provides support for a bunch of geoip functions, allowing you to look up any IP. This information is only available inside PHP.

• PHP library:
  This is a library installed and managed with Composer, making it possible to get started with minimal effort. Once included, the PHP library provides functions similar to the PHP extension.

Experiment:

For the test we used the free MaxMind GeoIP lite database as well as the "legacy" tooling, as they still seem the most popular. We used Siege (an open source HTTP load tester and benchmarking utility) to run our tests. Read on for a full description of our test environment, procedure, and results.

Software Versions & Configuration:

• Digital Ocean 2GB VM running Debian 7.
• Siege 2.70
• Apache 2.2.22
  Prefork MPM, modstatus disabled.
  Apache configuration:

<IfModule mpm_prefork_module>
    StartServers          50
    MinSpareServers       2
    MaxSpareServers      60
    MaxClients          100
    MaxRequestsPerChild   0
</IfModule>

• PHP 5.4.45-0+deb7u5 with OPcache v7.0.3.
  PHP Configuration:

opcache.memory_consumption=64
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=4000
opcache.validate_timestamps=0
opcache.fast_shutdown=1
opcache.enable_cli=0

• Script variations on:

Procedure

We used variations on the following script for each test, being careful to ensure that the output for each variation was identical.

<?php
require 'vendor/autoload.php';
$gi = geoip_open("/usr/share/GeoIP/GeoIP.dat",GEOIP_STANDARD);

var_dump(geoip_country_name_by_addr($gi, $_SERVER['REMOTE_ADDR']));
var_dump($_SERVER['REMOTE_ADDR']);

To execute the tests, we ran siege -c50 -t60s http:://<ip address>/ four times, throwing away the first result to allow for cache warming. We restarted Apache between runs for configuration changes.

Experimental Notes:

• By ensuring that the script output for each test was identical, we avoided having artificially fast results.

• For each run, we took care to avoid any script errors that would slow down execution.

• OPCache was configured to never validate timestamps. For this test, we could have set OPCache validation to either "always" or "never"; doing it on a timed basis would skew results as it might occur a different number of times during benchmarking. We settled on opcache.validate_timestamps=0 (never") because we weren't trying to measure how long it takes PHP to update the OPCache.

Results:

Analysis:

It doesn't matter which approach you use: the difference in the results between the Apache module, PHP extension and PHP library were easily within the margin of error for this test. So, use whichever tool you want: they're all really darn fast.

I was particularly surprised that the PHP library performed as well as it did as it requires many more files to be used. Kudos to the OPCache team for their fantastic work! Turning off the OPCache did result in lower performance, but it was a smaller reduction than I expected (~5700 hits in 60 seconds).

That being said, one major limitation of the test is that the script was repeatedly determining the GeoIP country for the same IP. While alternatives were possible for testing the PHP extension and library, the Apache module always uses the remote IP address. Figuring out a way to address this would be helpful for future benchmarking.

Repetition:

If you've got the time, we'd love to link to other people testing this out. Send me an email (paul at wonderproxy.com ) and I'll link your results here. Dire warnings about fatal flaws in my testing methods also gratefully accepted!

On Debian 7 Minimal Install, this should get you started setting up your environment:

apt-get install -y build-essential apache2 php5 php5-dev siege php-pear build-essential autoconf libtool apache2-threaded-dev

If you're using Vagrant, you'll find these instructions much more detailed.