MediaWiki 1.4 benchmarks

This page is kept for historical interest. Its content is outdated or may be wrong. You may find more up-to-date information at on the www.mediawiki.org website.

Conclusions

Using an opcode cache (eg Turck MMCache) gives a large constant speedup, about 2X for short pages. This speedup is relatively diminished for long pages but is still a significant chunk.
Using the parser cache nearly eliminates the difference in render time between short and long pages for cache hits. The relative benefits are small for small pages and increase for longer pages. (Cache misses will still take as long as without the cache.)
Particular CPU-intensive activities such as complicated diffs of long pages can be sped up dramatically by special-purpose C/C++ extensions. To the extent that these operations are either frequent or that their occasional use creates large load spikes, the plugin can be very beneficial.
The file cache is still several times slower for hits than it could be; for higher-traffic sites unable to use squid the savings in CPU time could be significant from optimizing this further. (cf Evan's experimental 404 handler mode)

The PHP extensions are something we can recommend, though we can't force them on and many people aren't in a position to be able to use such.

The parser cache however can easily be enabled, and doesn't require filesystem permissions (cf safe mode hell). I've tweaked it to support using the objectcache table for non-memcached sites; this should give a nice boost to lower- to medium-traffic sites at relatively little cost. It's now enabled by default in 1.4.

Test methodology

Simple benchmark using 'ab' tool. A page is loaded consecutively 20 times from a neighboring computer on the LAN; the times reported are median, minimum, and maximum total load times from each run, in milliseconds.

Test server is AMD Athlon XP 2400+ with 512 MB RAM, running Ubuntu Linux (Warty Warthog). MySQL 4.0.20, Apache 2, PHP 4.3.8, Turck MMCache 2.4.6, SWIG and ICU are installed from Ubuntu's packaging system. UtfNormal/ICU and wikidiff plugins are compiled from the 'normal' and 'wikidiff' subdirectories in the extensions module in CVS.

When changing configurations, test pages are loaded once each from a web browser before the ab test runs.

Comparing 1.3.8 release against 1.4 as checked out from CVS late-night UTC November 20, 2004. Results added later with subsequent revisions of 1.4 which have been further optimized are marked with '1.4+' and a comment.

Simple page view

Viewing the Main Page of a freshly installed English-Unicode wiki.

 Without Turck:
   1.3: 216 (213 to 235)
   1.4: 189 (185 to 208)

 With Turck:
   1.3:  88 ( 87 to 118)
   1.4:  74 ( 70 to  83)
   1.4+  61 ( 59 to  64) after some further tweaks

Simple page view with non-ASCII title

1.4 does extra checks for valid, normalized UTF-8. Even for short strings like a title this requires loading up some data tables which is a speed hit. Using a PHP extension to do the check using an external library, this speed hit is minimized.

 Without Turck:
   1.3: 215 (210 to 234)
   1.4: 212 (210 to 218)
 
   With ICU:
   1.4: 186 (184 to 206)

 With Turck:
   1.3:  86 ( 82 to  90)
   1.4:  92 ( 90 to 100)
 
   With ICU:
   1.4:  72 ( 71 to  88)

en:Zuiderzee Works

Copied full text, images, and the commons template. Did not create other pages, so it's full of broken links.

 Without Turck:
   1.3: 719 (715 to 726)
   1.4: 613 (607 to 693)
   1.4+ 355 (351 to 374) after further tweaks and regression fix
   
   With memcached/parser cache:
   1.3: 224 (222 to 247)
   1.4: 197 (194 to 207)
   
   With file cache:
   1.3:  155 (153 to 162)
   1.4:  151 (147 to 166)

 With Turck:
   1.3: 575 (563 to 589)
   1.4: 472 (467 to 490)
   1.4+ 238 (232 to 243) after further tweaks and regression fix
   
   With memcached/parser cache:
   1.3: 104 (102 to 119)
   1.4:  82 ( 78 to  91)
   1.4+  57 ( 56 to  79) after some further tweaks
   
   With file cache:
   1.3:  38 ( 35 to  40)
   1.4:  41 ( 37 to  50)
   
   Experimentally grabbing the cached .html file direct:
   raw: 200 ( 26 to 204)
   This is really, really weird. I don't understand.
   
   And via a short, special-purpose PHP passthrough script:
   raw:   7 (  7 to   7)

Zuiderzee Works minus links

Removed all brackets from the text. Note that 1.4 is rather faster on this version.

 Without Turck:
   1.3: 417 (414 to 473)
   1.4: 310 (307 to 314)
 
   With memcached/parser cache:
   1.3: 214 (213 to 220)
   1.4: 192 (189 to 197)
 
   With file cache:
   1.3: 153 (150 to 161)
   1.4: 148 (147 to 150)

 With Turck:
   1.3: 291 (286 to 327)
   1.4: 193 (188 to 199)
 
   With memcached/parser cache:
   1.3:  95 ( 93 to 110)
   1.4:  80 ( 76 to  99)
   
   With file cache:
   1.3:  37 ( 36 to  39)
   1.4:  39 ( 38 to  40)

Diff

From the linkless version of Zuiderzee Works to the linkful version.

 Without Turck:
   1.3: 1372 (1360 to 1393)
   1.4: 1225 (1218 to 1358)
   
   With wikidiff:
   1.4:  711 ( 706 to  717)

 With Turck:
   1.3: 1194 (1183 to 1209)
   1.4: 1050 (1038 to 1082)
   
   With wikidiff:
   1.4:  561 ( 558 to  575)

Anomaly

On my first checks of Zuiderzee Works with and without turck, I got these lower times:

   with: 1.3: 557 (553 to 605)
   without: 1.3: 422 (412 to 439)

All later checks give the slower times given in the tables above, and my checks of 1.4 continue to give the same results as I originally got at the time for it. I'm assuming I did something wrong the first time, but maybe there's some magic 'speed up!' option I stumbled on. ;)

Addendum: PHP5

I installed a second copy of Apache on my test box and set it up with PHP 5.0.3rc1. In a quick ab comparison benchmark against the PHP 4.3.8 installation already there, I found that PHP 5 was slightly slower, by a roughly 25ms margin. Profiling showed the difference to be mainly in the includes section of Setup.php, indicating that PHP 5's parsing/loading is more expensive.

Tests were done without Turck or other extensions, and not using memcached. (Parser cache test thus using the objectcache table.)

Zuiderzee Works

 Parser cache enabled:
   PHP4 trial one: 168 (165 to 181)
   PHP4 trial two: 168 (165 to 171)
   
   PHP5 trial one: 195 (192 to 377)
   PHP5 trial two: 194 (193 to 369)

 Parser cache disabled:
   PHP4 trial one: 339 (335 to 343)
   PHP4 trial two: 339 (333 to 471)
   
   PHP5 trial one: 362 (358 to 378)
   PHP5 trial two: 361 (358 to 379)

Addendum: DokuWiki

DokuWiki vs MediaWiki benchmarks