Slow MediaWiki: Optimize to Enhance Performance – Part 1

I run a popular website based on MediaWiki. For last couple of years it has been giving me nightmares because of its slowness. Despite all my efforts, I was not able to stabilize the web server and tackle slow MediaWiki. Most of the time server log would blame load.php and index.php scripts for causing crash. I did quite a bit of research and now that finally my website is in a much better shape, I thought of writing down my experiences for you all. Here I will write about a number of tips that I got from Internet and their effectiveness in enhancing MediaWiki performance.

While I was researching about how to enhance MW performance, I came across a number of fora where people were talking about their MediaWiki related problems and possible solutions. However, one thing that I almost always found missing from such problem/solution descriptions was the technical details about the website that was facing problems. We need to understand that different technical environments need different solutions. Therefore, presenting solution without giving any context may not be of much use.

So, first of all, let me tell you about the website I have so as to enable you to benchmark your own website, its problems, and effectiveness of possible solutions.

HARDWARE: VPS with about 1800MB RAM
SOFTWARE: Following is the software environment of my MediaWiki server:

• Uses UTF-8 charset for MediaWiki as well as MySQL database. It’s a non-English website
• MySQL database use innodb for most tables, some tables are MYISAM
• At present based on MediaWiki 1.20.3 (however, the problem of slowness began when it was running on MW 1.14)
• PHP version is    5.3.23 (cgi-fcgi)
• MySQL version is 5.1.69-cll
• Apache version is 2.2.24

My website has more than 60,000 pages, nearly 150,000 visitors per month and about 2 million page views per month. So, you may consider it a moderately high traffic website.

1) Slow performance. Pages will take 10 seconds to over 2 minutes to load. Server error log often tells me that load.php and index.php scripts were taking too long to finish. And that creates a bottleneck.

2) Unstable. Web server will often go down and will become as slow as dead (serving just 2-3 pages in a minute). Server will go into this unbearably slow mode for 15-20 minutes and then again spring back to life (serving 20-30 pages per minute).

3) Environment failure. I would often get email from  host that MySQL has failed or Apache has crashed or some other services went down.

Following tools will be very handy while investigating the cause of your server problem and implementation of solutions.

Tips for increasing MediaWiki performance.

Use Google Anayltics: Real time analytics provided by Google is a blessing. This shows you that, at present, how many people are connected to your website, how many pages are being delivered per minute and per second. This information is extremely handy in your trial and error round as you would will be able to see how various measures taken by you have affected traffic on your website.

Use a VPS: If you have a significantly big MediaWiki website and if you can afford it –go for a Virtual Private Server (VPS) instead of a Shared Server. On a VPS you can do a lot of things rather easily. You should get access to interfaces like cPanel, WHM and/or Virtuozzo.

Use SSH: Even with a shared server you can get SSH access to your account (however, in most cases, you have to request it from the host –they don’t give it on shared servers by default). So, go and request SSH access if you haven’t already got it. Once you get SSH access to your server, use a simple tool like putty to connect to your server account via SSH.

Let me say that it is very important to have SSH access if you hope to troubleshoot MediaWiki and enhance its performance.

We can try to enhance MediaWiki performance at several levels. I will write this article in three parts; one each for tweaking of MediaWiki, Apache Web Server and MySQL database.

Let’s begin with the first part of the process. First of all, you should set your MediaWiki installation to ideal settings. These settings are important for keeping larger MW site stable. These are the first steps toward high performance.

MediaWiki is a heavy web application both on Apache as well as MySQL. If you don’t often make changes in most of your pages, you can use file caching. When a user will request a page from your website, MediaWiki will store a copy of the page as an HTML file. Next time when someone will request the same page –instead of talking to MySQL and rebuilding the page –MW will send the HTML copy as response. This will make the response faster because database calls and PHP rendering will be bypassed. To enable file caching, put the following lines in LocalSettings.php file.

$wgUseFileCache = true;
$wgFileCacheDirectory = “$IP/cache”;
$wgShowIPinHeader = false;

You can also cache the sidebar content. This will speed up response from web server. To cache sidebar, add the following in LocalSettings.php file.

$wgEnableSidebarCache=true;

If you enable compression, MediaWiki will send the requested page to browser as a compressed file. This will reduce the file size of response and, therefore, transfer speed will be greater. All modern browsers are capable of uncompressing content and show it to the user. To enable compression, add the following line in the LocalSettings.php file.

$wgUseGzip = true;

MediaWiki keeps a track of how many times a page has been viewed. So, every time a page is requested –the page view count of that page is incremented in database. If keeping track of page views is not important for you, you should switch off this functionality to save database calls. To disable page view counters, add the following line in LocalSettings.php file.

$wgDisableCounters = true;

Note: If you’re using file caching, keeping counters on will not be of use anyway because in most cases the requested page will get delivered to the users from cache and database update will not take place.

Set Hit Counter Frequency
In case pageview counts are of great importance to you and you want to keep it on; you should make use of variable $wgHitcounterUpdateFreq

This variable sets the frequency of how often pageview counters are updated. By default, it has a value of 1 –which means update is done everytime a page is viewed. You should set it to something like 500. This would mean that the database will be updated only once after 500 pageviews. To set this variable, use the following in LocalSettings.php file.

$wgHitcounterUpdateFreq = 500;

MediaWiki can run in a miser mode wherein it disables a number of features to save resources and enhance performance. Following are the things that enabling miser mode does:

• Disables custom formatting of change sizes in Special:RecentChanges via MediaWiki:Rc-change-size (r48986)
• Disables selecting all pages that start with x box on Special:log (and leprefix option of list=logevents API module)
• Totally disables Special:MimeSearch, as well as aimime and famime option in the list=allimages and list=filearchive API module
• Disables showsizediff option of the RSS feed for Special:Contributions
• In the list=categorymembers and list=externallinks API module, use reduced sorting by namespace mode (returns only a few results when cmnamespace or elnamespace option is in use).
• Similarly disables the filter by namespace box on Special:LinkSearch
• Disables showing list of most popular pages on Special:Statistics (But still will show other page view statistics if counters are on)
• Doesn’t update number of active users quite as often
• Disables the search for images with x somewhere in their name box on Special:NewImages and Special:ListFiles
• Does not show how many previous versions of an image were uploaded on Special:ListFiles
• When running rebuildrecentchanges.php maintenance script, will not re-auto-patrol edits by users with auto-patrol flag.
• When running updateCollations.php maintenance script, will not tell you how many rows in total there are to update.

To enable miser mode, add the following in LocalSettings.php

$wgMiserMode = true;

Folks at MediaWiki development desk keep changing the code to make it faster and add features that could improve performances. So, you should upgrade your MediaWiki installation to the latest version. See how to upgrade MediaWiki.

MediaWiki maintains a queue wherein it schedules the jobs to be performed on database (e.g. addition of a new page, update changes, deletion of page etc.). $wgJobRunRate variable controls the rate at which jobs from this queue are executed. Ideally, we should execute jobs when there is least traffic (e.g. during night).

By default, this variable is set to 1; which means that one job will be executed on every page requested from your website. If you set it to 0.1, one job will be executed on every 10 page requests.

If real time updates are not important in your website, it is better to set job rate variable to 0 and then setup a cron job to execute all the accumulated jobs in off-peak hours.

To set job rate to zero, add the following in LocalSettings.php

$wgJobRunRate = 0;

Then you can set a crontab like this:

0 0 * * * /usr/bin/php /var/www/wiki/maintenance/runJobs.php > /var/log/runJobs.log 2>&1

This will cause jobs to run in batch mode at midnight on all days.

Pages like MediaWiki:Aboutsite and MediaWiki:Pagetitle refer to {{SITENAME}}. You should edit these pages and write your site’s name instead of {{SITENAME}}.

Now in part 2, I will discuss how we should tweak Apache webserver in a bid to make a slow MediaWiki installation run faster. Stay tuned.