In a previous blog post, I discussed about the occurrence of security bugs through software evolution. In this post we will examine their existence in a large software ecosystem. To achieve this, together with four other colleagues (Vasilios Karakoidas, Georgios Gousios, Panos Louridas and Diomidis Spinellis) we used the FindBugs static analysis tool, to analyze all the projects that exist in the Maven central repository (approximately 260GB of interdependent project versions).

Maven is a build automation tool used primarily for Java projects and it is hosted by the Apache Software Foundation. It uses XML to describe the software project being built, its dependencies on other external modules, the build order, and required plug-ins. First, we scanned the Maven repository for appropriate JARs and created a list that included them. After some project filtering, we narrowed down our data set to 17,505 projects with 115,214 versions. With the JAR list at hand, we created a series of processing tasks and added them to a task queue. Then we executed twenty-five (Unix-based) workers written in Python that checked out tasks from the queue, processed the data after invoking FindBugs and stored the results to a data repository.

FindBugs separates software bugs into nine categories. Two of them involve security issues: Security and Malicious Code. From the total number of releases, 4,353 of them contained at least one bug coming from the first category and 45,559 coming from the second.

The figure below shows how software bugs are distributed among the repository. Together with the Bad Practice bugs and the Style bugs, security bugs (the sum of the Security and Malicious Code categories) are the most popular in the repository (≥ 21% each). This could be a strong indication that programmers write code that implements the required functionality without considering its many security aspects; an issue that has already been reported in literature.

Bug Percentage in Maven Repository

Another observation involves bugs that we could call severe and they are a subset of the Security category. Such bugs are related to vulnerabilities that appear due to the lack of user-input validation and can lead to damaging attacks like SQL injection and Cross-Site Scripting. To exploit such vulnerabilities, a malicious user does not have to know anything about the application internals. For all the other bugs, another program should be written to incorporate references to mutable objects, access non-final fields, etc. Also, as bug descriptions indicate, if an application has such bugs, it might have more vulnerabilities that FindBugs doesn’t report. In essence, 5,501 releases (≈ 4, 77%) contained at least one severe security bug. Given the fact that other projects include these versions as their dependencies, they are automatically rendered vulnerable if they use the code fragments that include the defects.

Linus’s Law states that “given enough eyeballs, all bugs are shallow”. In a context like this, we expect that the project versions that are dependencies to many other projects would have a small number of security bugs. To examine this variation of the Linus’s Law and highlight the domino effect we did the following: during the experiment we retrieved the dependencies of every version. Based on this in- formation we created a graph that represented the snapshot of the Maven repository. The nodes of the graph represented the versions and the vertices their dependencies. The graph contained 80,354 nodes. Obviously, the number does not correspond to the number of the total versions. This is because some versions did not contain any information about their dependencies so they are not represented in the graph. After creating the graph, we ran the PageRank algorithm on it and retrieved all PageRanks for every node. Then we examined the security bugs of the fifty most popular nodes based on their PageRank. Contrary to Linus’s Law, thirty-three of them contained security bugs, while two of them contained severe bugs. Twenty-five of them were latest versions at the time. This also highlights the domino effect.

Future work could also involve the observation of other ecosystems, that serve different languages than Java, in the same manner such as, Python’s PyPy (Python Package Index) and Perl’s CPAN (Comprehensive Perl Archive Network).