Why to use automated PHP Code Analysis Tools ?

Traditional PHP Code Review demands an army of developers. Doing go PHP Code Review demands scarce experts. Sustainable PHP Code review demands tools that each uses to help manage the code.

With the increasingly important of technical debt, in PHP as elsewhere, we’ve seen the last two years the creation of multiple tools designed to automated code review and leverage this inside day-to-day developer activity.

The devops leaders face to different category of tools : either dedicated to framework like Insight from Sensiolabs, or generalist like Scrutinizer-CI, Codacy, or new entry in PHP like Code Climate.

These tools are a great shift towards the adoption of tools designed to detect compelling flaws as quickly as possible.

The main strength of automated code review (ie static source code analysis) is in the quick and automated checking of everything without the actual execution of the code. Because it targets issues that are boring to discover manually, it’s a perfect helper to the human eye.

Nevertheless, based on RebelLABS Study, more than 80% of code review is not following by a strict code fix …even these tools have a fair adoption more than 60%. Why this situation ?

After the workflow features to integrate the checks in the development lifecycle, the quality of these tools is based on  the quality of the PHP rules library (said in another way  PHP expertise).

What about the PHP expertise of commercial tools ?

We try to answer to the question, why to low code fix after inspection, in evaluating each rule.

In our study, we evaluate the 4 tools recognized as the best against our project Exakat : Scrutinizer-CI, Insight from Sensiolabs, Codacy and Code Climate.

As starting point, we focus only on PHP rules excluding the rules dedicated to others environment if available : C, CSS, Java or Python.

The global result is provided as global score stacked in bar for 4 different categories :
– Global for PHP project aims to classify all rules applicable for any php project excluding rules dedicated to a specific project or framework. This category includes security, bug risk, performance or deprecated code.
– Coding Convention aims to classify all rules to write code in the same style ( PEAR, PSR-1, PSR-2, …),
– Dedicated to framework aims to classify the rules applicable to a specific project or framework ( Zend Framework, Symfony, SPIP, …),
– Production environment aims to classify the rules dedicated to production matters.

 

The main lesson is that Scrutinizer-CI clearly exceeds all others commercials tools regarding the number of PHP rules and the balance between the compelling category. Exakat and Insight Labs are not so far, but Code climate and Codacy are clearly not applicable to reinforce the quality of a PHP project. These tools are either to young or late entry in PHP market.

 

Inside the 4 categories, a mandatory but not added value can be excluded: Coding Convention. Then, with a focus on the single category which is the flagship of PHP quality (Global for PHP project), the feeling is confirmed.

 

Except Scrutinizer-CI, it is difficult to use the commercial tools of the market to increase the quality of PHP quality. The tools are promising through their workflows features but provide poor knowledge or insight on the code because of the bareness of their PHP expertise.

And the price ?

We add (in red in the table below) the average price for a standard team requirement against the “Global for PHP projects “ category.

Code climate is twice as expensive as other tools. Codacy, Insight Labs and Scrutinizer-CI remain in the same slot of price. Insight is the less expensive of the commercial tools. With the specialisation on symphony and the level of workflow feature the pricing position is at level.

Why to launch Exakat

In conclusion, the scrutinizer-CI is the less worst commercial tool to start a compelling automated code review approach. Code climate, the last entry in the PHP market but the more expensive too, needs to improve its solution before to be clearly applicable to a PHP project.

In this context, we have identified since months an opportunity for the PHP Lead Devops to have a new tool to take advantage of the possibilities of new technology to analysis in real time tons of line of PHP code. We have define 3 characteristics for this new tool : Expertise, Focus and Open.