Application quality

Software quality can be categorized into code quality and application quality.

Systems like Sonarqube and Microsoft Code Analysis measures software quality, to ensure that the actual written code keeps high quality.

This page explains, in our opinion, the important aspects of ensuring application quality.

You can compare those analysis tools to spell checkers when writing documents. They make sure that you have no spelling errors, ensure that you use the correct grammar and that sentences seems to be constructed correctly.

Your story may have the correct grammar, no spelling errors and still be quite faulty in the readers eye.

It may not make sense, mixing up characters, or just be portraying totally different stories mixed together. Those things cannot be detected by the spell checker (i.e. code analysis tools). Those things can only be found by tests or your users.

This page is dedicated to explain the key metrics that you, as a manager, can use to make sure that not too many bugs goes into production.

Test coverage measures how large part of your code is covered by unit tests (code tests).

Measuring test coverage is vital to get an indication to learn how well tested it is. The higher coverage, the less likely that previously built functions stop working.

A common goal is to cover 80% of the code base with tests, which means that 20% of the code is untested.

What's important is not the actual goal itself (like 80% above), but that the coverage isn't decreasing over time. Therefore, measure in the end of every sprint to ensure your application quality.

Test coverage is a metric which is supported in most development tools.

Cyclomatic complexity

The simple function to the right requires four tests to ensure that it works per the requirements, no matter if the tests are in code or done manually by a tester.

It's quite common that only one test case is made for a function, making sure that the successful case works. By not testing the fault paths, the code might continue with invalid data, and those errors can be tough to debug.

The important aspect of cyclomatic complexity isn't as a test indicator, but a comprehension indicator. The higher complexity, the harder for a developer to manage all the different way that the code can take. Complex code is much more prone for errors.

Your team should strive to have at most 20 as complexity in their functions, as the risk of bugs increases too much over that.

function saveDiscount(User accountOwner, string id, decimal discount)
{
    if (id == null)
    {
		throw new ArgumentNullException("You must specify account id.");
    }
    if (accountOwner == null)
    {
		throw new ArgumentNullException("You must specify account owner.");
    }
	
    var account = LoadAccount(id);
    if (account == null)
    {
		throw new InvalidDataException("Failed to load account " + id);
    }
	
    account.Discount = discount;
    SaveAccount(account);
}

Summary

There is a lot of ways to measure your application quality. In our experience, the metrics above are those that have the most considerable effect on your application quality (as opposed to code quality). It's even possible to have low software quality and high application quality if you manage complexity and test coverage.

High complexity combined with a low (or not measured code coverage) is the worst you can have. In that case, you have several errors in your production environment that you are not aware of.

Having a low unit complexity combined with a high test coverage is the best way to maintain a high application quality over time.

Try our bug estimator to learn what you are missing out on.

Bugs calculator