Introduction

Attributes such as testability and security are key characteristics to determine the overall quality of any software system. I’m fairly sure that everybody would agree on this. However, just to reinforce the concept I’d love to recall that testability and security are expressly listed as nonfunctional requirements in any software architecture by the international standard ISO/IEC 9126. Since 1991, this standard defines a general set of quality characteristics required in software products. Based on this, testability and security are key attributes to guarantee the quality of a software system and therefore you should be planning for them very early in the design phase. In this article, I’ll focus on testability.

Testability in General

In the .NET space, topics such as software testing, unit testing, and testability have been overlooked for too many years. It’s only a few years ago that these arguments started gaining momentum; and it’s only recently that we could have a test environment in Visual Studio. What’s testability, anyway?

In software engineering, testability is an attribute that refers to the capability of a system to be tested, especially in an automated way. Testability measures how simple it is to test and troubleshoot a given piece of software. Testing is the process of checking software to ensure that it behaves as requirements dictate and contains no errors.

Design for Testability (DFT) was developed as a general concept a few decades ago. Goal of DFT was improving the process of building low-level circuits within boards or chips. DFT employed a number of design techniques and practices with the purpose of enabling effective testing via automated testing equipment. Automated testing equipment is nothing more than ad hoc programs that test expected functions of a board and report results for diagnosis purposes.

Design for software testability revolves around unit tests and suggest code design tips to make unit tests easier to write. More in general the widely accepted definition for software testability is “anything that makes your software easier to test in an automated way.”

Software Testing and Testability

In software, testing happens at various levels. There are unit tests to determine whether individual components of the software meet their functional requirements. There are integration tests to verify whether the software works well in the host environment and interacts properly with existing components. Finally, you have acceptance tests to ascertain whether the completed system meets the requirements set by the customer.

Unit tests and integration tests are performed by the development team. These are internal tests aimed at making the team confident about the quality of the software they’re producing. The end customer is typically not interested in the results of unit and integration tests. Acceptance tests, on the other hand, are all the customer cares about. Acceptance tests focus on the completed system and are often part of the contract set between the customer and the team. More often than not, acceptance tests are written after a strict collaboration between the customer and the team.

Software testability results from a number of characteristics of the code being tested that the development team should ideally guarantee: visibility, control and simplicity.

The attribute of visibility is defined as the ability to observe the current state of the software under test and any output it can produce. If testers have a way to programmatically observe a given behavior, then they can easily test it against expected and wrong values.

The attribute of control refers to the degree at which the code allows testers to apply fixed input data to the software under test. Any piece of software runs according to a virtual contract that includes preconditions. The easier you can configure preconditions, the easier you can write effective tests.

Simplicity is always a positive attribute for any system and in just every context. Testing is clearly no exception. Simple and extremely cohesive components with are preferable because the less you have to test, the more reliably and quickly you can do that.

In the end, design for software testability means writing the source code—preferably right from the beginning of the project—in order to privilege and maximize attributes such as visibility, control and simplicity. When this happens, writing unit tests that run within testing harnesses is easier and effective.

Before I get into more details about practical guidelines that smooth testing, let’s review the concept of software contracts. Having a contract clearly defined for each class you write will make your code inherently more testable.

Software Contracts

Bertrand Meyer first introduced the idea of software contracts in mid 1980s while he was working at the definition of the Eiffel programming language. The idea was just that of replicating in software the metaphor of a business contract that is established between two parties. In software, the parties participating to a contract are the method of a class and its caller.

Ideally, each method on a class provides a service and requires that a number of obligations are fulfilled by perspective callers. A software contract is expressed by the answers for the following three questions.

1. What are the initial conditions to be verified in order to invoke the method successfully?
2. Which conditions are verified after the method terminates?
3. Which conditions remain unchanged during the execution of the method?

These three points are also referred to as preconditions, postconditions, and invariants. Altogether, they form a software contract.

Your testing efforts greatly benefit from detailed knowledge of the software contract supported by all methods in a class. For example, when you intend to maximize testability, you design methods on a class by paying attention to validate all preconditions. Software contracts help you understand what needs to be tested and also make easier to implement whatever the method is expected to do.

Increase the Level of Visibility in Your Code

As mentioned, visibility is a code attribute that makes possible for testers to observe the state of the code and its output. To increase the visibility, you need to be accurate with software contracts.

For each method in a each class, you should have clear preconditions, postconditions and invariants. Preconditions indicate the feasible input data, including type and range of values. Preconditions also refer to the state of the object required for execution and which exceptions are thrown if an internal member is null or if certain conditions are not met. This information is key for arranging good tests.

Postconditions refer to the output generated by the method and the changes it produces on the state of the object. Postconditions help testers to figure out expected results.

Finally, invariants refer to property values (or expressions) that do not change during the method execution. In a design for testability scenario, you publish these invariants and testers assert them in tests. As an example of an invariant, consider the property Status of DbConnection: it has to be Open before you invoke BeginTransaction, and it must remain Open afterward.

Increase the Level of Control in Your Code

Knowing preconditions is a first key step to increase the control attribute over the code under test. Preconditions help you figure out exactly preliminary conditions to have a given method start and proceed correctly without throwing exceptions or just throwing the expected exception.

In terms of testing, exceptions are meaningful if they result out of failed preconditions, not as a postcondition. Testing a class, you should only pay attention to test that exceptions fire when invalid data is provided thus violating preconditions. Overall, you should design the body to have a prologue where you check preconditions and throw any possible exception here. If execution passes through this stage, then producing output should be a purely mechanical matter.

What if something bad happens during the execution of the method that causes the code to throw? If preconditions are correctly coded, this event is all in all occasional and should not be tested for. Or, you should test it elsewhere if it originates from an invoked component.

As an example, suppose that your code needs to read a file. The precondition to test is the file existence and you should throw a FileNotFoundException exception before attempting to read. If the file exists, there’s no deterministic cause that could raise an exception. A FileIOException is still acceptable, but only if during the test you lose the connection to the file, or if the file content is corrupted. Now, if you see these events as deterministic and know how to verify the state of the network and file beforehand, then you add both checks to the list of preconditions. Otherwise, they remain occasional failures not necessarily needing specific tests.

What if the method delegates some work to an internal component? For the purpose of testing, that component will be replaced with a fake one that is guaranteed to return valid data by contract. For this to happen, you should introduce proper interfaces to isolate the functionality of the component from its actual implementation. Dependency injection really comes in handy here and is a pattern that has a huge impact on testability. A class that depends on interfaces (the first principle of object-oriented design), and uses dependency injection to receive from the outside world any objects it needs to do its own work, is inherently more testable.

Increase the Level of Simplicity in Your Code

Two other general code attributes are never out of place: simplicity and stability. In a simple class with one primary responsibility is easier to identify software contracts for each method. A simple class is also easier to change and the overall number of changes you may introduce are anyway limited. So the fewer the changes, the fewer the disruption to testing.

Summary

The way in which you write your code may increase or decrease the visibility and control of the code and subsequently the effectiveness of tests. What makes a test successful? A nice guide to writing effective tests can be found here, but in general code isolation and good information about expected behavior and implementation are key factors.

Today, testing tools do a great work and let you build mocks and fakes for virtually every kind of class and method. In this way, software changes to enable or just favor testing are no longer necessary as it was only a few years ago. However, proper use of interfaces and inheritance attributes certainly make it easier. Thanks to powerful testing tools, the alternative to an inherent lack of testability is not manual testing. However, adding testability since the beginning inherently delivers cleaner code easy to read and maintain.

About Dino Esposito

Dino Esposito is one of the world's authorities on Web technology and software architecture. Dino published an array of books, most of which are considered state-of-the-art in their respective areas. His most recent books are “Microsoft ® .NET: Architecting Applications for the Enterprise” and “... This author has published 53 articles on DotNetSlackers. View other articles or the complete profile here.