Part 1 The focus in part 1 is on the fundamental concepts and tooling support in Visual Studio 2010.
Part 2 In this article, we will delve into unit testing controllers in ASP.NET MVC, especially in terms of ASP.NET MVC 3 related concerns.
In recent years, Test-Driven Development (TDD) has become a hot technique in software development industry. According to the TDD philosophy, we should first write unit tests based on product specifications, and then write the source code that is required to make the tests succeed. In response to this, Visual Studio provides a couple of related facilities, especially an excellent built-in unit testing framework (in short, MSTest). And also, the latest Visual Studio 2010 supports a new feature named "Generate From Usage", through which we can generate new types and members in the source code promptly when we first reference them in our test cases before they are defined.
In this series of articles, I will provide you a detailed tutorial in using Visual Studio 2010's built-in unit testing framework to develop ASP.NET MVC 3.0 based applications. In this first article, we are to focus upon the fundamental concepts and tooling support in Visual Studio 2010.
The test environments we'll utilize in the sample applications in this series contain:
1. Windows XP Professional (SP3);
2. .NET 4.0;
3. Visual Studio 2010;
4. ASP.NET MVC 3 RC (http://www.microsoft.com/downloads/en/details.aspx?FamilyID=a920ccee-1397-4feb-824a-2dfefee47d54);
5. Microsoft Northwind sample database;
6. The famous and simplest mocking library - Moq 4.0 (download address: http://code.google.com/p/moq/downloads/list).
Introducing Test Driven Development and Unit Test
Test-Driven Development (TDD) is a technique that adheres to the "Test First" principle of Extreme Programming (XP). TDD is a revolution for developers to shift from writing code first and then testing it to writing the tests first and then writing the code to satisfy the tests afterwards.
Test first programming bears a couple of benefits. First, if we have a better understanding with the target project requirements prior to starting coding, then we will be in a good position writing the tests for that code and therefore the set of tests represents the acceptance tests for that code. Second, having defined the set of tests you can then write the code to satisfy them. Knowing that you are looking to pass the tests really helps focus the developer's mind on writing the minimum code to satisfy those tests.
Unit testing is the cornerstone of Extreme Programming, which relies on an automated unit testing framework. This automated unit testing framework can be either third party, e.g., xUnit, or created within the development group.
As you can image, the tests (of course including unit tests) should be sufficient and comprehensive enough. Only this precondition is met can the above benefits be valid; otherwise, the unit tests will become empty words.
Following good Test-Driven Development practice, before you do anything else, you first write a test. You don't write any application code until you have a test for the code.
The A-A-A pattern
To write meaningful unit tests that can be well-read, developers are suggested to follow the A-A-A (Arrange-Act-Assert) pattern. In detail, first we arrange all necessary preconditions and inputs. Then, we act on the object or method under test. Finally, we assert that we have the desired result. In practice, this can be translated easily into testing ASP.NET MVC controllers:
1. Arrange: Instantiate a controller object (in DI scenarios, you may need to supply mock versions of any dependencies as constructor parameters).
2. Act: Run an action method, passing sample parameters and collecting the ActionResult.
3. Assert: Assert that the ActionResult describes the expected result.
Look at the following example.
The above loggic is east to follow. First, we create an instance of the controller class ProductController. Then, we invoke the action method
Index of the controller instance and get the returned result. Finally, we make two assertions: 1. is the view
Index rendered or not; 2. is the view name is empty or not.
Next, let's look at how Visual Studio 2010 provides support for test-based development, especially unit test related stuff.
Test Support in Visual Studio 2010
Visual Studio 2010, as the industry leader among tons of software development tools, has introduced a number of support tools for testing. In this section, let's make a short summary around them.
Test types support
Visual Studio 2010 defines a couple of tests support, including Basic Unit Test, Coded UI Test, Database Unit Test, Generic Test, Load Test, Ordered Test, Unit Test, Unit Test Wizard, and Web Performance Test. Some of these tests should be used together with TFS, covering a wide range of development knowledge.
Considering the scope of this series, we are only to introduce some of them.
1. Ordered Test. This kind of test is used to group all or some of the above tests and then execute them in the same order. The main advantage of creating the ordered test is to execute multiple tests in an order based on the dependencies. For example, Web Performance Tests might depend on the results produced by executing the unit tests. So we need to run these tests in an order so that the Unit Test can be executed before starting the Web Performance Tests.
2. Generic Tests. This kind of test provides ways to integrate third-party tests into Visual Studio. There could be applications which use third party components or services which have to be tested as part of the whole system testing. Do remember that, in this case, we cannot have our own custom test methods or testing created for the third party component as we may not know the logic and the details of how the component is built. In order to test the third party component, Generic Test types in Visual Studio act as wrappers for executing these third-party tests within the boundary of Visual Studio. Once they are wrapped, we can execute these generic tests like any other test through Visual Studio IDE.
3. Load Testing. This kind of test corresponds to the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system's behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation. When the load placed on the system is raised beyond normal usage patterns, in order to test the system's response at unusually high or peak loads, it is known as stress testing.
There is little agreement on what the specific goals of load testing are. The term is often used synonymously with software performance testing, reliability testing, and volume testing. For a detailed story related to load testing with Visual Studio 2010, you can refer to this article.
4. Web Tests. This kind of test allow you to simulate a user performing a set of operations – typically a defined use case – on ASP.NET Web applications, and validate the responses to see if the application is working as expected. Once you have our Web tests defined, you can knit them together to create a load test to see how well our application performs under stress. It's always better to test our Web applications early to detect performance problems.
5. Coded UI Test. This kind of test provides functional testing of the user interface and validation of user interface controls. Automated UI tests enable you to test that the user interface is functioning correctly after code changes. They are quicker to run than manual tests. Therefore, you can run them more frequently.
Next, let's focus upon the really interesting topics associated with Unit Test in Visual Studio 2010.
Creating test assertions
At most time, when you are writing the code for our test methods, you use the methods of the Assert class. The last line of code contained in most test methods uses the Assert class to assert a condition that a test must satisfy in order for the test to pass.
1. Static class - Assert
Visual Studio 2010 defines a static class Assert achieving these functionalities with a couple of static methods as follows.
The following lists some of the commonly-used members:
AreEqual: With several override forms, verifies that two specified objects are equal. The assertion fails if the objects are not equal.
AreNotEqual: With several override forms, verifies that two specified objects are not equal. The assertion fails if the objects are equal.
AreSame:Verifies that two specified object variables refer to the same object. The assertion fails if they refer to different objects.
AreNotSame: Verifies that two specified object variables refer to different objects. The assertion fails if they refer to the same object.
Inconclusive: Asserts that a test result is inconclusive. Visual Studio includes this assertion for methods that it generates automatically and that you need to implement.
IsFalse: Asserts that a given condition expression returns the value False.
IsInstanceOfType: Asserts that a given object is an instance of a specified type.
IsNotInstanceOfType: Asserts that a given object is not an instance of a specified type.
IsNotNull: Asserts that an object does not represent the Null value.
IsNull: Asserts that an object represents the Null value.
IsTrue: Asserts that a given condition expression returns the value True.
ReplaceNullChars: Replaces Null characters in a string \0 with \\0.
When an Assert method fails, the Assert class throws an AssertFailedException.
2. Static class - CollectionAssert
There is a special class for testing assertions about collections named the CollectionAssert class. In detail, the CollectionAssert class supports the following static methods:
AllItemsAreInstancesOfType: Asserts that each item in a collection is of a specified type.
AllItemsAreNotNull: Asserts that each item in a collection is not null.
AllItemsAreUnique: Asserts that each item in a collection is unique.
AreEqual: Asserts that the value of each item in two collections is equal.
AreEquivalent: Asserts that the values of each item in two collections are equal (but the order of the items in the first collection might not match the order of the items in the second collection).
AreNotEqual: Asserts that two collections are not equal.
AreNotEquivalent: Asserts that two collections are not equivalent.
Contains: Asserts that a collection contains an item.
DoesNotContain: Asserts that a collection does not contain an item.
IsNotSubsetOf: Asserts that one collection is not a subset of another collection.
IsSubsetOf: Asserts that one collection is a subset of another collection.
3. Static class - StringAssert
There is also a special class, named StringAssert, for performing assertions about strings. In detail, the StringAssert class supports the following static methods:
Contains: Verifies that the first string contains the second string.
DoesNotMatch: Asserts that a string does not match a specified regular expression.
EndsWith: Asserts that a string ends with a specified substring.
Matches: Asserts that a string matches a specified regular expression.
StartsWith: Asserts that a string starts with a specified substring.
There are many attributes defined inside the namespace Microsoft.VisualStudio.TestTools.UnitTesting. Let's enumerate them one by one.
1. The mandatory attributes
When we build tests, we will use the following two attributes:
[TestClass]: used to mark a class as a test class. Only classes marked with this attribute will run when you run our tests.
[TestMethod]: used to mark a method as a test method. Only methods marked with this attribute will run when you run our tests.
When building tests, you should always use the
2. The other important attributes
Besides the above two mandatory attributes, there are several other useful, but optional, test attributes. For example, you can use the following attribute pairs to setup and tear down tests:
[AssemblyCleanup]: used to mark methods that execute before and after all of the tests in an assembly are executed.
[ClassCleanup]: used to mark methods that execute before and after all of the tests in a class are executed.
[TestCleanup]: used to mark methods that execute before and after each test method is executed.
Consider such an example - you want to create a fake HttpContext that you can use with all of our test methods. You can setup the fake HttpContext in a method marked with the
[ClassInitialize] attribute and dispose of the fake HttpContext in a method marked with the
[ClassCleanup] attribute. To do with such a case, we will generally resort to some mocking tools, such as Moq (open source), Rhino Mocks (open source), and TypeMock(commercial).
Here is a simple example using the above two pairs of attributes.
In essence, this is rather standard behavior for test suites: setup and teardown before and after each test. The philosophy is that tests should not depend on each other. If you do want another behavior, you should probably use static objects that persist between each test.
3. Other optional attributes
There are still several attributes that you can use optionally to provide additional information about test methods. These attributes are useful when you are working with hundreds of unit tests and you need to manage the tests by sorting and filtering the tests:
[Owner]: Enables you to specify the author of a test method.
[Description]: Enables you to provide a description of a test method.
[Priority]: Enables you to specify an integer priority for a test.
[TestProperty]: Enables you to specify an arbitrary test property.
As for the first three attributes, from the naming, you can easily deduce their respective usages. But for the last one, I want to give more explanation. First of all, please look at the following code.
Running the above two tests, you will find they can both pass. Here, there are some points worth stressed. First, the
TestContext property is a special place to hold shared information provided to unit tests. Especially, its
Properties property is a good place to store name/value. Next, when you test the first method TestAgainstDefaultSetup the method
MyInitialize should run first. But at this time, the
TestContext.Properties does not contain a name named thing. So, the variable
_thing is assigned to "default". Hence, the method TestAgainstDefaultSetup can easily get passed. Third, when you test the second method TestAgainstCustomSetup the method
MyInitialize will again run first. But due to the attribute instance
[TestProperty("thing", "non-default")] decorating the method TestAgainstCustomSetup, the
TestContext.Properties is stuffed into a name called
thing. And also, the corresponding value is "non-default". So after the
MyInitialize executes, the variable
_thing contains a value "non-default". So, as you image, the second method can also easily get passed.
Now you can find where TestProperty shines is that the properties for the current test can become available during the
MyTestInitialize method. That means we can bung a whole bunch of common setup code into the
MyTestInitialize method, and parameterize its behavior based on properties defined on the tests themselves.
At last, please note the
TestProperty attribute can be specified on a test method. There can be multiple instances of this attribute to specify more than one item. Refer to code in Listing 5.
4. The [Ignore] attribute
Sometimes, you may want to temporarily ignore tests upon some methods. In this case, you can fall back upon an attribute named
[Ignore]. This attribute is useful when one of our tests has a problem and we just don't want to deal with the problem at the moment. In another word,
[Ignore] enables us to temporarily disable a test.
Note we can use this attribute on either a test method or an entire test class.
5. The [ExpectedException] attribute
Last but not the least attribute is
[ExpectedException], with which we can assert that a test method should throw a particular type of exception. Please consider the following example.
Herein, we expect to see a DivideByZeroException type of exception. In this case, the
ExpectedException attribute comes to help.
Unit Test with the Generate From Usage Feature
Visual Studio 2010 can generate the new types and members for us with minimal interruption to our workflow. This owes to a new feature named "Generate From Usage" introduced in Visual Studio 2010. In details, it can help us create stubs for types, methods, properties, fields, or constructors without leaving our current location in code.
Apparently, the Generate From Usage feature can be used with test frameworks that integrate with Visual Studio. In this section, we are going to demonstrate the Generate From Usage feature in combination with Microsoft Unit Testing Framework.
Follow the steps below to generate a sample solution sketch.
1. Start up Visual Studio 2010.
2. In the "New Project" dialog box, select the "ASP.NET MVC 3 Web Application" template to create a solution named Mvc3UnitTestDemo.
3. Click OK. And then, the "Create ASP.NET MVC 3 Project" dialog box is displayed, asking us to select the template, view engine, and whether to create an associated unit test project.
4. Make related selections. And also, make sure the checkbox "create a unit test project" is selected, as shown in Figure 1 below.
Figure 1: Select the MVC 3 template together with a unit test project
Note also, in the figure we select "Visual Studio Unit Test", the built-in test framework. As a result, the final solution Mvc3UnitTestDemo contains two projects, one named Mvc3UnitTestDemo and the other named Mvc3UnitTestDemo.Tests.
Generate a new class from a unit test
Suppose that we want to build an online MIS system based upon the Northwind database. Consider a case that we want to create a Details page displaying details for a specified Employee. We pass a query string that contains an employee Id to the Details page, and the Employee details are retrieved from the database and displayed.
Following good Test-Driven Development practice, we first write a test before doing anything else. We don't write any application code until we have a test for the code. The tests for the Details page act as a criterion for success.
To create a successful Details page, we design the following test targets:
- If an EmployeeID is not passed to the page, an exception should be thrown.
- The EmployeeID should be used to retrieve an employee from the database.
- If a matching employee cannot be retrieved from the database, an exception should be thrown.
- The Details View should be rendered.
- The employee should be assigned to the Details View's ViewData.
Now, let's start with the first test. According to the first test, if an EmployeeID is not passed to the Details page, an exception should be thrown. We need to add a new unit test to our test project.
Now, right-click the Controllers folder in the Test Project Mvc3UnitTestDemo.Tests and select "Add | New Test". Then, select the "Unit Test" template (refer to Figure 2), and name the new unit test EmployeeControllerTest.
Figure 2: Create a new unit test
Do remember to select the appropriate approach to add unit tests. If you right-click the Controllers folder in the test project and select "Add | Unit Test", then you enter into the Unit Test Wizard starting with a dialog shown in Figure 3. This wizard will generate a unit test that executes within the context of a web server. This type of unit test is not appropriate for Test-Driven Development because it just takes too long to execute.
Figure 3: Please avoid opening such a dialog to unit test our project
Next, we should replace the automatically generated method
TestMethod1 with the following:
Our initial aim is to test whether an exception is thrown when the Details page is requested without an
EmployeeID. Note the second attribute
[ExpectedException] sets up an expectation for the test. If executing the test method does not throw an ArgumentNullException, then the test fails. We want the test to throw an exception because we want an exception to be thrown when the Details page is requested without an
Obviously, up till now, we have not created the EmployeeController class. So this test will fail. Note that is just what we want. According to Test-Driven Development philosophy, we first write a test that fails, and then write code to fix it.
Now, let's click the menu "Test | Run | Tests in the Current Context" or press Ctrl+R+T to run the current test. As you can image, we will meet first a compile error before running the unit test, as shown in Figure 4 below.
Figure 4: A compile error before running the unit test
Then, click "Yes" to continue the process. Note Visual Studio will not continue any more, but stop with an "Error List" window showing the compilation error to be corrected.
Well, now we have to define the class EmployeeController first so as to run the test. Now, the "Generate from Usage" feature comes to help. Please refer to the following steps:
1. Right click the class EmployeeController.
2. From the popup menu select "Generate | New Type..." (Not "Generate |Class", which will generate a file under the current folder. This is not what we want).
3. In the popped "Generate New Type" dialog box, select and populate contents, as shown in Figure 5.
Figure 5: Use the “Generate from Usage” feature to generate the controller class
4. Click OK.
As supposed, a class named EmployeeController is automatically generated into a file EmployeeController.cs under the path Mvc3UnitTestDemo\Controllers. This is what we just want. The only drawback is the following two namespace references (they are nearly the MUST HAVE in each controller definition) require you to manually provide:
But a good thing is the following namespace reference is added to the unit test file automatically:
Generate a stub for a method
Now, rest the mouse pointer over the smart tag corresponding to the undefined method
Details to see an error message indicating that no method named Details is defined yet. Click the smart tag or press CTRL+. (CTRL+period) to open the "Generate From Usage" shortcut menu, as shown in the following Figure 6.
Figure 6: Use the “Generate from Usage” feature to generate the method stub
Then, click the prompt string. You will see a related method named
Details automatically added to the above class EmployeeController. Listing 8 gives the related code.
A related trick is you should not put any parameter in the above method
Details in the test class before using the "Generate from Usage" support; otherwise, you will not launch the good "Generate from Usage" functionality.
Now, return to the test class and run test again. You will fail the test, as shown in Figure 7.
Figure 7: The method test fails
The result is easy to understand - we want to the test method throws an ArgumentNullException. Since we've pass none argument to the method, the test fails. For this, let's modify the method
Details in the controller.
And, correspondingly, we modify the invocation in the test class, as follows:
Now, run the test again. As expected, the test passed, as shown in Figure 8 below.
Figure 8: The method test passes
Similarly, you can generate a property stub using the "Generate From Usage" feature. Due to space limitation, you can figure it out yourselves.
Till now, we can easily create corresponding tests for public class, methods, and properties. Then, how about testing private ones?
Testing Private Methods, Properties, and Fields
According to good Test-Driven Development practices, we should test all our code, including the private methods in our application. In the MSTest case, you can test both public and private methods using unit tests.
Although we can manually code unit tests to test any method, this task is more difficult for private than for public methods because it requires a deeper understanding of the intricacies of reflection. Then, how does Visual Studio come to help?
In Visual studio 2010, when we generate a unit test for a private method, a private accessor is automatically created. A private accessor is a method that the test method uses to access the private code. The unit test generates a call to the private accessor, and then calls the private method through the private accessor. The private accessor resides in a file that is part of our test project; therefore it is compiled into our test project assembly.
For example, imagine that you want to test a class named MiniCalculator that contains a private method named Add(). You can right click this class and from the popup menu click "Create Private Accessor | Mvc3UnitTestDemo.Tests", as shown in Figure 9.
Figure 9: Creating a Private Accessor
A tiny trick associated with the "Create Private Accessor" feature is you should not put the class that contains private/static members in the test project (in our case Mvc3UnitTestDemo.Tests); or else, you will not find the "Create Private Accessor" option from the shortcut menu.
After that, a Test References folder (see Figure 10) is added to your test project and an accessor is added to that folder. The accessor is also referred to in the logic of the unit test method.
Figure 10: A Test References folder is added to your test project
After you create the accessor, you can use it in the unit test code to test the Add method. But wait a moment; let's first look at the accessor class:
Listing 1: MiniCalculatorTest.cs (Accessor)
Now, we can right click the project Mvc3UnitTestDemo.Tests and click "Add | New Test...|Unit Test" to generate a test file MiniCalculatorTest.cs. The unit test in Listing 12 tests whether the Add method returns the right result for 12.12+55.43.
Listing 2: MiniCalculatorTest.cs (Accessor)
Notice that the
Add() method is called on instance of the MiniCalculator_Accessor class rather than the MiniCalculator class itself. Because the
Add() method is private, you can't call it directly on the MiniCalculator class. However, the generated MiniCalculator_Accessor class exposes the method just fine.
In essence, if you prefer, you can generate the accessor class from the command line. Visual Studio includes a command line tool named Publicize.exe that generates a public façade for a class with private members. Please refer to this article at MSDN.
The second method for testing private class methods is to use reflection. By taking advantage of reflection, you can bypass access restrictions and invoke any class method and access any class property. The test in Listing 13 uses reflection to call the private
The code in Listing 13 calls the private static
Add() method by calling the
Invoke() method on a MethodInfo object that represents the
The last note is, for static members you can also use the above means, but the related example should rest on yourself.
By the way, there are other non-recommendable ways in testing private/static methods. For example, you can use PrivateObject and PrivateType (both located in the Microsoft.VisualStudio.TestTools.UnitTesting namespace) to achieve the aim. For more details, please refer to this article at CodeProject.
Introducing Code Coverage
To see what proportion of your project's code is actually being tested, you are highly suggested using the code coverage feature of Visual Studio 2010. To do this, you should first edit the run configuration to indicate the assembly containing the code whose coverage you want to measure; then, run tests on that code. After all that, detailed code coverage statistics appear in a window.
The following illustrates the rough steps to make preparation for code coverage analysis- generate a code coverage related configure file, and make simple configuration.
1. Right click the solution root Mvc3UnitTestDemo, and select "Add | New Item...".
You can right click neither the Mvc3UnitTestDemo project nor Mvc3UnitTestDemo.Tests; or else, you cannot add the test settings file.
2. In the "Add New Item" dialog, from the Installed Templates click Test Settings, you will see a screenshot shown in Figure 11.
Figure 11: Generate the test settings file
3. Click the Add button. After that, a dialog Test Settings appears, as shown in Figure 12 below.
Figure 12: The Test Settings configuration dialog
4. Click the Apply button. Then a file named TestSettings1.testsettings is generated under an independent folder Solution Items. After that, you can double click this file to re-modify your configuration.
5. Click Roles to switch to the Roles page. Since we want to run our tests locally, we select Local execution from the Test Execution method list box.
6. To start up code coverage test, you should also click Data and Diagnostics panel to tick the Code Coverage checkbox. After that, select the check box for Code Coverage and then click Configure located immediately above the list of diagnostic data adapters. From the Code Coverage Detail dialog box to select the artifacts that you want to instrument. Figure 13 illustrates our interested selection.
Figure 13: Select the target artifacts to instrument
In the above sample, since all our test related files are located in the project Mvc3UnitTestDemo.Tests, we made the corresponding selection.
7. Now click the menu item "Test | Run | Tests in current Context" to test the method
DivideTest. After that, click the menu item "Test | Windows | Code Coverage Results" and you will see a similar screenshot as shown in Figure 14.
Figure 14: The Code Coverage Results window
Note that you can also click the "Show Code Coverage Coloring" icon to see what parts get executed in the current test context.
As you've seen, the Code Coverage tool is a great help in test driven development. In fact, we've only scratch the surface of Code Coverage related topics. For more details, you can refer to MSDN and all related links.
In this article, we have covered the basic aspects for test-driven development supported in Visual Studio 2010. As mentioned in the context, Visual Studio 2010 provides abundant and even complex supports towards various kinds of test related topics.
An important point that every developer should bear a clear understanding with is Test-Driven Development is neither applicable to all application development nor a cure-all universal immortality. Tons of tests might potentially add both complexity and bugs to the target system. Even under the cases that are applicable to Test-Driven Development, we should try our best to list the testing targets as clearly as possible, rather than carry out tests without purpose.
Anyway, our main interests lie in the ASP.NET MVC 3 related stuff. So, this article just serves as an elementary start to Visual Studio 2010 Unit Test novices. Starting from the next article, we will shift our attention to ASP.NET MVC 3 related unit test topics.
Part 1 The focus in part 1 is on the fundamental concepts and tooling support in Visual Studio 2010.
Part 2 In this article, we will delve into unit testing controllers in ASP.NET MVC, especially in terms of ASP.NET MVC 3 related concerns.
I'm a college teacher and also a freelance developer and writer from WeiFang China, with more than fourteen years of experience in design, and development of various kinds of products and applications on Windows platform. My expertise is in Visual C++/Basic/C#, SQL Server 2000/2005/2008, PHP+MyS...
This author has published 81 articles on DotNetSlackers. View other articles or the complete profile here.
Please login to rate or to leave a comment.