Everything You Need to Get Started with SpecFlow and WatiN

Posted by: Steven Smith, on 24 Jun 2011 | View original | Bookmarked: 0 time(s)

Im adding SpecFlow to an application Im working on so that I can add some acceptance tests that actually exercise the user interface.  Ive only spent a couple of hours on it thus far, but I have it working with a single specification running through the tests via WatiN.  I found the following resources helpful as I was going through this exercise:

Im assuming that youre just interested in getting up to speed with SpecFlow for acceptance testing and that you dont want to waste any time on hidden gotchas or visiting all of the above URLs just to figure out what you actually need.  Let me just give you the Stuff You Need To Know.

Installing SpecFlow

First, you need to install SpecFlow from this URL.  When I wrote this, the latest version was 1.6.1.  Alternately, you can get SpecFlow via Nuget here.  If you run the installer, specflow will be installed in your Program Files (x86) folder under TechTalk, like so:


Next, youll want to add a new Class Library project to whatever solution youre working with (Im going to assume you have an ASP.NET web project in the solution that youre trying to test).  You can name it Specifications or AcceptanceTests or whatever you prefer.  There isnt a special project template set up, but there are item templates, so after you have the class library, add a New Item and choose a SpecFlow Feature File:


This will give you a sample .feature file to start from.  Im not going to go into detail about the gherkin language, but rather will simply show you my first .feature file which is a bit more real-world than the one provided (this one will end up working against a scaffolded ASP.NET MVC 3 app):


Note that there is a generated C# (.cs) file that goes along with each .feature file.  This generated code is responsible for all of the test setup.  In my case Im using the default NUnit test runner, but you can also use MSTest.  To install NUnit, I simply used the Package Manager to add it, like so:

install-package nunit

Once you have NUnit installed, you can run the tests.  Im using ReSharper so this screenshot shows its integrated test runner:


Note that since I havent defined any Step definitions, my tests dont actually do anything yet.  However, part of the workflow of SpecFlow that you can follow is to run the test without the steps, and then copy and paste the step methods from the test results shown here.  You can just add a new class and replace its contents with one of the classes above, or you can Add > New Item and add a new SpecFlow Step Definition.  Im still figuring out how best to organize my Specifications project but for now Im naming things by feature name, so I have a CreatePublicationFeature.feature file and a CreatePublicationSteps.cs file, so the two will sort together.

Now that we have some yellow tests, we can work toward making the test pass.  We also can produce an HTML report, showing all of our features and our progress on them thus far.  Ill show the reporting last, but if youre defining a lot of features up front, having the reporting early on can be a great thing to share with project stakeholders, since theyll be able to see with each iteration how many features are going from yellow to green (and hopefully none turning red) as they are implemented and tested.

Testing ASP.NET MVC with WATIN

In my scenario I decided to give WatiN a try.  To install it, I once again turned to Nuget.  The install was easy, however, there were a few gotchas that I encountered.  The first one was the dreaded STAThread issue, which you can resolve by adding an attribute to your unit test class marking it as a Single Threaded Apartment.  However, since SpecFlow generates our test classes, this isnt ideal.  So the alternate approach is to add an app.config file with the following configuration to force NUnit to run in STA mode:

<?xml version="1.0" encoding="utf-8" ?>
    <sectionGroup name="NUnit">
      <section name="TestRunner" type="System.Configuration.NameValueSectionHandler"/>
      <!-- Valid values are STA,MTA. Others ignored. -->
      <add key="ApartmentState" value="STA" />


The other issue I encountered was related to the Interop.SHDocVw assembly, which is referenced and is in my /packages folder, but still my tests would fail saying they could not load the assembly.  The fix that worked for me (VS2010, Windows 7 x64, if that matters) was to change the Embed Interop Types property of the reference to False (it defaulted to True).  This eliminated that issue, and allowed my tests to launch an Internet Explorer window.  Heres the setting:


I mention the above up front because they were immediate roadblocks to my being able to use WatiN in my project, and I dont want you to get hung up on them when you get started with the next bit of code.  I did add one helper class that I found on one of the links I listed at the top of this post, which lets me reference the same browser instance between test methods.  Its a simple static class with a static method that leverages the SpecFlow ScenarioContext collection.  If youre following along with my code exactly, youll need this:

public static class WebBrowser
    public static IE Current
            string key = "browser";
            if (!ScenarioContext.Current.ContainsKey(key))
                ScenarioContext.Current[key] = new IE();
            return ScenarioContext.Current[key] as IE;

Now back to the Step .cs class.  We cut-and-pasted the empty step definitions from our test output so that we now have some [Given] [When] and [Then] methods in our class, CreatePublicationSteps.  For my scenario, I dont actually have any setup to do in my [Given], so I could omit it entirely.  But if I do keep it, its sufficient for it to look like this:

[Given(@"I am on the site")]
public void GivenIAmOnTheSite()

I opted to leave it in here for now.  Its likely that Ill update it at some point to include a requirement like Given I am on the site and logged in which might require some actual setup work.  Next I have [When] statements for the scenario.  Note that any and between your When and the Then is simply treated as another When statement.  So for my scenario, where I have

When I navigate to the Publication/Create page
And I enter "Test Publication" in the Publisher textbox

thats going to result in two separate [When] methods, which I was able to cut-and-paste scaffolded versions from my test output.  Once filled in with appropriate WatiN calls, they look like this:

[When(@"I navigate to the Publication/Create page")]
public void WhenINavigateToThePublicationCreatePage()
[When(@"I enter ""(.*)"" in the Publisher textbox")]
public void WhenIEnterTestPublicationInThePublisherTextbox(string publicationName)
    var pubName = publicationName + new Random().Next(1000);
    ScenarioContext.Current.Add("publicationName", pubName);

Note that you probably dont have a .TypeTextQuickly() method you can replace that with TypeText for now, which I recommend so you can see how things work by default.  Note that for this to work, your web application has to be running (if youre using the Dev Web Server).  If youre using IIS or actually hitting a site on the Internet, then this shouldnt be an issue.  Note in the second function that Ive changed the attribute value, replacing the original (C# escaped quotes) Test Publication with this (.*).  This is a regular expression, and .* basically will match any series of characters.  The result of this match is then provided as the string parameter on the method, string publicationName.  It will be populated from the scenario, so when the test is run, publicationName will be Test Publication.  Since at the moment Im not resetting the database between each test, I made the test a bit more robust by adding a random number to the name, and then storing the resulting modified name in the ScenarioContext.  I'll use that in the assertions within the [Then] methods.

By the way, Im using a simple ASP.NET MVC 3 application with some default controllers and view set up using Entity Framework.  You can see how to do this in ScottGus post on EF Code First and Data Scaffolding with the ASP.NET MVC 3 Tools Update.  If you want to follow along, just create a class called Publication with a Name property (and an ID column), build your project, and then Add Controller and (assuming you have the tools update referenced in Scotts post) you should then choose the Controller with read/write actions and views, using Entity Framework template, which will generated a Create controller action and view that you can use to run the exact tests Im showing.  Heres what my view looks like:


Before you can fully run the test, we need to implement the [Then] methods.  These are both pretty simple:

[Then(@"I am taken the the Publication/Index page")]
public void ThenIAmTakenTheThePublicationIndexPage()
    var currentUrlPath = WebBrowser.Current.Uri.PathAndQuery;
    Assert.That(currentUrlPath, Is.EqualTo("/Publication"));
[Then(@"I see the publication I just created")]
public void ThenISeeThePublicationIJustCreated()
    var pubName = ScenarioContext.Current["publicationName"] as string;

With these in place, you should be able to run your test, and with any luck it will come back green.  Two things to note at this point, however:

  1. The test is pretty slow.
  2. The IE window created sticks around, and a new one is made every time you run the test.

To fix the speed problem, you can add this extension method (create a class WatinExtensions and add it to your project):

using WatiN.Core;
namespace Specifications.Extensions
    public static class WatiNExtensions
        /// <summary>
        /// Sets text quickly, but does not raise key events or focus/blur events
        /// Source: http://blog.dbtracer.org/2010/08/05/speed-up-typing-text-with-watin/
        /// </summary>
        /// <param name="textField"></param>
        /// <param name="text"></param>
        public static void TypeTextQuickly(this TextField textField, string text)
            textField.SetAttributeValue("value", text);

This should speed up your test dramatically on my machine it makes it about twice as fast.  Its still slow compared to unit tests, but twice as fast is certainly an improvement, and in this case I dont need any keypress events or in fact any client-side events at all.

To fix the window problem, you can uncomment the call to WebBrowser.Current.Close() in the last [Then] method.  As you add additional features, youll no-doubt want to come up with a better approach to recycle the browser windows.  From what Ive been able to learn, it seems like the typical usage of WatiN is to use test class level setup and teardown methods to create and close the window.  With SpecFlow, if you want to go this route, you can either manually edit the .feature.cs file (which I dont recommend), or since its a partial class, you can create your own partial class and add the following to it:

public partial class CreateNewPublicationFeature
    public virtual void MyScenarioTeardown()

This will destroy the current WebBrowser at the end of each scenario.  With that in place, you should be able to run your tests, run them reasonably quickly, and not have any browser instances to clean up when you are finished.

Automation and Reporting

It would be nice if we could automate the running of these tests, so that we could schedule them or run them as part of a build process.  Likewise, one of the nice features of creating executable specifications is that they should produce documents that you can share with project stakeholders.  SpecFlow supports both of these scenarios, but getting it set up can take a little bit of work.  In order to use SpecFlow to create reports as part of your automated build process, youll need some of the SpecFlow DLLs and the specflow.exe file in source control, so they can work on any machine without requiring SpecFlow to be installed on that machine.  If youre using Nuget, I think everything you need should be in the /packages folder, but I havent tested that scenario.  In my case, I simply copied the DLLs I needed into a /lib folder in my projects source control repository.  Here are the files I ended up needing:


To create an HTML report, you need to first run your unit tests using NUnits command line tool, with a few switches enabled.  I like to use a combination of MSBuild and a ClickToBuild.bat batch file in the root of my source repository so that whenever someone grabs the source they can immediately run that batch file and get a working build of the software (with all tests run).  Here is my current build.proj file, that is in the root of my solution:

<?xml version="1.0" encoding="utf-8" ?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Target Name="DebugBuild">
    <Message Text="Building $(ProjectName)" />
    <MSBuild Projects="src\$(ProjectName).sln" Targets="Clean" Properties="Configuration=Debug"/>
    <MSBuild Projects="src\$(ProjectName).sln" Targets="Build" Properties="Configuration=Debug"/>
  <Target Name="BuildAndTest" DependsOnTargets="DebugBuild">
    <Message Text="Running $(ProjectName) Unit Tests" />
    <Exec Command="$(NUnitConsoleEXE) src\UnitTests\bin\debug\UnitTests.dll /nologo /framework=4.0.30319 /xml:UnitTestResults.xml"/>
      <Message Text="Running $(ProjectName) Acceptance Tests" />
    <Exec Command="$(NUnitConsoleEXE) src\Specifications\bin\debug\Specifications.dll /nologo /framework=4.0.30319 /labels /xml:TestResult.xml"/>
    <Message Text="Generating SpecFlow Report..." />
    <Exec Command="lib\SpecFlow\specflow.exe nunitexecutionreport src\Specifications\Specifications.csproj /out:AcceptanceTestResults.html" />
  <Target Name="ReleaseBuild" DependsOnTargets="BuildAndTest">
    <Message Text="Building $(ProjectName) Release Build" />
    <MSBuild Projects="src\$(ProjectName).sln" Targets="Clean" Properties="Configuration=Release" />
    <MSBuild Projects="src\$(ProjectName).sln" Targets="Build" Properties="Configuration=Release" />
    <Message Text="$(ProjectName) Release Build Complete!" />

Then I also include a build.bat and a ClickToBuild.bat file with every solution.  They look like this (one line each):


%systemroot%\Microsoft.Net\Framework\v4.0.30319\MSBuild.exe build.proj /t:%*


build.bat ReleaseBuild & pause


The main work is all performed in the BuildAndTest target of the MSBuild .proj file.  If youre just testing this out from a command prompt, you can pull the commands you need from the <Exec /> elements there.  For NUnit to generate a TestResult.xml file, you need to pass in the /xml:TestResult.xml switch.  Note that at this time ReSharper cannot generate an NUnit TestResult.xml file, so you have to resort to using the command line (or the NUnit GUI).  Once you have that XML file, you can generate the SpecFlow HTML report by running specflow.exe with the command nunitexecutionreport and providing it with /out:SomeFile.html.  Assuming thats all correct, you should end up with an HTML file that looks something like this:



Theres not a whole lot to getting SpecFlow and WatiN working with your ASP.NET (MVC) application.  There are a few hidden gotchas that Ive tried to cover in this post.  Hopefully this will provide all of the resources you need.  If theres something missing, please let me know and I will provide an update to address the issue.  The nice thing about the final HTML report you get is that you can sit down with the customer or project stakeholder and create all of the major features and many of the known scenarios prior to an iteration or release cycle, and then provide regular updates showing progress being made on a feature-by-feature (and scenario-by-scenario) level.  Assuming the documented executable specifications accurately reflect the customers needs, these acceptance tests provide a common definition of what done is for the project, reducing the frequency of the team delivering incomplete or incorrect features.

Category: Agile | Other Posts: View all posts by this blogger | Report as irrelevant | View bloggers stats | Views: 3528 | Hits: 40

Similar Posts

  • The Downside of Transparency more
  • Your idea will probably change more
  • In Response To DNR 476 more
  • Demeter Transmogrifiers To The Rescue more
  • ASP.NET disk-based caching more
  • Thoughts on agile design and platforms more
  • Script# (Script Sharp) writing javascript in C# more
  • Silverlight 3 Multi-touch: The Basics more
  • Mocking indexer getters with Moq more
  • The usual result of Poor Mans Dependency Injection more

News Categories

.NET | Agile | Ajax | Architecture | ASP.NET | BizTalk | C# | Certification | Data | DataGrid | DataSet | Debugger | DotNetNuke | Events | GridView | IIS | Indigo | JavaScript | Mobile | Mono | Patterns and Practices | Performance | Podcast | Refactor | Regex | Security | Sharepoint | Silverlight | Smart Client Applications | Software | SQL | VB.NET | Visual Studio | W3 | WCF | WinFx | WPF | WSE | XAML | XLinq | XML | XSD