cmatskas

Using NCrunch to enhance your tests in Visual Studio

Christos Matskas -

I have recently come across nCrunch during a code demo. nCrunch is an automated, concurrent testing tool for Visual Studio and I have to say that I am pretty impressed with this ‘little’ piece of software.

nCrunch is an 3rd party tool that allows you to continuously run tests as you write them. Actually, the tests are run every time you change any part of your code that is being monitored by nCrunch and may have an impact on your tests. The tool is responsible for building and running tests as it sees fit.

Installation was easy and painless. As soon as you install nCrunch you are presented with a wizard that takes you through the process of setting up nCrunch to run your tests. It is at this stage that you decide how to run your tests. Some of the settings include:

  • nCrunch Enabled/Disabled
  • Engine mode (concurrent, sequencial etc)
  • Which tests to run
  • Number of Threads to use

Once you complete all the wizard tests and as long as you enabled nCrunch to run your tests, nCrunch kicks off and runs in the background. The image below shows how to quickly check the test progress:

For example, the image above informs me that 2 of my tests are failing. To see the failing tests, you can either click on the little icon on the task bar or you can use the nCrunch Tests Window (Ctrl + Shift + M). The window should look similar to the one below:

You can click on each of the test to see the failure as nCrunch handily displays the exception below. It will also display reasons for failure if your test fails on a specific assertion. Double-click on any of the tests and can instantly go to the failing code, like in the picture below:

The beauty of nCrunch is that it provides immediate and continuous feedback. Here we can see the line where the code within the is failing - marked with a red x. Obviously I will need to do a bit more digging to fix the problem, but I know straight away where to look. Also, another thing worth noting in the example above is the line marked with the amber/orange circle. This is nCrunch telling me that the code executed in this line is taking longer and I may want to look at it! Isn't this great?

But this is not the only feature that nCrunch has to offer! In addition to continuous and seamless testing we also get:

  • Automatic Concurrent Testing
  • Code Coverage
  • Performance Metrics
  • Inline Exception Details
  • Intelligent Test Execution
  • Small Memory Footprint
  • Parallel Execution
  • Optimised Build Process
  • Smart Multiprocessor Support
  • Easy Debugging
  • Part Continuous/Manual Mode

I like the fact that straight out of the box you get all this. It is true that the memory footprint is small but at times I’ve found my machine working a bit harder when nCrunch goes through a full test cycle when the code changes impact a large number of tests. The tool is intelligent enough to work out which tests to target depending on your changes.

The parallel execution is a great feature as well, but there are a few caveats that you need to be aware of. If your tests rely on database operations and/or file access, then you need to work around the concurrency issues. There are two ways to approach this problem:

  • You disable parallel execution by assigning only one thread to nCrunch. This can be done easily through the nCrunch Menu and will result in tests being run sequentially.
  • You set up your tests to use unique databases and files (using Guids instead of hard-coded file names or database names). For example, my integration tests test my repositories to ensure that all CRUD operations work and that Entity Framework performs as expected. My tests have a setup and tear-down method that creates a database and then deletes it for each test fixture (I’m using nUnit as my testing framework). My original code wasn't optimized for nCrunch and as such most of the integration tests were failing as they were unable to obtain exclusive access to the database. So instead of relying on a connection string being passed from the app.config, I configured the tests to create a connection string on the fly using a Guid for the database name. Something like this:
public static string ConnectionName = @"Server=(localdb)\v11.0;Integrated Security=true;Initial Catalog=" + Guid.NewGuid() + ";";
public static QfContext QfContext = new QfContext(ConnectionName);
public static UnitOfWork UnitOfWork = new UnitOfWork(QfContext);

[TestFixtureSetUp]
public void TestSetup()
{
	if (QfContext.Database.Exists())
	{
		QfContext.Database.Delete();
	}
	QfContext.Database.Create();
}

[TestFixtureTearDown]
public void TestTearDown()
{
	QfContext.Database.Delete();
}

Note that my dbContext is designed to take either a connection string or a connection name and work out how to establish a connection to the database.

There are similar workarounds for file access and you will find that nCrunch has a nice website and great documentation and support. Since I installed nCrunch, I have barely run any unit test manually and it has significantly sped up my development. Instead of waiting around to run the tests, I can now dive into the code and focus on the things that matter. As soon as nCrunch complains about a failure, I make sure that the offending code is fixed and carry on. Finally, I have started focusing a lot more on Code Coverage without waiting on my CI or other tools to indicate that some piece of code in my class is not reachable and not covered by tests.

I would recommend nCrunch to anyone who’s serious about unit testing and wants to improve his/her development speed. I hope that you will find this tool as useful as I did.

Happy coding...