Visual Studio’s Greatest Test Runner? Take a Look at CodeRush 10.2

CodeRush 10.2 was released last week and the Test Runner that ships inside appears to be the greatest Test Runner available for .NET development.

So before I show you why the CodeRush Test Runner is the best, let’s first consider the criteria by which we might judge greatness. I suggest greatness can be measured in one or more of the following:

If you can think of important criteria that isn’t listed above please let me know in a comment below. However, if your criteria for greatness is listed above, we have really good news for you: The CodeRush Test Runner is going to impress you.

Native Test Framework Support

The CodeRush Test Runner natively supports NUnit, MSTest, xUnit, MbUnit and Silverlight. Native support means CodeRush fully supports each framework, so new extensions and new features added to the framework can be exploited immediately without needing an update from DevExpress. No other test runner out there has built-in native support for all these testing frameworks.

Performance

The CodeRush Test Runner is the fastest test runner available for .NET. With our new multi-threaded run option, you can run tests in multiple assemblies in separate threads, resulting in a test run that is up to three times as fast as the competition.

Like waiting? You don’t have to. CodeRush finishes your multi-assembly test run up to three times faster.

Run & Debug a Single Test

Seems like a simple request, but being able to run and debug a single test is important, especially to Silverlight developers. CodeRush is currently the only unit test framework that lets you run or debug a single test in all frameworks supported, including Silverlight. Not even Visual Studio’s built-in test runner can do this.

Clarity in the User Interface

The CodeRush Test Runner user interface delivers more clarity and less noise than any of the competing test runners, helping you see essential information quickly.

Here’s the output of a failed test in CodeRush:

Differences between expected and actual values are highlighted automatically.

Redundant details (e.g., “String lengths are both 28. Strings differ at index 4.”) are removed to reduce noise and enhance clarity. CodeRush is the only test runner that visually highlights differences between expected and actual results like this.

If you right-click the debug output…

you can change the call stack display from line view to tree view, which groups methods by parenting class, like this:

You may find this call stack view more interesting as methods are grouped by owning class.

You can change the syntax highlight colors in the DevExpress Options dialog on the Unit Testing\Test Runner Window options page:

Note: If you don’t see this options page, change the Level combo box from "New User” to “Advanced”.

Compare Results with the Competition

Compare the readability of the CodeRush Test Runner results with similar results from competing test runners.

Competitor #1:

This competing test runner (shown above) does not ship a test result tool window for Visual Studio, so test results can only be viewed through the Output window (the CodeRush Test Runner can send output here as well if needed). Note that only the first change is shown (the letter “f” in “fox”). Subsequent differences (e.g., “dog” vs. “log”) are much harder to see and must be determined by visually scanning and comparing the results.

Competitor #2:

This competitor makes the most important information harder to read by including redundant details (e.g., “String lengths are both 28”) in the results. Paradoxically, the essential information is rendered in black text on a gray background. Contrast for important information needs to be higher so it’s easy to see. And just like competitor #1, only the first difference is shown.

Competitor #3:

This competitor shows the results in a variable-width font, so the difference indicator actually points to the wrong character (e.g., the “h” in “The” – but there’s no difference there). Also, this competitor wraps the results in the window, which can make reading the call stack more challenging.

Here is the result from CodeRush Test Runner again:

Which presentation do you prefer to read?

In-source Test Result Icons

CodeRush also makes it easier to work with tests in the source code, with test result icons appearing next to each test case. In the screenshot below, the first test has not run yet (), however the next two tests have run, one passing () and the other failing ():

If your hand is on the mouse, you can click the icon to bring up a test case menu, which allows you to run or debug that test.

Test fixtures & namespaces containing unit tests have icons summarizing the run results of their contents. For example, the namespace below and the test fixture both have a red “X” (), indicating they contain at least one test which has failed:

If you hover the mouse over one of these test fixture icons, you can see a summary of results for unit tests in this class.

If you click one of these test fixture or test namespace icons, a popup menu appears that allows you to run or debug all tests contained within.

If your hands are on the keyboard, don’t worry – CodeRush gives you powerful commands to run test cases without reaching for the mouse.

Full Keyboard Control

CodeRush offers full keyboard control over your test runs from inside Visual Studio. The keyboard shortcuts for running tests that ship with 10.2:

If you prefer the Visual Studio shortcuts, CodeRush ships an alternate collection of bindings reflecting those commands. You can change the bindings for any of these commands if you like.

What’s Coming in 2011

OK, so we just shipped 10.2, so it seems a bit crazy to tell you what’s coming in 11.1. But, you know, things get a little crazy over here at DevExpress from time to time. Our intention is to continue to maintain test runner dominance among all .NET alternatives, including Visual Studio. With that in mind, we will add support for even more testing frameworks, add more commands for controlling which test cases to run, and we’re working on improving visualization of code coverage, as well as a bigger feature we are going to save for later discussions.

Free DevExpress Products - Get Your Copy Today