Mark Miller

December 2010 - Posts

  • Visual Studio’s Greatest Test Runner? Take a Look at CodeRush 10.2

    CodeRush 10.2 was released last week and the Test Runner that ships inside appears to be the greatest Test Runner available for .NET development.

    So before I show you why the CodeRush Test Runner is the best, let’s first consider the criteria by which we might judge greatness. I suggest greatness can be measured in one or more of the following:

  • Native support for your test frameworks
  • Ability to run unit tests quickly
  • A user interface prioritizing clarity over noise
  • Ability to run and debug a single test
  • If you can think of important criteria that isn’t listed above please let me know in a comment below. However, if your criteria for greatness is listed above, we have really good news for you: The CodeRush Test Runner is going to impress you.

     

    Native Test Framework Support

    The CodeRush Test Runner natively supports NUnit, MSTest, xUnit, MbUnit and Silverlight. Native support means CodeRush fully supports each framework, so new extensions and new features added to the framework can be exploited immediately without needing an update from DevExpress. No other test runner out there has built-in native support for all these testing frameworks.

     

    Performance

    The CodeRush Test Runner is the fastest test runner available for .NET. With our new multi-threaded run option, you can run tests in multiple assemblies in separate threads, resulting in a test run that is up to three times as fast as the competition.

    TestRunCompareTime3

    Like waiting? You don’t have to. CodeRush finishes your multi-assembly test run up to three times faster.

     

    Run & Debug a Single Test

    Seems like a simple request, but being able to run and debug a single test is important, especially to Silverlight developers. CodeRush is currently the only unit test framework that lets you run or debug a single test in all frameworks supported, including Silverlight. Not even Visual Studio’s built-in test runner can do this.

     

    Clarity in the User Interface

    The CodeRush Test Runner user interface delivers more clarity and less noise than any of the competing test runners, helping you see essential information quickly.

    Here’s the output of a failed test in CodeRush:

    CodeRushTestRunnerResultsLineView

    Differences between expected and actual values are highlighted automatically.

    Redundant details (e.g., “String lengths are both 28. Strings differ at index 4.”) are removed to reduce noise and enhance clarity. CodeRush is the only test runner that visually highlights differences between expected and actual results like this.

    If you right-click the debug output…

    TestRunnerMenu

    you can change the call stack display from line view to tree view, which groups methods by parenting class, like this:

    CodeRushTestRunnerResultsTreeView

    You may find this call stack view more interesting as methods are grouped by owning class.

    You can change the syntax highlight colors in the DevExpress Options dialog on the Unit Testing\Test Runner Window options page:

    OptionsUnitTesting

    Note: If you don’t see this options page, change the Level combo box from "New User” to “Advanced”.

    AdvancedLevel

     

    Compare Results with the Competition

    Compare the readability of the CodeRush Test Runner results with similar results from competing test runners.

    Competitor #1:

    TestDrivenDotNetResults2

    This competing test runner (shown above) does not ship a test result tool window for Visual Studio, so test results can only be viewed through the Output window (the CodeRush Test Runner can send output here as well if needed). Note that only the first change is shown (the letter “f” in “fox”). Subsequent differences (e.g., “dog” vs. “log”) are much harder to see and must be determined by visually scanning and comparing the results.

     

    Competitor #2:

    resharperResult2 

    This competitor makes the most important information harder to read by including redundant details (e.g., “String lengths are both 28”) in the results. Paradoxically, the essential information is rendered in black text on a gray background. Contrast for important information needs to be higher so it’s easy to see. And just like competitor #1, only the first difference is shown.

     

    Competitor #3:
    JustCodeResults2

    This competitor shows the results in a variable-width font, so the difference indicator actually points to the wrong character (e.g., the “h” in “The” – but there’s no difference there). Also, this competitor wraps the results in the window, which can make reading the call stack more challenging.

     

    Here is the result from CodeRush Test Runner again:

    CodeRushTestRunnerResultsLineView

    Which presentation do you prefer to read?

     

    In-source Test Result Icons

    CodeRush also makes it easier to work with tests in the source code, with test result icons appearing next to each test case. In the screenshot below, the first test has not run yet (TestTube), however the next two tests have run, one passing (TestPassed) and the other failing (TestFailed):

    TestMethodIcons

     

    If your hand is on the mouse, you can click the icon to bring up a test case menu, which allows you to run or debug that test.

    TestCaseMenu

     

    Test fixtures & namespaces containing unit tests have icons summarizing the run results of their contents. For example, the namespace below and the test fixture both have a red “X” (TestFailed), indicating they contain at least one test which has failed:

    TestCaseParents

     

    If you hover the mouse over one of these test fixture icons, you can see a summary of results for unit tests in this class.

    TestRunSummary

     

    If you click one of these test fixture or test namespace icons, a popup menu appears that allows you to run or debug all tests contained within.

    RunAllContainedTests

     

    If your hands are on the keyboard, don’t worry – CodeRush gives you powerful commands to run test cases without reaching for the mouse.

     

    Full Keyboard Control

    CodeRush offers full keyboard control over your test runs from inside Visual Studio. The keyboard shortcuts for running tests that ship with 10.2:

    Shortcut   Behavior
    Ctrl+T, S   Run all tests in the solution
    Ctrl+T, P   Run all tests in the active project
    Ctrl+T, F   Run all tests in the active file
    Ctrl+T, R   Run active test
    Ctrl+T, D   Debug active test
    Ctrl+T, T   Show the CodeRush Test Runner

    If you prefer the Visual Studio shortcuts, CodeRush ships an alternate collection of bindings reflecting those commands. You can change the bindings for any of these commands if you like.

     

    What’s Coming in 2011

    OK, so we just shipped 10.2, so it seems a bit crazy to tell you what’s coming in 11.1. But, you know, things get a little crazy over here at DevExpress from time to time. Our intention is to continue to maintain test runner dominance among all .NET alternatives, including Visual Studio. With that in mind, we will add support for even more testing frameworks, add more commands for controlling which test cases to run, and we’re working on improving visualization of code coverage, as well as a bigger feature we are going to save for later discussions.

  • Minority Report Meets Visual Studio

    One of the cool things I get to do here at DevExpress is explore new technologies to interface humans with machines, and more specifically, developers with their code. In the past you’ve seen us write code with chopsticks, super models, guitars, and our minds. With the release of Microsoft’s Kinect, we knew it wouldn’t be long before we’d be working on an interface for Visual Studio to let you write code with your hands, Minority Report style.

    SelectPreview

    Navigating, coding, declaring, refactoring, testing – you’ll do it all with your hands. As with everything else we build here at DevExpress, our intention is to craft a great user interface around this technology. An experience that's as efficient at writing code as using the keyboard and mouse. And we expect to make the source code available so you’ll be able to play with it yourself.

    Don't be surprised if you see this at a major developer conference in the very near future. Stay tuned.

LIVE CHAT

Chat is one of the many ways you can contact members of the DevExpress Team.
We are available Monday-Friday between 7:30am and 4:30pm Pacific Time.

If you need additional product information, write to us at info@devexpress.com or call us at +1 (818) 844-3383

FOLLOW US

DevExpress engineers feature-complete Presentation Controls, IDE Productivity Tools, Business Application Frameworks, and Reporting Systems for Visual Studio, along with high-performance HTML JS Mobile Frameworks for developers targeting iOS, Android and Windows Phone. Whether using WPF, ASP.NET, WinForms, HTML5 or Windows 10, DevExpress tools help you build and deliver your best in the shortest time possible.

Copyright © 1998-2017 Developer Express Inc.
All trademarks or registered trademarks are property of their respective owners