TestCafe Studio: A New Web Testing IDE

TestCafe Team Blog
30 January 2019

Two years ago, we announced a shift in our long-term plans for TestCafe. As some of you may already know, we chose to freeze the features shipping in v2015.1 and to focus our energies on two new versions of the product:

  • TestCafe: an open-source CLI tool
    The team always wanted to publish a free open-source version that doesn’t include advanced UI capabilities. You can now find this version on GitHub, where it’s been freely available to everyone for the past several months.
  • TestCafe Studio: the next step in GUI version evolution
    A commercial web testing IDE built around the same engine as the open-source version. Our work on this product is almost completed: you can download a Preview version today.

Below you’ll find all the information about the new TestCafe Studio: how to get started, the differences between the new version and v2015.1, and where to find the latest Preview version.

TestCafe Studio 101 & Getting Started

TestCafe Studio is a standalone application that allows you to create, edit, manage, execute, and debug tests with ease.

  • Runs on Windows, Mac and Linux.
  • Ships with a Visual Test Recorder, allowing you to create and edit tests without coding.
  • Includes a built-in code editor with auto-complete and syntax highlighting support (for those who prefer to script their tests in code).

If you’ve never used TestCafe before, watch this short Getting Started video.

What’s New in TestCafe Studio: Compare to v2015.1

Here’s a brief summary of the capabilities that differentiate the new TestCafe Studio from its predecessor.

TestCafe is Now a Desktop Application

TestCafe Studio is a desktop application that looks and behaves much like today’s most popular IDEs. It includes familiar UI elements such as a Toolbar, File Explorer, Main Menu, Tabs View, and Reports Panel.

TestCafe v2015.1 had a UI implemented as a website hosted on a local Node.js server.

TestCafe New Desktop Application UI

Visual Recorder: Now with Test Editing Capability

TestCafe Studio: the Visual Test Recorder now ships with a new “Continue Recording” option. As its name suggests, this feature allows you to edit previously recorded tests and add additional test actions visually.

TestCafe v2015.1: Visual Test Recorder could be used for initial test construction, but the only way to modify a test was to edit generated code.

TestCafe Visual Test Recorder

You will find the action list UI more user-friendly and easier to understand. Every action on the list can be edited according to your needs.

Tests recorded in TestCafe Studio are saved in a new "*.testcafe" file format. Recorded tests are no longer transformed into plain JavaScript code once recording is complete. This is what allows you to display your tests as a sequence of actions, edit them as needed, and modify them using the Visual Test Recorder.

Should you prefer to edit JavaScript code, we provide you with the option of manual "*.testcafe" to "*.js" conversion.

Smart Element Selectors

TestCafe Studio: You can pick a selector from a list of predefined types or build your own. The new Selector Builder UI allows you to locate an element in a variety of ways:

  • Use CSS selectors
  • Run custom JavaScript functions
  • Filter page elements by index, text, attributes or visibility
  • Find elements based on their position within the DOM tree

TestCafe v2015.1: Advanced Selector Builder UI wasn't available. Customization was limited to manual CSS selector construction.

Automatically-generated selectors:

TestCafe Page Element Selectors

Custom selector builder:

Custom Element Selector Builder

Concurrent Test Execution Improvements

TestCafe Studio: Can start several instances of the same browser, divide fixture tests into several batches, and execute each batch in its own browser instance.

TestCafe v2015.1: Only one browser instance was created per test run and all tests were executed sequentially.

Both TestCafe Studio and its predecessor execute tests in parallel if you specify multiple target browsers.

Improved Test Execution Configuration Management

TestCafe Studio: You can create multiple “configurations” that define target browsers and additional test execution settings. You can then run tests for specific configurations.

TestCafe v2015.1: You needed to set configuration settings each time you ran tests.

TestCafe Configuration Manager

Faster and More Stable Tests with the New Auto-Wait Mechanism

TestCafe Studio: Automatically suspends test execution while waiting for element availability, page load completion, XHR requests, etc. The new auto-wait mechanism also extends to assertions and thus makes tests more stable when you verify the page state. Customizable timeouts allow you to fine-tune built-in waiting mechanisms.

TestCafe v2015.1: Also waited for element availability, page loads and XHR requests automatically, but didn’t wait for assertions to pass, and the timeouts were not customizable.

Updated File Manager UI

TestCafe Studio: You can create, edit, and view various file formats including test files, test recordings, helpers, wrappers, and PageModels. You can launch tests directly from selected files or folders.

TestCafe v2015.1: You could create, edit, and view only .js test files. It was not possible to start an arbitrary selection of tests or fixtures - you could only launch an individual test or fixture.

TestCafe File Manager UI

Enhanced Reports

TestCafe Studio: The new UI includes a panel with information on past test runs and current sessions that can be cancelled if necessary. A click on an item within the Reports panel opens a detailed report on any given test. You can view test reports and export them into various formats (json, list, minimal, spec, xunit). You can install plugins that provide additional report formats.

TestCafe v2015.1: Displayed a summary of test runs on a separate page, instead of a panel integrated in the main UI. Tests that were currently in progress could only be stopped by closing the browser that ran them, but this led to an error. There was no way to add more report formats.

Interactive Test Execution Progress Panel

TestCafe Test Report Details

Feature-Complete Code Editor

TestCafe Studio: Features a fully integrated code editor with autocomplete and syntax highlighting. You can launch tests directly from the editor.

TestCafe v2015.1: Auto-completion wasn’t available, tests could not be launched from the code editor, and a minimap (outline view) was not displayed.

TestCafe Code Editor

Powerful Test Debugger

If an automated test fails, but you cannot reproduce the failure manually, there may be an issue with the script. The best way to catch those issues is by debugging the test session. For this purpose, TestCafe Studio implements all the means necessary to effectively troubleshoot test execution.

TestCafe Studio:

  • The Debug action stops a test at a desired moment.
  • The Debug Mode allows you to run the entire test step-by-step.
  • The “Pause the test when it fails” option stops a test the moment it fails.

Whenever a test pauses, you can explore the tested page with developer tools, interact with it or step over to the next action (if the test has not failed).

TestCafe v2015.1: You could stop a test with the ‘debugger’ keyword, but the tested page was frozen, you could not interact with it or step over to the next action. There was no way to stop execution automatically when a test failed.

Test API

TestCafe Studio: The new engine includes the following API changes:

  • a more intuitive interface that simplifies the task of writing a test from scratch;
  • test code can now include conditions and cycles;
  • built-in support for ES6 and TypeScript.

TestCafe v2015.1: Test code had to be organized into "steps", where each step could only contain a single test action. Test code logic had to be linear: cycles and conditions were not allowed. There was no support for TypeScript. The version of JavaScript was limited by the browser version in which you ran tests.

Continuous Integration Support

TestCafe Studio: Use the open-source TestCafe, with its extensive command line and JavaScript interfaces, as a test runner.

TestCafe v2015.1: You needed to use a separate Node.js module that provided only a minimal JavaScript API and no command line features.

Try the Preview and Let Us Know What You Think

TestCafe Studio maintains full backward compatibility - all tests you recorded in TestCafe versions up to 15.1 can be executed in TestCafe Studio without limitations. We do suggest, however, that you switch to the new API once we release the RTM version, so you can enjoy all the new features shipping inside TestCafe Studio.

TestCafe Studio Preview is free to download - sign up now to get your copy.

We look forward to hearing about your experience. Write to us at testcafeteam@devexpress.com or create a ticket in our Support Center to share your thoughts and feedback.

19 comment(s)
Peter Bossier
Peter Bossier

I need to take a decision between Selenium and TestCafe shortly, to be supported over the years to come. I see that DevExpress is on the Selenium references page, not sure why. Please confirm that TestCafe is going to stay and can this new version be integrated in Azure DevOps thru unit testing? Thanks!

18 February, 2019
Customer56436436
Customer56436436

TestCafe Studio GUI uses Electron? Or is it written in the .NET Framework?

18 February, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

@Peter,

Both TestCafe Studio and TestCafe OS versions will remain. You can export your recorded with TestCafe Studio tests into JavaScript and use the TestCafe OS version to integrate them into Azure DevOps!

19 February, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

@Customer37575,

Yes - TestCafe Studio uses Electron.

19 February, 2019
Peter Bossier
Peter Bossier

Dear Alex, thank you . One additional question.

Can Testcafé be ordered as an additional product (such as CodeRush)?

It's currently only included in the Universal Subscription, will that change?

19 February, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

You are welcome, Peter.

At the moment, TestCafe Studio is available free of charge and can be downloaded individually. We will likely ship it individually in the future as well.

20 February, 2019
Cesar Augusto
Cesar Augusto

It looks like still can't work on a XAF application, is it on roadmap?

27 February, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

Cesar,

The only issue we are aware of and which may occur in an XAF application is related to selectors with auto-generated (dynamic) ID attributes like '#Vertical_v5_27208920_MainLayoutEdit_xaf_l54_xaf_dviOffice_Edit_I'.

Each time you run your XAF application (or execute a test case on such an application), the ID attribute will be different. That is why a test will fail - it will not find an element with a dynamic ID attribute. To fix this, you can choose a different selector from the list of suggested selectors in the TestCafe Studio Test Editor. The fix to this behavior is on our TO-DO list.

If you are talking about a different issue, please submit it to our Support Center (https://www.devexpress.com/ask), so we can examine it and find a suitable solution.

28 February, 2019
Customer71783
Customer71783

Hello,

Can someone please provide details on how TestCafe studio can be used for Continous Integration with Azure Devops?

Thanks.

18 March, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

Hello,

I appreciate your interest in our testing tools. Have you seen the Integrate Tests with CI Systems topic? (docs.devexpress.com/.../integrate-tests-with-ci-systems)

After you recorded your test cases, you need to install the testcafe npm package that will run your tests in your favorite continuous integration system. You can use the .testcafe tests format or export them into JavaScript.

In the following topic, you can find more detailed information on how to setup your Azure DevOps CI environment with testcafe: Integrate TestCafe with Azure DevOps (devexpress.github.io/.../azure-devops.html)

18 March, 2019
Customer71783
Customer71783

Thank you Alex.

Also is there any plan to integrate a terminal with testcafe studio?

19 March, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

You are welcome. Would you please clarify how you would like to use such a terminal? Is it a standard command prompt terminal to run external commands? If so, which ones? Or, is it a different kind of terminal? If so, please describe what you are looking for in greater detail.

19 March, 2019
Shwetali Adnaik
Shwetali Adnaik

Hi Can anyone help me understand why I am not able to run migrated test cases from 15.1.3?

These are parameterized test cases.

29 March, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

Hi,

We're looking into this issue in the context of your support ticket.

You will be automatically notified once we make progress.

1 April, 2019
Customer74447
Customer74447

Hello,

Once a test is converted to a JS file and changes are made in that file, is there or will there be the ability to convert that file back to a TESTCAFE file and see those changes in the UI?

3 April, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress

Hello,

Thank you for your interest in our testing tools. We didn't plan to introduce this functionality. The reason is that when you edit your JavaScript test, you can write any custom JavaScript code. The TestCafe file format may not support your custom JavaScript code. Converting it back to the TestCafe format may lead to unexpected behavior because your custom code will likely be lost during the conversion.

3 April, 2019
Customer107284
Customer107284

Hello Alex,

Is there possible to convert .testcafe (recorded) files to javascript in commandline? 

It will be huge improvement when want to automate recorded tests to CI pipeline.

If not is there posibility to get that feature and when we can wait that coming?

-J

1 November, 2019
Customer107284
Customer107284

Hello Alex,

Is there possible to convert .testcafe (recorded) files to javascript in commandline? 

It will be huge improvement when want to automate recorded tests to CI pipeline.

If not is there posibility to get that feature and when we can wait that coming?

-J

1 November, 2019
Alex Skorkin - DevExpress
Alex Skorkin - DevExpress
Hello J,
You can run your recorded .testcafe files directly from the TestCafe command-line interface. There's no need to convert them into JavaScript.
3 November, 2019

Please login or register to post comments.