TestCafe Team Blog

The Fastest Way to Your Next Test – TestCafe + TestCafe Studio - Q&A's

Paul Usher (DevExpress) — Wed, 23 Sep 2020 11:00:00 Z

Thank you to all that attended the recent TestCafe Studio presentation, the questions raised have been answered by the team and listed below.

Watch the Webinar

Questions & Answers

Will the webinar recording on YouTube include the Q&A section?

We discussed the most popular Q&As at the end of the webinar. This part is included in the recording. The remaining questions are answered in this post.

Can I test Windows Form applications with TestCafe Studio?

TestCafe Studio can test web applications only.

Am I allowed to install TestCafe Studio on our customer’s site to test it in their environment?

TestCafe Studio is a desktop application installed on a developer’s/tester’s local computer, not on a website. You can install TestCafe Studio on any Windows, macOS, or Linux machine. If the machine where TestCafe Studio runs can access the website deployed on your customer’s infrastructure, you can simply specify its URL to test it in its native environment.

Does TestCafe Studio support multiple languages?

The TestCafe Studio UI only supports English. If you are asking about programming languages in which you can write tests, TestCafe Studio supports JavaScript, TypeScript, CoffeeScript, and of course tests recorded in TestCafe Studio (*.testcafe files).

Can I include TestCafe Studio in Continuous Integration?

You can use the open-source TestCafe to run tests recorded or written in TestCafe Studio in CI systems. See the Integrate Tests with CI Systems topic for more information.

What value does TestCafe Studio add to an automated CI pipeline in which tests are executed in headless mode?

TestCafe Studio allows you to create and modify tests with ease. In a CI system, the tests will run with an open-source TestCafe runner that doesn’t make any difference between TestCafe Studio recorded tests and JavaScript TestCafe tests. See the Integrate Tests with CI Systems topic for more information. So, TestCafe Studio’s goal is to facilitate recording and maintenance.

Is there a way to record tests in TestCafe Studio and export them to JavaScript to run individually?

Yes, you can do this. See the Convert Recorded Tests to JavaScript topic for more information.

Does TestCafe Studio have any known issues with SPA apps, specifically Angular?

No, we are not aware of any significant issues with SPAs including Angular applications. You can also make use of the following selector extension that enables you to test Angular applications even easier: DevExpress/testcafe-angular-selectors

Are there any recommendations on the best method for initializing a selector?

We plan to publish an article that details the best practices for TestCafe tests, including suggestions on how to build effective selectors. In the meantime, refer to the Element Selectors topic to learn more about them.

Does the open-source TestCafe have the reporting and screen capturing capabilities?

Yes, it has both. See the Reporters and Screenshots and Videos topics for details.

Can I test “concurrent” scenarios where two users use the app and modify the data simultaneously?

To simulate users interacting with the app concurrently, you need two browser windows. Support for multiple browser windows is currently available in beta for coded tests only. See the details in the open-source TestCafe documentation: Multiple Browser Windows. As an alternative solution, you can try running the same test that edits a data record in different browsers at the same time.

Can I run tests at specific moments in time, e.g. on schedule?

You can integrate your test with any CI service to run them on schedule.

Can I receive an email notification with test results once they are done?

You can search for a TestCafe reporter that sends an email with test run results or create a new custom reporter for your needs.

Can I deliberately slow down the test run in TestCafe Studio for more convenient viewing?

Yes, you can adjust the Speed option to slow down test execution.

How do you use the GUI to create a custom attribute for a webpage?

You can customize the selector generation mechanism to take into account custom attributes. Add a custom attribute name to the list of selector types. TestCafe Studio will then use this attribute to create selectors. On your website end, you should specify the custom attribute for page elements manually.

Is it possible to write a text assertion with wildcards in it, so that it matches multiple text values?

Yes, you can use regular expressions for this purpose: Match Assertion

Is it possible to load user credentials from a file to use them in tests?

You can use the Run TestCafe Script action. Write a script that imports the fs Node.js module, reads the file content and saves the credentials in the test context. Now you can read these values from `t.ctx` in other scripts within Run TestCafe Script actions. Currently, you cannot pass the test context to recorded action parameters.

Can I create variables to store secrets or URLs for different environments?

TestCafe Studio can save selectors and functions to variables. In Run TestCafe Script action code, you can create other types of variables and use them according to your needs. To share variables between different scripts, store them in the test context.

Can I use conditions, add conditional jumps or go-to statements?

In recorded tests, you cannot use conditions and go-to statements. You can record test scenarios and then convert them to JavaScript to add conditions.

How to organize tests in TestCafe Studio? How can I tag Smoke and Regression tests to distinguish between them? I tried metadata but found it ineffective. Is there an alternative way in TestCafe Studio?

This is no such capability built in so far. You can split your tests across different directories or convert the recorded tests into JavaScript and use metadata with the open-source TestCafe test runner.

Can I use data from external sources (text files or databases) in my tests?

Yes, but not in the Test Editor UI. If you convert the recorded tests into JavaScript, you can use any Node.js or npm module to read data from your files or load from a database to use it in your test cases.

How can I ensure temporary objects that pop up briefly after a button is pushed or the mouse hovers over something are displayed? The display doesn't last long enough for me to click on it after activating the "=" assertion.

In recoded tests, you can add the Define Function action that checks if the target element exists every n milliseconds and returns the result when the element is found (you can use a Promise). Then add an action for click. After that, use the client function’s return value in an assertion.

How would I test visuals that indicate drag-and-drop, since that visuals only exist while you are actively dragging?

TestCafe cannot execute an assertion while you are dragging. We may suggest a multi-step workaround where you inject client-side code that is triggered during the drag, checks for the visuals, and stores the result. When drag is finished, you can obtain this value via a client function and verify it in an assertion. For manual verification, you can use the open-source TestCafe to record a video of a test run to ensure that the drag-and-drop indicator is displayed.

How do I run the same test against my app versions deployed in different environments (at different URLs), e.g., FAT/UAT/production?

In coded tests, you can specify the test’s start page programmatically. To test multiple versions of your app with the same test suite, start the tests as many times as you need while passing a different URL each time.

In recorded tests, you can use the same approach, but instead of setting the start page (which is not possible programmatically), begin tests with a custom script that accesses the passed URL and navigates to the required page with the t.navigateTo method.

Can variables be parameterized?

Currently, you cannot use parameters for recorded test steps but you can write any custom code in the Run TestCafe Script action.

Can the JS scripts act as a REST client to obtain assertion values (via a service call to a helper endpoint)?

Yes, you can use a Node.js HTTP client like got.

In our tests, selectors throw errors occasionally. How can we get rid of this irregular behavior?

Let us know how we can replicate this behavior. Please submit an issue in the Support Center.

Can I record accessibility tests with TestCafe Studio?

No, but you can use axe-testcafe in coded tests for accessibility testing.

How do I integrate TestCafe reports with Xray?

Check out the following custom reporter plug-in for Xray: antreyes/testcafe-reporter-xray

Can you show an example of a much more complex real-life test?

We publish TestCafe examples in this GitHub repository: DevExpress/testcafe-examples. You can find both basic examples as well as samples that address some rare scenarios.

However, you wouldn’t find a really complex test among our examples because it’s not good practice to overengineer end-to-end tests. After all, we created TestCafe to keep your tests simple.

Can you please show an example with the DevExpress Grid Control?

DevExpress Grid Controls can be easily tested with TestCafe like any other web control/page. You can try to record your own test on JS Grid demo pages with TestCafe Studio.

Will TestCafe be able to generate tests in other languages, like C#, or using Selenium API, etc?

Not in the nearest future.

Do you offer a testing framework for desktop applications (specifically, .NET)?

Yes, we offer Coded UI for WinForms applications.

TestCafe Webinar - Your Questions Answered

Paul Usher (DevExpress) — Sun, 16 Aug 2020 15:30:00 Z

We recently hosted a webinar on TestCafe and found the response overwhelming. During the presentation a large number of questions were raised, and as promised, the team have been through and answered them all.

Watch the Webinar

Webinar Resources

Does the book (End-to-End Testing with TestCafe by Dennis Martinez) go over the examples from the Webinar?

This webinar is based on the book, so you will certainly find details about the examples we showed.

Is the webinar available in other languages?

We currently hold online events in English only.

In what language the application you show is written?

It is JavaScript, the ES6 standard.

TestCafe Benefits

What are the advantages of TestCafe over Selenium/Cypress/Nightwatch.js/Robot Framework/WebdriverIO?

TestCafe does not require browser plugins, SDKs or other tools and libraries - just Node.js and browsers.
You don’t need to configure testing environment. Install TestCafe from npm and you are ready to test.
TestCafe is truly cross-browser and cross-platform. It can run tests on Windows, macOS, Linux, iOS, and Android. Supported browsers include Chrome, Firefox, Safari, Edge, and Internet Explorer.
TestCafe features a built-in wait mechanism. You don’t need to insert manual waiting in your tests. TestCafe waits for all kinds of loadings and animations automatically, so your tests are stable even when they slow down for whatever reason.
TestCafe allows you to test even the most complex scenarios like those with cross-domain navigation, file upload, sophisticated iframe use cases, secure services (payment, geolocation) or multiple windows (in beta).

License

Do I need to purchase a separate license for TestCafe? Is it included in any of the DevExpress subscriptions?

TestCafe is open-source and free. It is licensed under the MIT license.

Desktop Application Testing

Does TestCafe only work in a browser? Can I use it to test desktop applications, like WinForms?

TestCafe is designed for Web applications only. Thankfully, if you use Electron.js for your desktop application, you can easily test it with TestCafe.

Parallelization

Can I start 100 TestCafe tests to simulate high load?

You can start 100 parallel browser instances if your computer has sufficient resources to run such a number of browsers. But we recommend using TestCafe for end-to-end rather than load testing.

Can I run the tests in parallel in multiple browser windows?

Yes, TestCafe allows you to execute tests concurrently. In concurrent mode, TestCafe invokes multiple instances of each browser. These instances constitute the pool of browsers against which tests run concurrently, i.e. each test runs in the first available instance. You can find information how to run tests in parallel in this topic.

How well does TestCafe scale when it comes to parallelization? What is the ideal number of tests that can be run in parallel?

You can run any number of tests in concurrent mode. Regarding the number of browser instances that can be used to run these test, it’s hard to name an optimal number. It all depends on the resources available on your machine and the tests themselves. It is best to choose this number on a case-by-case basis.

Can I have multiple TestCafe instances running tests?

Yes, you can use several TestCafe instances. There are no special restrictions here. But in this case you will have to make sure that the instances do not conflict for resources (open ports, access to files, etc.).

Is there a way to start more parallel tests in memory without running browsers?

Concurrent mode implies launching the specified number of browser instances. All instances will be launched even if the number of tests is not enough to run at least one on each of them. Therefore, now there is no way to use this mode without launching browsers.

Integrations

Does TestCafe integrate with BrowserStack?

Yes, you can run tests in Cloud Testing Services (BrowserStack, Sauce Labs) via the corresponding plugins. See more in the documentation.

Can I run TestCafe on Azure DevOps?

You can find a topic about Azure DevOps integration in the TestCafe documentation.

Can I run TestCafe on Jenkins where there are no local browsers installed? Should I use headless mode on Jenkins?

If the machine that runs Jenkins does not have local browsers installed, you can use portable browsers or cloud testing services (BrowserStack, SauceLabs).

Headless mode is supported for both locally installed and portable browsers. We recommend that you use headless mode to decrease testing time. You can run browsers in headless mode even on machines without graphical system.

This tutorial describes how to integrate TestCafe with Jenkins.

You may also wish to check the TestCafe plugin for Jenkins that attaches screenshots and videos to the Jenkins test results page.

Is TestCafe compatible with Angular applications?

Yes, TestCafe has full support for Angular applications. You can use the Angular selector plugin that adds selector extensions to make it easier to test Angular applications with TestCafe. These extensions allow you to create a Selector to find elements on the page in a way that is native to Angular applications.

Does TestCafe support Vue?

Yes, TestCafe allows you test Vue applications the same way as any other apps. You can use the Vue selector plugin that contains selector extensions to make it easier to test Vue components with TestCafe. These extensions allow you to test Vue component state and result markup alongside.

Can TestCafe pick elements by their React component name?

You can use the TestCafe selector plugin for React components. This plugin adds selector extensions that allow you to select page elements in a way that is native to React. For instance, you can use component names to identify page elements.

Is TestCafe compatible with Knockout?

Yes, TestCafe can test applications based on Knockout. There are no known issues.

Does TestCafe support Bamboo?

You can use "Script Task" in Bamboo setting to write a wrapper that invokes automated tests. Alternatively, you can use the TestCafe Docker image.

Blazor

Can I use TestCafe to test a Blazor application?

TestCafe supports testing for Blazor apps. You can test your Blazor app the same way as any other application. There are no known issues.

Action Chaining

Are there limitations as to what kinds of actions can be chained? Is it the best practice to chain as many actions as I can?

Almost all TestCafe test controller’s methods can be chained, except those that return a value and hence break the chain:

You can chain these methods in any desirable manner but we recommend adding blank lines between logical parts of the action chain to improve tests readability, e.g.

await t
. click(‘#id1’) //some settings
. click(‘#id1’)
. click(‘#id1’)

.expect(Selector(‘#result’).textContent).eql(‘expected text’);//assertion

Can I extend the TestController object with my own chainable methods? Or only helpers and pageobjects are the way to go?

TestController cannot be extended with custom methods at the moment. We may consider adding this feature, but it isn’t in our current roadmap. As you correctly suggested, you can use helpers and page objects to implement custom methods.

TestCafe Studio

Can TestCafe record my actions (some tests need many entries and writing them is very time-consuming)?

You can use TestCafe Studio that has a built-in test recorder. It allows you to record your actions without coding them.

Is there a user interface for building tests, or is it all JS-based?

You can use TestCafe Studio which is a UI tool to maintain your tests.

Is TestCafe Studio a desktop application or is it available as a Web service?

TestCafe Studio is a cross-platform desktop application (powered by Electron.js). You can download its installer for Windows, macOS or Linux and install on your machine. Offline registration is available for computers without internet access. Read more in the Installation topic in the Docs.

Does TestCafe Studio require a license?

Yes, this is a commercial product that requires obtaining a license. There are two options available: TestCafe Studio and TestCafe Studio Pro with DevExpress Technical support.

Can TestCafe Studio generate test steps in a more user-friendly form than code?

TestCafe Studio has the built-in Test Editor designed for codeless test creation.

Can you change the order of tests on the side panel on the left?

The Explorer Panel allows you to view and manage files in your testing directory. For recorded tests, the Explorer Panel displays fixtures and tests. You can create new tests, copy/paste them and more. But currently you can’t change the order of the tests.

Are there more formats of report logs in TestCafe Studio?

In the Report View you can see tests run report in the TestCafe Studio format - Grid View.

The following built-in reports formats are also available:

JSON
xUnit
List
Minimal
Spec

You can install TestCafe reporter plugins to add more formats: Get More Report Formats.

Is there any guide with comprehensive instructions and examples for TestCafe Studio?

See docs, guides, videos, and examples in the TestCafe Studio documentation.

What is the element locator strategy?

TestCafe Studio use some characteristic of the DOM element in order to identify it on the page, such as id, class, text etc. You can set the usage priority of these characteristics, switch them off (e.g. for auto generated id) or create new ones based on custom attributes. See the details in the documentation.

Is TestCafe Studio a part of the Universal Subscription?

Yes, TestCafe Studio Pro version is.

Does TestCafe Studio support console.log output? I am currently running tests from the command line so I’d like to see my console.log statements.

You can add the Run TestCafe Script action. In this action, you can use console.log() to print messages directly, or even call t.getBrowserConsoleMessages() to fetch messages from the browser’s console.

Can TestCafe Studio generate reports with statistics, graphs, pies, etc., illustrating execution results?

Currently there is no such capability.

In TestCafe Studio, moving a line becomes almost impossible over 100 lines. Is that a memory issue that can be changed via config?

We cannot reproduce this behavior on our side. Please create an inquiry on https://www.devexpress.com/ask and provide more detail about the issue you encounter.

We are using both DevExpress and custom controls in our application. We record tests in TestCafe Studio, but they fail when we run them. Can you tell why this happens?

To resolve the issue, please refer to the Troubleshooting section on our FAQ page. If suggestions describe there do not help or apply to your case, create a ticket on https://www.devexpress.com/ask and describe your issue in detail.

Page Model

Do you create the page models manually? Can the page models be dynamically generated?

Page Models should be created manually, there is no built-in way to generate page models automatically. A page model reflects the internal structure of the testing page: what is the container, what is the parent and what is the child, etc., which distinguishing characteristics are best used to identify elements. Any automatically generated model is unlikely to reflect these semantics.

Is it necessary to create selectors with the Selector constructor in a page object class?

No, you can use CSS strings:

class Page {
constructor(){
this.submitButton = ‘#submit’;
}
}

But if you want to extend or chain such a selector with selector’s methods, you will need to use the Selector constructor:

class Page {
constructor(){
this.button = Selector(‘button’);
this.submitButton = this.button.withText(‘Submit’);
this.cancelButton = this.button.withText(‘Cancel’);
}
}

Can I add additional methods to page object in TestCafe?

Yes, you can add methods to a page model. Usually these methods are the most frequently repeated actions specific to this page. See an example in the TestCafe docs.

Does TestCafe support the PageObjects library?

If you mean this PageObjects library, then TestCafe does not support it because this library is designed for Selenium.

Database Access

Can test results be sent to a database?

There's no built-in functionality for this, but you can certainly make a plugin that will do this.

Can TestCafe connect to databases to verify that the correct data was written?

TestCafe doesn’t have built-in functionality for interaction with databases. But you can use any npm module for it.

Browsers

Does TestCafe support Microsoft Edge? Can TestCafe create mocks and run tests in IE11?

Yes, TestCafe supports both browsers. You can find the list of supported browsers in the TestCafe docs.

Selectors

What type of selectors are available? Is it only HTML element IDs and/or classes?

You can use any pure CSS selectors or TestCafe Selector API that extends capabilities offered by CSS selectors.

Can I target elements based on text that is actually visible to the end-user?

TestCafe Selector’s withText method finds elements by text that is visible to the end-user (the element’s innerText property). You can also create a selector that uses custom logic to find an element. Pass a function to the selector’s constructor.

Is it possible to set the waiting time for an element? What if I mistyped the selector's "name" or the page is loading really slow? How long does TestCafe wait by default?

You can specify the Selector’s timeout option During this timeout, the selector waits for the element. The default timeout duration is 10000 ms.

I use auto-generated names for DevExpress control’s elements in my application. Can TestCafe interact with them?

Yes, you can identify these elements by their static parts. See an example in our docs.

Can I use XPath to locate elements?

XPath selectors are not built-in with TestCafe, but you can implement a simple helper method to use them. See an example here.

Can I verify a certain element does not exist or will that cause a failure?

If an element used in some action doesn’t exist, a test fails. A test run report then indicates that the test failed because a target element doesn’t exist on the page. To check if the element exists before TestCafe attempts to interact with it, use the following approach

if(await Selector('#element').exists)
    await t.click(Selector('#element'));

For instance, you would use this verification to close ad popups (should they appear).

Debug

Are there any approaches that help debug tests? How can I debug TestCafe and the Hammerhead proxy?

Check out this blog post on how to debug your application and tests.

You can also find the instruction about how to debug in other IDEs in our guide.

To debug TestCafe and Hammerhead, enable the development mode. Use the --dev flag when you run tests:

testcafe chrome my-tests --dev

What are the available debug options?

The following capabilities help you determine the cause of errors and failures:

Take screenshots automatically when tests fail
Capture videos of test runs
Set the speed of test execution
Enter debug mode automatically when the first test fails
Execute tests step by step in debug mode

Can TestCafe capture a screenshot when a test fails?

Absolutely, enable the takeOnFails option to do this.

Authentication

Can I log in once and execute multiple tests under the same account rather than logging in in every test?

Yes, you can use roles to perform actions that log you in only once. In the subsequent tests, you just activate the role you defined earlier to log in without repeating the authentication process each time. See User Roles in the documentation for details.

Do roles work with auth0/openid which has JWT in headers?

Roles can process the JWT if it is sent in the cookies or stored in the local or session storage.

Would we run into any issues if we're using Microsoft authentication tokens for permissions?

You should not have any issue. Please try this and see how it works. If you face any issues, let us know, so we can research it.

How does TestCafe log out the user to switch to a different role? Does it just re-open the page?

When you switch to a role that was active previously in the current session, TestCafe restores cookies and browser’s local and session storage where authentication data is located. If the new role wasn’t used yet, TestCafe just resets the cookies and storage and then performs authentication actions.

Can you create Roles with API calls instead of functional steps?

If your question is about REST API, then yes, you can call REST API to authenticate a user in a role. Just note that you need to use a client function to call REST API from the browser. You may want to inject a library that can call API into the tested page.

What are the options for smart card logins?

TestCafe only supports web-based authentication. That is, your website should be able to authenticate you by your actions in the web browser, without relying on additional hardware. The only way is to deploy a testing instance of your web app without smart card authentication.

Multiple Apps, Windows, and Tabs

Can TestCafe test integration across multiple applications?

Yes, you can test any number of web applications.

Can you test multiple browser windows at the same time? For example, I may need to access application A, then check application B and come back to A.

Yes, you can do this. We released multiple window support in beta in v1.9.0. Please find more information in the Multiple Browser Windows topic and don’t hesitate to leave your feedback. Your opinion helps us make sure you get the best experience when this feature is released.

Does TestCafe support multiple tabs?

You can open and switch between multiple browser windows with this new beta feature. TestCafe will open a new popup window instead of a tab when necessary. This is because tabs can be throttled which interrupts TestCafe scripts.

iframes

Does TestCafe support iframes?

Yes, TestCafe supports iframes and provides two simple methods to interact with their content: switchToIframe to switch the test's browsing context to iframe’s context and switchToMainWindow to switch it back to the main window.

Reports

How does TestCafe output test run reports? Where can I find test results?

TestCafe shows the test run report in the console by default. You can redirect output to a file or use dedicated reporters to post reports to a CI system or elsewhere.

What happens when a test fails? What logs are produced?

If a test fails, you will see a comprehensive report in the console. For example:

Multiple report formats are supported, including the following:

spec
list
minimal
xUnit
JSON

In what format does TestCafe generate the reports? Is it XML or HTML?

The format depends on the reporter. It can be XML, JSON, HTML, and many others.

You can explore the available reporter packages on npm.

Test Organization

Can I use logic operators to have different conditional "paths" during a test?

Yes, TestCafe can do conditional testing. You can use standard JavaScript if-clauses to make this happen.

Is it possible to share fixtures between test files?

TestCafe treats fixtures declared in different files separately. If you can share your use case when merging fixtures is useful, please open an issue on GitHub.

How TestCafe Works

Did I miss what the ‘async’ keyword means in code? Is that to say TestCafe commands can run as other tests are running?

‘async’ is a JavaScript keyword that indicates that the function is asynchronous. Technically, an asynchronous function means that this function returns a Promise. TestCafe actions are asynchronous because so are the page events. Page loading, animations, HTTP requests take different amounts of time. The test function is asynchronous as well because it uses asynchronous TestCafe actions.

‘await’ means that the next command should not execute until the previous (‘awaited’) command is completed. This is required to execute test actions in the right sequence (and, in the same manner, TestCafe ‘awaits’ tests in a fixture to run them one by one). Note that, by contrast, tests in different browsers can run in parallel, just like in concurrency mode.

Please provide technical insight about how TestCafe avoids installing special plug-ins in order to interact with the browser. How does it actually interact with browsers?

TestCafe proxies the tested webpage and injects automation scripts right inside the proxied copy. This way, browsers do all the work when they execute the injected scripts. You can find more details in the How It Works article.

Is it possible to access runtimeJS objects from application client libraries loaded in the browser?

TestCafe is designed to conceal itself from application’s JavaScript code. This is necessary to ensure that pages proxied by TestCafe run exactly in the same way as they do on the website. For this reason, application code cannot access TestCafe scripts or test code entities.

Waiting and Asynchrony

What if a portion of a page takes a while to render (for instance, the page uses AJAX). Can TestCafe take it into account?

Yes, TestCafe tests wait automatically until the page is rendered completely. You can also insert manual waiting, but it shouldn’t be necessary in almost all cases.

I set up my page in a jQuery callback: $(document).ready(function(){}). Will there be JS races with TestCafe?

TestCafe will wait until the page loads completely and run tests only after that. TestCafe wait mechanisms are designed to prevent races with JS code.

It is great that TestCafe handles waiting internally. Would there be a need to override default timeout functionality though? How should a user go about it?

You may need to override the default timeout if your page requires extra long waiting. All TestCafe auto-wait timeouts are customizable. You can find more information in the Built-in Wait Mechanisms topic.

Other Features

Does TestCafe support all ES6 or later features? Do I need plugins to support arrow functions or the async/await syntax?

TestCafe supports all modern JavaScript and TypeScript features. No plugins needed, everything works out of the box.

Can TestCafe work with the Service Worker?

We are currently in the progress of implementing full support for Service Workers. Stay tuned for our updates to be the first to know when we are ready.

Does TestCafe work with Python?

You can write tests in JavaScript/TypeScript/CoffeeScript only.

Does TestCafe support upload ( excel ) data functionality?

TestCafe can upload files, including Excel, to websites if it is a part of the test scenario. However, if you are asking about API to access Excel data sheets from test code, TestCafe does not provide one out of the box. You can search for npm modules that implement the functionality you need and use them from TestCafe tests.

What about functional test coverage? Does TestCafe support it?

Test coverage estimation is not available at the moment.

Miscellaneous

If TestCafe runs in headless mode, is it still possible to grab screenshots on failures?

Yes, TestCafe can take screenshots even in headless mode.

Does this require a test framework like jest/mocha/jasmine?

No additional frameworks are required for TestCafe.

How can I use TestCafe to test different resolutions?

You can use the t.resizeWindow action to change the browser window size and test adaptive UIs. TestCafe also supports Chromium device emulation to test on emulated mobile devices of various sizes.

Can we use Test Cafe to compare two documents? Like documents with logos, disclaimers and other information?

If you mean visual comparison, TestCafe does not provide this functionality out of the box. However, you can find multiple open-source packages that try to solve this task, like testcafe-blink-diff. Note that they are maintained by the community and are not related to the TestCafe team.

If your goal is to compare HTML, you can pass ‘innerHTML’ to a TestCafe assertion as shown in the example in this topic.

How do you change the base URL to test in different environments?

You can import the base URL from a configuration file or an environment variable, and then use it to build start page URLs in test code. See this example on the FAQ page.

Can I run TestCafe on a Linux computer without the UI?

Yes, you can run your tests on Linux and use headless browsers to run your tests without the UI.

Could you please share ideas about how to do localization testing? Does TestCafe support that?

TestCafe does not have built-in functionality for localization testing. As a suggestion, you can fetch the localization strings from a resource file or database with Node.js means or third-party npm modules. Then, you can create data-driven tests that check if the displayed strings are localized (present in the localization resources).

When do you plan to release version 1.9.0?

1.9.0 was released on August 6th. If you are asking about the multi-window support final release, we do not have an exact date yet.

How does TestCafe handle exceptions?

TestCafe considers the test failed if its code throws an exception. The report will indicate the line of code and call stack where this exception originated. Exceptions in JavaScript code on the tested page also make tests fail. However, this behavior is customizable. You can use the --skip-js-errors flag to ignore page errors. Similarly, the --skip-uncaught-errors flag allows you to continue tests when test code throws an exception.

Can we measure client-side performance using TestCafe (both HTTP and script execution time)?

TestCafe is not intended for this task. In fact, TestCafe impacts the webpage performance when it runs tests. We are constantly optimizing our code to minimize this slowdown, but we do not aim to precisely match the original performance.

Are tests executed in sequence within a fixture?

Yes, tests are executed as they are declared in code.

Can I add my custom element library to the context object?

You can add anything to the feature and test context objects. However, keep in mind that test code does not run in the browser, so adding browser libraries to the context objects might not help. If this is the case, you may be more interested in the custom script injection functionality designed for user’s client libraries.

Can TestCafe test HTML5 features like fullscreen, camera, audio or video interaction?

Yes, you can test these HTML 5 features with TestCafe.

Note that you might need to set the --autoplay-policy=no-user-gesture-required browser flag as explained in this article to test the fullscreen mode and audio/video. The camera can be mocked using ClientFunctions.

Does TestCafe support multiple domains in a test?

Yes, navigation and requests between different domains work out of the box. You do not need to care if you visit pages on one or several domains.

What happens if TestCafe loses network connection with a remote device that runs tests?

Tests cannot run without network connection because TestCafe must be able to send commands to the remote device. If the connection is interrupted, TestCafe will throw an error saying that a browser connection is lost.

Is it possible to run all JS in one command? Could you please specify this command?

If I understand you correctly, yes, you can run all your JS tests using a single console command.

Here is the CLI reference where you can see the testcafe command syntax.

I noticed non-standard code indentation. Is a respective formatter for VS Code available?

We didn’t release a specific formatter because you can use any indentation you prefer with TestCafe.

Can I use TestCafe to test the development and production instances of my app? If so, how can I configure different URLs for my instances?

Yes, you can import the base URL from a configuration file or an environment variable, and then use it to build start page URLs in test code. See this example on the FAQ page.

Does TestCafe have support for file download? Can you set the default file download location using TestCafe API?

Yes, if test actions on the page trigger file download, the browser will download it as usually. Then you can access the file from test code to perform verification. Currently, TestCafe API does not allow you to set the default download location.

TestCafe and TestCafe Studio: Learn How to Integrate Our Web Testing Tools into Your Workflow

TestCafe Team (DevExpress) — Wed, 15 Jul 2020 03:33:00 Z

We will soon host a TestCafe webinar that will cater to both first-time users and those who may have tried TestCafe (our end-to-end web testing tool) in the past. Paul Usher, our technical evangelist and live session host, will begin with first-time setup and then will go into a few of the more recent TestCafe innovations. He will also cover the differences between TestCafe Open Source and TestCafe Studio. See the details below.

Upcoming Webinar: 10 Ways to Simplify UI Testing

See how you can easily incorporate UI testing in your development workflow. In this presentation, Paul Usher will show you how TestCafe can be leveraged to deliver high quality web apps that always meet end-user expectations. You’ll discover:

How to setup TestCafe and start testing in under 10 minutes.
How to create customizable tests that can easily evolve as business requirements change.
The purpose of Custom Selectors and why they make web testing so much easier.
The purpose of the Page Model approach and why it helps increase productivity.
And you’ll learn how to master use of Smart Assertions.

What's So Cool about TestCafe?

TestCafe Open Source and TestCafe Studio are driver free and do not require you to manage complex plug-ins. Their flexibility and ease of use are probably best illustrated by the ability to run tests on remote browsers across multiple devices and platforms. TestCafe Studio generates a URL used to connect to the test runner... or a QR code so you can easily connect your mobile devices.

This functionality is not limited to TestCafe Studio. You can do the same if you use our open source TestCafe library and start your tests from the command line:

The URL provides access to run the tests the remote device. This minimal setup philosophy always has been our guiding principle in all areas of web testing - from installation to test debugging.

Something #webdriver can’t do... #testcafe pic.twitter.com/bgD8cgUwXX
— TestCafe (@DXTestCafe) December 1, 2016

If you'd like to know more - register for the webinar on July 29th. Even if you cannot attend the live session, we'll make sure to send out a recording to all registrants.

TestCafe Studio v1.3.0 - A New Way to Work with Selectors

The DevExpress Team — Wed, 10 Jun 2020 11:00:00 Z

TestCafe's Visual Test Recorder automatically creates selectors for all elements used in a test. This complicated algorithm balances many different variables, and we just recently added a few options so that you can customize it to meet your specific needs.

This article first covers what we consider to be a reliable selector. We then mention the pros and cons of various selector strategies, and custom attributes in particular. Finally, we'll show you how to customize TestCafe options so that it builds selectors that are optimized for your website.

How TestCafe Studio Generates Selectors

To record an automated test in TestCafe Studio, specify the tested page’s URL and start the recorder. TestCafe Studio launches the selected browser, opens the web page, and records your actions. Once the test scenario is ready, you can run it in any local browser, a mobile device, or as part of your CI pipeline.

TestCafe Studio not only records your actions (element clicks, text input), but it also creates a selector for every affected page element. The selector generation algorithm was developed to adhere to the following requirements:

Selectors shouldn’t be too specific. Otherwise, you might have to rewrite them after each page modification. For instance, Selector(‘body’).find(‘div’).nth(5).find(‘p’).nth(3) must be revised each time the number of elements on the page changes.
Selectors shouldn't be too generic. Otherwise, they may return different elements after each markup change. For example, Selector(‘div > button’) can match multiple elements at once.
Selectors shouldn’t rely on element parameters that are likely to change. For instance, Selector('[style*="background-color: red"]') uses a CSS style that changes frequently.
Selectors should remain readable. Selectors should be easy to understand (by you or another developer) for as long as the test is maintained. For instance, it may be difficult to understand which element corresponds to the following selector: Selector(‘div’).find(‘pre’).nextSibling(-1). If you use Selector(‘#topContainer’).find(‘.child’).withText(‘Add item’) instead, the selector is much easier to read.
Selectors should reflect the user’s point of view. Since TestCafe supports end-to-end testing, it’s a good idea to build selectors that identify elements as an end-user would. For instance, Selector(‘form’).find(‘[name=”btn-foo-123”]’) might be stable, but it is written from the programmer’s perspective rather than from the user’s point of view.

In addition to these general requirements for selectors, page organization is also vital in the testing process. Some pages may have stable selectors that rely on the ‘id’ attribute, while other pages use auto-generated ‘id’ attributes - so it does not make sense to use ‘id’ in a selector. This also applies to other attributes such as ‘class’. Text-based selectors can also be ineffective, depending on changes you introduce to the page over time.

Why Custom Attributes are Required

Modern web component frameworks, like React, complicate DOM element identification. This is why many developers prefer to assign custom attributes to page elements. Custom attributes enable tests to survive changes in HTML structure, script behavior, and styles.

Let’s consider the following Populate button on our example page:

<input type="button" id="populate" data-testid="populate-button" value="Populate">

You can build the following selectors for this button:

HTML-based

Selector: Selector('input').nth(1)
Pros and Cons: Too generic.
Summary: Might match the wrong elements because it is vague.
Selector: Selector('body').find('form').find('div').find('fieldset').find('input').nth(1)
Pros and Cons: Too strict.
Summary: May break due to minor changes in HTML structure.

ID-based

Selector: Selector('#populate')
Pros and Cons: Relies on ID only, which could be introduced for styling or event handling, or may be auto-generated.
Summary: Better options are available.
Selector: Selector('#main-form').find('input').nth(1)
Pros and Cons: Similar to the above selector, but relies on the parent ID.
Summary: Appears to be more specific than the above selector, but it is not.

Class-based

Selector: Selector('.column.col-1').find('input').nth(1)
Pros and Cons: The element has no class, so this selector uses the parent class. Classes can change because they are usually assigned for styling purposes.
Summary: Sensitive to CSS modifications; difficult to maintain.

Text-based (text matches the attribute value in this case)

Selector: Selector('input[value="Populate"]')
Pros and Cons: Mirrors how users perceive the page, so it may be stable. Detects actual issues caused by accidental text changes.
Summary: Satisfactory, but fails even when text changes are intended.

Custom attribute-based

Selector: Selector('[data-testid="populate-button"]')
Pros and Cons: The most stable selector, since it is not affected by changes.
Summary: The preferred selector type for most scenarios.

In summary, custom attributes have the following benefits:

Tests do not break based on markup or code changes. A test will not fail when you refactor HTML or implement new behavior in JavaScript.
Tests do not depend on page styles. You can switch themes as many times as you wish - tests will not fail because of a certain style or theme.
Text content modifications do not affect tests. Tests are not affected by changes to content, including localization changes. Note, however, that if an element’s text is essential to a test scenario, you should verify the text content or use a text-based selector: Selector(‘div’).withText(‘Add to Cart’).
Use a custom attribute to indicate that an element is used in tests.

A New Way to Customize Selectors

When you record a test, TestCafe automatically generates a selector. TestCafe also lets you choose an alternative selector from a drop-down list or create one manually. The latest TestCafe update (v1.3.0) ships with an adjustable selector generation algorithm. You can now configure the selector generator on the fly, based on the tested page:

Prioritize selector types. Use this option if you often need to choose a different selector from the suggestion list. The default selector is generated based on your preferences.
Disable unwanted selector types. For instance, your website can generate id, class, or other element attributes automatically, so that they change each time the page reloads. However, you cannot use these values in selectors because the test cannot be replayed. In this instance, you may want to exclude these attributes from selector generation rules. You can now simply disable the corresponding selector type - and switch back to it at any time - without restarting the recorder.
Add selector types based on custom attributes. These selector types are preferred because if you configure them, you won’t have to edit selectors due to changes. The only requirement is that you maintain unique attribute values during development.

Recap

TestCafe Studio generates selectors for every page element that takes part in a test. Good selectors must comply with strict guidelines to be stable and reliable. TestCafe algorithms use best practices to generate selectors that meet these criteria. You can also optimize selectors based on your understanding of how the website works. Our latest update (v1.3.0) allows you to instruct TestCafe Studio to use custom attributes in selectors or ignore dynamic IDs. The result is more stable selectors that are easy to read and maintain.

You can read more about the new adjustable selector generation algorithm in our documentation. As always, we are happy to answer your questions in the comments. Happy testing!

Automated Web Testing - When to Choose Recording Over Scripting?

The DevExpress Team — Tue, 24 Dec 2019 07:00:00 Z

As you may know, we currently offer both scripting-based (TestCafe) and visual recording-based (TestCafe Studio) platforms for web test automation. Though both options can get you where you want to go, we believe that visual test recording offers unique strengths for those who want to incorporate automated testing in the product delivery process.

Visual Recording in a Nutshell 

Unlike script-based test automation, visual test recording eliminates (or significantly reduces) the need to write test script. We like to think of visual test recording as “codeless automation.” To create a recorded test, you simply open a browser and click through your way to a web test (you interact with your web app in the same way end-users would). The visual recorder tracks actions and generates test scripts that you or a CI system can automatically replay later.

This leads to the obvious question - when does it make sense to select automated recording over manual scripting?

1. When You Have Limited Programming Experience

With TestCafe Studio’s visual test recorder, test authors don’t have to be software developers.  To create an automated test with the visual recorder, you simply open a web browser and execute standard user actions (click buttons, fill in forms, drag sliders). Once you’ve recorded actions, you specify the page elements that require verification (via point and click) within your test scenario.

2. When You Have Multiple Test Scenarios

If you have programming experience and you need to write ten or twenty tests – you can write your test scripts manually without much difficulty. If, however, you must write a few hundred tests, scripting could be rather time-consuming. In such instances, it may make sense to consider automated recording. Regardless of how fast you write code, point and click recording is always faster and less costly. This is especially true when the recorder allows you to re-record individual actions and modify selectors when the user interface changes over time.

3. When You Have to Test Apps Based on Different Technologies

Coding tests for large web applications requires that you examine the structure of every page involved. This can be hard if the application consolidates multiple services, includes third-party iframes, etc. If you write tests for more than one application, you may spend unnecessary energy on platform-specifics - specifics that can affect your test code.  Recording conceals these complexities and allows QA engineers to get the job done with the least amount of effort. When using an automated recorder, test script generation remains straightforward regardless of how technologies are used on a webpage.

4. When You Have to Perform Black-Box Testing

When you write a test with a web testing framework, you would normally call page code or browser API to perform actions against page elements. This approach can lead to situations wherein test authors make modifications that are unavailable to end-users. When recording tests visually, engineers are limited to the same screen elements available to end-users.

Visual Recording and You

Though visual recording cannot address every single test scenario, we think it offers important advantages over manual scripting – advantages that can save your team significant time. If you are currently using scripting alone or recording your test scripts, we’d love your feedback. Please tell us why you prefer manual scripting or visual recording in the comments section below.

About TestCafe Studio

TestCafe Studio is a cross-platform, end-to-end web testing IDE with the fully integrated Visual Web Test Recorder. TestCafe Studio requires no WebDrivers and allows you to create, run and maintain automated web tests with zero code. Download Free 30-day Trial.

Web Application Testing - 6 Tips to Make the Most of TestCafe Studio’s Web Test Recorder

The DevExpress Team — Thu, 05 Dec 2019 05:00:00 Z

TestCafe Studio allows us to test large pools of data and scenarios, thereby catching issues before we launch our products. It has transformed our launches and improved our product success rate.

Our customers love TestCafe Studio for many reasons – among them its accurate and easy-to-use test recording engine. By clicking through a target website, TestCafe Studio allows you to emulate end-user actions and automatically record stable web tests without writing any code.

In this post, we will show you 6 tips to boost productivity and leverage TestCafe Studio’s Test Recorder to address a variety of usage scenarios.

1. Hover Over an Element

TestCafe Studio’s recorder does not save mouse movements during most actions (except for drag-and-drop operations). This enhances test stability because recorded mouse movements usually generate random errors during playback (for instance, this occurs when an element is rendered at a different position).

If you need to perform actions against specific UI elements (when you hover over a menu item or button), TestCafe Studio requires you to explicitly add a hover action.

To add a hover action during recording, switch to the TestCafe Studio window and click in the Actions panel to add a Hover action:

Click Pick a target element to select an element to hover within the browser:

You can now switch back to the browser window and continue recording.

2. Save the Actual Click Point

TestCafe Studio clicks the center of the target element when it replays a recorded test. This produces stable tests even when a target element’s size changes over time.

If you need TestCafe Studio to click the exact position you clicked during recording, simply add an Offset parameter. To add an Offset parameter, expand the action within our Test Editor and click the +Add Option.

Select Offsets from the dropdown menu. This adds both X and Y options to the action.

TestCafe Studio initializes these with coordinates captured during the recording. You can use these values or specify a different offset.

3. Repeat Recorded Actions Before Each Test

Many test scenarios begin with similar actions (like login or opening a view). TestCafe Studio allows you to record these actions and configure them prior to each test run.

Action sequences executed before and after each test are called hooks. Here’s how to add actions to a hook.

Run the recorder as you normally would and record your actions:

Hold the Cmd/Ctrl button and select actions you need to repeat before each test:

Then click the Move to before each hook button.

TestCafe Studio creates a ‘before each’ hook with these actions or appends them to an existing hook.

4. Handle Native Browser Dialogs

TestCafe Studio allows you to define how tests should respond to native browser dialogs (alert, confirmations, etc.) invoked on a page.

Click in the Actions panel to add the Handle Native Dialogs action. Ensure that it is executed before the action that invokes the dialog.

Click +Confirm to add a confirmation dialog.

Specify OK as the close action, enable the Message switch and specify the expected message:

If any subsequent action invokes a confirmation dialog with the specified message, TestCafe Studio will emulate the OK click to close it.

5. Take a Screenshot of the Tested Page

TestCafe Studio allows you to take screenshots of your tested page throughout the test execution process.

Once you record actions that require a screen capture, switch to the TestCafe Studio window and click in the Actions panel.

TestCafe Studio can also capture screenshots each time a test fails. To enable this option, click the button next to the Run Configuration drop-down menu on the main toolbar and check ‘Take a screenshot if a test fails‘ in the dialog window.

You can view screenshots captured during test execution in your report.

Click the icon in the grid to open the screenshot window.

6. Add an Assertion and Verify Its Status

TestCafe Studio provides a straightforward way to create an assertion, configure it and check if it passes.

Select the assertion in the Actions panel to create your assertion.

Click Pick a target element to specify the element whose state you need to verify. This opens the browser where you can point to the target element.

Use the property grid to select the element property whose value you need to check.

TestCafe Studio retrieves and displays its actual value. Click the button to copy this value to the ‘Expected Value’ field. This is a reference value compared with the values obtained during the test run.

Finally, click the button to check if the assertion passes. The tick mark indicates that the assertion’s condition matches.

Your Feedback Counts

As always, we welcome your feedback. Please share your thoughts below and let us know how you’re using TestCafe Studio within your enterprise.

Automated Web Testing: How to Create a Web Test in 2 Minutes with TestCafe Studio

TestCafe Team (DevExpress) — Wed, 21 Aug 2019 16:55:00 Z

In this quick tutorial, I’ll describe how to get started with TestCafe Studio and how to test an existing webpage (in this instance, a relatively advanced web page with integrated file upload functionality). As you’ll soon see, I will select and upload two files and then run two assertions: The first for correct file name output and the other for correct file size label format.

I promise – this should only take two minutes of your time - TestCafe Studio’s Visual Test Recorder makes it easy to visually record web tests.

Step 1: Record a Script

I’ll first launch TestCafe Studio (if you’d like to install TestCafe Studio and follow along, simply download our 30-day trial), enter the website URL inside TestCafe’s Welcome screen and press ‘Start Recording a Test’. My browser will navigate to the appropriate address – and since TestCafe’s Visual Test Recorder is up and running in the background – I’ll be ready to record actions performed on this webpage.

I’ll use the Upload mode drop-down and specify that I wish to begin file upload manually.

I’ll then enable multi-file checkbox.

I’ll click Select file and make my file selections.

I’ll click Upload to complete the process.

Step 2: Review the Action List

Its time to switch to TestCafe Studio and review the commands generated by my actions.

Step 3: Add a Value Assertion

As you can see, site UI is rendered in an IFRAME, so the first action represents a context switch - click and upload steps follow thereafter (replicating my earlier site actions).

It’s time to finalize the test with a couple of assertions. I’ll first check whether the demo app displays the correct file name.

As you can see, you don’t have to write a single line of code to do this (unlike competing web testing tools which do require you to write lots of custom test code). I’ll simply click a toolbox icon to add an assertion with an “equals” condition.

I’ll then use the Element Picker button to select the target element within the browser.

This focuses the browser window so you can point to the appropriate element.

As you can see below, TestCafe Studio generates a selector for this element and opens a list of its properties. Select the property you wish to assert.

TestCafe Studio obtains the current property value and asks me to confirm its use (for comparison).

Step 4: Add a Format Assertion

In much the same way, you can assert a label’s font-size CSS property value.

Step 5: Stop the Recorder and Run the Test

My test is ready and recording can be stopped. To execute my newly recorded test, I’ll press the Run button and watch TestCafe replay my actions.

Step 6: Review the Test Report

Once TestCafe completes the test run, I can review test results within TestCafe’s intuitive UX.

That’s it. In just a few minutes and without a single line of code, I created a test that was ready to run across all modern browsers and on different platforms (Win, Mac, and Linux).

Get Started Today

Ready to compare TestCafe Studio to your current web testing platform and experience the TestCafe difference for yourself?

Download Free 30-Day Trial

Should you have any questions or need assistance, share your comments below.

TestCafe Studio is Here

TestCafe Team (DevExpress) — Fri, 26 Apr 2019 06:43:00 Z

We are proud to announce the immediate availability of TestCafe Studio – our next-gen automated web testing framework. To learn more about TestCafe Studio and experience its capabilities first hand, feel free to download a no-cost 30-day evaluation version today.

Online Webinar: Introducing TestCafe Studio

Join Paul Usher and Julian Bucknall on April 30th to learn more about TestCafe Studio and how you can leverage its capabilities to deliver more robust web solutions to your end users.

Webinar: Introducing TestCafe Studio

TestCafe Studio vs TestCafe v15.1

TestCafe Studio ships with unique features unavailable in its predecessor (TestCafe v15.1), including:

A more powerful test execution platform – TestCafe Studio is now based on hybrid client-server architecture instead of in-browser execution.
A redesigned Visual Test Recorder – TestCafe Studio can now visually record a test from scratch or edit existing test scripts.
A feature-complete code editor.
A new test debugger designed to run tests step-by-step and access page element data while doing so.

For additional information on the differences between TestCafe v15.x and TestCafe Studio, please refer to the following blog post:

TestCafe Studio: A New Web Testing IDE

Upgrade from TestCafe Open Source Edition

Needless to say, we remain fully committed to TestCafe as an open source web testing platform and are looking forward to extending TestCafe’s end-to-end web testing capabilities now and into the future. As you might expect, users have repeatedly asked us to create a Visual Test Editor/Test Recorder. With the release of TestCafe Studio, we’ve made certain to address this requirement and to deliver a straightforward way to visual record and edit test scripts.

Whether you’re looking to simplify the way you record your tests, or want to delegate testing to a QA team that does not want to write JavaScript, try the Studio edition and tell us what you think. For information on how TestCafe Studio can become part of your existing continuous integration workflow, read the following article:

What's Better than TestCafe? TestCafe Studio

Contact Us if You Have Any Questions

Should you have any questions about TestCafe, its capabilities or licensing terms, post a comment below or create a new ticket in our Support Center and we'll be happy to help.

TestCafe Studio: A New Web Testing IDE

TestCafe Team (DevExpress) — Wed, 30 Jan 2019 08:01:00 Z

Two years ago, we announced a shift in our long-term plans for TestCafe. As some of you may already know, we chose to freeze the features shipping in v2015.1 and to focus our energies on two new versions of the product:

TestCafe: an open-source CLI tool
The team always wanted to publish a free open-source version that doesn’t include advanced UI capabilities. You can now find this version on GitHub, where it’s been freely available to everyone for the past several months.
TestCafe Studio: the next step in GUI version evolution
A commercial web testing IDE built around the same engine as the open-source version. Our work on this product is almost completed: you can download a Preview version today.

Below you’ll find all the information about the new TestCafe Studio: how to get started, the differences between the new version and v2015.1, and where to find the latest Preview version.

TestCafe Studio 101 & Getting Started
Start here if you’re new to TestCafe. A short video will explain the basic concepts.
What’s New in TestCafe Studio: Compare to v2015.1
Start here if you’ve used TestCafe before. This part will bring you up-to-date on TestCafe capabilities.
Try the Preview and Let Us Know What You Think
Migrate your existing TestCafe tests or start working on new projects.

TestCafe Studio 101 & Getting Started

TestCafe Studio is a standalone application that allows you to create, edit, manage, execute, and debug tests with ease.

Runs on Windows, Mac and Linux.
Ships with a Visual Test Recorder, allowing you to create and edit tests without coding.
Includes a built-in code editor with auto-complete and syntax highlighting support (for those who prefer to script their tests in code).

If you’ve never used TestCafe before, watch this short Getting Started video.

What’s New in TestCafe Studio: Compare to v2015.1

Here’s a brief summary of the capabilities that differentiate the new TestCafe Studio from its predecessor.

TestCafe is Now a Desktop Application
Visual Recorder: Now with Test Editing Capability
Smart Element Selectors
Concurrent Test Execution Improvements
Improved Test Execution Configuration Management
Faster and More Stable Tests with the New Auto-Wait Mechanism
Updated File Manager UI
Enhanced Reports
Feature-Complete Code Editor
Powerful Test Debugger
Test API
Continuous Integration Support

TestCafe is Now a Desktop Application

TestCafe Studio is a desktop application that looks and behaves much like today’s most popular IDEs. It includes familiar UI elements such as a Toolbar, File Explorer, Main Menu, Tabs View, and Reports Panel.

TestCafe v2015.1 had a UI implemented as a website hosted on a local Node.js server.

Visual Recorder: Now with Test Editing Capability

TestCafe Studio: the Visual Test Recorder now ships with a new “Continue Recording” option. As its name suggests, this feature allows you to edit previously recorded tests and add additional test actions visually.

TestCafe v2015.1: Visual Test Recorder could be used for initial test construction, but the only way to modify a test was to edit generated code.

You will find the action list UI more user-friendly and easier to understand. Every action on the list can be edited according to your needs.

Tests recorded in TestCafe Studio are saved in a new "*.testcafe" file format. Recorded tests are no longer transformed into plain JavaScript code once recording is complete. This is what allows you to display your tests as a sequence of actions, edit them as needed, and modify them using the Visual Test Recorder.

Should you prefer to edit JavaScript code, we provide you with the option of manual "*.testcafe" to "*.js" conversion.

Smart Element Selectors

TestCafe Studio: You can pick a selector from a list of predefined types or build your own. The new Selector Builder UI allows you to locate an element in a variety of ways:

Use CSS selectors
Run custom JavaScript functions
Filter page elements by index, text, attributes or visibility
Find elements based on their position within the DOM tree

TestCafe v2015.1: Advanced Selector Builder UI wasn't available. Customization was limited to manual CSS selector construction.

Automatically-generated selectors:

Custom selector builder:

Concurrent Test Execution Improvements

TestCafe Studio: Can start several instances of the same browser, divide fixture tests into several batches, and execute each batch in its own browser instance.

TestCafe v2015.1: Only one browser instance was created per test run and all tests were executed sequentially.

Both TestCafe Studio and its predecessor execute tests in parallel if you specify multiple target browsers.

Improved Test Execution Configuration Management

TestCafe Studio: You can create multiple “configurations” that define target browsers and additional test execution settings. You can then run tests for specific configurations.

TestCafe v2015.1: You needed to set configuration settings each time you ran tests.

Faster and More Stable Tests with the New Auto-Wait Mechanism

TestCafe Studio: Automatically suspends test execution while waiting for element availability, page load completion, XHR requests, etc. The new auto-wait mechanism also extends to assertions and thus makes tests more stable when you verify the page state. Customizable timeouts allow you to fine-tune built-in waiting mechanisms.

TestCafe v2015.1: Also waited for element availability, page loads and XHR requests automatically, but didn’t wait for assertions to pass, and the timeouts were not customizable.

Updated File Manager UI

TestCafe Studio: You can create, edit, and view various file formats including test files, test recordings, helpers, wrappers, and PageModels. You can launch tests directly from selected files or folders.

TestCafe v2015.1: You could create, edit, and view only .js test files. It was not possible to start an arbitrary selection of tests or fixtures - you could only launch an individual test or fixture.

Enhanced Reports

TestCafe Studio: The new UI includes a panel with information on past test runs and current sessions that can be cancelled if necessary. A click on an item within the Reports panel opens a detailed report on any given test. You can view test reports and export them into various formats (json, list, minimal, spec, xunit). You can install plugins that provide additional report formats.

TestCafe v2015.1: Displayed a summary of test runs on a separate page, instead of a panel integrated in the main UI. Tests that were currently in progress could only be stopped by closing the browser that ran them, but this led to an error. There was no way to add more report formats.

Feature-Complete Code Editor

TestCafe Studio: Features a fully integrated code editor with autocomplete and syntax highlighting. You can launch tests directly from the editor.

TestCafe v2015.1: Auto-completion wasn’t available, tests could not be launched from the code editor, and a minimap (outline view) was not displayed.

Powerful Test Debugger

If an automated test fails, but you cannot reproduce the failure manually, there may be an issue with the script. The best way to catch those issues is by debugging the test session. For this purpose, TestCafe Studio implements all the means necessary to effectively troubleshoot test execution.

TestCafe Studio:

The Debug action stops a test at a desired moment.
The Debug Mode allows you to run the entire test step-by-step.
The “Pause the test when it fails” option stops a test the moment it fails.

Whenever a test pauses, you can explore the tested page with developer tools, interact with it or step over to the next action (if the test has not failed).

TestCafe v2015.1: You could stop a test with the ‘debugger’ keyword, but the tested page was frozen, you could not interact with it or step over to the next action. There was no way to stop execution automatically when a test failed.

Test API

TestCafe Studio: The new engine includes the following API changes:

a more intuitive interface that simplifies the task of writing a test from scratch;
test code can now include conditions and cycles;
built-in support for ES6 and TypeScript.

TestCafe v2015.1: Test code had to be organized into "steps", where each step could only contain a single test action. Test code logic had to be linear: cycles and conditions were not allowed. There was no support for TypeScript. The version of JavaScript was limited by the browser version in which you ran tests.

Continuous Integration Support

TestCafe Studio: Use the open-source TestCafe, with its extensive command line and JavaScript interfaces, as a test runner.

TestCafe v2015.1: You needed to use a separate Node.js module that provided only a minimal JavaScript API and no command line features.

Try the Preview and Let Us Know What You Think

TestCafe Studio maintains full backward compatibility - all tests you recorded in TestCafe versions up to 15.1 can be executed in TestCafe Studio without limitations. We do suggest, however, that you switch to the new API once we release the RTM version, so you can enjoy all the new features shipping inside TestCafe Studio.

TestCafe Studio Preview is free to download - sign up now to get your copy.

We look forward to hearing about your experience. Write to us at testcafeteam@devexpress.com or create a ticket in our Support Center to share your thoughts and feedback.

TestCafe Update (v14.1.2)

The DevExpress Team — Thu, 10 Jul 2014 05:32:00 Z

We have just released TestCafe v14.1.2 - a new version of our functional testing framework for web applications. There have been several changes to the product and I’d like to show off some of the more important new features and improvements.

(Not familiar with TestCafe yet? Read this FAQ)

Extended Reporter/CI API – The most popular continuous integration (CI) system report formats - JUnit and NUnit - are now supported.

Test Parameterization - TestCafe now provides a way to define data sets (test cases) and use them as inputs to run the same test with a different set of data each time.

Here’s how to specify several test cases within the test code and run the test with each of the different parameters:

'@test'['Data-driven testing'] = {
    '@testCases': [
        { "@name": "Peter", login: "Peter Parker", password: "12345", group: "admin" },
        { "@name": "John", login: "John Smith", password: "test", group: "user" },
        { "@name": "Olivia", login: "Olivia Taylor", password: "qwerty", group: "moderator" }
    ],
    'Type in input login': function () {
        var input = $('#login_input');
        act.type(input, this.login);
    },
    'Type in input password': function () {
        var input = $('#password_input');
        act.type(input, this.password);
    },
    'Click submit button': function () {
        var submitButton = $('#submit_button');
        act.click(submitButton);
    },
    'Check Result': function () {
        var header = $('#group_header');
        eq(header, this.group);
    }
};

If there are many test cases, you can save them to a separate JSON file. This is also useful if you are going to share these data sets with other tests and fixtures.

Mixins – We’ve also added support for mixins that represent reusable test code snippets.

This example shows how to use a mixin to log in to an account before viewing or editing a profile. The "Authentication” mixin contains three steps: enter login, enter password, and click the "Submit" button. When you need to log in to an account, just call the "Authentication” mixin in the "Log In" step. This will allow you to re-use code without duplicating it:

'@mixin'['Authentication'] = {
    'Type in input login': function () {
        var input = $('#login_input');
        act.type(input, 'Peter Parker');
    },
    'Type in input password': function () {
        var input = $('#password_input');
        act.type(input, '12345');
    },
    'Click submit button': function () {
        var submitButton = $('#submit_button');
        act.click(submitButton);
    }
};

'@test'['View profile'] = {
    'Log In': '@mixin Authentication',

    'Click link "My Profile"': function () {
        var link = $(':containsExcludeChildren(My Profile)');
        act.click(link);
    }
};

'@test'['Edit profile'] = {
    'Log In': '@mixin Authentication',

    'Click link "Edit Profile"': function () {
        var link = $(':containsExcludeChildren(Edit Profile)');
        act.click(link);
    }
};

Improved Architecture - The TestCafe Core engine has been significantly overhauled to provide greater reliability and performance and to close many outstanding issues.

Test Run Metadata - A new property (this.__workerName) has been introduced to the TestCafe API. This allows your test code to retrieve the name of the current worker.

New Installer with GUI Runner for Mac OS Users - TestCafe for Mac OS now ships with a simplified installer that initializes the testing environment as well as installing the product. There is no need any more to unpack the TestCafe distribution and configure it using the command line.

When you launch TestCafé on the Mac, it will display a specific icon within the menu bar. Click this icon to access a context menu. This context menu allows you to open the TestCafe Control Panel, to copy TestCafe's URL to the clipboard, or to exit TestCafe.

You can also run TestCafe from the application launcher by clicking the TestCafe icon listed in Launchpad.

Get started today

TestCafe is easy to get started with. Just download, install, and you're ready to record your first test.

You can download the new version of TestCafe from our site (http://testcafe.devexpress.com/Download). We look forward to your feedback as you use TestCafe v14.1.2.