Automation testing frameworks are basically the groundwork for automating test scripts. Different automated test frameworks offer different rules, guidelines, and procedures to help efficiently develop and execute tests.

The best choice for each tester is, of course, dependent on their preferences and needs. In this post, we’ll differentiate the multiple automation testing frameworks and open source automation tools that sit on top of Selenium and help you make the choice for better testing & development.

No Reason to Make Your Own

While it is possible to create your own framework, there are plenty of pre-defined automation testing frameworks to simplify test execution and help you meet your browser automation goals.

“I’ve seen some teams take it to the extreme by building their own, elaborate test framework from scratch,” said TestTalks Founder Joe Colantonio. “This drives me crazy, because they could have easily made use of existing open-source tools and libraries that would have met their needs without writing any code — in most cases with better results.”

Rather than wasting time trying to build an adequate test framework, check out some of developers’ trusted choices. These resources make it easier for your team to collaborate for better quality, security, speed and readability.

Tool Frameworks

The automated test frameworks that we discuss further along in this post require a prior knowledge of foundational browser automation tools. Selenium, Appium, Capybara, and Watir are the original open source tools that provide a basic infrastructure for other frameworks to add to for increased features, capabilities, and efficient test execution.

Selenium WebDriver is a tool and API framework for web browser automation. The big selling point of WebDriver for developers is that it allows them to test in any preferred programming language, which also makes it one of the most popular open-source automation frameworks.

Appium, on the other hand, is an open source automation framework for mobile apps. The framework allows native, hybrid, and mobile testing and drives iOS, Android, and Windows apps using WebDriver. Just like Selenium, it’s a cross-platform tool that can run scripts in any language.

Capybara is a library written in Ruby and is another framework that makes it simple to automate web application testing using an intuitive API that simulates an actual user.  It offers a user-friendly DSL (Domain Specific Language) and locates elements in the DOM (Document Object Model).

Watir (pronounced like water) is another family of Ruby libraries for automating web browsers in an app no matter what technology it’s been developed in. Watir only supports Internet Exploreer on Windows, but Watir-WebDriver (which is powered by Selenium) supports Chrome, Firefox, Internet Explorer, Opera and can also run in HTMLUnit headless mode

What all these tool frameworks have in common is that they communicate directly with the browser to perform tasks in the same way that a human would, so executing automated tasks is as accurate as manually testing them.

Automation Testing Frameworks

WebDriver.io – WebdriverIO makes it easier to setup, manage, and write Selenium tests in JavaScript by implementing the WebDriver protocol API. The custom libraries include a test runner for integration testing and provides a plethora of shortcuts to make code cleaner and easier to read. Learn more about testing with WebDriverIO here.

Istanbul – Istanbul is another JavaScript test coverage tool with line counters that make it easy to track how well your unit-tests exercise your codebase and generate coverage reports. Because of this, Istanbul is particularly helpful when it comes to unit testing and code coverage. Not to mention, it works well with most other JavaScript automation testing frameworks like Mocha and Ava.

Karma – Karma is a test runner that is particularly helpful for integration testing and providing an environment with instant feedback. It allows you to test code on real browsers as well as headless browsers like PhantomJS where you can generate, display, and watch test results.

Robot Framework – Robot Framework is implemented using Python as its core but can also run on Java and other languages through a remote interface. It utilizes keyword-driven testing, which makes it simple for testers to perform either normal test automation or ATDD. The modular architecture can be bundled with self-made test libraries in addition to the many included tools and libraries.

Nightwatch.JS – For end-to-end testing, this easy to use Node.js based framework uses Selenium WebDriver’s API to perform commands and assertions on DOM elements. Nightwatch.js also has a built-in command-line test runner that can execute tests sequentially or in parallel. Learn more about testing with Nightwatch here.

Jest – Jest is the test framework Facebook first used to test Javascript and is now used by other tech giants such as Instagram and Twitter. It’s known as a “zero configuration testing platform” that has an easy setup, instant feedback, and snapshot testing. Jest does everything from providing a test environment, structure, and assertion functions to generating code coverage reports, making it increasingly popular among testers outside Zuckerberg’s team especially since it’s so easy to get started with.

Jasmine – As a simple JavaScript testing framework for browser and node.js, Jasmine has an easy setup and broad Angular support with assertations and test double utilities, while it’s BDD style API makes it similar to Mocha

Cucumber – Cucumber is a Behavior Driven Development (BDD) framework that lets you execute plain-text functional descriptions as automated text. It’s written in Ruby but also tests in other languages including Java and Python. The point of Cucumber is that even people without highly technical knowledge are able to read the code, making it more accessible for collaboration across different business departments.

Lettuce – Another BDD test framework but for Python, Lettuce allows you to execute automated tests from plain-text descriptions that almost anyone can understand. It’s actually based off Cucumber, which has a few more features, but Lettuce aims to cover the most common tasks on BDD and is a fairly straightforward option. Learn more about testing with Lettuce here.

Protractor – Protractor is an end-to-end test framework for AngularJS applications that can simulate a user’s interaction with an Angular application running in a browser or mobile device. It has added hooks for running in Selenium that enable you to get more data and benefit from other Angular features. Additionally, a really cool feature is that it lets you separate your tests into various test suites, which allows you to enter a single command and run a group of test cases for different scenarios. Learn more about testing with Protractor here.

Mocha – Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser. It’s great for unit and integration testing, and self-described as making asynchronous testing “simple and fun.” It also includes test coverage reports and use of most JavaScript assertion libraries — most popularly Chai. Learn more about testing with Mocha here.

Tape – Tape relies on a very few number of APIs to provide a framework that’s good for producing test harness for node and browsers. Because it maintains a minimal core and is fairly low-level, additional features are usually added through another module and can be used for both unit and integration testing — some even feel it’s superior to Mocha.