Automation testing frameworks are basically the groundwork for automating test scripts. Different automated test frameworks offer different rules, guidelines, and procedures to help efficiently develop and execute tests.
The best choice for each tester is, of course, dependent on their preferences and needs. In this post, we’ll differentiate the multiple automation testing frameworks and open source automation tools that sit on top of Selenium and help you make the choice for better testing & development.
No Reason to Make Your Own
While it is possible to create your own framework, there are plenty of pre-defined automation testing frameworks to simplify test execution and help you meet your browser automation goals.
“I’ve seen some teams take it to the extreme by building their own, elaborate test framework from scratch,” said TestTalks Founder Joe Colantonio. “This drives me crazy, because they could have easily made use of existing open-source tools and libraries that would have met their needs without writing any code — in most cases with better results.”
Rather than wasting time trying to build an adequate test framework, check out some of developers’ trusted choices. These resources make it easier for your team to collaborate for better quality, security, speed and readability.
The automated test frameworks that we discuss further along in this post require a prior knowledge of foundational browser automation tools. Selenium, Appium, Capybara, and Watir are the original open source tools that provide a basic infrastructure for other frameworks to add to for increased features, capabilities, and efficient test execution.
Selenium WebDriver is a tool and API framework for web browser automation. The big selling point of WebDriver for developers is that it allows them to test in any preferred programming language, which also makes it one of the most popular open-source automation frameworks.
Appium, on the other hand, is an open source automation framework for mobile apps. The framework allows native, hybrid, and mobile testing and drives iOS, Android, and Windows apps using WebDriver. Just like Selenium, it’s a cross-platform tool that can run scripts in any language.
Capybara is a library written in Ruby and is another framework that makes it simple to automate web application testing using an intuitive API that simulates an actual user. It offers a user-friendly DSL (Domain Specific Language) and locates elements in the DOM (Document Object Model).
Watir (pronounced like water) is another family of Ruby libraries for automating web browsers in an app no matter what technology it’s been developed in. Watir only supports Internet Exploreer on Windows, but Watir-WebDriver (which is powered by Selenium) supports Chrome, Firefox, Internet Explorer, Opera and can also run in HTMLUnit headless mode
What all these tool frameworks have in common is that they communicate directly with the browser to perform tasks in the same way that a human would, so executing automated tasks is as accurate as manually testing them.
Automation Testing Frameworks
Karma – Karma is a test runner that is particularly helpful for integration testing and providing an environment with instant feedback. It allows you to test code on real browsers as well as headless browsers like PhantomJS where you can generate, display, and watch test results.
Robot Framework – Robot Framework is implemented using Python as its core but can also run on Java and other languages through a remote interface. It utilizes keyword-driven testing, which makes it simple for testers to perform either normal test automation or ATDD. The modular architecture can be bundled with self-made test libraries in addition to the many included tools and libraries.
Nightwatch.JS – For end-to-end testing, this easy to use Node.js based framework uses Selenium WebDriver’s API to perform commands and assertions on DOM elements. Nightwatch.js also has a built-in command-line test runner that can execute tests sequentially or in parallel. Learn more about testing with Nightwatch here.
Cucumber – Cucumber is a Behavior Driven Development (BDD) framework that lets you execute plain-text functional descriptions as automated text. It’s written in Ruby but also tests in other languages including Java and Python. The point of Cucumber is that even people without highly technical knowledge are able to read the code, making it more accessible for collaboration across different business departments.
Lettuce – Another BDD test framework but for Python, Lettuce allows you to execute automated tests from plain-text descriptions that almost anyone can understand. It’s actually based off Cucumber, which has a few more features, but Lettuce aims to cover the most common tasks on BDD and is a fairly straightforward option. Learn more about testing with Lettuce here.
Protractor – Protractor is an end-to-end test framework for AngularJS applications that can simulate a user’s interaction with an Angular application running in a browser or mobile device. It has added hooks for running in Selenium that enable you to get more data and benefit from other Angular features. Additionally, a really cool feature is that it lets you separate your tests into various test suites, which allows you to enter a single command and run a group of test cases for different scenarios. Learn more about testing with Protractor here.
Tape – Tape relies on a very few number of APIs to provide a framework that’s good for producing test harness for node and browsers. Because it maintains a minimal core and is fairly low-level, additional features are usually added through another module and can be used for both unit and integration testing — some even feel it’s superior to Mocha.