testing frameworks Archives | CrossBrowserTesting.com

Must-Have Components of Your Successful Web App Test Strategy

February 18, 2020 By Sarah Mischinger Leave a Comment

In the early days of the WWW era, websites were rather static and simple. Today though, many web applications are quite complicated and more so what people think of as “software.” This complexity comes with a lot of responsibility since you want your app to perform as expected and please your users. This is why proper and thorough testing practices are mandatory for releasing high-quality web apps.

Pave the Way for More Efficient Test Practices During Development

Not thinking about testing during the development process can lead to unnecessary frustrations for your testers. Because testing web applications is no longer optional, you should consider testing requirements right from your project’s start.

Here’s what you can do during development to make testing more efficient:

Leverage Your Past Experiences

Suppose you know that the component you are creating may be faulty in a particular browser if it’s not programmed in a certain way. If you share this knowledge right away and write the functionality accordingly, you can save time and headaches when testing.

Tip: Share your experience with colleagues and make comments in the code if you know about error-prone components.

Use a Solid Foundation

Instead of reinventing the wheel, use frameworks and libraries that are already set up and optimized for your project. For example, Bootstrap for your CSS, Angular, or Vue.js for your JavaScript, Laravel for your PHP backend, etc. With these tools, you can make sure that you are more efficient and end up with fewer errors in your code. Besides, most frameworks already include guidelines for writing code, tests, and so on. As a result, you and your team can save a lot of time in development and testing, as following these rules will help you build a clean and consistent codebase.

Reuse Code Snippets From Other Projects

If you’ve already done something, why not do the exact same thing again? By reusing previously tested code from other projects, you can save time in development and testing. Of course, you still want to write tests for these snippets, but you can expect them to be less of a problem for you.

Clean Code Saves Everyone’s Day

Writing clean code should be a no-brainer since it’s not only easier to test, but also improves your team’s productivity and motivation. If you need to work with someone else’s code, you sure don’t want to spend too much time deciphering what’s going on there. And clean code can help make it more readable and understandable.

The tricky part of achieving clean code is that every single programmer needs to contribute. A great way to improve code quality is to enforce style/code guidelines and let developers install linters in their IDEs that automatically check the written code for spelling mistakes and wrongly styled code.

Be Prepared and Identify Browsers and Devices to Test

While your web app must be all-around highly functioning, it is critical to make sure that it looks and works correctly on the many different browsers and devices, which is called cross-browser testing. After all, the interface of your web app is what users see and interact with. If you present users with a faulty UI, you may be missing out on some new loyal customers.

Before you start cross-browser testing – or any other tests that you must conduct in a browser – you first need to know where to test. For this purpose, you should work with your app’s existing analysis data and determine your users’ preferred browsers and devices. If you don’t have such data, you can find general figures and statistics on the internet.

Based on the results, you then create a matrix with the browsers to be tested, their versions, and various devices such as smartphones and tablets. Also, make sure to make this matrix an integral part of your web app test strategy and update it from time to time, so you don’t miss out on anything new!

Next, evaluate the environments in your matrix and identify the troublemakers, i.e., which are the most demanding to test on and which are more prone to errors? Make sure to test on these first and work your way up to the easier ones. If you do it the other way around, you will likely break previously working code, and then have to test everything again.

Which Test Types Should You Use?

As mentioned earlier, web apps tend to be complex and need to be thoroughly tested before you ship them to your users. Let’s take a quick look at the most essential tests you should include in your web app test strategy to produce a high-quality product your team can release with confidence:

Functionality Testing

Does your web app work as specified? You can make sure of this by performing a variety of tests:

Unit tests in your frontend and backend code. For example: Does a specific input produce the expected output?
Database integrity tests. For example: Do database operations such as updating or deleting rows cause unwanted side-effects?
HTML and CSS checks. For example: Does your code comply with the W3C standard, and are your elements accessible via keyboard?
Frontend functionality tests. For example, do forms and other elements that the user can interact with work as intended? A great tool for this kind of testing is Selenium, which you can learn more about in our webinar here.

Usability Testing

Your team has seen your web app every day for some time now, so it’s wise to get a fresh pair of eyes involved and check your web apps’ usability. You can do this by inviting a small group of external testers that match your app’s target audience and watch them use your application. As a result, you can see what is working well and which areas still need improvement.

Interface Testing

Most web apps get their data from the server via an API. Of course, you need to make sure that both endpoints are individually tested well, but also that the right data is being sent and retrieved. Besides, this is the perfect opportunity to review any error messages from your web application to ensure that they are accurate and understandable.

Compatibility Testing

As discussed earlier, you need to make sure that your app works on different browsers, major browser versions, and your users’ favorite devices. In contrast to functional tests, cross-browser testing mainly focuses on checking whether the user is presented with a consistent experience, regardless of the technology used. Manually performing these types of tests can be tedious. However, testers can save a lot of time by creating and running automated tests with Selenium, a toolbox for browser automation.

Performance Testing

Your users and search engines hate slow websites. Therefore, it is extremely important that you continuously try to improve the speed of your web application. However, other factors can impair the performance, which is why we recommend the following:

JavaScript and CSS: try to minimize your JS/CSS code and serve it in one file if possible. Also, check that you only load code that your app uses!
Assets: Always remember to keep your pictures, videos, etc. as small as possible – compress, compress, compress!
Backend code and database queries: Optimize your API code and database queries to keep waiting times for requests as short as possible.
Stress tests, load tests, and capacity tests: To improve and scale your app in the future, you need to know its metrics: What is the maximum number of users who can use your app at the same time? What does it take to make the system fail? How much data can the server process at the same time?

Be sure to check out LoadNinja to get accurate load tests in just minutes, using real browsers in the cloud.

Security Testing

Not only finance and e-commerce web apps have to be tested for security gaps! Take extra time and let testers look for loopholes and other vulnerabilities. Pay special attention to:

SQL Injection
Cross-Site Scripting (XSS)
Broken Authentication
Cross-Site Request Forgery (CSRF)
Components with well-known vulnerabilities
Encryption of sensitive data
Cookie and Session handling

Conclusion

As you’ve seen, there are many things to consider to ensure your web app is thoroughly tested. We recommend that you create a web app test strategy that helps you keep track of your tasks and makes testing more straightforward for the entire team.

Since testing can be quite time-consuming, you should also look for tools that can help you be more productive. Give our cross-browser test services a shot and see how we can help you deliver high-quality apps to your users more frequently.

Selenium vs. Cypress: Is WebDriver on Its Way Out?

June 12, 2018 By Alex McPeak 2 Comments

There’s a new kid on the block for open source test automation tools, and everyone’s talking about how Cypress may be an alternative to Selenium.

But while some have posed that Cypress signals the end of an era, we’re not so sure that Selenium is going anywhere anytime soon.

To better understand the similarities and differences between these two tools, let’s take a better look at what they both do.

Selenium

Selenium WebDriver is the open source automation tool we all know and love. With a history starting from the humble beginnings of IDE and transforming into Selenium 2.0 a.k.a. Selenium WebDriver, Selenium has become a foundation of many modern software teams’ test automation toolkits.

Most are familiar with Selenium’s use — it makes direct calls to the browser using each browser’s native support for automation. By writing a script in Selenese using the compatible programming languages, you can run tests that work as if you were driving the browser yourself.

While the learning curve can be steep for Selenium and requires the knowledge of at least one programming language, it has a vast community of active users and even boasts an annual user conference.

One of the foundations of Selenium is the ability to test on the most popular browsers, making it essential to cross-browser testing.

As testing with Selenium has grown in popularity, it’s not only become a basis of many automation strategies but has also become an integral part of many third-party tools such as CrossBrowserTesting. Once you know how to use Selenium, you can use these cloud testing tools to access thousands of browsers and leverage parallel testing to cut down on testing time.

Cypress

So how is Cypress different from Selenium, and how is it the same?

First, Cypress is said to have an easier onboarding than Selenium, where all you need to do is install the .exe to get up and running sine all the drivers and dependencies are automatically installed. With Selenium, on the other hand, you need to spend a little more time getting everything up and running.

One of the advantages of Selenium is that it supports most of the major programming languages including Java, Perl PHP, Python, Ruby, and C#, so if your team is already using one of those languages, Cypress may not be ideal since it only supports JavaScript.

In addition, since Cypress tests are written using Mocha and Chai, the syntax echoes what most Javascript users understand. This means that if you work with Javascript, it will be especially easy to start using Cypress. However, if you’re more familiar with another language, it will be fairly complex.

Cypress almost acts as a “freemium” model. Similiar to Selenium which is entirely open sourced unless you integrate with a paid tool, you can also add to Cypress’s core capabilities by leveraging the paid dashboard which includes a test runner that allows you to easily debug tests with DOM snapshots and includes video storage for private projects.

Selenium and Cypress differ in the way of their architecture, as well. Selenium is made up of bindings, or libraries, and the WebDriver, which controls the browsers. These two components work through the JSON network.

Alternatively, while Cypress is used for UI testing, it uses its own unique DOM manipulation and runs directly in the browser with no network communication. Cypress is often cited as a developer-friendly test automation tool for this reason since this often causes frustrations for them in Selenium.

Having a different architecture on top of an Electron app also allows Cypress to be so close to the web application under test that we can do things that you can’t do in Selenium such as stubbing DOM APIs.

One of the biggest gaps between Selenium and Cypress is that while Selenium runs against different browsers, as mentioned before, Cypress only supports Chrome. Selenium’s browser support is often integral to those who use it, making Cypress severely lacking in the area of cross-browser testing.

For software teams that find importance in checking their website on multiple browsers instead of just leaving their code up to chance, finding their way around this deficiency could prove challenging to Cypress users.

Cypress vs Selenium

Cypress and Selenium serve a similar purpose that is achieved in two different ways.

Selenium is a library but requires a unit testing framework or a runner plus an assertions library to build out its capabilities. Cypress provides a robust, complete framework for running automated tests but takes some of the freedom out of Selenium by confining the user to specific frameworks and languages.

If you don’t know what you’re doing when it comes to building out a test automation framework, Cypress may be a good option.

Another thing to keep in mind is that if you look at how Cypress is built, it is largely a unit testing tool that is will fit best in Javascript-focused development teams. Once you stray from these details and your team decides to experiment with other methods of test automation, you’ll find that Selenium can better accommodate those growing pains.

As a tool made by developers for developers, it’s less a replacement for Selenium and more so a way to introduce developers to test automation.

Like most instances when it comes to test automation, the tool you pick and the method you follow will largely depend on what your team decides and what works best for your organization. However, the notion that Selenium is on its last legs is misled.

If you’re still confused as to the differences between Cypress and Selenium, Automation Rhapsody provides a really great breakdown of how each performs in different categories.

The Criteria to Consider for Choosing JavaScript Testing Frameworks

January 18, 2018 By Alex McPeak 1 Comment

Since JavaScript is widely considered the “language of the web”, it comes to no surprise that test automation frameworks for the language are the most plentiful and most popular. However, this can also make it more difficult to choose the right ones for your team.

By considering the attributes of different frameworks, it’ll become more clear as to which one is the right fit for your current workflow. Follow our guide for selecting JavaScript testing frameworks in order to make the right choice that will lead your organization to lasting success.

Criteria for Selection

When choosing JavaScript testing frameworks, there are going to be many options and different factors to consider. The following criteria are going to most influential in your decision:

E2E or Unit – An end-to-end framework is specialized to run through an entire human interaction on a website. A unit testing framework will give you different capabilities since it’s for testing single functions.

Assertions – Every end-to-end testing framework needs an assertion library. Assertions make sure that our tests return expected results. When choosing a framework, you also need to have a reason for choosing which assertion library you’re going to use and why you’re going to use it.

BDD/TDD Friendly – Some frameworks are not as easy to use with a BDD Girken style language as others, and you will want to stay away from those if your organization is following a certain style. Choose BDD or TDD and pick a framework that makes sense for your team and fits your organizational flow.

Asynchronous testing – When testing JavaScript, it’s important to worry about asynchronous testing and promises. Some frameworks will do this better out of the box than others, which will take longer to grab a promise.

Visual Testing – Some frameworks are better suited for visual testing and capturing browser screenshots than others, and some actually come out of the box with it. This will give you the ability to look at visual differences in your regression suite.

CI Integrations – If your team is following Continuous Integration and Delivery practices, it’s no surprise that you’ll also probably want a framework that integrates with Jenkins, Team City, and Bamboo. Most of the CI systems run on a standard runner, so they work hand in hand with many of the larger JavaScript frameworks to make sure that they work together as well as possible, but it’s still something you want to double-check when choosing.

App Language – The app language criteria has risen recently with the dynamic nature of JavaScript. Previously it was fine to have test code and production code not be the same language. Now, with the way app languages are going in frameworks, we want to pick a framework that is best suited for the actual language that our application is written in.

Speciality – You may be searching for a framework with a certain specialty. Is it easy to set up and get running out of the box? Is there a clean and simple syntax? Is it compatible with Angular testing or a Selenium binding? Does it have good customization? Is it flexible, and can it fit into other assertion libraries, code coverage tools, and frameworks? These are all considerations you’ll want to assess with your team.

Cloud Support – Pay attention to which frameworks work with cloud providers because not every framework will out of the box. Running tests in the cloud will allow you to increase test coverage and speed while eliminating hassle, cost, and maintenance, so compatibility should be part of your consideration.

Size of the Community – The size of the community matters because it’s where you look for answers and find tutorials and documentation, but this also depends on personal preference. For example, one framework may be good because it’s newer so more people are talking about it, but some might prefer a more mature framework that already has a good community base and documentation.

Top JavaScript Frameworks

To get a better idea of specific frameworks and how they meet these criteria, we listed some of the top frameworks used by teams that work in JavaScript.

Assertion Libraries

Jasmine – Jasmine is a BDD/TDD compatible unit testing framework with an out-of-the-box assertion library, an easy to read syntax, and broad Angular support.
Should.js – Should.js has a BDD assertion library and often pairs with Mocha.
Assert.js – Assert.js works with Angular and often is used with Chai to provide a more simple set of assertions.

Unit Testing Frameworks

Mocha – The self-proclaimed simple, flexible and fun framework is one of the most popular choices. It depends on other frameworks like Chai for assertions, but it is BDD/TDD compatible and includes a runner. Mocha is a very flexible, extendable, and feature-rich configuration that’s also good for asynchronous testing.
Chai – Chai often goes hand-in-hand with Mocha as the most popular assertion library used alongside it. It is also BDD/TDD compatible.
Tape – Tape is also a newer framework for unit testing that some might uses as an alternative to Mocha. It uses a traditional syntax, making it very simple to use.
Jest – Jest is a runner for unit testing that can adapt to any JavaScript library or framework. It comes with its own assertion library and supports BDD. It’s a newer framework, so it’s less widely used than others.

E2E Frameworks

Protractor – Protractor has risen in popularity as a good choice for Angular apps because it puts a 10-second wait at the end of each command for a JavaScript single page app. This is an example of something you’d want to consider when thinking about app language. It would also depend on assertion libraries from another framework.
PhantomJS– Phantom.js deals with headless browsers but has had a drop-off in popularity since Chrome came out with their own and real browsers have been optimized to be comparably fast to headless ones.
Webdriver.io – Webdriver.io includes visual testing with WebDriver CSS. It also specializes in customization and is a flexible framework where you can fit assertion libraries, code coverage tools, and other frameworks.
Nightwatch – Nightwatch is an E2E test framework with its own assertion library and runners. It’s an easy, out of the box setup that a simple syntax, which couples together with a configuration file. Nightwatch is also a custom implementation of W3C WebDriver and works with the JSON wire protocol.
CapserJS – Casper.js also works for headless browser testing with a runner for Mocha, Jasmine, and Chai for end-to-end testing. It has added capabilities like assertions and testing utilities that PhantomJS lacks.
Karma – Karma is a test runner created by the AngularJS team, making it often the preferred test runner for apps written in Angular. It’s frequently used alongside Jasmine and integrates with popular CI servers as well.
Cucumber – Cucumber is a popular BDD framework for functional testing using the Gerkin syntax that works with a lot of these tools and runners, like WebDriver.io. It’s being used more and more as more teams use BDD.
InternJS – Intern is a fairly new framework that’s BDD/TDD compatible. As a test runner, it also integrates with CI tools like Jenkins and TeamCity as well the cloud. Don’t let the immaturity of this framework fool you, it has comprehensive documentation and a strong community.