CrossBrowserTesting.com

A Design, Development, Testing Blog

Real Device and Browser Testing Strategy for 2020

By Leave a Comment

Check out the latest blog from the CrossBrowserTesting and Bitbar team as we help you get off on the right foot for 2020!

With the new year coming in big strides, it’s time to start thinking about your real device and browser testing strategy for 2020. As you can imagine, testing without a proper plan often causes bottlenecks that hinder your apps’ timely release. Therefore, by defining a strategy, your teams can be more productive and publish highly functional apps regularly.

Even if you already have a plan on hand – next year could be the right opportunity to reevaluate the tools and methods you currently utilize and search for better fitting alternatives. Our goal for this article is to help you put your device and browser testing strategy in the best position for 2020.

Testing On the Right Devices and Browsers

No matter if you’re developing a web app or a native mobile app – you need to test on various real devices, platforms, etc. to make sure your product works perfectly for as many customers as possible. As discussed here, you shouldn’t rely on emulators to get reliable test results.

However, we can find a sheer unending number of possible test environments out there – especially on Android, as they are highly fragmented. That being said, you also can’t blindly pick devices and browsers for testing but need a proper plan. Ultimately, this plan should guide your testers and developers through testing so they can focus on the task at hand.

What does this mean for your testing strategy for 2020?

Depending on where you’re currently at, there are two different key actions you need to take:

  1. If you don’t have a list of devices and browsers to test yet, it’s time to establish one and make it an integral part of your device and browser testing strategy for 2020.
  2. If you already have a list of real devices and browsers that you need to test your app, you’ll need to revise this list to meet the changing mobile device and browser markets.

Use your product’s usage data to determine which devices and browsers your customers prefer. You can also find information in the quarterly report of the most popular mobile devices from DeviceAtlas. Also, look for brow distribution data for your target markets online.

After collecting and analyzing the data, you should be able to create a prioritized list of the must-test devices and browsers with confidence. Also, make sure to take action when older devices and browser versions disappear, and new ones appear and update your list.

Pick a Fitting Framework for UI Testing

As the new year approaches us, it might also be the perfect opportunity to switch to a different UI testing framework – or finally start using one.

Depending on the tech stack, team size, and available roles in your team, some frameworks may fit better than others.

What does this mean for your testing strategy for 2020?

Choosing and working with the right test framework enables your team to get the most out of their testing time, and release functional, high-quality applications. It doesn’t matter whether you’re building a web or mobile app – only the tools you can leverage are different.

Before choosing a framework, you need to make sure to analyze and understand your product’s and team’s needs. According to Capgemini’s World Quality Report 2019-20, 30%¹ of all respondents felt the lack of skills in all areas of testing for QA and test professionals. Hence, you need to make sure you have skilled experts on your team or have the resources to hire and retain them.

UI Test Framework for Web Apps

The framework that worked its way to the top of the most popular testing tools for web apps is Selenium. Although your developers and testers may need a while to get comfortable with Selenium testing and write efficient tests with it, you will save a lot of time and headaches in the future.

Basically, you can use Selenium to automate interactions with web browsers on desktop and mobile devices using different programming languages. Kick-start your automation journey by learning from our Selenium Toolkit here.

UI Test Frameworks for Mobile Apps

For mobile testing, we can find three popular frameworks that all have their pros and cons.

  • Appium – works similar to Selenium and supports both Android and iOS. You only need one codebase, but tests tend to be slower and flakier than tests run with the other frameworks.
  • Espresso – only supports Android and is part of the Android SDK. Thus, tests written with Espresso perform faster than with Appium and are less flaky.
  • XCUITest – only supports iOS testing and is part of the XCode integrated XCTest framework. As a result, tests perform well and deliver reliable results.

Tip: Read our comparison articles on Appium vs. Espresso and Appium vs. XCUITest to help you make a decision.

Make Use of Automation and Parallel Testing

Modern businesses aim to deliver apps in the shortest time possible – this can only be achieved if an appropriate automation strategy is set up.

However, most likely, not everything can be automated, and your team needs to decide which parts to test manually and which to test automatically.

In fact, according to Capgemini’s survey, 50%² of respondents named the “Inability to apply test automation appropriate levels” as one of the challenges that they are currently facing when integrating testing practices into agile developments. Surprisingly, this number hasn’t changed since the last survey in 2018. Therefore, the effective integration of test automation in 2020 will remain an essential part of the whole real device and browser testing strategy for companies.

What does that mean for your testing strategy for 2020?

Especially for testing the user experience of mobile or web apps, manual testing can be of great help. For everything else, try to use automation. However, this shouldn’t cause new bottlenecks, and you need to determine what you need to test how. As a rule of thumb, anything that needs to be checked regularly, or from which you can expect certain results, can be safely automated.

Naturally, this is the same whether you’re testing mobile or web apps. Though, again, the tools which can help you in this endeavor differ.

For example, you could use a tool that automates your test scripts and takes screenshots of your web app on different screen sizes. Mainly for testing mobile apps, it makes sense to select an environment that lets you run tests on multiple devices at the same time (parallel testing). This way, you not only save time by automating tests but also running them more efficiently. You can get the ultimate results out of this by testing on an unlimited number of devices.

Another key driver for businesses in 2020 is AI-powered testing. 16%³ of executives rate the use of AI in test execution activities as their top priority concerning AI plans. More and more businesses will rely on AI for testing in the coming years. Companies that adapt to these new technologies at an early stage have an advantage over companies that ignore them.

Decide Where to Test – On-Premise or in the Cloud

Do you have a device and browser test lab in-house, or do you leverage online-based test services? We can’t possibly favor one option over the other, as the right environment depends on your requirements.

With cloud-based services, your team can run tests anywhere, anytime, and you don’t have to buy and maintain devices yourself. This way, you can save a lot of money and the time of your employees.

Yet, many SaaS platforms don’t meet the modern enterprise security requirements, so some companies can’t use them. Besides, SaaS usually provides out-of-the-box features, so you can not customize the setup to suit your needs.

What does this mean for your testing strategy for 2020?

Considering what we’ve discussed so far, the optimal environment should:

  • be flexible and allow customization
  • support all known test frameworks
  • allow you to run tests in environments provided by Docker images
  • enable you to test on real devices and browsers in parallel
  • be scalable and adaptable to fit your future needs
  • let you use AI-powered testing tools
  • meet the security requirements of your business

Before you can choose the appropriate environment, you need to analyze your requirements – also take your potential future needs into account! You must remain flexible and scalable to survive in the ever-changing technical landscape.

For browser testing, you can utilize our over 2000 real browsers and mobile devices to perform manual, visual, and automated Selenium tests in the cloud, using real devices and browsers.

For mobile testing, you could utilize our friends at Bitbar for the most flexible real device cloud. Execute your tests automatically on as many devices as needed and leverage their real device concurrency functionality. If you need your data to stay on-premises, they’ll help you build a modern local mobile device lab with their enterprise solution.

6 Key Factors That Improve Cross Browser Testing

By Leave a Comment

Our friends over at Bitbar recently wrote a fantastic blog about 6 key factors that can improve your cross browser testing. Take a look below!

If you start thinking about testing after the completion of your web application, you’re already too late. The decisions you make before and during development impact and shape testing more than one might think. And not involving testing from the start may lead to some severe problems that can prevent a timely release. In particular, cross browser tests can be a bottleneck. That’s why we’ve put together a list of 6 essential factors that can help you improve cross browser testing and plan ahead!

1. New Isn’t Necessarily Better

Web technology advances swiftly, and innovative features are being introduced with each new browser version. We know the urge to use fancy new tech in a project can be massive. But you have to keep in mind that new doesn’t automatically mean better! Some of these functionalities are still in beta, experimental, or not yet supported in all major browsers. As a result, you deliberately introduce cross browser variations into your product and face a more difficult time testing.

Our advice:
If using a particular new technology that brings no real value to your users, think twice about integrating it into your app. Avoid unnecessary cross browser variations in your web application.

2. Stop Reinventing the Wheel

When your team plans the technology stack for your next web-based project, see if you can use existing frameworks, libraries, and so on. Use React, Vue.js, or Angular, and don’t depend on Vanilla JS in your app too much. You not only save a considerable amount of time in development but also during testing! The same is true for CSS frameworks like Bootstrap or Foundation. They are already tested on a wide range of browsers, and devices, saving you time testing your app.

Our advice:
Go with established frameworks and libraries to write your CSS and JavaScript code. They are usually thoroughly tested, and you can safely use them in your project. That way, you reduce the code you need to write and validate. Additionally, you can reuse already tested components, further reducing testing time.

3. Only Test Clean Code

Have your developers follow coding and style guidelines and let them deliver clean, consistent code. As a result, your code is less error-prone, and bugs can be found more efficiently. Additionally, anyone in the team can work on the source code regardless of who wrote it. If you don’t have guidelines or don’t have the resources to create them, use existing ones! Why not use style guidelines from other companies such as the Airbnb React/JSX Style Guide?

Our advice:
Using and enforcing coding and style guidelines makes developing and testing your app more straightforward. Ensure your team delivers clean code by having your developers install code linters in their IDEs. These linters should automatically check the written code against your guidelines and find minor errors, like spelling mistakes or missing brackets.
As a result, cleaner code – without the usual minor errors – arrives at your testers without investing too much extra time!

4. Make Use of Automation

Test automation can help you shorten testing time significantly so you can release updates and bug-fixes more often. Although automating cross browser testing can be a bit tricky in some places, we can’t not take notice of the benefits. Automating tests also mean that your devs and testers can eliminate repetitive and tedious tasks from their to do list. This gives them more time to create new features and perform necessary manual tests.

Our advice:
First, see what you can automate and which parts or browsers you should manually test. Here are some ideas to get you started:

  • Follow TDD for your JavaScript code. Write and automate unit tests by using a framework like Mocha.
  • Use browser automation and inspection tools like Selenium WebDriver to automate your end-to-end tests.
  • Take a look at the different tools in the Selenium suite that help you test your apps. Learn more about the suite in our blog article series about Selenium.
  • Make sure your app looks immaculate on different screen resolutions by automating image regression testing. Cloud-based services like CrossbrowserTesting provide the functionality you need.

5. Test on a Large Scale

Automating your tests is nearly worthless if you don’t run them on real browsers and devices, preferably at the same time. Sure, test automation saves time, but if you have to run these tests on one device and one browser at a time, the process takes forever. Also, keep in mind that your users are real people who use real computers, smartphones, tablets, and browsers. Can you imagine anyone using emulators to browse the internet? Consequently, make sure to run your automated tests on multiple real-world devices and browsers in parallel. This lets you test more efficiently so you can spend more time on other things.

Our advice:
Choose a cloud-based testing service, that lets you run tests on multiple real-world devices and browsers simultaneously. As a result, you can run more tests in less time while ensuring your app works as intended in a variety of real-world scenarios. In the end, you can release new features to your customers with more confidence.

6. Bring Continuous Integration into Play

If you’ve integrated all previous steps into your workflow to improve cross browser testing, then Continuous Integration (CI) is the icing on the cake. If you are unfamiliar with CI, be sure to read some information on this topic, as you will undoubtedly benefit greatly.

In a nutshell, CI works as follows: Whenever a developer pushes code into the project’s central repository, several jobs start running. Usually, these jobs perform a variety of tests. In the end, CI helps you determine whether the committed code is releasable – only commits that pass the validation get accepted.

Our advice:
Using CI does not require your developers to start the tests manually, so no test can be forgotten. Additionally, only perfectly valid code is accepted for release. Not only does CI automate repetitive processes, but it also makes sure your app performs at its best. You can find many different CI tools that meet your specific needs. For example, use Jenkins if you want to run jobs on a dedicated server. Or use TravisCI if you wish to use a cloud service instead. However, many, but not all, cross browser tests, can be integrated into a CI job. Consequently, you need to pick the service that’s most compatible with your other tools! Check out the integrations CrossBrowserTesting supports!

Wrapping up – Improve Cross Browser Testing by Using the Right Tools

Test early and often, use tools that fit your needs, and make sure you run automated tests on real devices as often as possible! Use different tools and services that match your situations and needs. Don’t hesitate to use a cloud-based service to make your test processes more efficient and resource-saving.

Join Over 600,000 Testers & Developers And Start Testing Today

Learn more Free Trial