11 Min. Read
While software testing has become a critical, growing part of the development lifecycle, it has traditionally relied on large teams executing manual test cases. This has changed in a recent years as teams have found the answer to facilitate a faster deployment cycle: test automation.
While QA teams are beginning to shift their strategy to be more inclusive of automation practices in order to increase speed, efficiency and coverage of the application under test, there’s still some who may be deciding if automation is truly the right choice for them or be struggling to transition their process.
To answer the questions teams are facing when it comes to test automation, we’ve summarized why we automate, when to use it, and how you can have the best automation strategy.
What Is It?
Automated testing uses the assistance of tools, scripts, and software to perform test cases on various levels of the application. To put it simply, automated testing takes pre-defined scripts or records an action and allows the tester to play it back repeatedly, and at a pace which is not determined by human resource limits. This is in opposition to manual testing, where a tester must repeat the same actions of a previous test case to check it after a code change or make sure it functions in a different browser configuration.
When Do I Need Automated Testing?
The short answer to this is that it’s ideal to have an automated script in place whenever you feel like you are repeating yourself with a test case more than once or twice. But the long answer is a little more nuanced.
Manual testing will always be a mandatory first step for creating original scripts. Exploratory testing and manual testing are crucial steps for young products and for setting up your eventual automated test cases. However, as your development strategy adopts a more agile development framework, your testing strategy will eventually have to scale in order to match that.
Of course, deciding when to automate testing doesn’t have to be rocket science. One tester’s blog details his fool-proof method for knowing when to switch to automation. “I call it the ‘I’m Bored’ heuristic. I don’t like to be bored, so when I get bored, I automate what I’m doing.”
When you feel like you are repeating a test over and over on a daily, weekly, or even monthly basis it may be time to automate it. You will improve your overall testing ROI and time by investing in the upfront cost of automation.
If you are still finding new bugs and defects when manually testing the feature or application, this process is probably not yet ready for automated testing. It is important to remember automated testing will only test what the script says, so it will not go “exploring” for edge cases or anomalies.
Test automation really shines though when conducting regression testing, or running a series of tests against an application each time new code is introduced to it. When we automate common test cases on a mature product or feature we can ship code faster which can be a huge advantage when bringing products to market.
Let’s say our development team commits a new feature and ships it over to QA. If the product is large enough, this may still mean 1 week full week of manual testing -- even for a larger team of 20-50 testers. That also means that in the feedback loop, developers have to wait a full week for bug reports and artifacts before being able to fix them. With test automation, we can cut 1 week of testing down to just a few hours, and massively decrease the feedback loop for the development team.
Manually testing regression tests can become monotonous and tedious, especially when these changes are made to code daily or hourly. You end up spending more time ensuring the basic functionality of your web application each time a small change is made to code and less time actually making valuable improvements.
Even the length of a test has an impact on whether to automate it or not. While unit tests are primarily automated, longer end to end acceptance testing sometimes are not. If you plan on running a 10 minute test even just a few times per month, it may be worth trying to automate some of it allowing you to run it faster and more often.
Of course, dull and laborious work like this will also lead to more human error. Manual testing becomes increasingly inefficient and will most likely end up costing you more in terms of ROI, resources and productivity.
Automating Browser Testing
You can already see how easily manually testing can become counterproductive. To make things more complicated, scalability becomes an issue when you start introducing new environments and realize how impractical it is to test one configuration at a time.
It’ll likely take hours or days to sequentially run a manual test through Chrome, Safari, Firefox, Opera, Edge and Internet Explorer on a PC, then repeat the same tests on the numerous other devices needed to ensure your procedure is complete and comprehensive.
By now, you should know that you have to test on as many hardware/software configurations as possible to make sure your web application is optimized across browsers to be inclusive of the largest possible consumer base.
Additionally, while buying all of these devices may have been realistic at one time, today’s diversification of devices, operating systems, browsers and browser versions mean that a tester would need to spend an enormous amount of money on hundreds or thousands of configurations. On top of that, there are new updates being released all the time.
Luckily, secure, third-party clouds allow automated testing across a wide bandwidth of devices and browsers to make sure that developers are able to effortlessly test the design, performance, and functionality of their web applications in parallel for faster and more accurate results, while saving money in the long run.
Tips for Best Practices in Automation Testing
Don’t expect 100 percent automation
While automation may seem like the best method for the majority of your testing needs, there is always a place for manual testing. In fact, there are few case where you should execute an automated test without first doing an exploratory test. This means you still have to design thorough and comprehensive test cases before automating them. Additionally, along the way, you’ll likely find that some tests are better suited for manual testing.
Continue to hire talented testers
Moving to automation doesn't mean you can get rid of QA staff. The point of automated testing is that allows more time and range for your testers to test. They still need to well informed in the world of programming, development, and testing, or your strategy will fall short. Also keep in mind, that browser testing platforms like CrossBrowserTesting allow sharing results so that developers, manual testers, and automation engineers can better collaborate, which can be an essential to a strong testing team.
Decide how you want to automate
Making the transition from manual to automated testing is one we fully support. However, there are still a few aspects you need to decide on. Especially as you’re just beginning to move to automation, you’ll have to pick and choose which tests you want to automate first. Consumer facing web applications, high traffic pages, regular regression tests and and high risk browsers should be prioritized.
Set automation goals
As we discussed, automating your testing is a great opportunity to save time and money while improving the quality of your web application and design. However, the difference automation will make in your approach will depend on the size of your team and the size of your company. Setting individual goals and tracking accomplishments when you begin automating will help you measure success and evaluate best practices for moving forward.