CrossBrowserTesting.com

A Design, Development, Testing Blog

6 Questions to Ask Yourself Before You Decide What Test Cases to Automate

By 1 Comment

decide what test cases to automate

Even just the idea of test automation is enough to get the attention of most software teams. But while the idea of faster testing may seem like the silver bullet, there’s a time and place for automation.

Before you jump in head first, you should evaluate how automation is actually fitting into your organization. By asking these questions, you can get a better idea as to whether automation is the right move, or if you should take another route.

  1. What’s the risk involved? If you’ve performed a risk analysis of your application, you probably have a good idea of which test cases are the most important to check in each code change and which are a lower priority. Consider how human error could affect test data if it’s not automated. Think about how different test cases that may critically impact the customer and/or impact internal team downtime if broken. Additionally, you might want to automate high traffic page, features that depend on security, and problematic areas and other functions that rank high on your risk analysis. Angie Jones talks about this in “Which Test Should I Automate,” which she’ll be presenting at SmartBear in a meetup on June 26. Register now or tune into the livestream.
  2. How often am I going to run this test again in the future? Running the same test more than once may seem repetitive and tedious, but if you’re not actually going to be running it frequently, it might not be worth it to automate. For example, if you know you’re going to have to test a feature two or three times but won’t be keeping it as a permanent part of the application or won’t be testing it after that, you might want to rethink automation. The tests you decide to automate should be a foundation of your regression testing strategy, so if you don’t feel like they’re a critical part of the application to be checked in every code change, it may be best to keep them separate.
  3. How much time am I willing to invest? When automation requires so much time and maintenance, you have to determine the ROI of your efforts. Some regression tests may seem like they’re a candidate for automation but upon closer inspection, might not be worth the energy for the value they’re providing. Automation is not a quick, one-and-done solution to your problems. Take a lesson from Elon Musk — you don’t want to drop automation into the mix without a second thought. Spend the time making sure your first test is checking all the boxes and meeting requirements. Then determine how often it will need to be updated or reevaluated in order to establish whether or not automation is worth your time investment.
  4. Are there parts of this test case that still need to be explored? You might think that you’re ready to automate, but keep in mind that automation comes after a thorough exploratory testing session. Just because you can automate doesn’t mean you should, and many tests are better off being performed manually. Are there still instances that could benefit from human observation? While you can’t expect to manually cover every part of the application, getting a foundation during exploratory testing is what leads to more successful, stable, and insightful automated tests.
  5. Does automating support the needs of my team? If you’re automating a test case, you want to make sure the tests you write are going to actually answer the questions your team is asking. First, take a hint from Bas Dijkstra, and discuss the “why” before firing up a test automation tool. While Selenium is a great tool for automation, understanding the nuances of the many commands and abilities of it can take some getting used to, so if you’re not able to write a script that encompasses all of the requirements your team is requesting, another method of testing may be better. If you still feel that you automation is the best process for the job but don’t feel confident in your Selenium skills, a tool like Record & Replay may help you achieve your testing goals. There are many choices when it comes to automated testing tools, so you need to make sure the one your organization uses will support the testing you’re trying to accomplish.
  6. Why are you automating? Sometimes the most simple questions can give you the most telling answers. If you’re automating because you think it’s what you should be doing or because your boss is telling you to, you might not be automating your tests for the right reasons. Your objective with test automation should be to provide insight to you your team and give faster feedback. If you have ulterior motives, your efforts won’t take you very far.

Conclusion

Even if you could automate every test, you would quickly find it’s not helping you or the rest of your team very much. Next time you’re deciding on whether or not to automate, ask the right questions first.

7 Reasons Your Testing Team Will Love Record & Replay

By Leave a Comment

Imagine there was a way to have your manual tests recorded so you could easily replay them on hundreds of browsers without needing to perform the same exact actions over and over again. Record & Replay does just that, allowing you to test once and play it back on the largest selection of browsers in the cloud.

This may seem too good to be true, but many testers have already experienced the power of a codeless automated testing tool. If you’re still a little lost, we’ll make it easy for you. Here’s exactly why you should be excited about Record & Replay:

  1. No coding knowledge needed – Have you been wanting to experience the speed of test automation but having trouble getting started with Selenium? If you don’t know how to code, you have a long way to go from learning a programming language to writing your first working script. Luckily, with Record & Replay, no coding knowledge is needed. Rather than writing script, all you have to do is perform them and CrossBrowserTesting will record the actions. If you’re struggling with complicated frameworks and commands to create a successful Selenium test, Record & Replay is a hassle-free way to automate with the skills you already possess.
  2. Ability to edit tests – What happens when you perform the wrong action in Record & Replay or want to change part of a test? No need to stress, all you have to do is go back and edit recorded scripts. Delete mistakes with one click, or go in and edit your script by clicking on the script name and “Edit Steps”. There, you’ll see all the recorded values for each step variable, and once you’ve made your adjustments, all you have to do is save it to put your changes into action. Whether it be a typo or missed step, there’s no reason to spend valuable time redoing an entire test when all you need to do is fix one part of it.
  3. Unbeatable browser selection – There’s no question — CrossBrowserTesting offers Record & Replay on the widest selection of browsers. In fact, most of our competitors aren’t doing it at all. But we aren’t here to brag, we just think that a test is only as valuable as the number of browser configurations it can be run on. That’s why it makes sense that Record & Replay users have access to our numerous configurations, which means whether you want to test IE 8 or the newest Chrome release, you have the resources to check cross-browser functionality where and when you need to.
  4. Parallel testing capabilities – What’s better than automating one test at a time? How about automating ten in parallel? With Record & Replay, it takes seconds to select your test suites and play them back at the same time against different browsers. Parallel testing gives you time back to focus on exploratory test and untouched code in your application. Stop running test after test in different browsers, and start running them all at once to make time for the tests you haven’t run yet.
  5. Access to recording tools – Since tests are recorded as a script, sometimes you may want to insert certain commands that you can’t do manually to get the results you want. For example, if you want to verify element properties to keep visual consistencies across different browsers, you might need extra capabilities to allow you to do that. The tools included in Record & Replay allow you to add quick commands such as expects, sleeps, and hovers to make sure you’re making the most of your tests.
  6. Start shifting left – Software development teams aren’t shifting left because it’s the cool thing to do, it’s actually just a really good way to make sure quality is accounted for earlier. Test automation has been a big part of the shift left movement, allowing teams to test faster and thus earlier and more often. For teams that haven’t yet made the shift or who are just getting starting with automated testing, Record & Replay provides an on-ramp to automation, and sharing capabilities through Slack and Jira means encourages open communication with your entire team so you can get testing involved sooner in the software development lifecycle.
  7. Testing for all – As the roles in the software development lifecycle begin to overlap and blur, testing isn’t just for testers anymore. While we do believe that organizations should depend on a dedicated QA team, there is more room for other software stakeholders to take part in some small testing tasks as well. Previously, the ability to automate was only available to automation engineers or testers with programming knowledge. Now anyone can test from developers and product managers to marketers and designers can take part, so your entire team can run like a well-oiled machine.

Conclusion

If your QA team is still playing a game of catch up to a development team that’s continuously integrating new code, it’s time to get on board with an automated testing tool.

Trying to understand automation can be daunting, but it doesn’t have to be. Record & Replay can help you take the first steps to faster testing and increased code coverage to support your team from planning to deployment.

How to Fix a Selenium Suite That’s Flakier Than Your Friends

By Leave a Comment

flaky selenium test
Selenium has changed the game for many software teams, but it doesn’t come without its many challenges. One of the most common headaches for testers is recurring flaky tests, or tests that fail randomly when there hasn’t been any change to the code.

It’s no surprise that this can get annoying since flaky tests affect progress. After a while, it can significantly slow down teams’ productivity when they are no longer able to differentiate between a test that has a bug and one that they’re wasting time on because of an invalid fail.

Eventually, too flaky tests may mean testers might ignore the results, which of course isn’t a good option either. In order to put an end to the flakiness, we have a few tips to keep your builds stable.

  1. Find the source – When your tests start going flaky, you first have to investigate why they’re going flaky.In order to identify the problem, it’s important to be familiar with common reasons for flaky tests so you can identify them. Oftentimes, this can be something really simple such as poor use of bad locators. Craig Schwarzwald stresses the importance of locators that are unique, descriptive, and unlikely to change. If they don’t meet that criteria, it’s a bad locator, which means your tests are likely to fail.  However, there are many other factors that can lead to flakiness. Richard Bradshaw says that lazy manual testing practices, poor knowledge of tools incomplete data, and bad assertions have all been culprits of flakiness, which goes to show that it simply comes back to how well your tests are written.
  2. Isolate and fix – Once you understand the root problem of the issue, it’s much easier to isolate the test and debug. You want to do this as soon as possible because the longer a flaky test sits in your suite, the more likely you are to attract more flaky tests. As Angie Jones points out, all it takes it one flaky test to ruin the rest of a build. She advises separating tests into different paths — one for stable tests that only fail when something is wrong, and one for unstable tests with flaky fails. This way you can divide attention between the red builds that need to be fixed or rewritten and the green builds that developers should focus on.
  3. Document flakiness – Good testing includes three things — documentation, documentation, and more documentation. Just because you’ve debugged a flaky test doesn’t mean you should be done with it. Documenting will make sure that the team is aware of the problems that caused the flakiness and has the resources to go back and fix it should the problem arise again. Additionally, if you have many flaky tests or a flaky test that is hard to determine the cause of failure, documentation helps recognize patterns that will prove critical for debugging. Nebojša Stričević says that you should document every flaky test in your ticketing system and add information as you discover it so your team can collectively figure out how to fix the suite. It’ll also do you well to keep organized reports before flakiness is found so that you rely on them when you run into an issue rather than trying to gather all your information once a bug is found.
  4. Use page object pattern – We’ve waxed poetic about page objects before, but they are especially useful when it comes to maintaining your Selenium suites. That’s because the page object model will help you make robust testing frameworks that are resistant to small tweaks in the UI. The clean separation between test code, page-specific code, and layout plus the single repository for the services or operations offered by the page allows any modifications required due to UI changes to all be made in one place. Nikolay Advolodkin goes into great detail about how page object patterns can often be the key to stabilizing test automation. If you truly want to eliminate flakiness, page objects will get you on the right track.
  5. Stabilize your environment – One common reason that your tests are unstable is that your environment is unstable. If you have a browser crash, server bug, or bad network latency, it’s going to affect your tests. This may be unsurprising, but it’s a commonly overlooked aspect of keeping your Selenium tests stable. You want to make sure you look at carefully the testing environment and separate automation from the rest of QA for maintainability purposes. Emanuil Slavov advocates that stabilizing his test environment helped go from a pass rate of 50 percent to 100 percent through this method of separation.
  6. Wait for it – Incorrectly using waits and sleeps will be a sure-fire way to have your tests switch from green to red. Not using them will make your life even more difficult. This is because page loads are unpredictable — some aspects will load quicker than others. If you’re trying to automate an action on a feature that hasn’t been loaded yet, you’re going to have inconsistent results. In order to avoid flakiness, differentiate between implicit waits, explicit waits, and sleeps, and add them to your script appropriately. Alan Richardson covers this aspect of synchronization as one of the top causes for failing automation — if your tests pass without waits, it’s probably because you got lucky. He says that state-based synchronization is the answer to making it work.
  7. Frameworks are your friend – Frameworks aren’t just helpful in writing tests, they’re also essential for maintaining a stable suite. Joe Colantonio says that pretty much any time you’re working with automation, you should have a framework in place for reuse. By following the guidelines that frameworks provide, your tests will be more repeatable and maintainable for your entire team.

Don’t Be Flaky

Flaky tests can result from a variety of reasons, but at the end of the day, it’s up to you as the tester to find the root and dissect the issue. Fortunately, everyone deals with flakiness from time to time, so there are many resources dedicated to helping you identify what went wrong.

Thinking about how to prevent flakiness before you start seeing red will be the difference between automation that actually helps you meet your goals and flakiness that just slows you down.

The Best Uses for Live, Visual, and Automated Testing

By Leave a Comment

uses for automated testing

From our humble beginnings as a manual testing tool with just around 50 browsers, CrossBrowserTesting has come a long way to be the reliable end-to-end testing tool you know today. With a range of offerings as old and problematic as IE6 and as new as Chrome 63 amongst our 1,500 configurations, we’re unquestionably the most complete cross-browser testing platform.

Another part of growing up and expanding was adding visual testing and automated testing capabilities in addition to live testing. Today, individuals and teams across the country depend on these features, but we want everyone to get the most bang for their buck. So, in order to truly understand our core functionalities and the best times to implement them, we’re explaining the best uses for live testing, visual testing, and automated testing.

Live Testing

We started with live testing to give people a way to manually test their website in a variety of browsers. Today, manual testing is as important as ever. As teams adopt Agile and Continuous Integration practices, it’s important that manual testing isn’t overlooked and continues to be a part of the SDLC into the future. Having dedicated testers who understand software and are focused on finding defects and reporting bugs is quintessential to proper live testing.

Best Use Case – Exploratory testing.

Exploratory testing is not a strict set of rules checking whether or not the application works, like automated testing. While there should be a plan going into exploratory testing, it’s also a somewhat ad-hoc process based on testing the limits of a new integration within the software, or even a completely new application, before it’s deployed. It takes a keen sense of observation and an understanding of the application design and purpose to be executed with value.

Using the live testing feature in CrossBrowserTesting, exploratory testing becomes more manageable with access to browsers, operating systems, and devices that teams may not have had previous access to, but their customers are using. By remotely accessing a variety of configurations rather than just the browser your developers are, owning this detailed observation for a broad user base is much more manageable.

Tips for Live Testing – Check out our tips for exploratory testing an eCommerce site and user personas playbook for ideas on what to look for during an exploratory test. Additionally, our native debugging tools allow you or any of your teammates to fix a bug, on the spot, in the very browser it was found on.

Visual Testing

Applitools Eyes Visual testing CrossBrowserTestingVisual testing is used to assess a web application’s responsiveness across browsers. By performing visual testing, you’re looking at the UI/UX components on the front end to decide whether the application under test is acceptable on a variation of browsers, devices, and screen resolutions since they all provide a different experience.

Best Use Case – Screenshot comparisons for visual validation.

Our screenshot comparison engine lets you take screenshots across multiple browsers in a matter of seconds, allowing you to easily compare full-page layouts. By highlighting the differences from your baselines browser, it’s easier than ever before to decide whether your web page is consistent across browsers without having to manually compare them. With the option to test on hundreds of devices, you can be confident that you’re accounting for every customer.

By performing visual regression testing, you can also compare new changes to historical versions to make sure any added integrations or application updates are supporting improvement rather than adding any glitches to the user experience.

Tips for Visual Testing – Run a risk analysis and pick five to ten of your highest priority pages. Find out what the most popular browsers are that your users are on, and run each of those targeted pages on those configurations to give you a basis as to whether or not the application is visually acceptable for the majority of your users. If you happen to find that a page is faulty, take to live testing to debug.

Automated Testing

Automated testing in CrossBrowserTesting relies on Selenium and Appium open source software to allow you to create test scripts in any major language that can be run across multiple browsers. While automated testing with these frameworks does require programming knowledge, teams prefer it to be able to run tests in a fraction of the time without needing to perform every step manually. Alternatively, Record & Replay provides a lightweight, codeless test automation solution for teams that have yet to fully learn how to write Selenium scripts.

Best Use Case – Functional and regression testing.

You can take all the screenshots in the world, but just because something looks great, doesn’t mean it also works like it’s supposed to. Automated testing digs deeper into the backend of a web application to analyze unit, integration, and end-to-end test cases and make sure that code renders properly on different browser configurations. By looking beyond the surface of the application, you can better insight on whether or not it will actually work correctly for users at different stages in their journey through your application.

Test automation is also a preferred method for regression testing since repeated actions of long test suites don’t have to be done manually. Instead, running an automated regression test can check that previously stable code still is functioning as it’s supposed to after a new integration, feature, or bug fix is added.

Tips for Automated Testing – Running an automated test across browsers in parallel can make testing up to twenty times faster, depending on how many configurations you include. While automation is already a great way to speed up repetitive or boring tests, parallel testing allows you to run that same script on other configurations at the same time rather than waiting until each is done to start the next.

Conclusion

While live testing, visual testing, and automated testing each have it’s own purpose, designing a strategy that involves all three is the key to truly comprehensive testing. As your team and product grow, using these capabilities to help your organization balance speed and quality will be what sets you apart from the competition and ensures your brand’s trustworthiness.

6 Ways to Speed Up Your Testing Cycles

By Leave a Comment

faster automated testing

In the battle between speed vs quality, teams that practice test automation know that there’s a practical way to aim for both. However, while testing inherently places value on precise observation and careful consideration over making mistakes for the benefit of speed, how can you make testing rapid enough the meet the needs of Agile development demands without skipping over essential steps?

Faster testing is easily attainable by refining your current automation strategy and adjusting the processes that are already in place. Not only will the following tips make your life easier, they’ll also accelerate your testing to help make the most out of your time.

  1. Get organized – Keeping your tests organized may not seem like a priority day-to-day, but it will make things much more streamlined for you in the future. You should be keeping your test cases lean and reusable, automating features at the unit level. By practicing good test management methods, maintaining accurate data, and keeping clean automation reports, it’ll be easier for you and your team to run tests whenever they’re needed.
  2. Talk to your teammates – Speaking of team, your teammates are some of your most important tools to driving testing that’s effective and efficient. Whether you’re the solo tester on a software team mostly made up of developers or you’re one of fifty other automation engineers, there are always other stakeholders in the software. Communicating with these people will help everyone stay on the same page as far as documentation and reporting, as well of the status of the software included what’s new, what needs to be tested, what’s been tested, and what needs to be fixed.
  3. Know when to automate – Of course, automating your tests will make things exponentially easier and faster, but automating the wrong test cases is just going to slow your progress down. While testing manually might seem to take longer, new test cases require exploratory testing, and taking the time to do it at the beginning will save you from having to go back and do it later after you’ve already tried to automate it. Ultimately, understanding the best cases to automate before you actually start running them will prevent you from wasting time — a software tester’s worst nightmare.
  4. Try parallel testing – While automation will unquestionably help you test more parts of your application faster, it falls short in the way that it is only able to accomplish this in a sequential fashion — no more than one automated test can generally be run at one time. However, with parallel testing, automation gets a much-needed boost by allowing different tests to run simultaneously. If you’re smart enough to understand the importance of the same tests across multiple browsers, operating systems, and devices, you can also see the value in doing this all at once with parallel testing instead of running the script over and over on different configurations.
  5. Use tools to your advantage – Being a great tester will only take you so far if you don’t know how to leverage the right tools to help you. Between adopting open source testing frameworks and investing in automation tools, there’s no shortage of ways to make your life a little easier. With the previously mentioned parallel testing, for example, you need to rely on a tool like CrossBrowserTesting for a cloud-based Selenium grid or Record & Replay.
  6. Cut out flakiness – While tools like Selenium are an asset in automation, it can also lead to flakiness. False negatives will just slow you down — no one likes seeing red all the time and dealing with inconsistent results. In order to avoid this, it’s best to attack the problem head-on by isolating flaky Selenium tests from the build to fix them before rerunning the test again.

Conclusion

Working towards faster test automation doesn’t have to mean starting over. Balancing speed and quality is something you just have to learn along the way, but getting to know test automation hacks will get you there even sooner.

From 0 to 100: How to Go From Manual to Automated Testing

By Leave a Comment

Manual tester to automated testing speedometer

Test automation is all the rage these days, but breaking into the role of an automation engineer is challenging, and sometimes it might seem like more trouble than it’s worth. When there’s still a clear need for manual testing, what’s the point of you uprooting your current ways just to follow the trend?

Though going from manual automated testing may seem daunting, it’s about much more than adding a skill to your LinkedIn profile. In fact, your future career could depend on your automated testing skills.

Why Do I Need Automated Testing?

  • Faster testing – Your current process might be working for you now, but as a tester, you should be striving for more ways to get faster and more effective at your job. And besides, manually testing repeat test cases must get pretty boring. Learning automation lets you get regression testing out of the way so you can focus on the fun stuff, while simultaneously allowing you to test more software in each sprint.
  • More and more teams are automating – Automation has been steadily adopted by companies to improve test efficiency, and today 44 percent of tests are automated on average, according to the SmartBear State of Testing survey. Not only that, but testers broadly believe it will become even more popular the coming years as more organizations shift to Agile.
  • High in demand – Because more teams are getting on board with the practice, test automation skills are becoming increasingly valuable. Testers that know how to code and are familiar with tools such as Selenium and Appium are the ones that will stand out when it comes time to make a career move. If you find yourself in the position where you’re looking for a new career, it’d be beneficial to have at least a base knowledge of automation and the corresponding technologies that these companies are embracing.

4 Steps for Moving to Test Automation

  1. Learn a programming language – As mentioned, learning a programming language is becoming a valuable skill for testers to have. It’s never too late to learn, but it’ll also be the biggest adjustment when adopting automation practices since it wasn’t previously needed for manual testing. In fact, learning to code will probably be one of the most difficult parts of learning test automation, as well. However, test automation is impossible without coding knowledge, so it will be something you need to know. We usually suggest starting with Python since it’s often easiest to pick up for beginners and there are countless resources for learning such as Treehouse or Codecademy. Additionally, other skills like XML, HTML, CSS, XPATH, and SOAP are often preferred.
  2. Adopt the right tools and frameworks – You’ll make the process a lot easier on yourself if you use a combination of frameworks, open-source tools, and third-party tools for automation. Sorry to break it to you, but you can no longer solely rely on your mouse and keyboard. Selenium WebDriver will probably be the foundation of your automated testing efforts as it’s by far the most popular, which is convenient because you can use any of the new languages you learned. You can learn more about getting started with Selenium here. You’ll also want to familiarize yourself with testing frameworks like Nightwatch.JS, Cucumber, and Protractor, libraries like Capybara and Watir, and Continuous Integration tools like Jenkins CI and TeamCity to get the most out of automated testing. On top of that, paid tools like CrossBrowserTesting will make your life a lot easier when it comes to run test across different browsers. Alternatively, you have the option of using tools like Record & Replay if you haven’t learned to write scripts in order to do some lightweight, codeless automated testing.
  3. Start small – The smaller your test cases are, the easier they will be to debug, maintain, and reuse. You don’t want to go crazy with automation and write heavy test cases, or you’ll just make your life harder in the end. Keep them separate and labeled so they’re easy to identify and share in reporting. On that note, you’ll also have to prioritize which tests you want to automate in general. Just because you have this newfound knowledge of automated testing doesn’t mean you should apply it to everything — in fact, it’s virtually impossible to automate everything, and many things are better off being done manually. Polish off those exploratory testing skills when a new feature is released and run risk analysis for which parts of your application should be automated. In addition, you’ll want to create a list of the browsers and devices that are going to be essential to your particular test automation plan.
  4. Back to the basics – Keep practicing, learning, and using your resources. Automated testing might seem intimidating when you’re first getting into it, but all it really takes is time and patience to learn the craft. Also, no matter how good an automation engineer you are, there’s always more to learn, so take comfort in knowing even the experts don’t know it all. Luckily, there’s also a lot of resources at your disposal. Between online guides, forums, conferences, meetups, communities, social media, and even Slack channels, there is no lack of help available at any time when you have a question or are just looking for some guidance. The testing community is robust, active online and offline, with many people who are happy to provide guidance and mentorship.

Additional Resource for Learning Automated Testing

Of course, this is a broad overview of what it takes to go from a manual to an automated tester, and there’s a lot more to actually learning automation than four steps. However, it’s likely that you’ll find the change worth it, both to heighten your career and enhance your abilities.

Upcoming Webinar: Accelerating Your Test Execution Pipeline

By Leave a Comment

parallel testing webinar smartbear

Do you find your testing process can’t keep up with the demands and expectations of quickening software development and delivery cycles? Do you mark tests as “done” just to discover certain users are still finding bugs? Maybe, you wouldn’t mind finding a solution that would cut your testing time and increase quality, without having to choose just one or the other.

On September 7th, SmartBear will be hosting a free webinar entitled “Accelerating Your Test Execution Pipeline” in two sessions (8:30 AM and 2:00 PM EDT) to talk about just that. Gabriella Grandgard and Daniel Giordano will discuss ways to identify key bottlenecks in your testing processes that will help to deliver software on-time without compromising quality.

As test automation processes increase among organizations, testing efforts can lag behind largely because of scalability issues. Test environments are often the root cause for a number of these problems due to the time it takes to create them and maintain them, and the cost to reflect real users. Incorporating a test automation platform can cut your testing time in half by leveraging the cloud to test multiple devices and browsers at one time, and fix bugs as fast as they’re found rather than waiting until after deployment.

This method referred to as parallel testing allows testers to run scripts in multiple environments concurrently to save time, increase coverage, and speed up tests. This means that instead of running one script sequentially across different device and browser configurations, it can be done at the same point in time. As browsers, devices, and operating systems continue to release and update, keeping up with these versions calls upon the need for both parallel testing and cloud testing to support scalability.

Join our software experts to learn how testing teams can accelerate their testing efforts by easily scaling and reporting on their tests across multiple environments using parallel testing techniques.

Highlights include:

  • Real world demonstration of testing across multiple environments.
  • Techniques that can help you easily speed up your testing efforts.
  • Strategies to enable parallel testing for both manual and automated testing processes.
  • New ways to quickly pinpoint problem areas in your tests.

 

Register for the free webinar here.

Sign Up

5 Steps for Taking Full-Page Screenshots of Your Website

By Leave a Comment

full-page screenshot automated cross-browser testing

Testers know the value of using browser screenshots for manual debugging and regression testing, but we take it a step further with capabilities for full-page automated screenshots. CrossBrowserTesting allows you to automate screenshots on real browsers and devices and compare them against a baseline to evaluate layout differences and find issues that need debugging more easily.

But what do we mean by a full-page screenshot? A full-page screenshot is more than what you see when you look at a web page on your device (a.k.a. a windowed screenshot) — it’s a capture of the entire web page above and below the fold.

This means full-page screenshots account for any area you have to scroll up, down, or sideways to view. As opposed to manually taking multiple windowed screenshots of one page and trying to stitch them together and compare them to multiple screenshots of another page; our algorithm scrolls the page, captures an image at each viewpoint, and merges them into one view in order to provide a fast and accurate screenshot of the entire page.

By evaluating different browser versions visually, side-by-side with our comparison engine, it’s easier to not only see layout differences but detect smaller problems that might go unnoticed.

CrossBrowserTesting full-page screenshots

Full-page screenshots are a great tool for

  • Exploratory testing – Exploratory testing is performed the first time a tester looks at new software or new features. Being able to multiply these efforts by visually exploring multiple browsers saves time while increasing coverage.
  • Regression testing – Any change to working code means the possibility that it’s been compromised. Regression testing is used to ensure that a recent change hasn’t impact any previously functioning features, and comparing them with pass/fail results makes the process easy, effective, and efficient.
  • Usability testing – Testing the UI/UX of a web page ensures that current and potential customers are happy with your application and will continue to come back. Usability testing across browser and devices configurations further considers diverse user experiences.
  • Responsive design – Responsive design by definition means that everything looks visually acceptable from screen to screen, device to device, and browser to browser, which means visually validating this on real devices is paramount.

Here are five easy steps to taking full-page screenshots of your website with CrossBrowserTesting:

  1. Log into the CrossBrowserTesting app, click screenshots, and start a screenshot test by entering the URL of choice. Keep in mind, you can also take screenshots locally or behind a firewall so you can test and debug before your app reaches the public eye.
  2. Choose from over 1,500 real browsers, devices, and operating systems to replicate real user scenarios. Hint: use Google Analytics to decide which browsers you should prioritize, or choose the “Popular Browsers” option.
  3. Use the full-page setting and view high-resolution results within minutes. Compare to your baseline, filter by device, and easily examine highlighted layout differents that anyone could understand to find bugs, spot UI issues, and identify elements that are hidden or misplaced.
  4. Download the results, save your reports, and share with your team through Slack, Hipchat, email, or Jira. These communication channel integrations make it easier to provide feedback, start discussions, and affect change by encouraging collaboration with different roles on the development team.
  5. Take to live testing for remote debugging.

Full-page screenshots prove to be a great resource in many stages of testing, and automating them with a tool like CrossBrowserTesting makes the entire process much more streamlined. Or, you can just photoshop your own full-page screenshots together if you’d like, but we prefer our way.

By increasing test coverage and speeding up debugging time, you can more easily identify gaps in quality and quickly deliver better software to your customers. Additionally, the CrossBrowserTesting Comparison Tool provides the best of both worlds, striking a balance between manual and automated testing by a visual representation of your web page while eliminating shortcomings of human error.

What Goes Into a Comprehensive Selenium Testing Strategy

By 4 Comments

Debunking 6 Automated Testing Myths

By Leave a Comment

Automated testing myths and misconceptions

Automated testing has been the answer for many teams who wanted to speed up and increase their testing efforts, but misconceptions surrounding automation mean that testers don’t always get the most out of their tools.

In an attempt to correct frequent misunderstandings, we’re busting common automated testing myths to give you the truth.

Myth: You should automate 100% of your tests.

Fact: You can’t test everything. Though the ability to automate testing is a great skill, it’s impossible to automate every test. And even if you could, it wouldn’t be an efficient testing solution. Automated tests are based off source code that’s built based on manual tests. Meanwhile, some tests are better off being executed manually. Anytime a new integration is added, a new test has to be done because automated tests will only check previous scripts. Experienced automation engineers know that prioritizing tests and knowing when to automate is just as important as knowing how to automate. However, though you’re not able to automate every test, automation will greatly increase the number of tests you’re able to run.

Myth: Developers should lead automated testing efforts.

Fact: Automation Engineers should drive automation. While it does take programming and coding knowledge, most of the time it’s the testers that are building automated test scripts. As the software industry begins to depend on new testing methods and development processes, test automation architects and engineers will be in high demand. In fact, most of the time having an automation engineer is preferred to a developer because testers have the mindset, training, and specialized skills for different automation tasks, which include much more than just coding.

Myth: Automation is more expensive than manual testing

Fact: If utilized correctly, automation can save money. While the initial investment in automated testing tools and hiring testers may seem more expensive, the ROI of test automation is often worth it since it saves time that would usually be spent on manual testing and increases productivity. For example, if you’re manually running tests that could be automated, it’s costing you in wasted time because your testers could be doing something more constructive. If you think about this scenario over weeks, months, or years, that’s a huge opportunity cost. Test automation is also a cost effective way to extend testing coverage, scale testing efforts, and mitigate risks as your application grows.

Myth: Automated testing is better than manual testing

Fact: Neither automated or manual testing is better or worse than each other, they just have different uses. The most effective testing strategy will find a happy medium between automation and manual testing. Manual testing is better for one-off tests or tests that require human observation, so exploratory tests, smoke tests, and usability testing should all be manual. Automation is usually best for repeated tests or things that need to be frequently tested, such as regression tests. Familiarizing yourself with the different uses and applying them to the appropriate test cases is the best way to get the most out of your time, money, and testing efforts.

Myth: Automated Testing is an easy fix.

Fact: Automated testing isn’t a one-and-done solution. You can’t set up an automated test and expect that you’ll never have to look at it again. In fact, the source code of your automated tests takes a good amount of maintenance to check that it’s doing its job. Not to mention that changing an organization’s testing and development processes will require new training and onboarding to familiarize stakeholders with automation and writing sophisticated tests. Though the ROI of test automation will likely be apparent over time, the transition to a mostly-automated process from a mostly-manual one will take commitment, patience, and financial investment.

Myth: Automation will solve all of your problems.

Fact: Automation is not a silver bullet. Automation engineers experience false positives and flaky builds that come from a lack of human observation, maintenance, and company-wide collaboration. For example, common challenges using Selenium for automated testing might include coding for cases with more complicated elements like dynamic content, multiple tabs, and pop-up windows that are easier to test manually. Testing will still require the specialized skills of a talented testing team to troubleshoot problems, write scripts, manage tools, and maintain test cases. For some best practices of Selenium testing, check out this guide.

Join Over 200,000 Testers & Developers And Start Testing Today

Learn more Free Trial