CrossBrowserTesting.com

A Design, Development, Testing Blog

Top Takeaways from Agile Testing Days USA

By Leave a Comment

Agile testing days usa

Agile Testing Days came to the US for the first time this year, and if we do say so ourselves, they chose one of the best city for the venue — Boston!

SmartBear was one of the event sponsors, so luckily we were able to sit in on some of the amazing speakers, and we learned a few things along the way about Agile testing. Here are just a few of the top takeaways from Agile Testing Days USA.

How Agile Are You?

If you’re at Agile TD, your team is probably working in an Agile fashion or looking to become more Agile. But Selena Delesie noticed something in her time working with different companies — many were saying they were Agile without actually working in an Agile manner.

Many times she found that people wanted to control everything and strictly plan out their sprints, which didn’t lead to the flexibility that Agile demands. Additionally, she found that many people were no longer engaged in their work and were not present with themselves and their coworkers.

“Stop doing Agile; stop treating it as a process,” said Selena. “Start being Agile. Doing and being are very different things”

Selena challenged the audience to reconsider what Agile means to them and think about whether their work processes were actually Agile.

Going back to the Agile Manifesto — specifically thinking about individuals and interactions, customer collaboration, working software, and responding to change — and even the definition of “agile” are good places to start. Agile is supposed to mean that you’re able to think on your feet, be active, and lively but many companies have lost that, and agile has become a heavier process than Waterfall.

Selena says that having more process and control does not help them do their best work, it makes them mediocre. She especially emphasized connecting with people and building relationships. Making the effort to have conversations with people, listen to them, and show compassion can make all the difference.

It’s especially important for management to instill these values and be transparent in order for an Agile organization to work effectively. Selena suggested to stop having meetings behind closed doors and instead, show trust in your team. Let go of feeling the need to control things and make more of an effort to discover and learn.

 

Look Beyond the Tool

Test automation is has proven essential for Agile teams, which means you need certain test automation tools. But when implementing automation into your workflow, it’s important to consider a few factors besides just which tools you’re using. In “Automation for the People”, Christin Wiedemann suggested that you additionally think about the “what, why, and who” of automation.

Before even starting with automation, you should first determine why you’re automating and how it’s benefitting the rest of your team.

Christin notes that automation is much more than running script using a tool — you should expand the scope of what you define as automation to understand what you want to automate. This is the best way to avoid waste, whether it’s waste of time, waste of effort, waste of money, or waste of trust in automation.

Oftentimes, automation is used to take over repetitive tasks to eliminate human error that comes with doing things over and over, but a lot of people start automating without specific intention.

Saving time isn’t always the answer because you might end up replacing the time it takes to execute a test with coding and maintenance. The most important thing is to have a reason for automation and set goals.

You also want to think about who will be automating. This isn’t limited to thinking about who will be writing the scripts and running them. Testing is a team effort, and everyone has different skills and ways to add value, which means those people should be contributing to automation efforts. “At the end of the day, we all want to contribute to the same objective,” says Christin.

Additionally, Christin emphasizes the importance of seeing automated and manual testing as supporting practices rather than contrasting them against each other. Meeting your objectives will require a mix of both manual and automated testing, and she says that it’s important to remember that the results are more telling than the method of achieving them.

By broadening our definition of automation to think beyond tools, we can get a better understand of test automation to further the success of testing in our teams.

Musicians and Testers Aren’t so Different

Have you ever thought about how much test automation has in common with music history? If not, Angie Jones’ talk “Owning Our Narrative” gives a lot to think about.

Angie took us through the evolution of music, starting with Thomas Edison’s phonograph in 1877 and explaining how that idea developed to lead way to new recording instruments like the player piano and jukebox, then more recently to the cassettes, CDs, and MP3s that we know today.

As these changes took place, people began to develop new preferences when it came to music. Rather than being the social aspect it once was, music was now being sorted in genres. Record labels were recording electronically to get better sound, and automated machines were able to play one song instead of making people buy a whole album.

With all this innovation, musicians were forced to make adjustments and change the way they made music to meet the requirements of the different ways it was recorded and played. However, while the role of the musician changed in many ways, the skills remained the same.

If we think about the timeline of test automation, it’s very similar. We’ve found new ways to write and record tests, which has required testers to learn new skills as we embrace agility and shift left, but at the core, the role of the tester also remains the same.

Angie advocates that testers need to find a way to stay true to this role no matter the changes that are going on around us. Musicians were upset about automated machines taking jobs while everyone else was excited, and it was only by adapting to the new environment of music they were able to be successful.

She encourages us not to make the same mistakes as musicians. Embracing industry changes and making it work to your advantage will be far more productive than resisting it.

Angie notes that sometimes it can be difficult to convince others the value of our role in the changing landscape, but the more conscious testers are to those changes, the more prepared they are to own their narrative.

BONUS: Strategize Test Automation

We love Angie Jones so much, we wanted to make sure we invited her to SmartBear while she was in Boston. Luckily Angie accepted our invitation and taught our attendees about “Which Tests Should I Automate” at our Ministry of Testing meetup.

In this interactive session, Angie gave us a look at how she would perform a risk analysis of different test cases at Twitter to decide which tests should be automated, which tests you might want to automate, and which you would skip.

Many times we make the decision of automation off a gut reaction without considering the different aspects of each test — which are the most user critical? Which break the most often? Which would take the longest to fix?

By evaluating these factors against each other, we can make better decisions when it comes to test automation.

See the video below to watch the lesson for yourself.

The Top 8 Things We Learned From the QA Team at America’s Test Kitchen

By Leave a Comment

Americas test kitchen quality assurance

With three core websites, America’s Test Kitchen has a lot of content to cover. Yet when Megan Collins came on as QA manager, she was the only tester. Since then, America’s Test Kitchen has exponentially scaled their testing, adopted an automation strategy, and expand the QA team to three members, including Automation Engineer Michael Ng.

We recently had Megan and Michael join us for a webinar to talk a little bit about the web testing strategy behind America’s Test Kitchen and how they handled their shift to automation with the help of CrossBrowserTesting.

Here are the top 8 things we learned in the webinar:

  1. Fast deployment schedule demand testing — Most organizations dream of achieving the agility that America’s Test Kitchen has reached, deploying code to production every two days. But when you’re integrating as often as they are, you can’t just rely on developers to test each others code or you’ll never achieve consistent test coverage. By bringing in Megan Collins as QA Manager, she was able to spend her time on something that had been severely lacking — testing. This way, proper exploratory tests could be performed when there are any changes to code, and the development team can continue working doing their jobs instead of worrying about the intricacies of whether a new feature works or not.
  2. Real devices matter – When discussing emulators, simulators, and real devices, there’s a time and place for each one. In fact, Megan mentions that the developers on her team use emulators. However, the only way to see exactly what your users are seeing is to use real devices, which is why they are so important to the QA team. In fact, they even ran into a bug where membership details weren’t displaying for a user that wasn’t found on the emulator they were using — they noticed it once they performed the test on a real tablet device. While emulators are highly accurate, they do not replace real devices, and this is a large reason why they need a tool like CrossBrowserTesting that includes all three in order to prioritize both speed and accuracy during mobile testing.
  3. It’s important to understand different user journeys – The success of the America’s Test Kitchen QA team has a lot to do with their ability to identify the different users they have and how it affects their testing. Because a lot of the web content they have is for members only, there are paywalls that occur when someone accesses from a non-paid account, for example. By separating user states into two focused areas for their automated tests, static and dynamic, Michael is able to create more precise tests for different parts of the website. Understanding that they need to test different based on whether users are new, registered, on unsubscribed helps them build out different test cases and scenarios.
  4. Capture tools can help prove a point – Sometimes seeing is believing, and it can be hard to convince other team members of a bug they haven’t seen for themselves. Tools like CrossBrowerTesting’s screenshots and video capture help the America’s Test Kitchen QA team build out their test reports to include visual evidence of different occurrences during tests. This way, Megan doesn’t have to repeat the test case or find it on the right browser when she shows it to development, she can just show the screenshot or video instead so bugs get fixed faster. Additionally, by using the screenshot comparison engine, Michael has been able to create an archive of each homepage over time so it’s easy to show what changes in each regression.
  5. Use automation to make your life easier – America’s Test Kitchen isn’t automating every single test on their website. In fact, manual testing is still a large part of their strategy. Automation comes in mostly with repetitive tasks that take up a lot of time. For example, they use automation to sign in as a member in order to see whether content can be viewed instead of going through the actions of logging in in every time. This saves Michael valuable time in performing different test cases under different user scenarios — instead of replicating the same actions every time, he can focus on how each test differs.
  6. Collaboration is key – We say it over and over again, but communication with your team is important to productive testing. By leveraging tools for organizing your tests, taking notes, and tagging, it’s easier to remember which bugs you’ve encountered before and which are new. Additionally, keeping open communication with the development and design teams has been essential to America’s Test Kitchen for understanding priorities, risks, and hot spots for more insightful testing. Megan and Michael have found collaboration with the development team to be especially crucial in the shift to automation — learning new skills, choosing what to automate, and being notified when new code is added.
  7. Test while you sleep – Many of the automated tests that the America’s Test Kitchen QA crew create run at night, meaning that you literally can test in your sleep. Overnight testing is at the crux of their Selenium strategy, meaning they can check tests in the morning to see if they pass or fail. Because they have so many tests that run for an extensive amount of time, they rotate their browsers each night to get coverage of all the different configuration their users are on to see which tests are failing on which browsers. By running these regression tests, they’re able to test a subset of test cases across different pieces of functionality that they need to make sure are working day to day.
  8. Tests are only as good as how much time you put into them – Michael runs a lot of automated tests, but he also spends a lot of time maintaining them. The America’s Test Kitchen QA team emphasizes that maintenance is crucial to the success of the project. This is one of the main reasons they have a dedicated QA automation engineer. In order to ensure that Selenium tests aren’t just been created left and right, Michael spends a lot of time on each one to make sure it is accomplishing his goals (and may even be a few steps ahead of Elon Musk in this way).

Megan and Michael gave us a great look at how they use automated and manual testing to further the goals of America’s Test Kitchen. By taking the time to learn from other successful QA teams such as this one, you can apply some of their practices to your own testing strategy.

If you want to view the full webinar, you can watch below.

Selenium vs. Cypress: Is WebDriver on Its Way Out?

By 2 Comments

selenium vs cypressThere’s a new kid on the block for open source test automation tools, and everyone’s talking about how Cypress may be an alternative to Selenium.

But while some have posed that Cypress signals the end of an era, we’re not so sure that Selenium is going anywhere anytime soon.

To better understand the similarities and differences between these two tools, let’s take a better look at what they both do.

Selenium

Selenium WebDriver is the open source automation tool we all know and love. With a history starting from the humble beginnings of IDE and transforming into Selenium 2.0 a.k.a. Selenium WebDriver, Selenium has become a foundation of many modern software teams’ test automation toolkits.

Most are familiar with Selenium’s use — it makes direct calls to the browser using each browser’s native support for automation. By writing a script in Selenese using the compatible programming languages, you can run tests that work as if you were driving the browser yourself.

While the learning curve can be steep for Selenium and requires the knowledge of at least one programming language, it has a vast community of active users and even boasts an annual user conference.

One of the foundations of Selenium is the ability to test on the most popular browsers, making it essential to cross-browser testing.

As Selenium has grown in popularity, it’s not only become a basis of many test automation strategies but has also become an integral part of many third-party tools such as CrossBrowserTesting. Once you know how to use Selenium, you can use these cloud testing tools to access thousands of browsers and leverage parallel testing to cut down on testing time.

Cypress

So how is Cypress different from Selenium, and how is it the same?

First, Cypress is said to have an easier onboarding than Selenium, where all you need to do is install the .exe to get up and running sine all the drivers and dependencies are automatically installed. With Selenium, on the other hand, you need to spend a little more time getting everything up and running.

One of the advantages of Selenium is that it supports most of the major programming languages including Java, Perl PHP, Python, Ruby, and C#, so if your team is already using one of those languages, Cypress may not be ideal since it only supports JavaScript.

In addition, since Cypress tests are written using Mocha and Chai, the syntax echoes what most Javascript users understand. This means that if you work with Javascript, it will be especially easy to start using Cypress. However, if you’re more familiar with another language, it will be fairly complex.

Cypress almost acts as a “freemium” model. Similiar to Selenium which is entirely open sourced unless you integrate with a paid tool, you can also add to Cypress’s core capabilities by leveraging the paid dashboard which includes a test runner that allows you to easily debug tests with DOM snapshots and includes video storage for private projects.

Selenium and Cypress differ in the way of their architecture, as well. Selenium is made up of bindings, or libraries, and the WebDriver, which controls the browsers. These two components work through the JSON network.

Alternatively, while Cypress is used for UI testing, it uses its own unique DOM manipulation and runs directly in the browser with no network communication. Cypress is often cited as a developer-friendly test automation tool for this reason since this often causes frustrations for them in Selenium.

Having a different architecture on top of an Electron app also allows Cypress to be so close to the web application under test that we can do things that you can’t do in Selenium such as stubbing DOM APIs.

One of the biggest gaps between Selenium and Cypress is that while Selenium runs against different browsers, as mentioned before, Cypress only supports Chrome. Selenium’s browser support is often integral to those who use it, making Cypress severely lacking in the area of cross-browser testing.

For software teams that find importance in checking their website on multiple browsers instead of just leaving their code up to chance, finding their way around this deficiency could prove challenging to Cypress users.

Cypress vs Selenium

Cypress and Selenium serve a similar purpose that is achieved in two different ways.

Selenium is a library but requires a unit testing framework or a runner plus an assertions library to build out its capabilities. Cypress provides a robust, complete framework for running automated tests but takes some of the freedom out of Selenium by confining the user to specific frameworks and languages.

If you don’t know what you’re doing when it comes to building out a test automation framework, Cypress may be a good option.

Another thing to keep in mind is that if you look at how Cypress is built, it is largely a unit testing tool that is will fit best in Javascript-focused development teams. Once you stray from these details and your team decides to experiment with other methods of test automation, you’ll find that Selenium can better accommodate those growing pains.

As a tool made by developers for developers, it’s less a replacement for Selenium and more so a way to introduce developers to test automation.

Like most instances when it comes to test automation, the tool you pick and the method you follow will largely depend on what your team decides and what works best for your organization. However, the notion that Selenium is on its last legs is misled.

If you’re still confused as to the differences between Cypress and Selenium, Automation Rhapsody provides a really great breakdown of how each performs in different categories.

Webinar: Meet the QA Team Behind America’s Test Kitchen

By Leave a Comment

america's test kitchen QA

America’s Test Kitchen is all about perfecting the recipe for success, literally.

With two TV shows, three print magazine, and four different websites, they’ve built a customer base that is hungry for quality culinary content. But their users are also paying close attention to how they’re being served their online experiences and want to consume it in a way that works best for their lifestyle.

This means the QA team is under a microscope when it comes to releasing a Michelin star web application that works across a large selection of devices and browsers. Whether users are taking a cooking course, shopping for kitchenware, or watching one of the pros whip up a delicious dinner, America’s Test Kitchen strives to satisfy from the backend to the front of the house.

Join CrossBrowserTesting and America’s Test Kitchen QA Manager Megan Collins & QA Engineer Michael Ng as we go behind the counter to learn about what kind of testing is needed to keep up with a company that aims empower and impress millions of home cooks.

In this webinar, you’ll get a taste of:

  • What the America’s Test Kitchen QA strategy looks like
  • The incredible growth of the testing team and their shift to automation
  • How they use CrossBrowserTesting to serve up their cross-compatible web app on a silver platter

Save Your Seat

6 Questions to Ask Yourself Before You Decide What Test Cases to Automate

By 1 Comment

decide what test cases to automate

Even just the idea of test automation is enough to get the attention of most software teams. But while the idea of faster testing may seem like the silver bullet, there’s a time and place for automation.

Before you jump in head first, you should evaluate how automation is actually fitting into your organization. By asking these questions, you can get a better idea as to whether automation is the right move, or if you should take another route.

  1. What’s the risk involved? If you’ve performed a risk analysis of your application, you probably have a good idea of which test cases are the most important to check in each code change and which are a lower priority. Consider how human error could affect test data if it’s not automated. Think about how different test cases that may critically impact the customer and/or impact internal team downtime if broken. Additionally, you might want to automate high traffic page, features that depend on security, and problematic areas and other functions that rank high on your risk analysis. Angie Jones talks about this in “Which Test Should I Automate,” which she’ll be presenting at SmartBear in a meetup on June 26. Register now or tune into the livestream.
  2. How often am I going to run this test again in the future? Running the same test more than once may seem repetitive and tedious, but if you’re not actually going to be running it frequently, it might not be worth it to automate. For example, if you know you’re going to have to test a feature two or three times but won’t be keeping it as a permanent part of the application or won’t be testing it after that, you might want to rethink automation. The tests you decide to automate should be a foundation of your regression testing strategy, so if you don’t feel like they’re a critical part of the application to be checked in every code change, it may be best to keep them separate.
  3. How much time am I willing to invest? When automation requires so much time and maintenance, you have to determine the ROI of your efforts. Some regression tests may seem like they’re a candidate for automation but upon closer inspection, might not be worth the energy for the value they’re providing. Automation is not a quick, one-and-done solution to your problems. Take a lesson from Elon Musk — you don’t want to drop automation into the mix without a second thought. Spend the time making sure your first test is checking all the boxes and meeting requirements. Then determine how often it will need to be updated or reevaluated in order to establish whether or not automation is worth your time investment.
  4. Are there parts of this test case that still need to be explored? You might think that you’re ready to automate, but keep in mind that automation comes after a thorough exploratory testing session. Just because you can automate doesn’t mean you should, and many tests are better off being performed manually. Are there still instances that could benefit from human observation? While you can’t expect to manually cover every part of the application, getting a foundation during exploratory testing is what leads to more successful, stable, and insightful automated tests.
  5. Does automating support the needs of my team? If you’re automating a test case, you want to make sure the tests you write are going to actually answer the questions your team is asking. First, take a hint from Bas Dijkstra, and discuss the “why” before firing up a test automation tool. While Selenium is a great tool for automation, understanding the nuances of the many commands and abilities of it can take some getting used to, so if you’re not able to write a script that encompasses all of the requirements your team is requesting, another method of testing may be better. If you still feel that you automation is the best process for the job but don’t feel confident in your Selenium skills, a tool like Record & Replay may help you achieve your testing goals. There are many choices when it comes to automated testing tools, so you need to make sure the one your organization uses will support the testing you’re trying to accomplish.
  6. Why are you automating? Sometimes the most simple questions can give you the most telling answers. If you’re automating because you think it’s what you should be doing or because your boss is telling you to, you might not be automating your tests for the right reasons. Your objective with test automation should be to provide insight to you your team and give faster feedback. If you have ulterior motives, your efforts won’t take you very far.

Conclusion

Even if you could automate every test, you would quickly find it’s not helping you or the rest of your team very much. Next time you’re deciding on whether or not to automate, ask the right questions first.

How Every Team Member Can Leverage a Test Automation Tool

By 1 Comment

test automation tool

With the introduction of test automation tools such as CrossBrowserTesting, it’s easier than ever for teams to meet the demands of Agile, CI/CD, and DevOps. However, while testing used to be a job strictly for QA, software teams are finding that this no longer applies to their development process. As organizations continue to shift left, the lines continue to blur between different roles, and testers aren’t the only ones testing anymore.

If your entire team is ready to take on more testing, here’s a few ways each member can best leverage a test automation tool.

Digital Marketers – Think that your marketers have no use for testing? Think again. Anyone on your team that cares about Search Engine Optimization (SEO) will likely be aware that mobile responsiveness is a factor when it comes to domain authority. In fact, Google has stated that responsive design is the recommended design pattern, which means that web applications that are optimized for mobile will actually show up higher in peoples’ search results. Your marketers may want to take advantage of a visual testing tool so that they can evaluate priority pages on different devices and screen sizes to determine whether or not they’re responsive. Ensuring mobile-friendliness will also benefit SEO by accelerating site speed, decreasing bounce rate, and boosting social sharing, making testing a marketer’s secret weapon.

Product Managers – Testing doesn’t have to (and shouldn’t) wait until the end of the SDLC. In fact, it can take place as soon as the planning stage, especially when shifting left. Product Managers can expertly use the data of past test results to influence the direction of the product, but they can also perform their own testing to make immediate improvements and determine what changes should be made in the application next. By focusing on the usability of the application, product managers can perform exploratory tests to better understand what forms could be more intuitive or determine if the site is optimized for accessibility, for example. Additionally, product managers might find value in testing newer features recently implemented by the development team that impact usability (home page layout, checkout process, etc.) to make sure it fits into the buyer journey they way they intended.

Designers – If your designers have been manually testing your website to make sure it looks good across browsers and devices, that’s a good first step, but they shouldn’t stop there. Visual testing allows them to automatically capture screenshots of a URL on tens of browsers and compare them side-by-side. By picking a base browser, they can see where there are differences in each layout and decide which are acceptable and which need to be fixed. Whether an app is off by one pixel or a hundred, your designers can determine for themselves whether everything looks the way they want it to across configurations. It’s also important to make sure everything works in addition to looking exceptional, which is why Record & Replay helps test across the same browsers to make sure things like buttons, navigation, and link are all working as expected.

Developers – While we believe that every software development team should have it’s own dedicated QA team, that’s not to say that there are not certain times where developers should test their own code. It goes without saying that when a new feature is added, change is made, or bug is fixed the developer should ensure basic functionality of that new implementation after the code is adjusted by testing it. For longer test cases or more elaborate projects that restructure large areas of the application and require more extensive, end-to-end testing (such as you might see with a website rebrand), developers might want to also run one-off tests without taking up too much time. By leveraging prior programming knowledge to learn Selenium commands, developers can ensure functionality without spending hours on testing. Instead of manually executing a basic test case, they could write an automation script in a fraction of the time and still cover large areas of code to check whether it passes or fails.

Manual Testers – Manual testing will always be a crucial part of QA because it depends on the curiosity, observation, and skill of the individual to execute tests that will provide insight to the rest of the test. However, while exploratory tests can be the best part of the job, manual regression tests can be the bane of their existence. Despite the name of the role, manual testers no longer have to be confined to strictly manual testing. With test automation tools like Record & Replay, manual testers can automate the tedious, boring, and repetitive testing that they’ve come to dread without having to learn a programming language or getting up to speed with complex testing frameworks.

Automation Engineers – For the tester that’s mastered Selenium and has too many UI tests to count, how can they speed up testing to take it from days to hours? Parallel testing is the key to making your automation time even faster. By simultaneously testing in multiple browser configurations at once instead of one after another, parallel testing allows you to take your suite and divide the time spent running tests. Just by running a test in two browsers at once, you’re already cutting your automation time in half because you’d be testing them all at the same point in time. You can see where this would really pay off as you continue to add more tests, create longer suites, and run them on more browsers.

Conclusion

When everyone in the SDLC is moving fast to implement and deliver new features, testing is no longer a one-man job. In order for other team members to meet their deadlines, they must also be held accountable for some of their own testing.

While we can never replace the important work of our QA teams, there are tools that can help ensure that everyone plays a part in ensuring speed and quality from planning to deployment.

From Bad to Brutal: Performing a Risk Analysis of Your Web Application

By 1 Comment

risk analysis in software testing

Test automation is great, but even with the ability to cover more parts of your application in less time, testing everything is impossible. This is why you have to perform a risk analysis of your web application in order to decide when certain test cases get performed and how often.

There are many levels to creating a risk analysis, and deciding which tests go to the top level depends on largely on your business needs and the state of your application. However, these five considerations will give you a good place to start when trying to decide your most important tests.

  1. Most Popular and Problematic Browsers – Take a look at Google Analytics or Mixpanel to see which browsers, operating systems, and devices are being used most often to access your web page. The number of configurations you include in your testing will depend on how fragmented your user base is. You want to ensure basic functionality for at least 80 percent of your users, so if 80 percent are between five browsers, you can get away with including those five in your first tier of testing. However, if it’s split between ten configurations, you’ll want to allow more time for cross-browser testing. If you do find browser use is highly fragmented, parallel testing is a good way to check off your boxes in multiple configurations at once. But we’re talking about risk here, so you also want to think about which configurations cause the most problems for you. IE 8 always breaking? Make sure to include it in your first tier.
  2. Most Critical Unit and Integration Tests – Which tests will have the biggest impact on your bottom line if broken? For example, if you’re running an e-commerce website, the ability to enter payment details and submit them for processing is clearly important for allowing users to make a purchase. If they aren’t able to make a purchase and the application fails when they try to submit this information, they won’t be giving you their business and will probably go to a competitor. On the other hand, this will be different if you’re working in healthcare, finance, or government, for example. Identify the unit and integration tests that make up the foundation of your business as well as your application and associated risk on a scale from minimal to critical in order to determine which place at the top of your risk analysis.
  3. Most Common Landing Pages – Ask any marketer — a big part of the success of any web application is a user’s first interaction with it. This means that you should be looking at the pages with the heaviest traffic to determine which should be tested most often in regression. Choose five to ten pages that users land on most often. Whether that includes the homepage, pricing page, or the “about us” page, you want to make sure these are consistently up to par with a healthy mix of functional testing and visual testing.
  4. Security and Privacy – Ensuring security in any instances where personal information is being requested from your users should be of top importance in your risk analysis. Take note of any times where payment is being received, passwords are being entered, contact information is input, or confidential information is requested. In addition, any part of the application that may have an impact on the safety or financial status of your users should always be a priority to ensure that their data is uncompromised. Just ask the companies that have been victim to some of the worst software bugs in history — once you lose user trust, it’s hard to get it back.
  5. Extended Downtime – Which functionalities will take the longest for your team to get up and running again? While your users should usually be your first priority, sometimes risk goes beyond their inconveniences when it puts the entire team on a backlog. If you already know it’s not going to be an easy fix that could affect the application downtime for days and team productivity for weeks, it should probably be high on your list when thinking of risk. Where have you had issues in the past? History repeats itself in life and your application. Log and report areas that have a high probability of failure so you can prepare your team and avoid excessive repair time.

You have to be familiar with the risk associated with different functions and features in an application to know what should be tested first, what should be tested again, and what you can deprioritize.

While you can never guarantee that code will be deployed with zero bugs, you can develop a strategy to ensure that any issues that do slip through the cracks won’t be detrimental to your application.

7 Reasons Your Testing Team Will Love Record & Replay

By Leave a Comment

Imagine there was a way to have your manual tests recorded so you could easily replay them on hundreds of browsers without needing to perform the same exact actions over and over again. Record & Replay does just that, allowing you to test once and play it back on the largest selection of browsers in the cloud.

This may seem too good to be true, but many testers have already experienced the power of a codeless automated testing tool. If you’re still a little lost, we’ll make it easy for you. Here’s exactly why you should be excited about Record & Replay:

  1. No coding knowledge needed – Have you been wanting to experience the speed of test automation but having trouble getting started with Selenium? If you don’t know how to code, you have a long way to go from learning a programming language to writing your first working script. Luckily, with Record & Replay, no coding knowledge is needed. Rather than writing script, all you have to do is perform them and CrossBrowserTesting will record the actions. If you’re struggling with complicated frameworks and commands to create a successful Selenium test, Record & Replay is a hassle-free way to automate with the skills you already possess.
  2. Ability to edit tests – What happens when you perform the wrong action in Record & Replay or want to change part of a test? No need to stress, all you have to do is go back and edit recorded scripts. Delete mistakes with one click, or go in and edit your script by clicking on the script name and “Edit Steps”. There, you’ll see all the recorded values for each step variable, and once you’ve made your adjustments, all you have to do is save it to put your changes into action. Whether it be a typo or missed step, there’s no reason to spend valuable time redoing an entire test when all you need to do is fix one part of it.
  3. Unbeatable browser selection – There’s no question — CrossBrowserTesting offers Record & Replay on the widest selection of browsers. In fact, most of our competitors aren’t doing it at all. But we aren’t here to brag, we just think that a test is only as valuable as the number of browser configurations it can be run on. That’s why it makes sense that Record & Replay users have access to our numerous configurations, which means whether you want to test IE 8 or the newest Chrome release, you have the resources to check cross-browser functionality where and when you need to.
  4. Parallel testing capabilities – What’s better than automating one test at a time? How about automating ten in parallel? With Record & Replay, it takes seconds to select your test suites and play them back at the same time against different browsers. Parallel testing gives you time back to focus on exploratory test and untouched code in your application. Stop running test after test in different browsers, and start running them all at once to make time for the tests you haven’t run yet.
  5. Access to recording tools – Since tests are recorded as a script, sometimes you may want to insert certain commands that you can’t do manually to get the results you want. For example, if you want to verify element properties to keep visual consistencies across different browsers, you might need extra capabilities to allow you to do that. The tools included in Record & Replay allow you to add quick commands such as expects, sleeps, and hovers to make sure you’re making the most of your tests.
  6. Start shifting left – Software development teams aren’t shifting left because it’s the cool thing to do, it’s actually just a really good way to make sure quality is accounted for earlier. Test automation has been a big part of the shift left movement, allowing teams to test faster and thus earlier and more often. For teams that haven’t yet made the shift or who are just getting starting with automated testing, Record & Replay provides an on-ramp to automation, and sharing capabilities through Slack and Jira means encourages open communication with your entire team so you can get testing involved sooner in the software development lifecycle.
  7. Testing for all – As the roles in the software development lifecycle begin to overlap and blur, testing isn’t just for testers anymore. While we do believe that organizations should depend on a dedicated QA team, there is more room for other software stakeholders to take part in some small testing tasks as well. Previously, the ability to automate was only available to automation engineers or testers with programming knowledge. Now anyone can test from developers and product managers to marketers and designers can take part, so your entire team can run like a well-oiled machine.

Conclusion

If your QA team is still playing a game of catch up to a development team that’s continuously integrating new code, it’s time to get on board with an automated testing tool.

Trying to understand automation can be daunting, but it doesn’t have to be. Record & Replay can help you take the first steps to faster testing and increased code coverage to support your team from planning to deployment.

The 5 Ways Parallel Testing Can Instantly Improve Your Workflow

By Leave a Comment

benefits parallel testing

You’ve probably heard of parallel testing before, but the ability to multiply your testing power without multiplying your testing time seems something of a QA fantasy if you haven’t tried it yet. But once you experience parallel testing, you’ll never want to go back to life before it.

While there are probably hundreds of benefits from parallel testing, everyone will gain the value in different ways. In case you need to be convinced (or you need to convince your boss), we rounded up the top five advantages you can expect from parallel testing.

  1. More Browser Coverage – Take one test, run it across multiple browsers at once — that’s the beauty of parallel testing. If you’ve been running tests on different browsers one after the other, or even just skipping browser testing all together, you’ll realize how valuable parallel testing is after your first time trying it. Instead of only testing on one or two browsers, you can substantially increase your cross-browser coverage with parallel testing, hitting configurations you didn’t even realize were an issue. Plus, with CrossBrowserTesting’s broad selection of browsers, operating systems, and devices, you have seemingly endless options for checking the functionality for any and every user.
  2. Less Manual Testing – We’re not saying manual testing is a bad thing — in fact, it’s a pillar of a successful testing strategy. However, when you’re manually performing the same regression tests over and over on different browsers, you’re not making the most of your time. Not only is this boring, tedious work, you’re also risking human error and inaccurate test data by trying to repeat the same tests. Parallel testing allows you to repeat these tests in regression without worrying about skipping a step so you can be confident you’re reaching all the right parts of your application.
  3. Saves Valuable Time – Besides increasing test coverage, parallel also decreases the amount of time you’re actually spending on each test. Since you’re running multiple tests at the same time, rather than one after the other, you’re testing gets done in the fraction of the time it normally takes. In fact, just running 2 tests in parallel doubles your speed, and the more you run in parallel, the more time you save that can be spent elsewhere. This means you can redirect that energy to go back to performing the exploratory tests and looking at new parts of your application at while still meeting your deadlines.
  4. Improving the Development Workflow – Not only does parallel testing make your job easier, it allows for better communication between departments and improves the productivity of your entire team. By allowing for faster feedback and better reporting, parallel testing equips you with the ability to relay results from testing back to your team earlier in the development cycle. This means that parallel testing will be useful in your organization’s effort to shift left and implement Continuous Integration and Delivery practices that will get iterations to market faster and with fewer bugs.
  5. Go Scriptless – With CrossBrowserTesting, you now have even more ways to run your tests in parallel. With our Selenium grid, you can use your Selenium skills to write tests and run them in parallel. With Record & Replay, you can also achieve parallel testing with zero coding knowledge needed by executing a test manually then rerunning your suites across the browsers your choose. This means teams who are both new to automation and experienced in Selenium can benefit from the capabilities of parallel testing.

We could talk all day about how amazing parallel testing is, but don’t take our word for it. Whether you’ve automated 100 tests or 1, you can easily start running tests in parallel now to experience more code coverage with less time spent on manual regressions.

Try your hand at running a test in two browsers at once then take it to the next level by adding more, and witness pure testing magic right before your eyes.

Record and Replay Testing vs. Scripting: Which is Right for You?

By Leave a Comment

record and playback testing
With so many tools on the market claiming to help you automate your tests, what do you choose? Between Selenium and record and replay testing, it can get a little confusing trying to differentiate and pick the best one for the job.

When it comes down to it, the most effective way to determine which path is better is to understand what the differences are and what you and your team need in a testing tool. All test automation tools have certain strengths; once you know how to leverage them, the answer becomes clear.

What is Record and Replay Testing?

Record and Replay, otherwise known as codeless automation, is a way to run tests without programming knowledge. This is done using a tool, like CrossBrowserTesting, that allows you to manually perform actions in the browser and save them as a test. You can then take this test and rerun it with a click, making regression and browser testing infinitely faster.

This means that manual testers or testers that are still new to Selenium can easily create automated tests. In addition, this allows other roles such as product managers, developers, designers, and even marketers participate in testing without having to learn an entirely new tool.

Record and Replay is the lightweight solution for test automation. The value is the most prominent for teams that are still transitioning from mostly manual testing to include some automation in order to speed up testing and help integrate it earlier in the software development lifecycle.

While Record and Replay may not be the ideal solution for teams who are well-versed in the intricacies of Selenium and who have lofty goals for code coverage on a multi-faceted web application, it is a great way to get started with automation.

Record and Replay is good for:

  • Individuals with little or no programming knowledge
  • Filling in the gaps of Selenium tests and transitioning from mostly manual testing
  • Lightweight automation for smaller tests
  • Non-technical roles doing one-off tests
  • Teams where members outside of QA take part in some testing

What’s Scripting?

Scripting is what you might traditionally think with test automation — writing a test using Selenium commands and coding knowledge. In order to do this, you do need to have a foundational understanding of at least one programming language as well as what makes up a Selenium script and Selenium commands.

As teams focus on shifting left to incorporate testing earlier and integrate more often, Selenium should eventually be a part of this strategy. However, while Selenium is an invaluable tool for testing, it does have a steep learning curve. Scripting is a great option if there are existing team members that know how to write them, but you might want to seek out an alternative if not.

Fortunately, there are plenty of resources for learning Selenium because anyone who uses Selenium will tell you how it’s worth it. If your team knows how to leverage frameworks, maintain tests, and create stable scripts that provide insight for your team, you’ve probably already witnessed how valuable Selenium is. Additionally, you can multiply the effectiveness of your tests by leveraging cloud tools like CrossBrowserTesting to extend your browser library and capabilities.

Selenium is good for:

  • Highly technical practitioners that know the ins and outs of writing a Selenium script
  • Enterprise teams with an experienced QA team, expansive web applications, and a high volume of users
  • Creating robust suites and end-to-end tests
  • The largest reach in terms of automation speed and code coverage

Automation for Everyone

While Record & Replay and Selenium have different uses, they do share features in CrossBrowserTesting that will benefit anyone.

A stand-out detail is the ability to test on one of the largest browser selections in the cloud. This means that whether you’re a seasoned automation engineer or first-time tester, you can run a single test across an enormous array of browsers.

Additionally, you also have the ability to test in parallel. We can’t say it enough, parallel testing is one of the best ways to effortlessly accelerate the speed and coverage of your testing.

Once your tests are saved in Record & Replay, you can run up to 10 scripts in parallel across dozens of browsers with a few clicks. With Selenium, you use multithreading, which you can read more about in this blog about running your Selenium test in two browsers at once.

But what if you’re still having troubling choosing? What if some people on your team are automation novices, while others are Selenium professionals? With CrossBrowserTesting, you don’t have to choose. We have both Record & Replay and a Selenium grid, which means that every organization, every team, and every tester can discover the best way to achieve their automation goals.

Join Over 200,000 Testers & Developers And Start Testing Today

Learn more Free Trial