CrossBrowserTesting.com

A Design, Development, Testing Blog

Running Automated Testing Software in an Agile Environment

By Leave a Comment

Claire Mackerras Automated Testing Agile Environment

There is no doubt that automated software testing is critical for maintaining agility and establishing continuous improvement. As more teams adopt both automated testing and Agile development, it becomes clear they are best practiced together.

Any testing that is being performed for more than two or three times in a short span of time can be automated with paid and/or open source tools. Execution of unit tests, functional tests, continuous integration, and continuous deployment are common ways of utilizing automation for the greatest benefit.

Automating software testing is especially preferred in an Agile environment because it accomplishes the following tasks:

  • Identifies test productivity problems
  • Swiftly responds to the testers
  • Helps testers discuss and examine solutions
  • Implements technology to improve test processes
  • Assess automation possibilities for upcoming projects

Benefits of Automated Testing in an Agile Environment

Automation technology accounts for repeated testing and delivers extensive functionality and coverage through the software testing life cycle. If automation software is correctly executed, it provides the following benefits:

  • Improved Software Quality – Consistent, repetitive and extensive automation tests helps to deliver high-quality software
  • Improved Documentation – Automated tests produces instant documented quality proofs, comprises extensive flop/pass consequences, brings down testing manpower and time
  • Bring down testing manpower and time – As automated testing is quicker than manual testing, as these are labor-intensive, and can do verification of thousands to millions of test permutations from minutes to hours.
  • Cheaper – It also helps bring a remarkable reduction in the expenses related to the production of best-in-class software.

Automated Testing Trends

Agile environment automated testing

Organizations have largely merged automated testing and Agile development due to major trends that have arisen in the last few years.

Let’s take a brief look at some of the trends as follows:

1. Mobile Testing

Mobile testing will continue to remain at the top of the list for automation trends, owing to the remarkable shift in the adoption of mobile technology. Mobile application testing will indisputably have a major impact on the software testing industry with functionality, compatibility, performance, usability and security testing as major areas of focus.

In the next few years, mobile test automation will match web test automation if the market for mobile continues to rise at the same pace. There’s a great possibility in the open-source framework, mobile development, cross-browser testing, and app markets to expand for the mobile test automation landscape.

2. Big Data Testing

With an increase in Big Data analytics, quality management for Big Data implementations will emerge. Applications, networks, and devices will generate a huge source of unstructured data, ultimately leading to faulty and costly decisions.

Testing quality ahead of product release will continue to remain critical. With this type of testing, businesses will aim to get hard numbers and statistical indicators on whether the application behavior will meet the demand of customers or not.

3. Testing with TDD & BDD

Enterprises are likely to adopt TDD and BDD to deliver applications as per the customer needs, without any defects or delay. By engaging in this technology, teams can get an actual idea of what is required.

As a result, enterprises are better off in terms of cost and time since the longer it takes to identify bugs, the more it costs to fix them.

4. DevOps

Alliance among IT professionals, developers and operation engineers has increased since the emergence of DevOps. More and more organizations are merging the role of testers with operational engineers and developers due to overlapping competencies in the continuous Agile cycles.

Experts predict that testers, developers, and operation engineers will similarly need to utilize the same tools to increase the collaboration. This technique will promote better communication and feedback among teams to achieve Agile development, testing, and delivery.

6. Micro Testing

Enterprises financing in Micro Services make it easier to test an independent piece of functionality without altering other applications.

This trend will inevitably rise in the coming years, as it certifies testing complex applications which comprise small, independent processes and architectures.

The Verdict

Automated Testing is considered by many software testers as one of the most forthcoming trends practiced in the overall software development lifecycle today. With the help of automated testing, one can get more effective and efficient testing results in an Agile environment.

About the author: Claire Mackerras is a Senior Quality Assurance Engineer & Editor at BugRaptors. BugRaptors is a CMMi5 certified company with extensive experience as a third party testing vendor for tools in Software Testing & QA. Claire enjoys creating, uncovering, and disseminating new and interesting perspectives on technology and software testing.

Debunking 6 Automated Testing Myths

By Leave a Comment

Automated testing myths and misconceptions

Automated testing has been the answer for many teams who wanted to speed up and increase their testing efforts, but misconceptions surrounding automation mean that testers don’t always get the most out of their tools.

In an attempt to correct frequent misunderstandings, we’re busting common automated testing myths to give you the truth.

Myth: You should automate 100% of your tests.

Fact: You can’t test everything. Though the ability to perform agile automated tests is a great skill, it’s impossible to automate every test. And even if you could, it wouldn’t be an efficient testing solution. Automated tests are based off source code that’s built based on manual tests. Meanwhile, some tests are better off being executed manually. Anytime a new integration is added, a new test has to be done because automated tests will only check previous scripts. Experienced automation engineers know that prioritizing tests and knowing when to automate is just as important as knowing how to automate. However, though you’re not able to automate every test, automation will greatly increase the number of tests you’re able to run.

Myth: Developers should lead automated testing efforts.

Fact: Automation Engineers should drive automation. While it does take programming and coding knowledge, most of the time it’s the testers that are building automated test scripts. As the software industry begins to depend on new testing methods and development processes, test automation architects and engineers will be in high demand. In fact, most of the time having an automation engineer is preferred to a developer because testers have the mindset, training, and specialized skills for different automation tasks, which include much more than just coding.

Myth: Automation is more expensive than manual testing

Fact: If utilized correctly, automation can save money. While the initial investment in automated testing tools and hiring testers may seem more expensive, the ROI of test automation is often worth it since it saves time that would usually be spent on manual testing and increases productivity. For example, if you’re manually running tests that could be automated, it’s costing you in wasted time because your testers could be doing something more constructive. If you think about this scenario over weeks, months, or years, that’s a huge opportunity cost. Test automation is also a cost effective way to extend testing coverage, scale testing efforts, and mitigate risks as your application grows.

Myth: Automated testing is better than manual testing

Fact: Neither automated or manual testing is better or worse than each other, they just have different uses. The most effective testing strategy will find a happy medium between automation and manual testing. Manual testing is better for one-off tests or tests that require human observation, so exploratory tests, smoke tests, and usability testing should all be manual. Automation is usually best for repeated tests or things that need to be frequently tested, such as regression tests. Familiarizing yourself with the different uses and applying them to the appropriate test cases is the best way to get the most out of your time, money, and testing efforts.

Myth: Automated Testing is an easy fix.

Fact: Automated testing isn’t a one-and-done solution. You can’t set up an automated test and expect that you’ll never have to look at it again. In fact, the source code of your automated tests takes a good amount of maintenance to check that it’s doing its job. Not to mention that changing an organization’s testing and development processes will require new training and onboarding to familiarize stakeholders with automation and writing sophisticated tests. Though the ROI of test automation will likely be apparent over time, the transition to a mostly-automated process from a mostly-manual one will take commitment, patience, and financial investment.

Myth: Automation will solve all of your problems.

Fact: Automation is not a silver bullet. Automation engineers experience false positives and flaky builds that come from a lack of human observation, maintenance, and company-wide collaboration. For example, common challenges using Selenium for automated website testing might include coding for cases with more complicated elements like dynamic content, multiple tabs, and pop-up windows that are easier to test manually. Testing will still require the specialized skills of a talented testing team to troubleshoot problems, write scripts, manage tools, and maintain test cases. For some best practices of Selenium testing, check out this guide.

How To Take an Automated Screenshot With SeleniumBuilder

By Leave a Comment

 

Selenium Builder was a life saver for many testers when SeleniumIDE became…a bit deprecated. With Selenium Builder, manual testers and less technical team members were still able to record and replay automated tests against a variety of browsers. When we came out with our integration for Selenium Builder and CrossBrowserTesting’s Cloud Grid, we also saw how awesome it would be to integrate it with our Automated Screenshots. So we did, and this is how you can use it to easily take automated browser screenshots of your website after a complex set of actions.

First, let’s begin by installing Selenium Builder and getting familiar with it. Selenium Builder is a Firefox Add-On you can find in their Add-On store. Once you have downloaded it, you’ll have to restart your Firefox browser. After you’ve done that, let’s head on over to Google.com.

In this example, we’ll want to take an automated screenshot of the second search results page for “Cross Browser Testing” on Google. This is a fairly complex action, and hey – it might help you with tracking SEO.

Right-click your screen, and select Launch Selenium Builder from the menu.

That will bring up Selenium Builder in another window:

Let’s click record and start recording our test. Head back to the Chrome window and type “CrossBrowserTesting” into the Google search bar, and click enter. Once that page loads, click Page 2. After you’ve finished, go back to the Selenium Builder window and click Stop Recording.

Now that we’ve finished recording our script we’ll want to export this script in a file format compatible with CrossBrowserTesting’s Automated Screenshots. By clicking File, and then Export, we’re given a few different options formatting our export. We’ll choose JSON, save the file with the title “google-screenshot.json”, and then open the file and copy the text <– we’ll need this soon!

Here is my JSON file after recording our test.

Now let’s head back to CrossBrowserTesting to feed our test into the Automated Screenshot tool. Once logged in, click Screenshots in the left menu, and then Advanced Options. You should see a list of toggles – click the one called “Selenium Script”, then click New.

This will bring up a modal for us to name our script and paste it in from our clipboard. After you can see your script in the entry modal, click Save.

Okay, now we have our script created and loaded into CrossBrowserTesting. Now, the only thing left to do is run the test. Click Run Test and we’ll start running your test against our real browsers and return you a high-def, full-page automated screenshot!

Everything You Need to Know About Load and Performance Testing

By Leave a Comment

Performance Testing 30 Days of Testing Challenge

We love following the Ministry of Testing #30DaysofTesting challenges. For May’s 30 Days of Accessibility Testing, we participated by talking about how top tech companies are integrating accessibility into their services, platforms, and routines. For July’s 30 Days of Performance Testing, we wanted to discuss the fundamentals of performance testing.

Hopefully, by going over the issues, approaches, and outcomes of performance testing, you’ll be able to check more than one box off your list for the challenge.

What is Performance Testing?

Performance testing is used to see how well a software can handle user traffic. By putting a simulated demand on an application or website, it’s possible to analyze breaking points and evaluate expected behavior. Specifically, performance testing is used to measure responsiveness, stability, scalability, reliability, and speed.

As a code change from a team that is continuously integrating new features and bug fixes can impact how an application looks and functions on different browsers and devices, it can also affect how quickly that application loads across machines.

This is why performance testing is so crucial to a well-rounded QA strategy — checking an application’s performance and ensuring consumers are experiencing acceptable load time and site speed is foundational to high-quality software.

Performance testing also helps teams analyze activity so they predict traffic trends. This allows them to better prepare for breakpoints and site dropouts for the future.

Load Testing vs Stress Testing

When we think of performance testing, we can usually separate methods into two categories — load testing and stress testing.

Load testing looks at normal conditions of your application — how quickly does it load when one user visits your website? How quickly will it load when 15 people are on your website at once? Load testing is used to make sure that software works as expected in ordinary circumstances.

Alternatively, we have stress testing. Stress testing challenges the limits of your application to see how much traffic it takes to slow down performance or break completely. By simulating 10,000 users, for example, a website will likely perform differently under that pressure than it will in everyday use. While stress testing encompasses unlikely test environments and user scenarios, it’s important for analyzing risk and breaking points.

Additionally, many B2C companies and online e-commerce sites depend on stress testing before events where high traffic is expected such as Black Friday, the Super Bowl, and Election Day. If you aren’t stress testing in these situations, just a few minutes of downtime could result in thousands of dollars in missed sales at a time that is critical to the business’s success.

Monitoring

While performance monitoring is has a similar goal to performance testing — to ensure the that an application is functioning under expected traffic conditions — it’s executed in an entirely different way. Rather than simulating user behavior, a performance monitoring tool like AlertSite is set up to watch the application, measure response time, and send alerts if the site goes down.

This is important for the same reasons that testing is important. While you want to be able to test the performance of your web application before your users encounter a problem, you also want to know if and when your users encounter a problem so you can promptly fix the issue before downtime noticeably affects your reputation or revenue.

What Happens When You Neglect Performance Testing

While some organizations are slower to adopt a complete performance testing strategy, they’ll quickly realize that having no strategy will negatively impact bottom line goals.

Here are a few of the ways that performance testing affects business ROI, according to Radware:

  • 51% of online shoppers in the US say that site slowness is the top reason they’d abandon a purchase.
  • Shoppers remember online wait times as being 35% longer than they actually are.
  • A 2-second delay in load time during a transaction results in abandonment rates of up to 87%.
  • The total cost of abandoned shopping carts for online retailers has been estimated at more than $18 billion per year.
  • 64% of smartphone users expect pages to load in less than 4 seconds.
  • When faced with a negative mobile shopping experience, 43% of consumers will go to a competitor’s site next.

As we witness a rise in mobile device use and multi-screen behavior, regular app updates, and demand from users for instant gratification with technology, it’s evident consumers are less likely to use applications that don’t meet their high expectations and more likely to leave your site and buy from faster competitors.

In turn, through regular load and stress testing, users will be more likely to stay on your site from the first time they land on the homepage through checkout. Additionally, by establishing trust as a fully-functioning application, they’ll be more likely to come back.

Performance Testing Tools

Performance testing is not complete without the assistance of a few tools, whether those be open source, paid, or a combination of both.

These are the tools that teams find are integral for load testing, stress testing, and web application monitoring.

  • JMeter – Apache JMeter is a go-to open source tool for load testing and measuring performance. As a purely Java-based application, it’s used for recording, building, monitoring, and debugging on different applications, servers, and networks. People prefer JMeter because of its user-friendly installation and GUI, multi-threading framework, and visual results.
  • Gatling – Gatling is another open source tool for performance testing on web applications and based on Scala, Akka, and Netty. It’s used to help predict and detect issues with features that allow you to automate tests, record tests, edit scenarios, analyze bottlenecks, and share results.
  • Fiddler – Fiddler is a free debugging proxy tool used to monitor traffic from web applications. It can be used on any browser, system, or platform and lets you see the total page weight, isolate bottlenecks, record traffic, and troubleshoot.
  • Selenium – Almost all things testing are possible through Selenium, and while it’s not particularly made for load testing, Dave Haeffner gives a pretty good rundown on how to use the beloved open source tool to make it work.
  • WebLoad – WebLoad also has both a free and a paid version. Similar to Neoload, the free version offers 50 virtual users. Additionally, the Load Generation Console simulates large user loads on a local cloud, the analytics dashboard provides shareable report templates, and it integrates with other open source software like Selenium and Jenkins.
  • LoadComplete – LoadComplete is used as a desktop tool for load and stress testing websites without requiring advanced programming or automation skills. It includes features such as record and playback as well as visual programming and allows you to generate load from VMs, local computers, and the cloud for a comprehensive performance testing and monitoring strategy.

Conclusion

As you continue the 30 Days of Performance Testing challenge with books, blogs, and podcasts, keep in mind the fundamentals of performance testing and the ways in which you need to leverage certain tools to apply specifically to your own web applications.

Keep in mind the high risk of not monitoring your software, load testing, or stress testing, and make sure your QA team has a strategy in place. By evaluating user behaviors and workflows, you can prepare for real-world scenarios without sacrificing scalability or customer loyalty.

What new knowledge have you gained about performance testing for the #30DaysofTesting Challenge so far? What would you still like to learn? Tell us in the comments!

What is Regression Testing?

By Leave a Comment

what is regression testing

When a developer goes to make adjustments in their software, the smallest change to code can disrupt previously flawless functionality. You know it worked before, so if it doesn’t work now, it must be something we introduced with our latest push, right?

Regression testing is the process used to ensure that a recent code change hasn’t impacted these features that were already working, and is also used to monitor and maintain production software that has already passed testing.

Not only do you want to make sure everything looks good visually, but you also have to make sure that it performs quickly and functions without bugs. By re-executing these test cases, it’s easier to evaluate whether everything still works the way it’s expected.

When thinking about regression testing, there are three types including unit, partial, and complete. Unit regression tests are used to test a single unit of code, partial regression tests are used for code modification, and complete regression is for on-going code changes.

While many testers acknowledge the importance of comprehensive regression testing, it may be difficult for a team to agree on the best method for its execution.

The Problem with Regression Testing

While regression testing is a necessary part of QA, it’s probably not your tester’s favorite part of the job. In fact, the repetition involved with manual regression testing can be really tedious after running cases the first few times.

While you’ll likely have to run the first regression test manually, executing regression tests manually is a huge waste of time and energy over an extended period of time. And it’s really boring to repeat those same actions over and over – really boring. Manually running regression tests is an open window for human error, since it’s difficult to perform the actions exactly the same over and over.

In addition, the best QA teams know that an accurate regression test has to be done across more than one browser to ensure functionality across different configurations, increasing the time and energy involved in the process multiple times over.

Fortunately, testers that take advantage of automated testing can program those repetitive regression test cases to run in the background on a nightly basis, or with every deployment. By using tools likes Selenium, Appium, Record & Replay, or other testing tools and frameworks it becomes much easier to run regression tests, while your testers have more time to focus on their other priorities.

Additionally, automating parallel tests in Selenium takes this a step further, allowing you to approach cross-browser testing by running separate tests at the same time.

To read more about parallel testing with Selenium, read this resource.

Should I Automate All of My Tests?

No. Automated testing will only assert whether the actions you tell it to are true or false, which is why it’s great for regression tests — you’re going back to make sure nothing has changed over time.

However, automated tests cannot and will not find new bugs in unexplored areas of software. As your system scales, so must your exploratory and unit tests, as well as your regression tests. And, as we mentioned, your regression test cases will have to stem from manual test execution.

Once a test case is stable, it’s a good idea to perform regression tests on them in the future, but they will also need maintenance and validation as the system changes over time. You’ll probably have to create new regression test cases to compliment this change as well.

Additionally, a great benefit of automated regression testing is that it frees up a lot of time that can be spent manually testing other unexplored areas. You don’t want to use automation to take over testing completely, but to aid testing efforts that need to be maintained more than they need to be inspected.

Regression Testing Today

While regression testing is not a new term in software testing, new test management practices, tools, and methodologies have changed the way testers approach these tasks.

As organizations become more Agile and begin to continuously integrate new code, it’s important that those changes are verified and approved before new features and functionalities are delivered to end-users.

In fact, the industry shift from Waterfall to Agile has actually made more regression testing possible more often.

Since Agile teams have also adopted the idea of testing early and often, it’s allowed smaller regression tests cases to be performed intermittently rather than running long case at the end of development.

Instead of re-doing an entire test suite, running regression test cases as needed saves a lot of time a creating a baseline of quality. Additionally, regression testing becomes a crucial part of reducing risk and eliminating bugs before they reach the customer.

Luckily because of the high demand for regression testing, there are many tools that strive to make this easier. Whether you’re parallel testing or using a visual regression tool to evaluate inconsistencies, there’s no excuse to skip out on regression tests, and as your organization grows, you won’t want to.

By crafting a testing strategy, prioritizing automation, and using the right tools, you can utilize regression testing in the best interest of your software development and QA teams.

How much have you experimented with regression testing? What are the best ways you’ve found to streamline the process? Tell us your experience in the comments!

12 Reasons to Pursue a Software Testing Career

By Leave a Comment

Reasons to pursue a software testing career

Software testing gets a bad rap. But, people that don’t think a software testing career is rewarding, challenging, and fun surely aren’t software testers. In fact, most testers really love their jobs, and wouldn’t give it up for any job in the world.

When considering which direction to take your career next, here are a few reasons you should think about pursuing a software testing career.

  • It’s challenging – Testing is not easy — there are constantly puzzles and problems to solve. The job will likely bring something new every day. If you prefer a boring job where you don’t have to think too much then don’t pursue a software testing career. But if you want a job that keeps you on your toes, anyone will tell you that testing is a really great choice.
  • It’s important – Testers don’t always get enough recognition for the work they do, but we’d be lost without them. As a tester, you’re advocating the end user and making sure that they’re being delivered a quality product. Without someone to find bugs before software is delivered, many businesses would be suffering from poor reputations and unloyal customer bases.
  • It’s creative – You have to get a little innovative when testing. The process isn’t going to be spelled out for you — in fact, it takes a little detective work. By acting as the end-user, you’re the one who has to get creative when thinking of places there may be inconsistencies.
  • It’s data-driven – One of the coolest things about a software testing career is that it’s just as technical as it is creative. While most testers need to have a foundation of developing and coding as a baseline, they’re also analyzing day-to-day data and product trends. Having a knowledge of computer sciences is imperative,  as you have to be someone that knows the ins and outs of software and the way it works.
  • You’re constantly learning – Whether you’re starting to code, automate, or security test, there’s always more to learn in a software testing career, and you likely have a very supportive team behind you to make sure you have all the resources you need to be the best. Plus, your work will never be stagnant, as you’ll be continuously growing and improving your practice.
  • You get to test the limit – You can’t go around smashing people’s laptops, but you do get to explore software to see where it stops working. Of course, testers don’t actually break software, but it can be really fun to find a bug no one thought existed in software everyone thought was working perfectly. It takes especially critical eyes to find these kinds of problems and solve them.
  • It’s in demand – If you want a high growth, high paying career, QA is the way to go. As a software tester, you’ll always be needed and will find no lack of leading companies trying their hardest to recruit you, and there are constantly opportunities to grow in your career to reach a managerial level.
  • There are many paths – Every company that uses software needs software testers, which is to say, pretty much everyone needs software testers. Testers are valuable in basically any industry, from healthcare to retail to video games. Additionally, you can choose whether you want to go into manual testing, automated testing, performance testing, etc.
  • It’s a specialized skill – Despite misconception, not anyone can test. Most testers start in a similar field and find themselves being drawn to the role, but it requires in-depth knowledge of UI/UX design and development patterns and practices, as well as analytical and communication skills. Not everyone will find that they have what it takes to be a tester, but those who find it’s their calling are sure to fall in love with it.
  • It’s rewarding – You’re essentially helping your company build a better product. If you take pride in your work and these people you work for, then testing is an exceptional way to make a measurable difference in your organization’s goals, objectives, and bottom lines. In fact, you’ll probably see your impact every day on the job.
  • You’ll love your colleagues – As the software industry increasingly trends toward concepts like Agile development, Continuous Integration, DevOps, and automation, communication becomes a highly important skill for software testers. This means that throughout the day you’ll always be collaborating with smart, interesting, and passionate people who share the same interests as you at every level in the organization.
  • There’s a strong community – Additionally, one of the best things about being a tester is the insanely supportive and robust community. From StackOverflow to Twitter, testers are a tightly knit group who have each other’s backs and always enjoy discussing best practices and trending topics in the industry.

Why did you become a software tester? What’s your favorite thing about your job? Share with us in the comments!

How to Test a Pop-up or Alert in Selenium

By 1 Comment

how to test a pop-up in selenium

Whether it be an advertisement, a call-to-action, or a warning, pops-up are a common way to draw the attention of users.

However, when these alerts and overlays are implemented, it may interfere with testers who are keen on test automation. So, how do you use Selenium to close out of a window, or automate testing to interact with a pop-up? We demonstrated the commands for basic pop-ups to make test automation as seamless as possible, without reverting to manual.

Alert Types

There are a few distinct types of alert messages that pop-up with varying user asks.

Simple – A simple alert will usually give minimal information or a warning and only require one, such as clicking an “OK” button to confirm you understand the message.
Prompt– Usually a prompt alert will require the user to input some information such as a name, email, or password before being able to click “OK” to continue.
Confirmation – Confirmation alerts will request permission, which can be answered by clicking “OK” or “Cancel”.

Pop-ups and alerts are usually either Windows-based or web-based. Windows-based alerts are beyond Webdriver capabilities because they’re part of the OS instead of the browser, so you’ll probably need to rely on third-party tools such as Sikuli Script or Robot class.

Using WebDriver for Web-based Pop-ups

WebDriver, on the other hand, does the ability to interact with multiple windows, including alerts, with a method called switchTo. This allows you to switch control to the pop-up while keeping the browser in the background:

  popup = @driver.switch_to.alert

From there, these are the simple alert commands for web-based pop-ups:

  • To click “Cancel”
       popup.dismiss
  • To click “OK”
       popup.accept
  • To capture the alert message
       popup.get_text
  • To enter information
       popup.send_keys('Text')
  • To exit
       popup.close


An example of this might be:


   require 'selenium-webdriver'
   require 'rspec/expectations'
   require 'TimeoutException'
   include RSpec::Matchers

   def setup
      @driver = Selenium::WebDriver.for :firefox
   end
   run do 
      @driver.get 'URL'
      @driver.maximize.window
      @driver.find_elements(css: 'button')[1].click

      popup = @driver.switch_to.alert
      popup.accept

      textreturned = driver.find_element 'result'

      print(textreturned.text)

      assertTrue(textreturned.text, 'You Clicked OK')
      
   except TimeoutException:
      print 'no alert'
  
   def teardown
   end


These commands should take care of any functions you want to perform when automating testing where there are any simple, prompt, or confirmation alert-pop-ups in your browser.


What are the most common challenges that you face testing with Selenium? Tell us in the comments!


9 Signs You Should be QA Testing in the Cloud


 By  Leave a Comment 
cross-browser testing cloud testing


We’re all familiar with the cloud. As a topic that’s clearly trending and growing in software, our appreciation of the cloud testing stems from its innovation as well as its effectiveness.


The cloud is only projected to become more popular and more common in years to come. In fact, 95 percent of organizations are using a cloud in some way, and the market is supposed to increase 22 percent annually according to SD Times.


In software development, cloud testing provides an easy-access, remote testing environment to produce real user scenarios. As organizations consider the benefits of the cloud testing, here are some signs that tell you it’s time to use it for browser testing.


    

  • You’re starting a business – One of the best times to check your web design across browsers are in the early stages a.k.a the exploratory tests. New companies should ensure they establish the UI of their website to look and function responsively across multiple devices when quality can make or break a business reputation.
    • 

  • You’re moving from manual to automation – Cloud testing offers an easy onboarding for organizations whose testers are a little newer to programming and automation. As you continue to recognize the benefits of automating your tests, the cloud is a practical way to make the process pain-free. With on-demand access, it’s easy for your team to test whenever they need and learn as they go. Additionally, browser screenshots, extensions, frameworks, debugging tools, and integrations are included to optimize usability and make testing more enjoyable. Plus, customizable preferences and customer-centric user interface assure an optimal user experience.
    • 

  • You’re realizing there aren’t enough hours a day to test everything – If you find yourself with test cases that are taking more time than there are hours in the day, it might be time to think about adding some more machines to your parallel testing. Parallel testing is a feature exclusive to cloud testing, which will allow you to significantly cut down your testing time by testing different cases on multiple machines at the same time, instead of one after the other. Additionally, the cloud is available all day every day, so you can test whenever you want without having to be on-site. In fact, you can even run automated tests in your sleep. This will make it easier to test all the configurations you need to, while still meeting deadlines.
    • 

  • You have a hard time keeping up with all the tech – Software testing is not what it used to be. With so many new browsers, OS versions, and devices coming out all the time, it can be difficult to keep track of all the configurations you need to test on. With cloud testing, new releases are added for you so you don’t even have to think about it.
    • 

  • You think new smartphones are way overpriced – If your device lab budget isn’t quite allowing all the configurations you need, cloud testing might be just the right answer. When an average Android is about $670 and an iPhone rounds out at around $700, the numbers can add up fast when you try to test on every device your user has — not to mention maintenance.
    • 

  • You recognize the benefits of simulators, emulators, and real devices – While emulators and simulators have their place in testing, real devices are crucial in order to test on the machines that your users are on. To get the most comprehensive breadth of simulators and emulators plus real devices for testing, cloud testing is the best way to access all three for different purposes, and the combination ensures that you always have access to testing.
    • 

  • You’re scaling your team – You had QA all figured out when you had just one (albeit really talented) tester. But, as your company scales, so must your testing team. When everyone’s trying to complete different tasks that all seem to be of top priority at the same time, the cloud gives you more room to accommodate growing pains. Since everyone has access to the same devices at once, you can ensure you’re supplying your growing team with the resources they require.
    • 

  • You have testers in different offices – Similarly, it’s a little bit difficult to share the capabilities of your device lab among a QA team that’s spread out among different cities, countries, and continents. Cloud testing provides one platform that further encourages communication between different departments and remote teams with integrations like Slack sharing.
    • 

  • Your customers are getting demanding – Has your customer support team been extra stressed out lately? Are you getting user reports saying that your website is slow, or there’s a bug, or it’s not working on their browser? Your customers have high expectations of you. They might be using a wider variety of devices than you think, and it’s compromising their interaction with your web application.
    • 



What pushed you to make the decision to adopt a cloud testing service? Tell us in the comments!


Deciding What to Automate When You Can’t Test Everything


 By  Leave a Comment 
 


prioritizing when you can't automate everything


Though automation is a great method for executing more frequent tests, teams that try to automate everything will quickly find that it’s a herculean task. GUI tests should be carefully prioritized with business-critical critical processes, revenue-generating flows, and common test cases.


As testers new to automation gain more experience, they should be able to prioritize tests, saving organizations time and money.


When Not to Automate


Usability Testing


User experience design (UX) is a human-driven evaluation of the usability of a product, application, or feature. This means the UX can’t be evaluated by automation very accurately because that would take out the aspect of human observation. Using our Screenshot Engine, you can visually verify the UX across different devices and browsers, but we recommend serious Usability Testing be done manually.


One-time tests


This one is pretty self-explanatory, but if there’s a feature you only plan to test once or twice, there’s no reason to try and automate it. Automation is primarily used for regression and repeated tests, and best saved for more stable code bases. For example, an ad-hoc test may just be used to check something without the need for extensive documentation and a formal test structure.


Brand new features


Smoke testing and exploratory testing are better for testing a new feature than writing full end-to-end tests for it. Automation is great for reducing testing time, but it doesn’t act as an end-user the way a manual tester is trained to do. Automation scripts won’t seek out new bugs; see below. Once the UI of that feature is stable, it’s in a better state to be tested across browsers and when small code changes are made.


Exploratory testing


Automated tests follow a script, they don’t explore the software to experiment with an application and break features. Unfortunately, since users aren’t robotic they won’t always know to use software the way it’s intended. By exploratory testing, manual testers can go in and consider more creative possibilities of how a user will handle it and execute negative testing.


What Test Cases to Automate


Regression tests


Again, the most basic use of automation is for regression tests. Are there certain test scripts that you know you always execute with every deployment? Did you just get to a stable point in your integration tests of a new feature and want to speed testing. That’s where automation comes into greatly reduce the time it takes, thus effectively assisting the ability for quality deployments in a CI/CD environment. Regression testing allows you to build a test suite that ensures quality with every iteration and change to the code, pinpointing where possible new bugs may be when integration systems or deploying new features.


Unit tests


As the base of Mike Cohn’s Test Pyramid, unit tests are great for automation because they’re the smallest kind of test (Hint: you should have a lot of them). This also encompasses the motto of testing “early and often” that’s a foundation of any good testing strategy.


High traffic pages


You want to have a good reputation in the eyes of your users, and a bug or broken link on your homepage can be embarrassing. If your analytics show that your traffic is converting through your webinar section then you should prioritize that page before your blog, making sure customers are able to sign up and join them.


Impact on customers


This is similar but slightly different than targeting highly trafficked pages. Will your customers be more likely to leave your site if they don’t have a good experience during check out compared to browsing inventory pages? Consider pages with high bounces rate, or pages that have a high ROI – like your free trial or contact us page.


Data-driven tests


Manual testing is great for UI, but it’s not a very data-driven approach, more so relying on human intuition and curiosity. If you are looking for specific test inputs to tell you exactly where bugs lie automation will give you a more accurate portrayal.


Cross-browser testing


You’ve tested your web app, but if you only tested in one browser, you’re missing the mark for broader coverage. This is a simple way to make sure that your web application looks acceptable to a wide array of users on different devices, operating systems, and browsers, which is cross-browser testing. However, unless you want this to take hours or days of your valuable time, your best option is to automate parallel tests. To learn more cutting down test time, read our guide to parallel testing.


Stop Trying to Automate Everything


Deciding between manual and automation can be pretty straightforward. Manual testing is most often used for more detailed human observation, while automated tests are a faster and more data-driven approach for repeated tests.


In addition, sometimes the line can blur between which tests you should execute first in both situations. Our best advice is to always consider the end-user first. Think about which tests will best ensure usability, functionality, and performance for them specific to the stage of development your software is in.


By making both manual and automated testing part of your ongoing QA strategy, you can establish that your software continuously is known for optimal quality among your user base.


How do you decide what tests should be automated first? Tell us how you prioritize in the comments!


Anticipating Selenium Conference Austin 2017


 By  Leave a Comment 
Selenium Conference Austin 2017


In just a few days, the CrossBrowserTesting team will be be traveling to Austin, Texas as an official sponsor for this year’s Selenium Conference to join developers, testers, and enthusiasts from around the world in a discussion of all things Selenium.


Between workshops, keynotes, and track presenters, we’ll surely be busy exchanging ideas and hearing new insights from major thought-leaders in the development and open-source community. In anticipation of Selenium Conference, here are a few of the sessions we’re most excited to hear.


Selenium: State Of The Union


Simon Stewart is the Lead Committer of Selenium Project and the Creator of WebDriver, so as a major contributor to Selenium’s history, this guy definitely knows what he’s talking about concerning the subject at hand. He is also co-editor of a W3C (World Wide Web Consortium) specification process, which he’ll talk more about in this keynote in relation to WebDriver. In addition, Stewart will facilitate discussion on the Software Freedom Conservancy and touch on how a “normal” person can approach Selenium.


Catching Waves in the Ocean of Quality


You know a keynote from the Senior Engineering Manager at Uber is going to be good. With over 2 billion rides taken and an estimated value of $70 billion, there’s a lot of brain power that must go into the wildly successful, on-demand transportation app. Denali Lumma’s keynote will help teams answer questions like “Does it compile and run correctly, quickly, and securely?”, “Can the development team make changes easily?”, “Can it be debugged, extended, and scaled”, and explore many more pressing concerns in the world of quality engineering, security, and building out teams.


The Good Software Project: A Personal History


Another keynote speaker, Jim Evans is the Lead Member of Technical Staff at Salesforce.com. Evans will cover how the computer software industry has changed over the last 25 years and share what he’s learned from own experiences as a core contributor to the Selenium project.


The Build That Cried Broken: Building Trust in Your Continuous Integration Tests


In this presentation, Angie Jones will reference the classic “The Boy Who Cried Wolf” story and relate it to teams’ continuous integration builds and the issues that arise when tests give false alarms. Jones is a Consulting Automation Engineer at LexisNexis who spends her time advising scrum teams on automation strategies, developing automation frameworks, founding over 20 patented inventions, and teaching Java programming as an adjunct college professor. We look forward to this clever developers’ storytime session and hope there’s a happy ending.


CI Excellence With Test Automation Engineers


Greg Sypolt is the Senior Engineer at Gannett | USA Today Network and will share how test automation has evolved and will continue evolving. His session will also focus on Continuous Integration — best practices, building the pipeline, working with your team, and maintaining the infrastructure will be discussed, among many other relevant topics in CI.


Embrace Innovation: Designing A Protractor Framework For Angular2 Applications


As a Frontend UI Automation Engineer, Meredith Bayne has been hard at work on an innovation project using technology that has never been tested before. Her behind-the-scenes peek at this intriguing venture will follow her experiences and provide suggestions, while giving insight into how she’s achieved success as a woman and disabled person in the tech field during what we expect will be a very inspiring session.


Security Test Driven Development (STDD) Using Selenium/Appium


A topic that’s on everyone’s mind when it comes to test automation is security. During Surendran Ethiraj’s presentation, he will examine how developers can use existing test driven development tools and practices to incorporate higher security into the development process.


 


This, of course, is only a preview of Selenium Conference. We look forward to joining as many conversations, listening to as many speakers, and learning as many new strategies as is possible in during our time in Austin.


Selenium Conference is taking place from April 3rd through 5th. Make sure to stop by and say hello to the CrossBrowserTesting team if you’re attending. If you can’t make it to this year’s conference, follow along with us by signing up for our newsletter and following us on social media as we cover the prominent trends, developments, and topics.