Selenium automates the browser to mimic real user actions on the web. As the web gets more complex, using Selenium to test has become increasingly popular. Although it has made web testing far easier for many teams and organizations across the world, it still has its fair share of challenges due it’s open source nature.
Most of the issues that testers experience have fairly straight forward solutions, which is why we outlined the most common Selenium challenges and how to fix them.
When a simple, prompt, or confirmation alert pops-up, it can be difficult to automate it to either accept or close. Additionally, Windows-based alerts are beyond Selenium’s capabilities since they’re part of the operating system instead of the browser. However, since Selenium can operate multiple windows, web-based alerts can usually be handled with the switchTo method to control the pop-up while keeping the browser in the background.
Learn more about handling pop-up windows with Selenium here.
A web page that has dynamically loaded content has elements that may not be at first visible when you visit the site. This can mean that the page is user specific and will display different content based on different users, new content appears on the page after a certain amount of time, or it appears after clicking something on the page. For when a locator cannot identify a new element present at that time, Selenium comes with an integrated Explicit Wait where you can specify an amount of time to wait before automating a command. This should help give the page enough time to load and identify the element to interact with.
Learn more about testing dynamic content with Selenium here.
Sometimes Selenium will give you flaky tests, which means they’ll come back as positive or negative when they actually are the opposite. Eventually, too many flaky tests may mean testers might ignore the results, though this isn’t a great option either. Oftentimes, unstable builds can be a product of one of many factors: poor test isolation, flaky external services, and timeouts. By examining the elements of your Selenium tests, it’ll be easier to find out why your builds are unstable so you can approach the problem head-on.
Learn more about managing flaky tests here.
Can’t Test Mobile Apps
While Selenium can test on any operating system and browser on desktops, it’s limited when it comes to mobile testing in that it cannot run on native operating systems like iOS and Android. So, for example, the version of Facebook you pull up on Safari on your Mac can be tested with Selenium, but not the one on mobile Safari – on your iPhone 7. The Selenium family has a solution. It’s cousin, Appium, is an open source test automation framework that drives iOS and Android native, mobile, and hybrid apps using the WebDriver protocol and can be used for this purpose for testing mobile apps instead of web applications. As users increasingly move to mobile devices, it’s no surprise that developers and testers are taking advantage of the functionalities of Appium for this purpose.
Learn more about Appium here.
Limited Reporting and Documentation
While Selenium will exponentially increase your automated testing capabilities, because it’s an open source tool it is limited and features and does not support much reporting on its own. For this purpose, Selenium is best complimented with a third-party tool like CrossBrowserTesting that can capture screenshots and share reports through integrations like Slack and HipChat. However, you can also set up a framework to generate an output folder after a test with report information like errors, execution time, pass/fail count, etc. like TestNG.
Selenium has the ability to test in multiple tabs, but it might present as an obstacle if you don’t know the correct commands. For example, if you want to open a new tab to perform a specific action without leaving the original tab, you’ll still want to be able to control both tabs and go back to the parent tab if users are likely to perform these kinds of actions. The best method for this is to store the previous window handle in a variable and then continue in the new window and store that in a second variable. Then, you can use the switchTo method to go between both windows as needed.
Learn more about how to work with multiple windows here.
No Manual Testing
Many testers will get excited about Selenium’s capabilities and expect that they can automate everything when the reality is that this just isn’t possible. In fact, a lot of automation relies on basic manual testing efforts. Rather, learning to prioritize which tests to automate will be beneficial to any tester. Additionally, as we mentioned, Selenium lacks capabilities for visual testing, which is important for testing beyond functionality to assess overall user experience. The most efficient way to do visual testing is by outsourcing to a third-party screenshot tool that integrates with Selenium.
Again, while Selenium allows you to test on pretty much any browser or operating system, it’s still limited in how many tests it can run at once and how fast it can run them based on how many hub/node configurations the tester has. And again, as your software scales, so must your testing. Without a Selenium grid, you can only test sequentially. However, with a Selenium grid and a device lab or a third party cloud tool like CrossBrowserTesting, you can test in parallel. This means multiples tests on different configurations can run at the same time, reducing the time it takes to run automated tests and increasing the configurations you can test on.
Learn more about parallel testing here.
One of the best things about being a software tester is that there’s always something new to learn. As the industry shifted from manual to automation, testers were tasked with learning new skills such as programming.
Now, as they become more familiar with automation techniques and tools such as Selenium, it’s admissible that they’ll encounter more challenges as they explore and familiarize themselves with new testing trends. Fortunately, as software continues to evolve, experts can depend on insights and support from the rest of the testing community to make sure they’re keeping up. Pick your preferred programming language to get started with Selenium testing here.
Ashley Hunsberger is a Test Automation Architect at Blackboard, a virtual learning environment and course management system for top institutions worldwide.
Her Selenium Conference presentation, “Transformative Culture” reviewed how Blackboard established departmental goals, built teams that would ensure quality and success, and slashed execution times, while transforming the way the company thinks about software quality.
Not only was this presentation a favorite among conference attendees, it also serves as highly valuable information for any company that’s undergoing a culture change.
A Cause for Change
A year ago, it took an hour to run Blackboard’s 400 tests, and 370 were unreliable. So, not only was the time to execute high, but the builds were constantly failing and developers had stopped caring about them.
Hunsberger quickly realized that the root of the issues they were having centered around culture and the way different departments thought about quality and communicated with each other to deploy projects.
She wanted to make a company-wide change that would enable the development team to own testing and quality and hoped to get the entire organization to put more care into test suites and maintenance.
The very first step to transform the department was a simple one. They would not longer be referred to as QA, but instead Engineering Productivity.
Hunsberger also wanted to re-establish the department mission to “Reduce the time from concept to deliverable by providing our product development teams with the tools, practices, and support to increase their productivity while maintaining high quality standards.”
She also wanted to clarify that Engineering Productivity would not take over writing tests for other teams, but instead be a resource to support the automation strategy.
The next move was to set goals including:
- Provide an easily maintainable and extensible framework that would enable scrum teams to add and remove tests.
- Enable the automatic and early detection of failures within the software under development.
- Prevent the source of detected failures from moving any further downstream.
- Accommodate all of this without impacting the engineer’s time.
Enter, Team Shangri-la
After realizing this transformation would only be possible with a proper leadership team to own the automation strategy, Hunsberger introduced Team Shangri-la, where she acted as the project owner alongside a scrum master, SETs, a feature engineer, and a CI/CD team member.
Team Shangri-la had to first establish their definition of “Done”, as Hunsberger found that in the past, a lot of teams were saying projects were done without them actually being testing, reviewed, documented, and released.
They also had to outline test suite definitions to get everyone speaking the same language:
- Goals – What was the intent for each suite?
- Trigger – What would trigger tests?
- Gates – What happens if tests fail?
- Requirements – What are the requirements for each suite?
Once the guidelines were determined, they had to work on containerizing environments for faster feedback and implementing project guardrails to provide support for nine different scrum teams.
Additionally, one prominent area of Shangri-la’s strategy was risk analysis, where they aimed to prioritize what tests would be executed and how they would be written.
By rating the likelihood of a bug occurring and what the impact on customers would be if the feature didn’t work, it became easier to evaluate and what areas need to be included in the test suite and which did not.
Charting and tracking the results also helped the team decide what kind of testing needed to be executed, as well as how the tests would be monitored and maintained.
Hunsberger and her teams saw the 33 stable tests and 370 unreliable tests that ran in an hour turn on its head to now run 165 stable tests in 30 minutes.
Along the way, Hunsberger learned that not everyone has the same idea about what’s critical, and you can’t test everything. These small realizations ultimately initiated a complete culture change that encouraged them to be better communicators so that they could establish goals and meet them.
By focusing departments to be a company-wide effort, Hunsberger was able to affect better testing to support development, optimize testing, and ensure uncompromised quality across Blackboard’s projects.
Security is a topic that’s on every tester’s mind this year. Whether you’re testing behind your company’s firewall, improving your applications security against hackers, or ensuring privacy in a consumer-facing application, many of us know that security is important but neglect it as part of our strategy.
During Selenium Conf Austin, test automation experts Denali Lumma and Surendran Ehiraj shared their insights on the best practices for more efficient security testing.
It’s no surprise Uber has a prominent concern for security testing with over 2 billion rides taken. Considering the amount of user information they acquire, confidentiality, integrity, and availability all serve a vital role in quality engineering efforts.
Denali Lumma is the Senior Technology Manager and the Founder of People Engineering at Uber tasked with hiring the engineers that will ship high quality, high security libraries through identification, authentication, and authorization user processes.
In “Catching Waves in the Ocean of Quality,” she posits that like successful quality work, successful security must be integrated into each step of the development process including design, RFC, code, and review.
Lumma and her team first built a secure customer authentication platform to protect user identity, defend against defrauding, embed privacy, and support the company ecosystem. It also improved usability, scalability, maintainability, and extensibility.
The way data is handled is also essential through practices including deletion, filtering, and encryption. This means making sure data is deleted without reappearing, sharing data internally and externally without compromising individual privacy, and regularly changing encryption keys are critical.
To approach this Lumma uses a differential privacy to ensure indistinguishably and limit the amount of information someone can gain about individuals and prevent insufficient anonymization while preserving trends and data.
In response to Uber’s growing consumer base, Lumma and her team were able to meet the high priority need for thorough security testing, while simultaneously increasing user acquisition, improving performance, and reducing losses due to fraud.
- Do you think in this digital world NFR’s such as security are important?
- If yes, are you doing security testing as part of an agile process?
- If yes, what is the frequency of your test?
These are the questions that Surendran Ehiraj posed at the start of his presentation, “Security Test Driven Development (STDD) Using Selenium/Appium”.
“Security is important. We all know, but we don’t do it,” says Ehiraj. “We compromise on security.”
Instead of including it in our agile development process, Ehiraj noticed that it becomes a concern after there’s an issue, referencing the infamous Ashley Madison hack as an example of a time where security was considered after it was a little too late.
To combat breaches, Ehiraj asserts that static, dynamic, and forensic application security testing can be inserted into the TDD process using existing tools and practices to include a greater focus on security into the development process.
For example, using Zed Attack Proxy (ZAP) as an open source solution for finding vulnerabilities on a web applications, you can continue to use existing Selenium tests while advancing security measures.
Ehiraj promoted ZAP as something he uses quite often to act like a proxy, and while automation experts might be hesitant to begin security testing, the tool is fairly easy to use even for beginners to collect comprehensive results on how secure a web application is.
Even better, we played around a little with ZAP and it works against CrossBrowserTesting, which means you can execute cross browser security testing with Selenium. Very fancy.
We know the importance of automated testing and cross-browser testing. However, one area we often disregard is security testing.
To ensure that your engineers are deploying high-quality and trusted web applications, security testing needs to become a part of the testing and development process. By prioritizing the security of your company and your customers, you can avoid the past pitfalls, data breaches, and security hacks that less fortunate applications have suffered from in profit and reputation.
We’ve discussed the evolution of Selenium, and the major shift from Selenium 1 to Selenium 2 in “Intro to Selenium Testing”. We also noted that latest version of developers’ preferred open source automation testing tool is actually Selenium 3.
But what are the differences present between the two versions, and how did it affect development and testing? In this post, we’ll examine the features that are components of Selenium 3.
Announcing Selenium 3.0 Beta
When Simon Stewart, the founding contributor of WebDriver, announced a new beta version of Selenium would soon be released at SeleniumConf in 2013, it caused quite a stir among the already large community of open source contributors.
After advising users to switch from the original RC APIs to WebDriver, Selenium 3 would stop supporting RC completely to be replaced by a similar, revised function that was integrated with WebDriver.
Of course, since most people had already abandoned RC at this point, this change didn’t affect too many developers. Those who still were using RC, however, would have to prepare the transition to WebDriver.
The update also required Java 8 to be running to access certain elements and stopped supporting IE prior to version 9.
In a second release note, Stewart specified with the following details:
- WebDriver users will just find bug fixes and a drop-in replacement for 2.x.
- Selenium Grid users will also find bug fixes and a simple update.
- The WebDriver APIs are now the only APIs actively supported by the Selenium project.
- The Selenium RC APIs have been moved to a “legacy” package.
- The original code powering Selenium RC has been replaced with something backed by
- WebDriver, which is also contained in the “legacy” package.
- By a quirk of timing, Mozilla have made changes to Firefox that mean that from Firefox 48 you must use their geckodriver to use that browser, regardless of whether you’re using Selenium 2 or 3.
However, developers continued to wait three years for the release of Selenium 3, which was supposed to come out at Christmas 2013.
The Official Release of Selenium 3.0
Finally, in October of 2016, Selenium 3 was released.
Before, Selenium was responsible for driving each browser. With the update, all major browser vendors would ship their own implementations. While the specification was still in the works, it meant that Selenium was moving more toward becoming a testing framework that would operate as the default browser.
“Because the browser vendors know their browsers better than anyone, their WebDriver implementations can be tightly coupled to the browser, leading to a better testing experience for you,” said Stewart in the final release note.
Selenium 3.0 would also be integrating a W3C WebDriver specification for automation. In a follow-up interview, testing specialist Juho Perälä noted that this was a relevant tool in mobile testing through Appium and iOS Driver and it would “encourage compatibility across different software implementations of the WebDriver API.”
The purpose of the update wasn’t going to make any huge changes to the common use of Selenium. Instead, it was largely to make Selenium Core obsolete in exchange for an alternative implementation old RC interfaces to meet a rapidly advancing environment of web testing.
Looking Forward with Selenium
Selenium 3 continues to be exceedingly favored in test automation today, and SeleniumConf returns for 2017 as developers and enthusiasts maintain conversation around best practices in testing and advancing the future of Selenium. And we can only hope that when Selenium 4 is announced, it doesn’t take as long to launch.
Selenium is an open source testing tool that allows users to test web applications across different browsers and platforms. Selenium includes a suite of software that developers can use to automate web applications including IDE, RC, WebDriver and Selenium grid, which all serve different purposes.
In this post, we’ll explore the multiple components of the Selenium framework and its evolution as well as how it’s used and why developers and testers depend on it for browser testing.
What’s the Use?
A Brief History of Selenium
Selenium has a dynamic history since it came to being in 2004 when Jason Huggins, who was a ThoughtWorks programmer at the time, decided to make a tool for testing web applications. By the end of the year, it had become a wildly popular open source tool.
The combination of Selenium and WebDriver in 2006 became a master web application tool know as Selenium 2.0 and is one of the leading free testing tools for programmers around the world.
While Selenium 3.0 has been released, it’s basically a new and improved version of Selenium 2.0 with the same fundamental purpose and structure.
Breaking Down the Selenium Suite
The Beginning – Selenium IDE
Selenium IDE (Integrated Development Environment) is the Firefox plug-in tool that allows you to record, playback, edit and debug test scripts to export them in a preferred programming language. By providing a test domain specific language, Selenese, IDE lets you test without learning a new scripting language and makes it easier to send commands to the browser and automate testing. However, IDE is limiting in the way that you can only use it to test web applications in Firefox, and therefore it cannot be used to test other browsers.
Selenium IDE is mostly used now for bug reproduction scripts, exploratory testing, learning about Selenese commands and exporting simple test cases to WebDriver. Selenium Builder is also a Firefox extension that can import HTML files recorded in Selenium IDE to run tests locally or on a Selenium server and further assist automation.
The Evolution – Selenium 1 and 2
Selenium RC (Remote Control), or Selenium 1, was a method used to run the test script in different browsers by launching them as a proxy server, but has since been abandoned since the creation of WebDriver, or Selenium 2.0. Selenium 2 is what you would use in testing today, as RC is no longer supported.
Selenium 2 is popular among developers and QA teams for regressions tests and cross browser testing, since it allows you to test on a spectrum of browsers and operating systems using the native operating system functionality. This means that commands perform as a human would if they were using the same browser.
Optimization – Selenium Grid
Selenium Grid allows you to multiple run tests at the same time on different machines and browsers in order to speed up testing. For CI motions, testing demands to be done at a rapid pace, and running tests sequentially can be painfully slow. Running in parallel on a grid can improve the speed of your testing greatly. Selenium Grid is comprised of one machine called a hub and multiple nodes for execution. The more nodes you have, the more tests you can run simultaneously.
While you can make your own Selenium grid with a costly device lab or VMs, many have found moving Selenium grid to a third-party cloud is faster and more productive, allowing more tests to run on more OS systems and browsers simultaneously.
The Selenium automation framework is a great tool-set for automating interactions with the browser. Because of its powerful open source features and standard adoption across many browsers, Selenium has become the most popular web testing framework in the world. You can run your automated testing scripts in a variety of languages against local browsers, a lab of browsers and devices using Selenium Grid or a device cloud like CrossBrowserTesting’s. To get the most out of Selenium automation, we’ve put together five best practices for automated browser testing.
Use The Right Locators
At the heart of the Selenium framework is interaction with the browser, letting you navigate, click, type and check different objects within the DOM using a few set actions. It does this using several different types of locators:
- Link Text
- PartPartial Link Textial
- Tag Name
- X Path
Selecting the right locators can be the different between a test script that is flexible and successful, and a brittle test that breaks on the slightest UI change. If they are present, use unique Classes or IDs as Selenium locators, as they are less likely to change without someone on your QA or Dev team knowing. But locators like link text can change all the time – think about a dynamic button based on logged in state, or your marketing team running A/B tests. You can also use Xpath to navigate the webdriver, you can read more about Xpath to CSS conversion here.
Communication Between QA, Developers, Marketers, and Designers is Key
There are two aspects of collaborating with different teams that can make your Selenium testing a success: constant communication and instilling the importance of testing to every department. Instilling the importance of the QA process on your development team can save time and efficiency when building a testing framework. Make sure they know how important unique CSS selectors are to your testing efforts, and ask them to write semantic and human-friendly code. Having constant communication across teams can help QA plan for changes to the UI of a website or application mandated from Marketing or the Design team. Having small conversations about the front-end aspect of your app decreases the chance of surprise UI changes that will go untested.
Page Object Model. Use it.
A popular test automation design pattern, the page object model will help you make robust testing frameworks that are resistant to small tweaks in the UI. The page object model has two core advantages:
- There is clean separation between test code and page specific code such as locators (or their use if you’re using a UI map) and layout.
- There is single repository for the services or operations offered by the page rather than having these services scattered through out the tests.
In both cases this allows any modifications required due to UI changes to all be made in one place. See SeleniumEasy’s explanation and example:
Make Tests Independent Of Each Other
Testing one single action or process, independent of any other tests is known as atomic testing. This testing strategy will keep you from making chained, brittle tests. Chaining together tests, while saving time upfront, can be a huge blocker for a truly agile or CI workflow. Keep tests as small as possible.
Use Your Resources
Selenium can be tough to learn because it requires a certain amount of tangential knowledge. You have to have some understanding of programming and to truly be a skilled Selenium tester, an understanding of test automation best practices. Luckily, there are some fantastic resources available on the internet to learn from. While we run monthly webinars to get you up and running with Selenium, here are a few of our favorites:
It’s Open Source…And Free.
Selenium is an Open Source automation framework built for web-browser testing by some of the brightest developers tired of scripting with expensive legacy automation tools not truly built for the web. Because of this, Selenium’s community is vibrant, growing and is seen as one of the real strength’s of the tool. It has become the most reliable web automation tool because of the ease of generating test scripts to validate functionality. Selenium also works with testing frameworks like WebDriver.IO, Nightwatch.js, and countless others. Oh, and it’s free, with no upfront costs.
One Script for True Cross Browser Testing
With only one script Selenium can test Chrome, Firefox, Safari, Edge, Opera, and Internet Explorer on almost any platform available: Windows, Mac, Linux, Android, and iOS. Customers are using an increasing amount of devices, as they become cheaper and more powerful. Keep up and improve your test coverage with just one script.
Works with The Languages You Use
Easy to Learn and Scale
You’re not alone! With an incredibly vibrant, growing community, Selenium tutorials, testing, and development help is usually just a short search away. The documentation at SeleniumHQ is helpful, and there countless tutorials and how-to sites for you to quickly learn “Selenese”. You can even use Selenium IDE, a Firefox Record and Replay extension, to generate scripts! Selenium was built to scale with their local testing Grid, so you can be assured that as your application grows, so too will your test coverage. Here are a few of our favorite tutorial sites for Selenium:
Integrations for Every Stage of Testing
Selenium has a solid foundation of standards that make it able to be used easily with popular software development and testing tools. Integrate easily with SC compilers like Maven and ANT, CI systems like Jenkins or TeamCity, cloud-grids like CrossBrowserTesting, and almost any test management tool.