CrossBrowserTesting.com

A Design, Development, Testing Blog

Anticipating Selenium Conference Austin 2017

By Leave a Comment

Selenium Conference Austin 2017

In just a few days, the CrossBrowserTesting team will be be traveling to Austin, Texas as an official sponsor for this year’s Selenium Conference to join developers, testers, and enthusiasts from around the world in a discussion of all things Selenium.

Between workshops, keynotes, and track presenters, we’ll surely be busy exchanging ideas and hearing new insights from major thought-leaders in the development and open-source community. In anticipation of Selenium Conference, here are a few of the sessions we’re most excited to hear.

Selenium: State Of The Union

Simon Stewart is the Lead Committer of Selenium Project and the Creator of WebDriver, so as a major contributor to Selenium’s history, this guy definitely knows what he’s talking about concerning the subject at hand. He is also co-editor of a W3C (World Wide Web Consortium) specification process, which he’ll talk more about in this keynote in relation to WebDriver. In addition, Stewart will facilitate discussion on the Software Freedom Conservancy and touch on how a “normal” person can approach Selenium.

Catching Waves in the Ocean of Quality

You know a keynote from the Senior Engineering Manager at Uber is going to be good. With over 2 billion rides taken and an estimated value of $70 billion, there’s a lot of brain power that must go into the wildly successful, on-demand transportation app. Denali Lumma’s keynote will help teams answer questions like “Does it compile and run correctly, quickly, and securely?”, “Can the development team make changes easily?”, “Can it be debugged, extended, and scaled”, and explore many more pressing concerns in the world of quality engineering, security, and building out teams.

The Good Software Project: A Personal History

Another keynote speaker, Jim Evans is the Lead Member of Technical Staff at Salesforce.com. Evans will cover how the computer software industry has changed over the last 25 years and share what he’s learned from own experiences as a core contributor to the Selenium project.

The Build That Cried Broken: Building Trust in Your Continuous Integration Tests

In this presentation, Angie Jones will reference the classic “The Boy Who Cried Wolf” story and relate it to teams’ continuous integration builds and the issues that arise when tests give false alarms. Jones is a Consulting Automation Engineer at LexisNexis who spends her time advising scrum teams on automation strategies, developing automation frameworks, founding over 20 patented inventions, and teaching Java programming as an adjunct college professor. We look forward to this clever developers’ storytime session and hope there’s a happy ending.

CI Excellence With Test Automation Engineers

Greg Sypolt is the Senior Engineer at Gannett | USA Today Network and will share how test automation has evolved and will continue evolving. His session will also focus on Continuous Integration — best practices, building the pipeline, working with your team, and maintaining the infrastructure will be discussed, among many other relevant topics in CI.

Embrace Innovation: Designing A Protractor Framework For Angular2 Applications

As a Frontend UI Automation Engineer, Meredith Bayne has been hard at work on an innovation project using technology that has never been tested before. Her behind-the-scenes peek at this intriguing venture will follow her experiences and provide suggestions, while giving insight into how she’s achieved success as a woman and disabled person in the tech field during what we expect will be a very inspiring session.

Security Test Driven Development (STDD) Using Selenium/Appium

A topic that’s on everyone’s mind when it comes to test automation is security. During Surendran Ethiraj’s presentation, he will examine how developers can use existing test driven development tools and practices to incorporate higher security into the development process.

 

This, of course, is only a preview of Selenium Conference. We look forward to joining as many conversations, listening to as many speakers, and learning as many new strategies as is possible in during our time in Austin.

Selenium Conference is taking place from April 3rd through 5th. Make sure to stop by and say hello to the CrossBrowserTesting team if you’re attending. If you can’t make it to this year’s conference, follow along with us by signing up for our newsletter and following us on social media as we cover the prominent trends, developments, and topics.

Top Automation Frameworks Testers Love

By Leave a Comment

test automation Frameworks

Test automation frameworks are basically the groundwork for automating test scripts. Different frameworks offer different rules, guidelines, and procedures to help efficiently develop and execute tests.

The best choice for each tester is, of course, dependent on their preferences and needs. In this post, we’ll differentiate the multiple frameworks and open source automation tools to make the best testing and development.

No Reason to Make Your Own

While it is possible to create your own testing framework, there are plenty of pre-defined frameworks to simplify test execution and help you meet your browser automation goals.

“I’ve seen some teams take it to the extreme by building their own, elaborate automation framework from scratch,” said TestTalks Founder Joe Colantonio. “This drives me crazy, because they could have easily made use of existing open-source tools and libraries that would have met their needs without writing any code — in most cases with better results.”

Rather than wasting time trying to build an adequate framework, check out some of developers’ trusted choices. These resources make it easier for your team to collaborate for better quality, security, speed and readability.

Tool Frameworks

The test writing frameworks that we discuss further along in this post require a prior knowledge of foundational browser automation tools. Selenium, Appium, Capybara, and Watir are the original open source tools that provide a basic infrastructure for other frameworks to add to for increased features, capabilities, and efficient test execution.

Selenium WebDriver is a tool and API framework for web browser automation. The big selling point of WebDriver for developers is that it allows them to test in any preferred programming language, which also makes it one of the most popular open-source automation frameworks.

Appium, on the other hand, is an open source automation framework for mobile apps. The framework allows native, hybrid, and mobile testing and drives iOS, Android, and Windows apps using WebDriver. Just like Selenium, it’s a cross-platform tool that can run scripts in any language.

Capybara is a library written in Ruby and is another framework that makes it simple to automate web application testing using an intuitive API that simulates an actual user.  It offers a user-friendly DSL (Domain Specific Language) and locates elements in the DOM (Document Object Model).

Watir (pronounced like water) is another family of Ruby libraries for automating web browsers in an app no matter what technology it’s been developed in. Watir only supports Internet Exploreer on Windows, but Watir-WebDriver (which is powered by Selenium) supports Chrome, Firefox, Internet Explorer, Opera and can also run in HTMLUnit headless mode

What all these tool frameworks have in common is that they communicate directly with the browser to perform tasks in the same way that a human would, so executing automated tasks is as accurate as manually testing them.

Test Writing Frameworks

WebDriver.io – WebdriverIO makes it easier to setup, manage, and write Selenium tests in JavaScript by implementing the WebDriver protocol API. The custom libraries include a test runner for integration testing and provides a plethora of shortcuts to make code cleaner and easier to read. Learn more about testing with WebDriverIO here.

Robot Framework – Robot Framework is implemented using Python as its core but can also run on Java and other languages through a remote interface. It utilizes keyword-driven testing, which makes it simple for testers to perform either normal test automation or ATDD. The modular architecture can be bundled with self-made test libraries in addition to the many included tools and libraries.

Nightwatch.JS – For end-to-end testing, this easy to use Node.js based framework uses Selenium WebDriver’s API to perform commands and assertions on DOM elements. Nightwatch.js also has a built-in command-line test runner that can execute tests sequentially or in parallel. Learn more about testing with Nightwatch here.

Cucumber – Cucumber is a Behavior Driven Development (BDD) framework that lets you execute plain-text functional descriptions as automated text. It’s written in Ruby but also tests in other languages including Java and Python. The point of Cucumber is that even people without highly technical knowledge are able to read the code, making it more accessible for collaboration across different business departments.

Lettuce – Another BDD framework but for Python, Lettuce allows you to execute automated tests from plain-text descriptions that almost anyone can understand. It’s actually based off Cucumber, which has a few more features, but Lettuce aims to cover the most common tasks on BDD and is a fairly straightforward option. Learn more about testing with Lettuce here.

Protractor – Protractor is an end-to-end test framework for AngularJS applications that can simulate a user’s interaction with an Angular application running in a browser or mobile device. It has added hooks for running in Selenium that enable you to get more data and benefit from other Angular features. Additionally, a really cool feature is that it lets you separate your tests into various test suites, which allows you to enter a single command and run a group of test cases for different scenarios. Learn more about testing with Protractor here.

Mocha – Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser. It’s great for unit and integration testing, and self-described as making asynchronous testing “simple and fun.” It also includes test coverage reports and use of most JavaScript assertion libraries — most popularly Chai. Learn more about testing with Mocha here.

7 Things to Consider Before Hiring Your Next Tester

By Leave a Comment

Hiring a test team

How do you hire your testers?

So, you’ve finally decided you need to hire someone for software testing. We’re very excited for you. There are, however, a few steps you should follow before you settle on a final decision and extend an offer, and we have a few tips on how you can make the best hire.

    1. First Thing’s First. The first thing that you should look out for is whether or not the person you’re considering has any errors in their resume. While this should be a given for any job or position, it’s especially crucial in industries where an attention to detail is essential to your job (a.k.a. testing). If the person in question used the wrong “your vs you’re” or spells the company name wrong, they’re probably not going to notice a small, discreet bug.
    2. Know what you want. Are you expecting this person to do a lot of manual testing, or are you looking more for an automation engineer? Are you transitioning to an agile methodology? What’s the company’s preferred coding language? While many qualified testers will come and go, you want someone who is going to be right for the specific jobs and projects you need completed. It’s not just about finding a good tester, it’s about finding the right one.
    3. Do they know the basics? Can they explain functional testing? What’s the difference between unit and integration testing? Have they ever coded before? While you shouldn’t expect your interviewee to be giving exact dictionary definitions, they should be able to convince you that they know what they’re talking about and are familiar with the software development and testing industry.
    4. Ask about their process. Giving your potential new tester scenarios and asking them how they’d go about finding bugs or how they’d get rid of them is a more in-depth way to assess their testing knowledge. It’ll also give you a more expansive insight into their work style and help you recognize if they’ll be the right fit for the jobs that you need them to do.
    5. Test them. You can challenge someone to define testing terms all day long, but the evidence lies in how well someone can actually execute that test in real life. Whether you have them complete an assignment before you meet, in-person at the interview, or request that they complete one at home afterwards, having them perform a coding or testing job will give a more accurate understanding of their skills, method and technique. This is also a good way to evaluate how much effort they’re willing to put in. A senior tester that makes sloppy mistakes should be re-considered if you have a junior tester that took the request seriously and clearly put more time and energy into the task.
    6. Have they worked as part of a team? You might find the perfect candidate with years of experience who checks all the boxes. However, being a tester isn’t always a one-person job. Keep in mind that the person you hire will have to regularly collaborate with various people on your team, from designers and programmers to management, engineers and other testers. Even the best tester will be potentially damaging to your organization’s goals if they are unwilling to take feedback and work with others. You want to make sure they fit the company and not just the job description.
    7. Are they interested in the job? You need to know what you want from your candidates, but they need to be invested in the opportunity, as well. Do they seem interested in the projects they’ll be assigned and eager to bring new testing strategies to your QA game? Your new tester should be willing to learn and be invested in the development process. Many people don’t go into testing and stay without an end goal to transfer to a programming role, but make sure that either way the person is in for the long haul and is going to be an asset to your objectives.

Hiring a new tester is a great opportunity to grow your company, learn new testing methods, improve current procedures and advance the efficiency of your software development strategy. In this way, it’s essential to the success of your organization, your product and your team that you put careful time and thought into choosing your next tester.

Image by สุวรรณา วิเศษแก้ว via Wikimedia Commons

The Difference Between Manual and Automated Testing

By Leave a Comment

Manual vs Automated Testing

Manual vs Automated Testing

When you begin web testing, one of the primary decisions you’ll have to make is whether you’re going to test manually or use automated testing.

While neither of these options are technically better or worse than the other, the size, budget and time allowance of a project will certainly be factors that affect which method will work best in your testing procedure.

Below, we’ll outline the best times to use manual and automated testing, and the advantages and disadvantages of each.

Manual Testing

Manual testing is essentially self-explanatory — testing of a web application is done manually, by human action. This means that someone actually goes on a device to evaluate numerous components including design, functionality, and performance by clicking through multiple elements or units of a web application without support from a tool or script.

Advantages

The method of manual testing is inherently valuable to consumer-facing websites because the tester is able to have the full user experience, making it easier to assess visual components such as text size and layout as well as UI.

Additionally, most QA processes will begin with some degree of manual testing because it’s an ideal way to perform one-off testing and exploratory test cases as well as usability and ad-hoc testing.

Disadvantages

The limitations of manual testing are that it’s not the best method for test cases that have frequent repetition. For code that’s updated by the day or hour, manual testing would take an exponential amount of time and human resources, and therefore, would not be an efficient way to approach agile CI.

Furthermore, because manual testing is performed by a human, there’s more room for human error as opposed to the accuracy of automated testing. It can also be a boring, time consuming and tedious task for developers to continually perform manual tests every time they update a code, rather than spending their time actually improving the code itself.

Manual Testing v Automated Testing

Manual Testing v Automated Testing for CI

Automated Testing

Automated testing uses the assistance of tools, scripts and software to perform test cases by repeating pre-defined actions. All automated testing, however, requires some manual testing at first to create initial testing scripts.

Advantages

The biggest draw of automated testing is that it’s noticeably faster and more accurate than manual testing. This is because automation allows multiple tests to be run in parallel, since the test cases can be distributed across different machines.

In comparison, manual testing has to be done sequentially, and so only one test can be run at a time. In this way, the accuracy of automated testing makes it ideal for finding bugs when running tests repeatedly over a long time period, or running multiple tests on different browsers.

Additionally, Automated testing is preferred for large projects that require regression, load and performance testing and repeated execution. Specifically, developers that use an agile CI methodology depend on automated testing to be able to retest scripts as they continually change code because manually testing every time a small fix was made would be exceptionally time consuming.

Disadvantages

The downfall of automated testing is that, unlike manual testing, there is little human observation involved. While automated testing is more efficient in many ways, developers will sometimes find that they need to execute manual testing in order to add a human element and meet specific consumer needs.

Often people will attribute automated testing as being more expensive because it requires a greater investment in tools. However, if your project calls for automated testing, this investment will most likely pay for itself and be more cost effective over time.

automated testing vs manual testing

ROI of Test Automation

Complimenting Testing with Tools

Whether you’re deciding to pursue a manual or automated testing strategy, there are a number of ways to make the process faster, easier and more accurate for your QA team.

Tools such as CrossBrowserTesting offer solutions and features that complement manual and automated testing alike. These value-add tools including live testing, automated screenshots, visual comparisons, selenium testing and local testing all contribute to a well-rounded project execution.

However, the success of your testing process will first depend on your ability to measure which testing method is best for your project. By evaluating the pros and cons of manual testing and automated testing as well as their best uses and what you would like to accomplish through testing, you can launch a solid strategy that will continue to improve your web application and users’ experiences.

To learn more about beginning test automation, check out our eBook.

Read Now

What Is Parallel Testing?

By Leave a Comment

Parallel testing is your ticket to faster testing and a quicker turn around in deployments. When testing websites or applications, it is important to remember that time is a factor- you always have a finite amount of time to test before deployments or to increase your coverage. Testing 100% of an application is a noble cause, but no developer wants to spend more time testing than developing their product. Parallel testing lets you get more testing done in a tighter window.

With Continuous Integration, testers and developers are asked to be constantly writing new test scripts for different features and test cases. These scripts take time to run, and the more test cases running against an increasing number of environments can spell doom for a deployment. Think that testing is only going to take 2 days? Wrong, your tests have been firing off for a week, and they still have a day left. So how do we speed up testing and do more QA’ing with less time between deployments? The answer is Parallel Testing.

Sequential Test Execution:

Sequential Test Execution

 

Parallel Test Execution:

parallel-test-execution

Instead of running tests sequentially, or one after the other, parallel testing allows us to execute multiple tests at the same point in time across different environments or part of the code base. You can do this by setting up multiple VMs and other device infrastructure or by using a cloud test service like CrossBrowserTesting.

Parallel Environments

The growing number of devices and browsers your customers are using can be a challenge when trying to test quickly and efficiently. Let’s imagine a real world example: in release 3 of your product, you have 8 hours of sequential regression testing to perform before the team feels confident to deploy.

By release 5, this may be twice the amount of hours you’ll need to run your tests and as a bonus, your product is getting popular and is being used by more users on an increasing number of different devices. Before you were testing for only Chrome and FireFox, but now you see that you need Android and iOS devices, Safari and multiple versions of Internet Explorer. So you have 16 hours of tests and 10 different devices or browser to cover. This would take us 160 hours for complete test coverage before our deployment. With parallel testing environments, we can run our 16 hours of tests on 10 different devices at the same time, saving us 146 hours of testing time.

Parallel execution also has the distinct advantage of isolating test cases and runs to one specific OS or browser, allowing for the testers and developers to dedicate their meaningful resources of serious problems with cross-platform compatibility.

Parallel Testing In CBT

The maximum number of automated Selenium and Javascript unit tests (API initiated tests) that can be run concurrently is limited according to your subscription plan.

We understand the importance of running as many tests concurrently as possible to increase the velocity of your continuous integration flow. To support this, we do not place arbitrarily low constraints on the number of parallel tests.

A major advantage of our service is that we provide a combination of real operating systems and an extensive array of physical phones and tablets. While we do maintain a large number of mobile devices, they are not unlimited. Ensuring the load is spread across the configurations is important in providing availability for all our customers.

If you want a toolkit to help you, we like TestNG for Java, there is a GitHub repo below that you can look at to get started

Python Examples
NodeJs Examples
Java Examples
Ruby Examples

Best Practices of Selenium Testing

By Leave a Comment

The Selenium automation framework is a great tool-set for automating interactions with the browser. Because of its powerful open source features and standard adoption across many browsers, Selenium has become the most popular web testing framework in the world. You can run your automated testing scripts in a variety of languages against local browsers, a lab of browsers and devices using Selenium Grid or a device cloud like CrossBrowserTesting’s. To get the most out of Selenium automation, we’ve put together five best practices for automated browser testing.

Use The Right Locators

At the heart of the Selenium framework is interaction with the browser, letting you navigate, click, type and check different objects within the DOM using a few set actions. It does this using several different types of locators:

  • Class
  • ID
  • Link Text
  • PartPartial Link Textial
  • Tag Name
  • X Path

Selecting the right locators can be the different between a test script that is flexible and successful, and a brittle test that breaks on the slightest UI change. If they are present, use unique Classes or IDs as Selenium locators, as they are less likely to change without someone on your QA or Dev team knowing. But locators like link text can change all the time – think about a dynamic button based on logged in state, or your marketing team running A/B tests. You can also use Xpath to navigate the webdriver, you can read more about Xpath to CSS conversion here.

Communication Between QA, Developers, Marketers, and Designers is Key

There are two aspects of collaborating with different teams that can make your Selenium testing a success: constant communication and instilling the importance of testing to every department. Instilling the importance of the QA process on your development team can save time and efficiency when building a testing framework. Make sure they know how important unique CSS selectors are to your testing efforts, and ask them to write semantic and human-friendly code. Having constant communication across teams can help QA plan for changes to the UI of a website or application mandated from Marketing or the Design team. Having small conversations about the front-end aspect of your app decreases the chance of surprise UI changes that will go untested.

Page Object Model. Use it.

A popular test automation design pattern, the page object model will help you make robust testing frameworks that are resistant to small tweaks in the UI. The page object model has two core advantages:

  1. There is clean separation between test code and page specific code such as locators (or their use if you’re using a UI map) and layout.
  2.  There is single repository for the services or operations offered by the page rather than having these services scattered through out the tests.

In both cases this allows any modifications required due to UI changes to all be made in one place. See SeleniumEasy’s explanation and example:

Make Tests Independent Of Each Other

Testing one single action or process, independent of any other tests is known as atomic testing. This testing strategy will keep you from making chained, brittle tests. Chaining together tests, while saving time upfront, can be a huge blocker for a truly agile or CI workflow. Keep tests as small as possible.

Use Your Resources

Selenium can be tough to learn because it requires a certain amount of tangential knowledge. You have to have some understanding of programming and to truly be a skilled Selenium tester, an understanding of test automation best practices. Luckily, there are some fantastic resources available on the internet to learn from. While we run monthly webinars to get you up and running with Selenium, here are a few of our favorites:

Benefits of Test Automation With Selenium

By Leave a Comment

It’s Open Source…And Free.

Selenium is an Open Source automation framework built for web-browser testing by some of the brightest developers tired of scripting with expensive legacy automation tools not truly built for the web. Because of this, Selenium’s community is vibrant, growing and is seen as one of the real strength’s of the tool. It has become the most reliable web automation tool because of the ease of generating test scripts to validate functionality. Selenium also works with testing frameworks like WebDriver.IO, Nightwatch.js, and countless others. Oh, and it’s free, with no upfront costs.

One Script for True Cross Browser Testing

With only one script Selenium can test Chrome, Firefox, Safari, Edge, Opera, and Internet Explorer on almost any platform available: Windows, Mac, Linux, Android, and iOS. Customers are using an increasing amount of devices, as they become cheaper and more powerful. Keep up and improve your test coverage with just one script.

Works with The Languages You Use

You can use your favorite language with Selenium, so testing starts right away. With support for Java, JavaScript, C#, PHP, Ruby, Python and Perl you’ll avoid the hassle of learning another programming language just to start browser testing.

Easy to Learn and Scale

You’re not alone! With an incredibly vibrant, growing community, Selenium tutorials, testing, and development help is usually just a short search away. The documentation at SeleniumHQ is helpful, and there countless tutorials and how-to sites for you to quickly learn “Selenese”. You can even use Selenium IDE, a Firefox Record and Replay extension, to generate scripts! Selenium was built to scale with their local testing Grid, so you can be assured that as your application grows, so too will your test coverage. Here are a few of our favorite tutorial sites for Selenium:

Integrations for Every Stage of Testing

Selenium has a solid foundation of standards that make it able to be used easily with popular software development and testing tools. Integrate easily with SC compilers like Maven and ANT, CI systems like Jenkins or TeamCity, cloud-grids like CrossBrowserTesting, and almost any test management tool.

Using SeleniumBuilder and CrossBrowserTesting

By Leave a Comment

We have released a CrossBrowserTesting plugin for Selenium Builder that allows you to record interactions with your site and play them back against any of the selenium enabled browsers at CrossBrowserTesting. This makes it easy to get started with Selenium scripting – you just point and click to record your script, saving individual scripts or test suites. You can then choose to execute these scripts against our remote browsers. To get started, check out http://crossbrowsertesting.com/faq/how-do-i-record-and-run-selenium-test….

In addition, you can use our Selenium Builder plugin to save a selenium script to our screenshot system. This allows you to script the screenshot engine to a certain point, then take a windowed, full page, and full page chromeless screenshot of this page. You can then use the automated screenshot comparison engine to look for difference between pages.

Using the Selenium Builder extension with Automated Screenshots

Using the Selenium Builder extension with Selenium

Joining The SmartBear Family

By Leave a Comment

We are happy to announce we are now part of the SmartBear Software family.

Who is SmartBear Software?

Glad you asked. SmartBear builds software quality tools used by more than four million software professionals and over 25,000 organizations in 194 countries. They are the folks behind the popular testing tools TestComplete and SoapUI, as well as the killer open source API definition platform Swagger. Impressive, right?

What’s going to happen to CrossBrowserTesting?

The logo is going to change. What? You were expecting everything to change? One of the critical reasons we decided to join the folks at SmartBear is because they are equally pumped about turning CrossBrowserTesting into the #1 cloud testing platform in the world. The resources and investment from SmartBear will enable us to build a bigger and better product much faster than doing it on our own.

To give us full autonomy, the team at SmartBear has decided to maintain CrossBrowserTesting as a standalone service with Memphis continuing to serve as our headquarters. In fact, we just signed a 5-year lease for a larger office right here in Memphis. So don’t worry, we aren’t going anywhere.

SmartBear has an exceptional track record in acquiring companies and allowing them to focus on making their products better. Take the recent acquisition of Swagger. Since acquisition, Swagger has continued to innovate and accelerate its product development process with new offerings such as SwaggerHub.

The SmartBear team brings their deep expertise and understanding of the testing market and we look forward to working with them.

What happens to my customer account?

Nothing. Nada. Zilch. Your account does not change in the slightest. If you have any questions, don’t hesitate to contact us, we will be here to answer just as we always have.

A very genuine thank you to all our customers!

Without the support and love of our customers, we wouldn’t have made it this far. So we want to thank all of you. Your comments, suggestions, and feedback have been critical in developing a tool that people love. We look forward to continue serving you and showing you the new stuff we are working on.

Ken, Mike & Tony

Join Over 200,000 Testers & Developers And Start Testing Today

Learn more Free Trial