test automation Archives | CrossBrowserTesting.com

6 Key Factors That Improve Cross Browser Testing

October 15, 2019 By Sarah Mischinger Leave a Comment

Our friends over at Bitbar recently wrote a fantastic blog about 6 key factors that can improve your cross browser testing. Take a look below!

If you start thinking about testing after the completion of your web application, you’re already too late. The decisions you make before and during development impact and shape testing more than one might think. And not involving testing from the start may lead to some severe problems that can prevent a timely release. In particular, cross browser tests can be a bottleneck. That’s why we’ve put together a list of 6 essential factors that can help you improve cross browser testing and plan ahead!

1. New Isn’t Necessarily Better

Web technology advances swiftly, and innovative features are being introduced with each new browser version. We know the urge to use fancy new tech in a project can be massive. But you have to keep in mind that new doesn’t automatically mean better! Some of these functionalities are still in beta, experimental, or not yet supported in all major browsers. As a result, you deliberately introduce cross browser variations into your product and face a more difficult time testing.

Our advice:
If using a particular new technology that brings no real value to your users, think twice about integrating it into your app. Avoid unnecessary cross browser variations in your web application.

2. Stop Reinventing the Wheel

When your team plans the technology stack for your next web-based project, see if you can use existing frameworks, libraries, and so on. Use React, Vue.js, or Angular, and don’t depend on Vanilla JS in your app too much. You not only save a considerable amount of time in development but also during testing! The same is true for CSS frameworks like Bootstrap or Foundation. They are already tested on a wide range of browsers, and devices, saving you time testing your app.

Our advice:
Go with established frameworks and libraries to write your CSS and JavaScript code. They are usually thoroughly tested, and you can safely use them in your project. That way, you reduce the code you need to write and validate. Additionally, you can reuse already tested components, further reducing testing time.

3. Only Test Clean Code

Have your developers follow coding and style guidelines and let them deliver clean, consistent code. As a result, your code is less error-prone, and bugs can be found more efficiently. Additionally, anyone in the team can work on the source code regardless of who wrote it. If you don’t have guidelines or don’t have the resources to create them, use existing ones! Why not use style guidelines from other companies such as the Airbnb React/JSX Style Guide?

Our advice:
Using and enforcing coding and style guidelines makes developing and testing your app more straightforward. Ensure your team delivers clean code by having your developers install code linters in their IDEs. These linters should automatically check the written code against your guidelines and find minor errors, like spelling mistakes or missing brackets.
As a result, cleaner code – without the usual minor errors – arrives at your testers without investing too much extra time!

4. Make Use of Automation

Test automation can help you shorten testing time significantly so you can release updates and bug-fixes more often. Although automating cross browser testing can be a bit tricky in some places, we can’t not take notice of the benefits. Automating tests also mean that your devs and testers can eliminate repetitive and tedious tasks from their to do list. This gives them more time to create new features and perform necessary manual tests.

Our advice:
First, see what you can automate and which parts or browsers you should manually test. Here are some ideas to get you started:

Follow TDD for your JavaScript code. Write and automate unit tests by using a framework like Mocha.
Use browser automation and inspection tools like Selenium WebDriver to automate your end-to-end tests.
Take a look at the different tools in the Selenium suite that help you test your apps. Learn more about the suite in our blog article series about Selenium.
Make sure your app looks immaculate on different screen resolutions by automating image regression testing. Cloud-based services like CrossbrowserTesting provide the functionality you need.

5. Test on a Large Scale

Automating your tests is nearly worthless if you don’t run them on real browsers and devices, preferably at the same time. Sure, test automation saves time, but if you have to run these tests on one device and one browser at a time, the process takes forever. Also, keep in mind that your users are real people who use real computers, smartphones, tablets, and browsers. Can you imagine anyone using emulators to browse the internet? Consequently, make sure to run your automated tests on multiple real-world devices and browsers in parallel. This lets you test more efficiently so you can spend more time on other things.

Our advice:
Choose a cloud-based testing service, that lets you run tests on multiple real-world devices and browsers simultaneously. As a result, you can run more tests in less time while ensuring your app works as intended in a variety of real-world scenarios. In the end, you can release new features to your customers with more confidence.

6. Bring Continuous Integration into Play

If you’ve integrated all previous steps into your workflow to improve cross browser testing, then Continuous Integration (CI) is the icing on the cake. If you are unfamiliar with CI, be sure to read some information on this topic, as you will undoubtedly benefit greatly.

In a nutshell, CI works as follows: Whenever a developer pushes code into the project’s central repository, several jobs start running. Usually, these jobs perform a variety of tests. In the end, CI helps you determine whether the committed code is releasable – only commits that pass the validation get accepted.

Our advice:
Using CI does not require your developers to start the tests manually, so no test can be forgotten. Additionally, only perfectly valid code is accepted for release. Not only does CI automate repetitive processes, but it also makes sure your app performs at its best. You can find many different CI tools that meet your specific needs. For example, use Jenkins if you want to run jobs on a dedicated server. Or use TravisCI if you wish to use a cloud service instead. However, many, but not all, cross browser tests, can be integrated into a CI job. Consequently, you need to pick the service that’s most compatible with your other tools! Check out the integrations CrossBrowserTesting supports!

Wrapping up – Improve Cross Browser Testing by Using the Right Tools

Test early and often, use tools that fit your needs, and make sure you run automated tests on real devices as often as possible! Use different tools and services that match your situations and needs. Don’t hesitate to use a cloud-based service to make your test processes more efficient and resource-saving.

Why Should I Run My Selenium Tests in Headless?

October 8, 2019 By Daphne Magsby Leave a Comment

Headless testing is greatly underutilized – here are three quick ways it can increase performance

Selenium has grown to be one of the most popular automation tools available today. It automates many tasks needed for online testing, including checking titles, navigating pages, clicking links, and much more. Many developers already use the full set of capabilities Selenium offers, but few know about running their tests in headless browsers. And even fewer actually deploy this approach in their daily testing. With headless testing you can increase your testing performance to an all-time high.

What is Headless testing?

Headless testing is simply running your Selenium tests using a headless browser. It operates as your typical browser would, but without a user interface, making it excellent for automated testing.

How does Headless testing benefit developers?

There are several benefits, actually. They can create new testing opportunities, as well as accelerate tests you’re already running.

Benefits include:

Greater testing reach
Improved speed and performance
Multitasking

Greater testing reach

When running Selenium tests, you typically need a machine that supports the graphics of the web browser that you’re testing on. With headless testing we get rid of this need and open up a whole new set of devices to test on. Ex. Servers, docker containers, etc.

Improved speed and performance

Selenium tests can take a while to complete, due to elements on a page that the browser needs to load. Headless testing gets rid of this load time, allowing you to cut your testing times significantly. In our tests with headless testing, we’ve seen a 30% reduction of test execution times. And you can use other techniques, like running more tests in parallel, to amplify that benefit.

Multitasking

Running normal Selenium tests take up your screen time, keeping you from being able to accomplish anything else on that device. With the UI disabled, headless testing lets you continue to use your computer while the tests execute in the background.

In our conversations with customers, we’ve found that headless testing has let developers experience all of these benefits – and many more!

How have you used headless testing to improve your processes? And where do you see opportunities to take advantage of them? Respond in the comments!

To help you get started Headless testing with CrossBrowserTesting, visit our help page: Headless Testing Documentation

Are These 5 Gaps Missing From Your UI Test Automation Strategy?

April 1, 2019 By Alex McPeak Leave a Comment

As more and more organizations integrate test automation into their workflows, teams have benefitted from shortened feedback loops and less time between development and delivery.

However, while automation has been an asset for effective and efficient testing, many of us are still getting acquainted with the skills required to build out tests that give us maximum value for our time.

What many don’t realize is that there could be gaps in your test automation strategy. These common oversights could be the difference between valuable automation suites and inadequate test results.

Recently, SmartBear hosted a webinar to address a few of these gaps and their solutions. Here are the top insights.

Spending too much time maintaining test scripts – If you’re putting too much time, effort, and energy into creating test scripts, it may be time to reconsider your approach. Often this can be caused by improper planning or poor structuring of testing frameworks. However, by making tests more reusable, maintenance time can be reduced significantly. Rather than making large tests in an attempt to cover as much of your application as possible, break them up into smaller tests cases. You can then use these tests to build out larger, more comprehensive test cases. This way if the UI or the user flow changes, it’ll be easier to go back and use your components.
Zero or little mobile browser testing – In 2018, half of website traffic came from mobile devices, yet companies still tend to neglect testing on mobile. Just having a mobile version of your web application is not enough, you need to verify that it also works correctly on different devices. Without testing on real devices and tablets, you have no idea what your users are experiencing every time they visit your website on a smaller screen. This is not something you want to leave up to chance as mobile web usage continues to increase.
Lack of environment options – Even the best intentions to increase test coverage through automation can fall short without a substantial number of environment options to test on. Different environments can render code in different ways, which means it’s crucial to look at your website on different browsers and devices. Having limited environment options can lead to issues going unnoticed. If you’re only testing one or two browsers and devices, you’re not going to know where there are issues in the others your customers are using. You don’t want preventable bugs slipping through to customers when the fix is as easy as increasing the number of environments you test on.
Spending too much time and money maintaining devices – Some may assume that the best answer to a lack of environment options is to build a device lab. In theory, this seems like a good idea — you’re team has 24/7, onsite access to all the machines and configurations they could possibly need. However, the reality is that these devices are expensive to buy and maintain. Tens of new mobile devices are released every year, meanwhile, existing devices must be repaired and maintained. Not only is the time commitment not practical for most organizations, but it’s also expensive. Most of the time, it’s more trouble than it’s worth to build out a device lab when there are expansive cloud options available.
Lack of parallel testing – Parallel testing is your key to increasing test coverage without compromising time. The issue? This practice is often underutilized by teams. While thinking about the time it takes to test in all these environments might be overwhelming, parallel testing allows you to significantly cut down on the hours it would normally take to run one test after the other. By testing multiple configurations simultaneously, you can condense 30 hours of testing on three different machines down to 10. In this way, parallel testing will reduce the time spent running tests, while still allowing you to hit the environments your users are on.

While automation is no easy task, there are steps we can all take to improve efficiency. Bridging these five gaps in your test automation strategy with the help of tools like TestComplete and CrossBrowserTesting will allow you to get more value out your time.

If you’d like to learn more about these common UI test automation gaps, check out the full webinar below.

Maximum Concurrency with Nightwatch.js

January 9, 2019 By John Reese Leave a Comment

You made it. After weeks writing a dozen Selenium tests with Nightwatch.js, you are done. Your elements have been found, your buttons have been clicked, and your page objects have been modeled. The only thing left is to run your suite of tests across all the browsers you care about. That’s where parallel testing comes in handy. Twelve tests across five platforms mean 60 test cases, but you have a 25-parallel Automation plan, so this shouldn’t take long.

$ nightwatch --env ie,edge,chrome,ff,safari

1	$ nightwatch --env ie,edge,chrome,ff,safari

You press enter and…

Started child process for: ie environment
Started child process for: edge environment
Started child process for: chrome environment
Started child process for: ff environment
Started child process for: safari environment

Started child process for: ie environment

Started child process for: edge environment

Started child process for: chrome environment

Started child process for: ff environment

Started child process for: safari environment

Huh, that’s funny. It looks like it’s only running five tests at a time, not 20.

Yup, CBT shows that there are only five tests running. What gives?

It turns out that if you use the --env flag to specify the browsers to run on, then Nightwatch will run each environment in parallel, but each test case will execute sequentially.

That’s not what we want! What’s the point of having a big beefy Automation plan if you can’t use it?

So the question becomes, how do we run with MAXIMUM CONCURRENCY?

Well, I couldn’t figure out how to do it in Nightwatch, but I hacked together a workaround!

The following Node.js code will take the --env (or -e) flag and launch an instance of Nightwatch for each specified environment. If no environment is specified, then it will use all environments specified in your Nightwatch configuration file.

parallel-nw.js:

const cbt = require('cbt_tunnels');
const nwConfig = require('./nightwatch')

const spawn = require('child_process').spawn;

const tunnelOptions = {
  'username': nwConfig.test_settings.default.username,
  'authkey': nwConfig.test_settings.default.access_key
}

async function startTunnel(tunnelOptions){
  return new Promise( (resolve, reject) => {
    cbt.start(tunnelOptions, function(err, data) {
      if (err) {
        reject(err)
      } else {
        resolve(data)
      }
    });
  })
}

function envNames(config){
  if(config && config.test_settings){
    return Object.keys(config.test_settings);
  }
  else {
    return [];
  }
}

async function spawnNW(envName){
  return new Promise( (resolve, reject) => {
    let err;
    let proc = spawn('nightwatch', ['-e', envName], {shell:true});
    proc.stdout.on('data', data => {
      if (data.toString('utf8').replace(/\s/g, '') !== ""){
        console.log(`${envName}:\n ${data.toString('utf8')}`)
      }
    })
    proc.stderr.on('data', data => {
      if (data.toString('utf8').replace(/\s/g, '') !== ""){
        console.error(data.toString('utf8'))
      }
    })
    proc.on('close', (code) => {
      if(code === 0){
        console.log(`${envName}: Finished!`)
        resolve(code)
      }
      else {
        reject(new Error(`${envName}: failed ` + err));
      }
    })
  })
}



( async () => {

  // make sure tunnel is started before starting selenium tests
  await startTunnel(tunnelOptions);


  // look for environment names in command line args...
  let names;
  for (let i=0; i < process.argv.length; i++){
    if (process.argv[i] === '--env' || process.argv[i] === '-e'){
      names = process.argv[i+1].split(',');
    }
  }

  // or pull from the config file
  if (names === undefined){
    names = envNames(nwConfig);
  }

  // spawn nightwatch for each environment
  let procs = names.map(spawnNW)

  console.log(`Starting to run Nightwatch on ${names.join(', ')}`)

  // wait for all nightwatch processes to return
  Promise.all(procs)
    .then( () => {
        console.log('Nightwatch finished!')
        // stop tunnel when all tests are done
        cbt.stop()
    })
    .catch( (err) => {console.log(err)})
})()

const cbt = require('cbt_tunnels');

const nwConfig = require('./nightwatch')

const spawn = require('child_process').spawn;

const tunnelOptions = {

'username': nwConfig.test_settings.default.username,

'authkey': nwConfig.test_settings.default.access_key

}

async function startTunnel(tunnelOptions){

return new Promise( (resolve, reject) => {

cbt.start(tunnelOptions, function(err, data) {

if (err) {

reject(err)

} else {

resolve(data)

}

});

})

}

function envNames(config){

if(config && config.test_settings){

return Object.keys(config.test_settings);

}

else {

return [];

}

async function spawnNW(envName){

return new Promise( (resolve, reject) => {

let err;

let proc = spawn('nightwatch', ['-e', envName], {shell:true});

proc.stdout.on('data', data => {

if (data.toString('utf8').replace(/\s/g, '') !== ""){

console.log(`${envName}:\n ${data.toString('utf8')}`)

}

})

proc.stderr.on('data', data => {

if (data.toString('utf8').replace(/\s/g, '') !== ""){

console.error(data.toString('utf8'))

}

})

proc.on('close', (code) => {

if(code === 0){

console.log(`${envName}: Finished!`)

resolve(code)

}

else {

reject(new Error(`${envName}: failed ` + err));

}

})

}

( async () => {

// make sure tunnel is started before starting selenium tests

await startTunnel(tunnelOptions);

// look for environment names in command line args...

let names;

for (let i=0; i < process.argv.length; i++){

if (process.argv[i] === '--env' || process.argv[i] === '-e'){

names = process.argv[i+1].split(',');

}

// or pull from the config file

if (names === undefined){

names = envNames(nwConfig);

}

// spawn nightwatch for each environment

let procs = names.map(spawnNW)

console.log(`Starting to run Nightwatch on ${names.join(', ')}`)

// wait for all nightwatch processes to return

Promise.all(procs)

.then( () => {

console.log('Nightwatch finished!')

// stop tunnel when all tests are done

cbt.stop()

})

.catch( (err) => {console.log(err)})

})()

Once that file is in place, we can use it to run our tests!

node parallel-nw.js --env ie,edge,chrome,ff,safari

1	node parallel-nw.js --env ie,edge,chrome,ff,safari

Much better!

Nightwatch.js is an End-to-End testing platform for writing and running Selenium tests with Node.js.

SmartBear Named a Leader in Gartner Magic Quadrant for Software Test Automation

January 8, 2019 By Alex McPeak Leave a Comment

SmartBear was named a Leader in the 2018 Gartner Magic Quadrant for Software Test Automation. Gartner, the world’s leading information technology research and advisory company, has recognized SmartBear for the fourth consecutive year, with this being the first as a Leader.

Amidst emerging trends in DevOps and Agile Development, SmartBear has solidified its commitment to accelerate delivery by enabling continuous testing, making it a key player in this space.

SmartBear has encouraged teams to adopt intelligent automation practices across the entire software delivery lifecycle, providing a portfolio of tools to easily design, test, and monitor at both the API and UI layer such as TestComplete, TestLeft, Hiptest, ReadyAPI, and CrossBrowserTesting.

CrossBrowserTesting is proud to be part of SmartBear since being acquired in 2016. The CrossBrowserTesting provides the ability to run Selenium- and Appium-based tests across over 1500 real desktop and mobile devices in the cloud. Additionally, capabilities for record and replay, remote interactive testing and debugging, and visual testing in CrossBrowserTesting make it an asset to teams are practicing both manual and automated testing.

The strong support for open source communities like Selenium in CrossBrowserTesting as well as the OpenAPI Initiative, Swagger, and SoapUI has further propelled the success of SmartBear. By contributing to these initiatives, SmartBear has a crucial hand in the open source tools that are used by thousands of software teams every day.

Additionally, new offerings over the last year have broadened the portfolio and enabled SmartBear to lead the test automation market. With the acquisitions of Zephyr for test management and Hiptest for continuous testing with native BDD support, as well as the development of LoadNinja, SmartBear is innovating at an unparalleled speed.

“We’re proud to be recognized by Gartner as a Leader in the Magic Quadrant,” said Christian Wright, Chief Product Officer and Executive GM at SmartBear. “We believe our Leader position reaffirms our vision of enabling quality throughout the software development lifecycle through a comprehensive test automation portfolio. We see Gartner’s recognition reflects the excellent feedback we receive from our vast customer base and our teams’ ability to deliver open, collaborative, and easy-to-use tools to our users.”

To access a complimentary copy of the 2018 Gartner Magic Quadrant for Software Test Automation, visit https://smartbear.com/resources/white-papers/gartner-magic-quadrant-2018/.

Gartner, Magic Quadrant for Software Test Automation, Joachim Herschmann, Thomas Murphy, Jim Scheibmeir, 27 November 2018

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from [insert client name or reprint URL].

3 Methods for Hands-free Continuous Testing

December 4, 2018 By Alex McPeak Leave a Comment

In order to strike the perfect balance between speed and quality, teams that hope to achieve Continuous Integration and Continuous Delivery must also implement Continuous Testing.

This is often easier said than done. It’s virtually impossible to have your team running tests at every moment — they do have lives outside of work after all, and it can be difficult to put dinner on the table while running a Selenium script at the same time. So how do you run your scripts, even if you aren’t at your desk?

Fortunately, there are various hands-free options for continuous testing that you can implement, which will keep tests running for you while you eat, sleep, or maybe even get some exploratory testing done.

Why Continuous Testing

What’s the point of continuous testing? There are a few major advantages that will enable you to achieve more in your day-to-day.

Test while you sleep – With continuous testing, you can literally test in your sleep. By setting tests to run overnight, you can experience the luxury of coming in the next morning to have your test results waiting for you and knowing with a glance whether they passed. Just ask America’s Test Kitchen about how convenient it is.
Keep up with Agile, DevOps and CI/CD – When software development teams are constantly integrating, you need a testing strategy to ensure those new features, fixes, and changes are put through the quality process, no matter how frequently they’re implemented. Continuous Testing is the only way QA teams can keep up with constantly changing features, frequent integrations, and fast workflows.
Catch bugs in each regression – When there are constantly small changes to an application’s code, it can break something that was previously functioning. And you don’t want your customers to be the one to catch it. Continuous Testing will help you be first to spot new bugs, so you can quickly debug and release to users with confidence.
Level up your automation game – Automate your automation with continuous testing, and set your tests to run as often as you need. Then come back to test results or screenshots that tell you exactly what you need to know about the health of your applications. Plus, testing in parallel allows you to further accelerate Continous Testing to increase browser coverage without compromising runtime.

Continuous Testing With a CI Server

Selenium WebDriver has long provided teams with the ability to automate tests across browsers. But in order to run those tests continuously, a Continuous Integration tool such as Jenkins, VS Team Services, Buildbot, Bamboo, CircleCI, TravisCI, Codeship, or TeamCity is needed.

By far, the most popular option is Jenkins, an open source automation server written in Java. In fact, according to the SmartBear 2018 State of Testing Survey, 66 percent of participants choose Jenkins to be their CI server. For development teams adopting Agile and DevOps workflows, Jenkins is a reliable tool for continuous integration and delivery.

But what about testing teams that want to leverage Jenkins for Continuous Testing with Selenium? The CrossBrowserTesting integration with Jenkins lets you run automated functional tests with Selenium, Appium, and visual screenshot tests, which means that every time you run your build process, you can simultaneously run your testing suite across thousands of browsers.

This is ideal for teams that already work with Selenium and a CI tool for deployment to start testing on a continuous basis. While Jenkins tends to be the most popular choice for CI and automated test scheduling, CrossBrowserTesting also integrates with TeamCity and VS Team Services, so no matter what tools you’re already using, it’s easy to integrate CrossBrowserTesting into your current workflow.

Continuous Testing With a Record and Playback Tool

For teams that may be less familiar with Selenium but still wish to automate, a record and playback tool might be a more practical option.

Record & Replay enables teams to achieve automation across browsers, just like they would with Selenium, without requiring knowledge of scripting and frameworks. This has provided teams with a more accessible option for less technical team members, or those who are just getting started with automation, to create automated tests. As tools like Selenium IDE and its alternatives are brought back into the spotlight, these options are becoming more and more attractive to modern testing teams.

But where record and playback tools will normally only run the test once to check if the test passes, how can testers implement Continuous Testing?

With new Scheduling in CrossBrowserTesting, you can schedule a Record & Replay suite to run as often as you need them. For organizations that have less technical testers, this allows you to achieve continuous testing without requiring the skills to run Selenium and Jenkins or another automation server.

Continuous Screenshot Testing

Teams that solely focus on functional testing without incorporating visual testing will be disappointed when all their tests pass and users are still pointing out visual bugs.

When visual elements can change from browser to browser or device to device and make the user’s experience difficult or impossible while still functioning correctly, it’s important to run compare screenshots in regression. Comparing these pages side by side and with their historical versions means you can be more aware of changes that break the application in order to debug before they reach the end user.

While you could always run automated screenshots in CrossBrowserTesting, it required a manual motion to rerun those tests on a certain cadence. You can now set those screenshot tests to run with the new Scheduling feature, just like you can with Record & Replay.

Continuous Testing for Every Team

No matter what tools you’re already using, what technical level you’re at, or how far your team is in their test automation journey, there are options to help every team achieve continuous testing in CrossBrowserTesting.

With options to schedule tests with Selenium, Record & Replay, and automated screenshots, you can take your automation to the next level and receive test results through email or Slack on a daily, weekly, or monthly basis that you decide.

The State of the New Selenium IDE

November 19, 2018 By Alex McPeak Leave a Comment

Last week, Applitools teamed up with Simon Stewart for “Introducing Selenium IDE: The Next Generation”.

Selenium IDE is part of the Selenium suite and has been a depreciated record and playback tool, largely because it could only be used in Firefox and had stopped being supported by contributors of the Selenium project.

But don’t call it a comeback — Selenium IDE 2.0 (still to be officially named) is more so a new and improved version of its former self.

What’s New?

Some of Selenium IDE you might recognize from the first version, such as their UI. However, while it may look similar at first glance, you’ll soon notice many improvements including better locators and support for common tasks.

Potentially the biggest differentiator with the new Selenium IDE is that it’s built as a web extension, which is standardized by the WC3, so it no longer only works in Firefox.

This means that the new Selenium IDE can run in any browser that supports web extensions such as Chrome, making record and playback cross-browser testing much more achievable than in the previous version.

How to Install the New Selenium IDE, Record, and Play

In the webinar, Simon showed us how we can install Selenium IDE and run our first test.

In Firefox:

Go to Tools
Click Add-ons and Get add-ons
Search Selenium IDE and click Add to Firefox
Go to the Menu bar and select Selenium IDE

In Chrome:

Go to Chrome Web Store
Search Selenium IDE
Add the extension (make sure to select the one from SeleniumHQ.org)

In the example, Simon does a Google search for “cheese,” which is as easy as firing up the new IDE, choosing to record a test, entering the URL you want to start with, and then going through the actions that you want recorded manually.

The new IDE also has options during testing such as assert and verify. If you want to edit the test steps after, you can do that, too.

As mentioned, there are quite a few value-add features and enhancements in Selenium IDE gathered from users over the years.

If you want to run tests with a command line, you can put tests into a CI system, or allow them to run locally with the Selenium IDE runner. The runner supports additional features such as parallel execution through a Selenium grid or a server like CrossBrowserTesting.

There are other enhancements with the new IDE, as well. Control flow statements enable more maintainable tests, whereas before, it was only available through plugins.

Additionally, fallback locators will provide better test stability and make tests more resilient despite changes. Previously, Selenium has been known as brittle and unstable because locators have been hard to write and maintain.

There currently is no option to export recorded tests like there was with the original IDE, but this is immediate on the product roadmap.

Selenium IDE Alternatives

As we discuss Selenium IDE, the conversation around record and playback tools is brought back into light.

More and more individuals and organizations are realizing the value of record and playback tools like Selenium IDE and Selenium IDE alternatives, such as Record and Replay in CrossBrowserTesting.

Both of these tools are ideal for the use cases that Simon reviewed in the webinar. Record and playback is helpful for:

Team members that don’t aren’t familiar with programming languages
Someone who wants to put together a bug report
Someone just getting started with automating a test suite
Automating boring and repetitive tasks

Of course, the benefits of Selenium IDE and its alternatives don’t stop there — the more teams adopt record and playback tools the more uses they’ll find.

So, whether you decide to stick with Selenium or try a tool like Record & Replay, it’s worth exploring the possibilities of a record and playback tool.

Next Steps for Teams Who Want to Move to Agile Project Management

November 7, 2018 By Alex McPeak Leave a Comment

It seems that the companies we all aspire to be more like are moving quicker and releasing faster than ever before. So, how can the rest of us strive to follow in the footsteps of software giants that release innovative software on the hour? While there may be no single, clearly-defined answer, we do know that many are seeing success with agile project management and development methods.

By focusing on feedback from stakeholders throughout the development process, not only is agile found to be a faster development method, it’s also more effective in the way that it allows and encourages changing requirements.

However, as waterfall is the traditional methodology for software development, teams that are stuck in their old ways might find the transition to agile hard to wrap their head around, so here are 7 tips for teams who want to take the next steps and move to agile project management.

Identify what or who is missing to make agile work. In order to move to agile, you first have to recognize that it will be process your entire team will have to get on board with and fully embrace in order to make it work. Identify the roles and people you need on your team to make a successful transition. It might be helpful to onboard a project manager, scrum master, or agile coach to help adjust the current workflow. Look at your current team, as well, to understand where there are silos and barriers to communication. Teams that don’t communicate will find it hard to be successful. Agile project management works best when this is cross-departmental collaboration, so it’s important to work from the inside out.
Prepare to shift left. Agile will require more testing earlier in the software development lifecycle. Because each phase of the SDLC is done in short iterations, rather than long incremental ones like with waterfall, testing shouldn’t wait until the end when new features are fully developed. With a focus on continuous integration and continuous delivery that comes with agile development, continuous testing will be the key to fast feedback and iterative changes that are needed to release high-quality software. Evaluate whether you will need to introduce test automation, additional QA team members, new tools, or something else to help you reach maximum test coverage.
Poke holes in what you’re doing now. Look at your current planning process, and outline how your strategy will have to change in agile development to allow for more feedback and flexibility. Where are the bottlenecks in your current process, and what prevents your team from delivering high-quality software or meeting short deadlines? This will be crucial to shifting to a new way of thinking and transforming your team’s workflow. Familiarize yourself with the different agile methodologies such as Scrum, Kanban, and Extreme Programming. While these all follow agile principles, they achieve it in different ways. Based on the makeup of your team and the objectives you discussed, it will affect the way you carry out agile project management.
Get on board with sprints and standup. No matter which methodology you choose, a few things will probably be the same, and they’ll help you properly implement agile project management in your organization if you take the time to do them correctly. Sprints will be the duration of you planning — usually they’re two weeks long but can be anywhere from one to four — and standups are where you talk about what you’ll be doing in each sprint. Take advantage of these opportunities for strategic planning and collaboration with the entire team.
Backlog issues and tasks. Identify the projects that currently challenge your team and what you’d like prioritize when you transition to agile. Create a backlog of tasks, gaps, objectives that need to be addressed so you can start ranking them on most to least important and having conversations about future goals. Discuss what is needed from each team member, how it will be accomplished, and what information is currently needed to make it as “done”. Once you begin to organize your team and the work that needs to be done, you can start building out detailed tasks and sprints.
Don’t skip your retrospective. As you adjust to agile and the many changes that come along, you’ll be constantly learning about what’s working well and what isn’t, as well as what individuals are successful with and what they’re struggling with. Ensure that everyone has an equal say in each sprint, so it’s not just leaders and managers giving their two centrs. Focus on how you can improve in each sprint. However, don’t just wait until retrospectives to collect feedback from your team — encourage communication and checkins throguhout the sprint.
Be open to change. Requirements change, tasks get de-prioritized, clients and customers change their mind, and feedback will alter the sprint process. The nice thing about agile project management is that it encourages these adjustments. Embrace agile and understand that these frequent changes are improvements that will help your team achieve continuous integration, testing, and delivery. If you feel like you’re getting lost or confused, always resort back to the agile manifesto to realign your team with the principles behind agile.

Finding your groove with agile won’t happen overnight, but there are certain steps your team can take to make the transition more seamless. For more, take a hint from what agile teams can learn from Netflix.

The Spectrum of Testing Practice: Why Are Some Companies Stuck in the 90’s?

October 24, 2018 By Lisa Crispin Leave a Comment

Those of us who are passionate about delivering valuable, high-quality software to our customers frequently and at a sustainable pace are living in exciting times.

Many are embracing “modern testing” principles. We’re acquiring new skills such as how to help non-testing teammates learn to test, how to analyze production use data, and how to use that data for testing. Testing is at the heart of DevOps culture, providing new opportunities for testers. We have amazing tools to help us with activities such as regression test automation and learning from production monitoring, logging, and observing.

At the same time, I still encounter many companies who are doing testing the way most people approached it 20 or more years ago. They have siloed testing/QA teams who don’t collaborate with development teams, operation teams, or even product and design teams. They have no automated regression tests or are struggling mightily to get traction on that automation. They’re doing only manual regression testing, working from written scripts, and no exploratory testing.

Why??!!!

I’ve seen leading agile practitioners scratch their heads and wonder – after two decades of sharing agile values, principles, and practices with the world, why are so many people not using them?

Teams that do use a whole team approach to testing and quality are successful at improving their processes and their product. The State of DevOps report shows correlations between the use of modern testing approaches and high team performance. So why isn’t everyone trying to use what we’ve seen work well for 20 years now?

I have no actual evidence as to why this is, but I have some unscientific theories which I’d like to share. I’d love to hear your theories too.

Lots of Newbies

The number of new software professionals is growing fast. The Bureau of Labor Statistics predicted a 24% increase in software developers alone from 2016 to 2026. Despite having heard “testing is dead” for the past 15 or so years, I see more and more testing conferences with more and more people attending them, so I surmise our profession is also growing fast.

Universities are still generally poor at teaching modern development and testing approaches, so people come out of school without expertise in agile or DevOps values, principles, and practices. They certainly don’t learn much about testing in university. So, they have to somehow discover all this once they’re on the job. If they join a company whose software process is still stuck in the 90’s, doing poor waterfall at best, they’re unlikely to be exposed to modern testing.

Culture is Hard to Change

In my experience, it’s extremely difficult to change an established company culture, especially in a large organization. Even big enterprise companies who “go agile” often transition from role-based silos to having dozens of siloed Scrum teams that don’t talk to each other. All too often, an IT organization transitions to “agile” and either leaves the testers on their own test/QA team or sticks them in a cross-functional delivery team with no training or support to figure out how they should now work with people in other roles.

Large companies often have a complicated power structure. Upper managers may be more interested in protecting their domain than in delivering better software to their customers more frequently – and they don’t always prioritize a sustainable pace. If nobody educates them on how an investment in software quality – doing things like giving teams time to experiment and learn — pays off in the long run, they just keep imposing unrealistic deadlines while their software teams burn out.

Change is hard. Even when management is receptive, maybe not all team members are willing to try something new. It often takes only one naysayer to kill an effort to move away from “the way we’ve always done it”. Companies that are strapped for money may not be able to see how an investment in learning pays off.

I once worked in a company with a “hero culture” where the person who fixed the problem that brought down the website was lauded and rewarded – so why try to prevent production problems from happening? Even after leading a successful agile project to meet an “impossible” deadline, I couldn’t affect change in that culture.

Life is Challenging

If your management doesn’t support you learning how to improve the way you deliver your software, you have to do it on your own time. As you learn, you can be a change agent and try to help your team improve.

But that takes time, and we all have many demands on our time. Some people have to work two or more jobs to support their family. Some people must spend much of each day caring for a family member. Others have health issues that limit their activities. Perhaps they can’t afford to go to conferences. Perhaps they can’t take an evening off to go to a meetup or watch online video courses. There are so many reasons people aren’t able to learn on their own time.

So, how do we promote adoption of modern software delivery principles and practices?

I don’t have any easy answers, but I’d like to start more conversations about this. I think we can raise awareness that there are better ways to work, that it’s possible to make our customers happier while enjoying our work a lot more. Here are some ideas:

If you can, make time to educate yourself with the many resources available to us these days: online courses, webinars, blogs, books, articles, podcasts, local meetups, conferences.
Share what you learn with your teammates. Help them learn about different types of testing. Try small experiments together to improve.
When you meet other software professionals, for example in a social situation, encourage them to join you at your local tech meetup
Write about your own experiences, share them at meetups and conferences, to show others that improvements are doable and effective (if public speaking scares the pants off you, check out the SpeakEasy mentoring program)
Contribute to scholarship programs that help people attend conferences and access online content such as webinars and videos of conference talks

We have new people joining our profession all the time. What ideas do you have to help them embrace 2018 rather than 1988?

About Lisa Crispin: Lisa is a tester who enjoys sharing her experiences and learning from others. She works as a Testing Advocate at mabl, is a co-author of Agile Testing: A Practical Guide for Testers and Agile Teams as well More Agile Testing: Learning Journeys for the Whole Team, she’s a frequent conference attendee and organizer, and she is an avid donkey lover. To find out more, find her on Twitter or LinkedIn.

The Top 5 Things We Learned in Our Webinar with Bas Dijkstra

October 9, 2018 By Alex McPeak Leave a Comment

Recently, Bas Dijsktra of On Test Automation joined the CrossBrowserTesting team in the webinar “Lighten Up: Methods for Making Your Automation Suite Leaner ”.

Bas’s presentation was jam-packed with important lessons and actionable takeaways. To get the most out of it, you should take the time to watch the full webinar at the bottom of this post.

But we’re also rounding up a few of the most important lessons we learned from Bas. Here are our top five:

There’s such thing as too much automation – With the introduction of tools like Selenium that allow us to automate at the UI level across different browsers, it can be tempting to automate anything we know how to write a test for. However, automation isn’t always the answer, and if you’re automating too much, it may be more of a problem than a solution when your automation suite becomes bloated. This can mean slow tests, shallow feedback, and high maintenance costs that reverse the benefits of automation. Additionally, when your suite is too heavy with automation, it no longer helps to shorten your feedback loop, and will likely add time instead. In order to get the most value out of your automation suite, Bas says that it’s important to understand where you should be testing at the UI level, when you should be testing at a lower level, and when to skip automation altogether.
We have some unrealistic expectations around automation – Automation should never be the end goal of our testing. Bas advocates that we stop trying to automate everything and start focusing on automating the right things. Prioritize the business value rather than automating just to automate. Understand that not everything testers do can be automated, and not all test should be automated. Know the limitations of automation and your testing tool, and familiarize yourself with the tests that could be taken out of your suite to lighten your load to improve your feedback loop.
Automation requires resources – One of the reasons why it may not be worth it to automate is return on investment, or ROI. Automation requires resources to write, maintain, run, and analyze tests. Bas says that this is too often forgotten when teams strive for “complete automation”. You have to decide whether it’s worth it to expend this energy and resources on automation, or if it would be best spent somewhere else. For example, maybe it would be more valuable to do an exploratory test of your application and do a deeper dive into areas you can’t assess with automation. Ask, “Are we comfortable not having the information provided by this test?” If you are, it might be a reason to redirect your resources away from automation.
Not everything should be tested from the end user perspective – UI testing is great to test the end user perspective, but it’s not always the best method. You should automate as close to the implementation as possible and only do user interface-driven end-to-end automation when verifying end-user journeys. Ask, “where is the logic that we’re verifying actually implemented?” Often times you can run a test in the UI, but it won’t give you the feedback you’re looking for. In his example, Bas shows there are instances where testing at a lower level will give you more accurate test results.
You might not need to run your automation suite as often as you think – UI tests can take a long time to run, and with multiple browsers and operating systems, it can take even longer. While tools like CrossBrowserTesting allow you access to a seemingly unlimited amount of browsers, that doesn’t mean you have to test in them all every time. Bas encourages you to decide which browsers you need to test and how often by looking at which your users are on and which are most problematic. If you think there’s value in testing in a browser or operating system but not adding it to your regression suite, CrossBrowserTesting lets you look at those same configurations in on-demand so you can test manually, as well. Bas also suggests moving tests out of the build pipeline and into an overnight job, so you’re not spending as much time running your suite.

Conclusion

As teams become better acquainted with automation, they have certain choices to make about what to automate, how to automate it, and why it should be automated. Unfortunately, sometimes we don’t ask these questions first and end up with a bloated test automation suite. Bas Dijkstra gives valuable advice for considering will actually set you up for success and give you more valuable test results.

To watch “Lighten Up: Methods for Making Your Automation Suite Leaner ” play the video below.