CrossBrowserTesting.com

A Design, Development, Testing Blog

How to Prove the ROI of Your Test Automation Efforts

By Leave a Comment

test automation roi

The reason for test automation usually boils down to the fact we all want to get more testing done, faster. But when it comes to proving the return on investment, or the ROI, of your automation efforts, things can become a bit blurry.

Whether you’re trying to convince management, your teammates, or whoever’s in charge of your budget, prescribing an exact value can be difficult. But this doesn’t mean you should give up on the dream of implementing automation across your organization.

The Trifecta

When we talk about software testing and development, we want to address three major elements: speed, quality, and cost. As we evolve our processes to encompass Agile, CI/CD, and DevOps in our workflows, this becomes especially true.

We want to create and test more features in less time, rather than releasing on a monthly, quarterly, or yearly basis. We want those integrations to be innovative and functional at the same time, rather than releasing updates that cause the software to break. We also want to do all of this in a way that’s cost-efficient.

Often we get stuck and are only able to focus on one or two out of these three priorities. But the true return on investment of adopting automation and doing it successfully is addressing all at once.

So how does automation help answer speed, cost, and quality all at the same time?

Speed

We know that automation is supposed to save us time, but how does it do that? Especially when it takes additional time to write the code or create the test, how can we be sure the investment of time is worth the time we save?

Where automation really come into play is with repetitive tasks and repeatable scripts. Say you have a test case that takes about an hour total. Realistically, this isn’t that much time — if you’re only doing it once.

But if this test is at the top of your risk analysis, it means you should be performing it during regressions. In addition, it means that you want to perform it in multiple browsers and devices.

If your team is integrating new code twice a week and you identify 5 configurations that are necessary to run as part of your regression suite, this means this test is taking 10 hours of your work week.

And that’s only one regression test. If you have 5 of these, that’s 50 hours just to check that everything is still working as intentioned; it doesn’t include testing any new features or areas that should be explored.

Of course, you could skip regression testing completely. But there’s always the possibility that when the new integration is pushed live, it will break something that will cost the entire team even more time to fix.

Automated testing takes care of time-consuming tasks so you can focus on testing new areas and building new tests, rather than manually executing the same tests over and over.

In this example, if you were able to automate those 5 tests, you would be saving the time it takes to repeat them in every browser and during every regression. While the initial time investment is there, the return on that investment is greater because you can reuse the scripts instead of manually repeating them.

Not only does this save time, gives you more time to perform new tests and create more automated tests. It also decreases the time it takes to get the test results back to the development team and thus helps to speed up the entire SDLC.

The best way to prove the ROI of automated testing for speed if it’s part of your current strategy is to add up the time it takes to do manual regression testing, then compare it to the time it takes to automate those regression suites.

Cost

It can be hard to determine the true cost-benefit of automation, but there are a few ways you can understand ROI when it comes to budget.

A good place to start by thinking about cost in relation to time — as they say, time is money. Quite simply, if you’re paying one or more software testers, you’re getting more for your money with an automation strategy in place to alleviate time-consuming work by replacing it with new tests of value, as we previously discussed.

However, another way to consider cost is to look at how much money you’re saving every time you find a bug before it reaches your customers.

The earlier on you find each bug in the software development life cycle (SDLC), the better according to IBM research, which found that cost to fix a bug increases as it moves through the SDLC.

In fact, the cost to fix an error found after release was about 15 times higher than if it’s uncovered during the design phase. It’s up to 100 times more expensive if it’s identified in the maintenance phase.

For example, if a bug costs $100 to fix in Gathering Requirements phase, it would be $1,500 in the testing phase and $10,000 once in Production.

The idea here is that it’s harder to fix an issue and more expensive as it approaches the end of the development cycle. Once it reaches the customer, it’s the most costly — not to mention intangible costs to your reputation and trust as a company when customers find bugs that you missed.

By getting more testing done sooner and verifying that old and new code is functioning, you avoid the cost that comes with finding a bug as you move down the SDLC.

Quality

Improved application quality comes with finding more bugs before new integrations reach customers as we just discussed, but it also accounts for the fact that time allows for more exploratory testing ideally being conducted during each development cycle.

The job of the tester isn’t simply to find bugs. An often overlooked element of the tester’s role is to provide feedback and insight when going through the application to make suggestions and affect changes for future changes and additions to the application. It’s impossible to do this if the tester is spending all their time checking whether the code is stable and functional.

By making more time exploratory testing and leaving automated scripts to check regressions, testers can focus on suggesting improvements, collaborating with programmers, and helping the software evolve, rather than spending the bulk of their time manually going through the same actions on different browsers every time they need to check core functionalities.

Additionally, automation takes out the factor of human error. While manual testing is essential to make observations about the application, tests that are merely for conducting the same steps over and over is left up to risk when done by a person. Automation is the answer to ensuring that your results are consistent and will provide a “Pass” or “Fail” when run.

Automation should never replace manual testing but support it in regression checking to allow for more time for manual tests that require human observation. Not only will this go towards confirming the quality of the current application, it will also aid in making changes to support improved quality in the future.

Additionally, by allowing more time for these manual tests and regression checks, automation helps address broader code coverage, meaning that the application that reaches the customer is more likely to work the way it should.

How to Prove ROI

You can see where speed, cost, and quality come together to help create a more productive development workflow and a better application.

But, if you’re still confused on how to prove ROI, here’s a few actionable items you can take:

  • Relate it Back to Your Business – It’s impossible to give one blanket statement for ROI that works for every company in terms of effectiveness. If you’re finding a challenge in convincing members of your team that automation is the right option, do your best to add up the manual hours that could be solved with automation. Identify which tests could make a good case for automation and where automation could help you could save time in your day-to-day.
  • Offer a Success Story – There’s a reason why everyone’s talking about test automation. Plenty of individuals, teams, and organizations have found success in adopting automation practices. Take America’s Test Kitchen. Their team found that they weren’t able to cover all the tests they wanted on all the configurations they needed to and keep up with the continuous iterations of the development team. By adopting a Selenium testing strategy and bringing on an automation engineer, they were able to find a solution to some of these pain points.
  • Leverage Parallel Testing – Parallel testing allows you to run the same test in multiple browsers at once, which means it’s extremely easy to prove the ROI. With every test you run in parallel, you’re increasing your test coverage that many times. For example, if you run two tests in parallel, you’re testing twice as much in the same amount of time. If you up it to 10 tests in parallel, you’re testing 10 times faster than running them on after another. Those are numbers that anyone can agree to.
  • Do Your Homework – Take the time to learn tools and automation practices on your own time. Understand what will make automation successful and what skills you can learn to make the strategy effective, so once you’re given the green light you can start to see the benefits. Learning any test automation tool does take onboarding and time to learn, but staying ahead of the curve means you’ll be more ready to jump in and start making tests. If you’re already automating to an extent, share the results and insights with your team. Record the amount of time it takes to create tests and how much time it saves, so you can have tangible proof of the ROI.
  • Look at the Browser Gap – A major benefit of automation is that it takes out the manual work of repeating a test in different browsers. If you’re not currently testing more than a few browsers, it’s worth it to investigate and see where you could be compromising quality. Check out which are the most popular browsers and devices your customers are using — you may have a gap that automation could help fill.
  • Do the Math – If you’re looking for hard numbers, this guide on measuring the ROI of automated testing may be a good place to start. By calculating costs that relate to automation of new tests, automation of prior tests, coverage across environments, reduction of defects, test redundancy and reusability, you can back up your argument for test automation with real data.

The ROI of Test Automation

Yes, automation does take some investments to get started. It’s easy to see why teams can be hesitant to hop on board with automated testing when looking at these factors.

The value comes with the ROI. Why it may not be immediate, you will see the ways that automation aids with time, costs, and quality over time.

As the team becomes more well-acquainted with the ways automation can help move the business forward, it will continue to be more of an asset. In fact, you might begin to question how you ever managed without the help of test automation.

Launch Countdown: What to Test Before Your Website Goes Live

By Leave a Comment

how to test a website before launch checklist

Planning for the launch of your website is an exciting time, but it can also be stressful if you start thinking about what could go wrong. The best way to make sure your first impression stays in people’s heads for the right reasons is to prepare.

First things first, you’ll want to access your website behind a firewall, on a staging server, or locally with CrossBrowserTesting’s secure tunnel technology to make sure what you’re testing is for your eyes only.

Once you have all that figured out, it might help to have a countdown calendar to strategize what to test and when. Take a look at our advice for how to test a website before launch, then fill in the gaps where you see fit.

One Month to Go…

You have some time to get into the nitty-gritty, but for now, you want to look at the basic functionality of your website because you’re not going to impress many users if key functions don’t work properly.

A few things you might want to check out include:

  • Forms – If your website includes any pages that have form fields, including check out, payment, contact, email sign up/subscriptions, etc. you want to validate that they are correctly accepting and processing information.
  • Buttons & Navigation – You want to make sure that when users are clicking around the site, it’s taking them to where they want to go. Click through or automate tests for buttons and navigation.
  • Links – There’s nothing more frustrating to a user than finding a broken link, and they’re extremely easy to miss during development. Google recommends broken link testing, but going through to click all of them manually is not the most productive use of your time. Try using a tool like Screaming Frog to do a crawl of your site — your SEO will thank you.

Keep in mind that you have a while to go until your website launch. If there are any integrations, bug fixes, or changes to code during this time, you might way to visit some of these test cases.

Two Weeks to Go…

Now that we know everything works, it’s time to pay attention to usability. Here, you want to look at content like text and copy, layout and design, and images and media to make sure that everything looks as intended.

This is a good time for cross-browser testing. Since you don’t know yet which browsers and devices your users will be on, it might be best to pick a few of the most popular browsers and manually go through to make sure your site looks good.

Pay special attention to responsive design. Test your site out on a few mobile devices — preferably at least one iPhone and one Android, but it’s also good to see how pages look on a tablet.

To get a good overview of what your site looks like on different screen sizes and devices, you may want to use a screenshot comparison engine to run a few of your most critical pages. But it might also help to manually go through on a few real mobile and desktop browsers.

One Week to Go…

What good is testing if you don’t prepare for the worst? You should be fairly confident in the basic use of your application, but you have to remember, you’re preparing for a launch day.

Hopefully, you’re implementing ample promotional strategies in preparation for the launch. If so, you should be expecting a high amount of traffic that will likely be significantly more than an average day.

Load testing will help you understand how much traffic you can handle at one time and will arm you with the information to make changes to avoid a site crash.

In addition, you’ll want to come up with a few ways to test your application beyond what’s expected. While you’re accustomed to using the functions and features of your site as intended, most of the people visiting will be first-time users, which means they don’t have a great idea of how everything works.

When testing, we usually want to perform the actions we think our users will follow to get to an end result, such as checking out or clicking a button. Negative testing looks at unexpected user behavior instead to explore the edge cases and determine what would break the application.

You don’t want your site crashing on launch day just because someone typed an invalid input into a form, so it might do you well to check out some test cases that will push the limits of your application.

A Few Days to Go…

With all the strategic test planning you’ve done, you should be in a good place to unveil your site to the public eye.

If you haven’t yet, outline a risk analysis and make sure that you’ve checked all high-risk areas. If you have, move on to the second tier of medium risk items that you might want to test.

Also, make sure to perform some last minute regression tests to validate that any code you changed or updated has remained stable. Running these tests in parallel across browsers is a good way to save time and make sure that your site functionality has held up.

Launch with Confidence

Sit back, relax, and watch your hard work pay off as your site traffic skyrockets. Once you make a checklist for how to test a website before launch, you’ll be much more prepared to make a good first impression on all your new site visitors. Rather than hearing about all the bugs on your site, enjoy the compliments you receive as your users note the impeccable visual and functional proficiency of your brand new application.

Move Fast, Break Things: How to Test the Limits of Your Web App

By Leave a Comment


When going through an application, it makes sense to perform some of the actions you would think your users would perform. But it may also makes sense to perform some of the actions you wouldn’t expect them to perform.

Your users are unpredictable, and the actions that cause complications with your application aren’t always what you would expect or even do yourself. Sometimes, you have to test the limits of your application in order to properly break it.

Clearing Misconceptions

Before we get started, we should make one thing clear: software testers don’t actually break things. Although the reference may be common and it’s a simple way to understand that a certain action resulted in a functional failure, it’s not an accurate way to think about testers in real life.

There’s a quote where James Bach says, “We don’t break the software. We break illusions about the software.”

The job of a software tester is to find where a product is broken. While something may work fine for the programmer who made it and is using it according to its intent, an application will have infinite areas for bugs can hide and users to find.

The make of a good tester comes down to being able to report these bugs, help stakeholders identify and fix issues, and contribute to the release of a product that’s of a higher quality than it was when it started out. Users don’t always go down the “happy path,” so it’s important that testers consider these different scenarios because at the end of the day, finding a problem is a good thing.

So although we might refer to it as “breaking software,” it’s important to understand that it’s not the testers who are breaking the software, it’s the application that’s broken.

Negative Testing

Often in testing, we want to follow a user journey by performing the actions that get us to an end result. For example, if you fill out the fields in a “Contact Us” box, that information will be sent to the site owner so they can follow up. If the application is working correctly, this will give us a passing result.

This passing result is a good thing because it means that in an ideal scenario, the user will fill in the information being asked as they are supposed to. But it doesn’t necessarily tell us much about the edge cases — what happens when a user doesn’t use the application as intended or planned. What happens if a user leaves a field blank or puts in an incorrect email address?

When we think about trying to see where an application fails, we want to look at negative testing. Negative testing ensures that your application can handle invalid inputs or unexpected user behavior. This means that you’re thinking about where a user can venture off path, rather than just confirming the application works with normal behavior.

By negative testing, you can get a more insight into where bugs are hidden. And by knowing what’s broken, you can build out your risk analysis to come up with a strategy for more comprehensive testing. How many of these edge cases do you feel are likely for users to follow? How detrimental would it be to the application if they were to follow them?

A good place to start with negative testing may be thinking about different personas. Everyone uses an application differently, and by paying attention to the details that give different end users unique experiences, you can curate a more intelligent approach to testing for them.

However, even when thinking about persons it can be difficult to truly put yourself in someone else’s shoes and come up with those edge cases. Brainstorming a list of scenariosmight be the best way to come up with new ideas for negative testing.

How to Test the Limits of Your Web App

Inspired by this post on Reddit that contains reliable strategies for breaking stuff (plus a few of our own favorite methods), we’ve rounded up a few good ways to test the limits of your application.

  • Start performing an action on the page. Leave it open for a long time, then try to come back and finish out the action.
  • Mess around with form fields. See what happens if you enter a bunch of random numbers, hold down a key continuously, or enter emojis and gifs.
  • Put a fake address where it asks for email.
  • Try any illegal input you can think of.
  • Enter the least likely inputs or large data strings.
  • Enter invalid data into promotional or payment fields.
  • Leave fields completely blank.
  • Cancel and remove items from your cart.
  • Add as many of one item as you can to your cart.
  • Reload the page.
  • Close out of the page. Reopen the page.
  • Shut down your computer. Reopen the page.
  • Automate data from the big list of naughty strings
  • Hit a submit button as many times as you can before the page changes.
  • Try and get through a test case as fast as possible.
  • Try and get through after navigating away from the page multiple times or opening up new tabs.
  • Test after a few beers.
  • Don’t follow directions; do the opposite.
  • Take it to mobile and see what happens when you have multiple apps open or are low on battery.
  • Turn it horizontal.
  • Have someone call or text you while using the application.
  • Simulate 10x the number of users that regularly go to your site.
  • Go backward. Then forwards. Then back again. Resume.
  • See what your website looks like in Internet Explorer
  • Try using your application with different plugins, integrations, and third-party apps.

Break Responsibly

While the goal of developing and testing is to release high quality, functional software, sometimes you have to find the places it’s broken first.

If you’re looking to try out negative testing, this is a good place to start. But the true value (and the fun) of negative testing comes from using your curious mindset and creativity to test the limits, especially during exploratory testing.

Try coming up with a few of your own ways to test the limits of your application, and leave your favorite method in the comments.

Making Your Site Pixel Perfect with Visual Regression Testing

By Leave a Comment

visual regression testing
When we think of regression testing, we tend to think about functionality and how new code affects the way previously working elements behave. Will a new integration cause something to break? Will everything still work the same after updating backend code?

But just because you’re looking at functionality doesn’t mean you should let visual design testing fall through the cracks during regressions.

In fact, visual regression testing is necessary to make sure that style issues don’t pop up and your web application continues to look just as good as it works.

What is Visual Regression Testing?

Imagine trying to use a web application on your phone and not being able to find certain elements because of an overlapping image or text. Visual testing isn’t just important for making sure that your application looks good but also ensuring that users are able to navigate it without running into style issues that functional testing might pass as technically correct.

But while we might know the need for visual testing, what’s visual regression testing?

While visual testing compares screenshots of a web page in its current state side-by-side with different browser versions of the same page, visual regression testing is used to look at historical differences.

As we know regression testing is used to check that new code has disrupted the functionality of a previous version, visual regression testing allows to do the same with a visual testing tool to make sure that a web page still renders as expected across different browsers after changes to the code.

Many organizations, such as America’s Test Kitchen, use visual regression testing to archive versions of a certain page to record how it changes from iteration to iteration.

With software teams that are shifting left and moving towards more continuous integration and continuous testing, visual regression testing is important to make sure that new changes don’t cause chaos to your layout. This is important to consider as the application evolves from version to version as well as across browsers.

Especially with mobile testing where smaller screen sizes can throw off responsive design and browser rendering, visual regression testing helps spot major differences before your users do.

How Can You Automate Visual Testing?

Tools like CrossBrowserTesting use a screenshot comparison engine to take automated screenshots of the same page across multiple browsers and devices. From here, you can choose a baseline browser, then compare highlighted layout differences side-by-side.

However, this process is not completely automated. Since pages are not passing or failing like in a Selenium test, it’s up to you to decide what differences are acceptable and which aren’t. For example, an image may be off by a few pixels in one browser but not the other. This is probably fine unless it’s throwing off other elements.

The same goes for visual regression testing. CrossBrowserTesting can automatically spin up the recent and previous versions of a web page, but you have to analyze where there are bugs or inconsistencies.

Tools and Tips for Automated Visual Regression Testing

There are a few ways to do visual regression testing that are worth considering.

  • PhantomCSS – PhantomCSS is basically a combination of PhantomJS or SlimerJS, CasperJS, and ResembleJS. This tutorial on CSS Tricks gives a good look at how to use PhantomCSS and Grunt for visual regression testing. However, it’s worth noting this requires installation and JavaScript knowledge.
  • Wraith – You can also use Wraith, which works by crawling two websites, taking screenshots, and comparing them. Like PhantomCSS, it also does take installation and scripting to use, but Wraith is more useful for testing sites with dynamic content in comparison. You can see this tutorial using Wraith for visual regression testing.
  • Selenium – Selenium isn’t traditionally known as a visual testing tool, but that doesn’t mean you there’s no way to take screenshots. Our Selenium 101 tutorial shows you how to take screenshots during automated tests. This way you can have, automated, functional, visual, and browser testing all-in-one.
  • CrossBrowserTesting and Applitools – For visual regression testing that requires no installation, setup, or coding knowledge, our integration with Applitools is your best bet. You can run visual comparisons of the same web page on the same browser or mobile device from a previous test session and get alerted of regression errors upon deployment or new releases.

Making Your Website Visually and Functionally Flawless

You don’t have to be shallow to care about how your application looks.

A working web application is important, but to ensure it’s also visually appealing takes its own type of testing.

Visual regression testing allows you to pinpoint every pixel at the UI level so that HTML, CSS, and JavaScript differences don’t affect your user experience.

Webinar: How to Increase Test Coverage Over Time with Automation

By Leave a Comment

abstracta webinar test coverage automation federico toledo crossbrowsertesting

When your application has endless possible test cases, do you know how much coverage is enough?

Test coverage gives us ideas of what to test and helps us know the extent of our tests. We often use automation a tool to increase test coverage by checking more parts of the application under test, faster.

But complete coverage is nearly impossible to achieve even with automation. And sometimes, more coverage isn’t always better — it’s a great measuring stick, but iti doesn’t guarentee that our application is bug-free.

The ideal amount of test coverage should be based on your priorities and analysis of risk. The good news is once we know which test cases to prioritize, we can apply long-term coverage criteria to optimize test coverage over time.

If you want to know more about how to get the most our of your test coverage, join us on August 7th at 2 PM ET.

CrossBrowserTesting joins Abstracta Co-founder and Director Federico Toledo in a webinar to discuss strategies for measuring test coverage and maximizing it for the long-term.

In this webinar, you will learn:

  • What test coverage is and how to measure it
  • Laying out a plan to optimize test coverage in the long-term
  • Demonstration with CrossBrowserTesting
  • Q&A

Save your seat

Exploratory Testing Techniques for Mobile Devices

By Leave a Comment

Exploratory testing mobile devices

For those of us who use our devices for everything, from buying coffee to connecting with friends, the mobile experience holds high priority despite smaller screens and limited capabilities.

Due to the sheer popularity of smartphones and tablets, the importance of mobile device testing doesn’t come as much of a surprise. As mobile testing becomes the industry standard, testers must increasingly incorporate test cases that consider the needs of users that are on the go.

However, while you may be familiar with which tests to perform with a keyboard and mouse, you have to come up with a new strategy for mobile devices, especially during exploratory testing.

Here are a few ways to test mobile devices when manual testing:

  • Touch Screen – Scrolling, clicking, and swiping are different when you’re using your fingers compared to a computer mouse. You want to perform a variety of these actions throughout your application to ensure that you can do everything on mobile that you’re able to do on a desktop and that it functions with the same accuracy. Are buttons big enough to click? Is it easy to scroll from top to bottom and back? How does it look when you zoom in or out? Does swiping work without too much lag? These considerations will be critical in assessing usability.
  • Keypad – Rather than they keyboard we’re all used to, mobile devices have much smaller on-screen versions that also use touchscreen technology, which can pose a difficulty to users. Because they’re generally less user-friendly, you want to look at instances that might use require the keypad such as search boxes. Pay special attention to form fields to make sure they’re not too narrow or hard to use. You don’t want your users becoming frustrated because of a keypad that makes using the app impossible.
  • Navigation – Many times certain navigations will be different for mobile compared to desktop because of the smaller screen space. For example, where a drop-down menu might normally be labeled “Menu” on the web application, it might turn into a hamburger menu on a phone screen. Ensure that these icons and elements are organized in a way that makes their function apparent to the user. Because some of these navigation items may be less obvious, you want to go through and make sure that you’re still able to accomplish the same journey.
  • Portrait and Landscape – When you’re testing on a desktop, the web page will always be horizontal. Mobile phones are most often in portrait mode, but they can also be turned to view in landscape. Visual testing will allow you to evaluate and compare responsive layouts to make sure images, text, media, and other content is digestible in both portrait and landscape modes on mobile.
  • Devices – It’s not enough to test on one mobile device — in fact, you should probably be testing a selection of devices. Because of mobile fragmentation, different models, brands, screen sizes, and resolutions mean different mobile experiences. Additionally, you want to test on both iOS and Android operating systems to get a better understanding of how it differs for each.
  • Performance – Performance may be in good shape on your desktop, but that doesn’t mean it’s consistent for mobile. When 85% of mobile users expect pages to load as fast or faster than on the desktop, testing for performance is not something you want to skip. Test out different pages and record their load time. You also might want to test out different networks, locations, and even battery charges to see how it affects speed.
  • Popups and Alerts – Checking pages that have popups and alerts is important because while it may look fine on a desktop, it could be intrusive on mobile. Alerts that are too large and unresponsive or popups that are layered over each other may even make the app unusable. Additionally, it’s a good idea to see how phone calls, notifications, and other interruptions affect the functionality application.

Once you know which devices you want to test on, you can build out a more comprehensive mobile testing strategy. Keep in mind the ways that devices differ, both from each other (tablet vs Android vs iPhone) and from desktop, to determine which test cases you should focus on during exploratory testing.

Your users aren’t ditching their devices any time soon. Exploratory testing on mobile devices means that you can better understand how users interact with your application.

 

Make Your Voice Heard in the 2018 State of Testing Survey

By Leave a Comment

SmartBear State of Testing Survey 2018

SmartBear’s State of Testing Survey has commenced for 2018, and we can’t wait to hear from our valued community.

The purpose of this survey is to get an benchmark of common practices, emerging trends, and future predictions in the software testing industry. And, it only takes about 15 minutes of your time to complete.

Over 3,400 respondents took part in last year’s survey. We’re hoping this year brings even more participants to give us deeper insights, but we can’t do it without your help.

As a thank you, we will be giving away $100 Amazon Gift Cards to five lucky survey respondents!

To qualify to win, you must complete all of the questions and provide your contact information at the end of the survey by 11:59 PM PST on Tuesday, July 31st.

The results will be analyzed and turned into our annual ‘State of Testing’ report. All respondents who complete the survey will receive a copy of the final results if they are interested.

Some of the highlights included observations on browser testing, mobile testing, parallel and cloud testing, as well as other takeaways surrounding Agile and automation. Additionally, we received some feedback on the top challenges testers face in their day-to-day.

To see what it’s all about, you can check out the full State of Testing 2017 Industry Report.

Make your voice heard — take SmartBear’s 2018 State of Testing Survey here.

Here’s What CrossBrowserTesting Customers Can Look Forward to at #SBConnect18

By Leave a Comment

smartbear connect 2018 crossbrowsertesting customers

Hey there CrossBrowserTesting customers!

We’re all getting ready for SmartBear Connect, and while October 29 might seem far away, it’ll be here before you know it.

That’s why we want you to get just as excited as we all are to join us for two days of learning, mingling, and growing with the rest of SmartBear community and staff.

SmartBear Connect will bring together the world’s testing, development, and technology communities for two days of training workshops, best practice sessions, and visionary content in addition to some really great speakers that will provide their own unique insights as customers and industry experts. If you went last year,

We’re going to have classes specifically for CrossBrowserTesting customers again this year! CrossBrowserTesting Software Engineers Chase Cook and Sree Vallabhaneni will be giving a CrossBrowserTesting 101 as well as a session about Integrating Selenium Tests with CI/CD Deployment and Reporting.

The former will be an introductory class for getting started with CrossBrowserTesting, while the latter will cover automated testing combined with the most popular tool for continuous integration, Jenkins, and will show how to do this both quickly with accurate reporting to help make your overall testing process more efficient.

So whether you’ve just started onboarding with CrossBrowserTesting or have been with us for years, there’s something for all of our customers at this year’s SmartBear Connect.

Get your ticket today to download the event app and start putting your schedule together. We can’t wait to meet you in October!

The Div That Looks Different on Every Browser

By Leave a Comment

the div that looks different in every browser

If you’ve been around the internet lately, you may have heard of the div that looks different in every browser.

But if you haven’t or unsure of what it’s all about, here’s the situation:

A guy named Martijn Cuppens coded a div in CSS which made a fairly simple green square. However, when you look at it across different browsers, it transforms completely into another shape.

In fact, almost every browser gives you something different than the last. The shapes range from a square with a hole in the middle, to two green squares, to a solid square, to different versions of the original shape. The browser even affects color properties, changing the hues and shades of green depending on Firefox, Chrome, IE, Edge, etc. You can see a few versions in this CrossBrowserTesting screenshot test:

This phenomenon has gone a bit viral on Twitter, Reddit, and HackerNews with designers and devs scratching their heads and examining what went wrong.

If you look in CodePen, the HTML is pretty basic, but the CSS is where things get more tricky. If you have any experience with CSS, you know this is where browser rendering can go haywire.

Most devs are aware of these idiosyncrasies when developing for cross-browser problems. We can see from Martin’s code that he uses outline-offset, which seems to be what causes the issue.

Since each browser renders the offset value differently, it causes the shape to morph according to those varying rules.

“I wanted to know if there is a way to inset outlines (like box-shadow). Turned out ‘inset’ was a outline-style and a negative outline-offset can be used to inset outlines. The combination of these properties led to this result,” said Martin in a Twitter thread.

So what does this teach us? For one thing, it’s probably a good idea to avoid outline-offset. But ultimately, it reinforces the importance of cross-browser testing your code.

Martin’s div gives a visual representation of why cross-browser testing matters, and what can happen when your code isn’t cross-compatible. We can plainly see that there are so many ways for different browsers to interpret a few lines of code.

His example provided a fun insight into how CSS can go wrong, but you wouldn’t want this happening to your web application with elements looking different in every browser.

CSS can be complex, but by testing your site in a selection of browsers, you can avoid the surprise that comes when you haven’t checked how your site looks in Safari.

Top Takeaways from Agile Testing Days USA

By Leave a Comment

Agile testing days usa

Agile Testing Days came to the US for the first time this year, and if we do say so ourselves, they chose one of the best city for the venue — Boston!

SmartBear was one of the event sponsors, so luckily we were able to sit in on some of the amazing speakers, and we learned a few things along the way about Agile testing. Here are just a few of the top takeaways from Agile Testing Days USA.

How Agile Are You?

If you’re at Agile TD, your team is probably working in an Agile fashion or looking to become more Agile. But Selena Delesie noticed something in her time working with different companies — many were saying they were Agile without actually working in an Agile manner.

Many times she found that people wanted to control everything and strictly plan out their sprints, which didn’t lead to the flexibility that Agile demands. Additionally, she found that many people were no longer engaged in their work and were not present with themselves and their coworkers.

“Stop doing Agile; stop treating it as a process,” said Selena. “Start being Agile. Doing and being are very different things”

Selena challenged the audience to reconsider what Agile means to them and think about whether their work processes were actually Agile.

Going back to the Agile Manifesto — specifically thinking about individuals and interactions, customer collaboration, working software, and responding to change — and even the definition of “agile” are good places to start. Agile is supposed to mean that you’re able to think on your feet, be active, and lively but many companies have lost that, and agile has become a heavier process than Waterfall.

Selena says that having more process and control does not help them do their best work, it makes them mediocre. She especially emphasized connecting with people and building relationships. Making the effort to have conversations with people, listen to them, and show compassion can make all the difference.

It’s especially important for management to instill these values and be transparent in order for an Agile organization to work effectively. Selena suggested to stop having meetings behind closed doors and instead, show trust in your team. Let go of feeling the need to control things and make more of an effort to discover and learn.

 

Look Beyond the Tool

Test automation is has proven essential for Agile teams, which means you need certain test automation tools. But when implementing automation into your workflow, it’s important to consider a few factors besides just which tools you’re using. In “Automation for the People”, Christin Wiedemann suggested that you additionally think about the “what, why, and who” of automation.

Before even starting with automation, you should first determine why you’re automating and how it’s benefitting the rest of your team.

Christin notes that automation is much more than running script using a tool — you should expand the scope of what you define as automation to understand what you want to automate. This is the best way to avoid waste, whether it’s waste of time, waste of effort, waste of money, or waste of trust in automation.

Oftentimes, automation is used to take over repetitive tasks to eliminate human error that comes with doing things over and over, but a lot of people start automating without specific intention.

Saving time isn’t always the answer because you might end up replacing the time it takes to execute a test with coding and maintenance. The most important thing is to have a reason for automation and set goals.

You also want to think about who will be automating. This isn’t limited to thinking about who will be writing the scripts and running them. Testing is a team effort, and everyone has different skills and ways to add value, which means those people should be contributing to automation efforts. “At the end of the day, we all want to contribute to the same objective,” says Christin.

Additionally, Christin emphasizes the importance of seeing automated and manual testing as supporting practices rather than contrasting them against each other. Meeting your objectives will require a mix of both manual and automated testing, and she says that it’s important to remember that the results are more telling than the method of achieving them.

By broadening our definition of automation to think beyond tools, we can get a better understand of test automation to further the success of testing in our teams.

Musicians and Testers Aren’t so Different

Have you ever thought about how much test automation has in common with music history? If not, Angie Jones’ talk “Owning Our Narrative” gives a lot to think about.

Angie took us through the evolution of music, starting with Thomas Edison’s phonograph in 1877 and explaining how that idea developed to lead way to new recording instruments like the player piano and jukebox, then more recently to the cassettes, CDs, and MP3s that we know today.

As these changes took place, people began to develop new preferences when it came to music. Rather than being the social aspect it once was, music was now being sorted in genres. Record labels were recording electronically to get better sound, and automated machines were able to play one song instead of making people buy a whole album.

With all this innovation, musicians were forced to make adjustments and change the way they made music to meet the requirements of the different ways it was recorded and played. However, while the role of the musician changed in many ways, the skills remained the same.

If we think about the timeline of test automation, it’s very similar. We’ve found new ways to write and record tests, which has required testers to learn new skills as we embrace agility and shift left, but at the core, the role of the tester also remains the same.

Angie advocates that testers need to find a way to stay true to this role no matter the changes that are going on around us. Musicians were upset about automated machines taking jobs while everyone else was excited, and it was only by adapting to the new environment of music they were able to be successful.

She encourages us not to make the same mistakes as musicians. Embracing industry changes and making it work to your advantage will be far more productive than resisting it.

Angie notes that sometimes it can be difficult to convince others the value of our role in the changing landscape, but the more conscious testers are to those changes, the more prepared they are to own their narrative.

BONUS: Strategize Test Automation

We love Angie Jones so much, we wanted to make sure we invited her to SmartBear while she was in Boston. Luckily Angie accepted our invitation and taught our attendees about “Which Tests Should I Automate” at our Ministry of Testing meetup.

In this interactive session, Angie gave us a look at how she would perform a risk analysis of different test cases at Twitter to decide which tests should be automated, which tests you might want to automate, and which you would skip.

Many times we make the decision of automation off a gut reaction without considering the different aspects of each test — which are the most user critical? Which break the most often? Which would take the longest to fix?

By evaluating these factors against each other, we can make better decisions when it comes to test automation.

See the video below to watch the lesson for yourself.

Join Over 200,000 Testers & Developers And Start Testing Today

Learn more Free Trial