Test Automation Archives | Page 2 of 7 | CrossBrowserTesting.com

Balancing Your Automation Strategy with Manual Testing

September 21, 2018 By Alex McPeak Leave a Comment

You’ve heard it a million times before — you can’t automate everything. But if your organization is just starting into the swing of automation, they’re probably thinking about all the ways they can automate. So how do you make sure you maintain the proper balance of manual testing and automation?

Whether you have an entire QA team whose roles are split between automated and manual testing, or there is one tester whose jobs is to do both, striking the right balance will be the difference between a testing strategy that provides value to your team and one that gives you more frustration than feedback.

By understanding how to best support your manual strategy with automation, you can ensure you’re getting the best of both world.

Know the Limitations of Automation

No matter how skilled you are in automation, how many Selenium commands you know, or what testing frameworks you use, some things simply cannot be automated.

For example, CAPTCHAs are often very popular with ecommerce sites. However, they’re inherently impossible to automate. This means that any test case that includes a CAPTCHA can’t be automated, and manual testing has to step in.

This, of course, isn’t the only case where you won’t be able to use automation. There are plenty of instances that it will be possible to automate, but it will be so complex to script, that you’re better off doing it manually.

Assess the limitations and challenges of test automation so you know where to step in with manual testing.

Solidify Your Regression Suite

The bottom line for automation is that you want to automate tests that are repetitive and time-consuming, so think about which tests you find yourself going through the most often. Solidifying your regression suite will help you determine which tests you want to cover after every integration or change to code.

Keep in mind that this will change over time as your application changes, but it’s helpful to have a suite that ensures basic functionality.

Additionally, deciding which environments you want to test on will be key to understanding which browsers, devices, and operating systems will be included in your regression suite.

Don’t Sweat the Small Stuff

Small test cases are best for automation and give you tests that are reusable and maintainable. The results of your automation are dependent on the state of your application, which means the smaller and more specific the test is, the more resistant it will be to changes in the UI. This is why unit and integration tests are ideal for automation.

But longer, more complex test cases and entire user journeys are often better left for manual testing. This gives you more room to explore the application and requires less time for scripting. Moreover, it means that if the UI of the application changes, you won’t have to go back into your script and figure out where it needs to be adjusted.

What’s the Risk?

When you’re thinking about whether or not to automate, risk should be a leading factor for consideration. However, while there are many methods for evaluating which tests are the most at risk, identifying these doesn’t mean you want to leave all other test cases that aren’t high risk out completely. In fact, these are perfect for manual testing.

When you considering risk, there are usually three tiers that you sort test cases into — high, medium, and low. As we discussed, most high-risk test cases will probably be included in your automation suite.

But, if you’re finding medium and low-risk cases that you still find valuable to test in order to provide feedback, then they might be something you want to run through manually.

Is It Providing Value?

Automation is a great way to speed up testing, but it also takes time for coding and maintenance, and that may not be worth it if you implement a test that isn’t adding value.

Say you run a test suite every day, and it passes 100 percent of the time. Is it worth it to keep automating if your results never change?

Just because a test was once automated doesn’t mean it should stay that way. Automation requires making changes, updates, and edits as the application code changes.

If a test is no longer providing value through automation, it might be better to take it back to manual testing to see if there are new parts to explore.

Communicate With Developers

Developers can’t tell you what to automate and what not to automate, but they can give you insight into the state of the application.

Keep open lines of communication with developers and other team members so that you are clear on what’s being changed in the application during development cycles. Depending on whether integrations are minor changes to the code, completely new features, or a revamped UI will make a difference in whether the test that follows will be part of your automation suite or should be tested manually.

To get additional insight, it may be helpful to pair with developers when going through the application to get a more in-depth look at what has changed and what areas or functions need to be tested.

Exploratory Testing

Exploratory testing is crucial to visually and functionally verifying new features and changes to the UI. Automation frees you up to test parts that you wouldn’t normally get to, so take it as a challenge to find a new bug or test for a different persona.

Human observation, curiosity, and creativity are paramount to being a tester. Automation can only take you so far when it comes to finding new bugs. That’s why manual testing, and especially exploratory testing, will always have a place in the QA process.

Visual Testing

Functional automated testing is not the only thing you have to worry about. Oftentimes, a functional test will pass because your application is working correctly, but it could still be visually inconsistent across browsers. Not only could this affect your layout and design, it could also affect the way your users are able to interact with your application.

In addition to functional testing with Selenium, you should incorporate automated screenshot testing into your workflow. This will help you account for visual cross-compatibility issues that you may have missed with functional testing by allowing you to compare screenshots of your web application side-by-side in different browsers.

Increasing Test Coverage

Automation will take care of repeated sequences, but in order to increase test coverage, you want to use manual testing to look at areas that have not yet been tested.

In this way, automation can be used as a tool to gradually help you increase test coverage over time. Test high-level functionality with automation, then fill in the gaps with manual testing. Look at edge cases and try negative testing to go beyond pass and fail results to gain deeper insight into your application.

This is also a good time to figure out what new tests you would like to add into your automation suite, so you can continue to increase coverage. Evaluate manual test cases to determine whether they would be valuable in regression, and assess which tests may be missing from the suite already.

Considering ROI

At the end of the day, it’s best to consider return on investment when deciding whether or not to automate. Automating takes time to analyze, code, and maintain.

If the time it takes to automate isn’t worth the results that you’d get from the test, it may be something you want to do manually, or not at all.

Webinar | Lighten Up: Methods for Making Your Automation Suite Leaner

September 19, 2018 By Alex McPeak Leave a Comment

Selenium has been the testers’ tool of choice for automated functional UI testing, but it’s too often misused. This results in inaccurate results, slow tests, and shallow feedback — and no one wants that.

If you’re struggling to write Selenium tests that meet your goals, it might be time to take a step back and get a better understand of the test cases that are best for Selenium as well as those that are better suited for testing at a lower level, or perhaps not automated at all.

Test Consultant Bas Dijkstra of On Test Automation joins CrossBrowserTesting to provide his expertise on how to de-bloat you UI tests and get your automation suite into shape. In this webinar, Bas will arm you with solutions to tighten up your end-to-end automation and streamline your Selenium strategy.

In this webinar you will learn:

What makes a good Selenium test, and what doesn’t
What is better tested at the API level, or not automated at all
How to future-proof your Selenium tests for better end-to-end testing
Live Q&A with Bas

More about Bas Dijkstra: Bas Dijkstra a Test Automation and Service Virtualization Consultant at On Test Automation who’s always looking for more intelligent ways to use tools to improve test processes and software quality. He’s been in the field for over 10 years, designing and developing test automation and service virtualization solutions that enhance and improve test teams and test processes. You can read more about him on LinkedIn or Twitter.

Challenges of Being the Team’s Sole Tester (Cont.)

September 11, 2018 By Alex McPeak Leave a Comment

PART TWO

Recap

This is part two of a look with my testing colleagues in Datacom at the challenges of being “that sole tester” on an agile team. What do you mean you missed it? Catch up now!

Last time we looked at some of the fundamental theory which underpins so much in IT, “The Iron Triangle”, as well as look at how cross-functional teams evolved an understanding that everyone had to support testing for it to be effective. Teams who embraced this had leveled up from being partly conflicted “forming” teams to “performing teams”.

Today we’ll look at how my peers found their voice and dealt with many of the challenges of being “that sole tester” …

Advocating for testing

A commonly recited tale within “forming” teams involved occasions where a story had been handed to them with just an hour to go until sprint end – and hence any testing performed would be rushed. This usually reflected a bias to see a story as “done” if development was finished.

It was typically these stories which, when passed into production would be the ones with missed issues.

As a tester it’s your responsibility to contribute to a “forming” team’s evolution to “performing” by helping the rest of the team understand your role and your needs in that role. The key to doing this is advocating for testing throughout the sprint and bringing your needs to the team.

During pre-sprint elaboration of stories when the team talks about future work requests and tries to size them:

If part of a story was going to be difficult to test, then ask developers to also add tools aid with the testability of the feature, and factor that into the size
If a story was going to be onerous to test, bring it up – it’s possible that the product owner might be comfortable accepting less testing, especially if it was an early part of a feature that would grow over a number of sprints. Looking at our “iron triangle”, this is essentially accepting there will be less scope to make it fit.
Alternatively, discuss with the team how a large testing task could be shared amongst the whole team. Looking at our iron triangle, this is accepting more resources/cost for testing in order to get the scope of testing you want.

An example of a shared team approach to aiding testing involved a number of sprints which involved page redesign – during the initial stories, the pages were only tested in IE, Chrome, an Android phone, and an iPhone. These items represented the most popular browsers/devices currently used by the system.

As all the pages neared completion, testing was performed across the finalised pages using a larger suite of tests. The project tester drew up a matrix of items to be tested, with members of the team helping with allocated browsers or phones to test in more depth according to the instructions given to them.

During stand up:

It always helps to have a rough expectation of how long you expected to need to finish testing a story. As the sprint end came closer, it helps to remind the team about how long you expect to need to test, especially if several pieces of work are due to complete close to the sprint end. The team may have help or solutions for you.
Some teams have embraced this in their planning, mixing up the size of story they prioritise to keep a good flow of work to best tested, rather than deliver several large stories at the end. Likewise, there is a level of maturity by accepting that a story won’t get finished this sprint and bringing it to the scrum master and product owner’s attention.

Communication

A key skillset that was talked about by sole testers was that of being effective and influential within their communications to the rest of the team. When talking about concerns or problems, it was important to put together a strong argument.

This often felt daunting to testers, especially in forming teams where there was better numerical representation of other disciplines.

The following elements were considered the core features of effective communication when making a case for a risk:

Explain why you consider it to be a problem. Frame it in terms relevant to the business.
Highlight some examples if need be. People are highly visual, so anything you can show will often make your case strongly.
Outline what you can / cannot do.

As a tester it’s important to recognise you need to make the strongest case you can, but often others – whether the wider group as a consensus or the product owner themselves — will make a decision based on that risk.

As time went on in a team, this pressure relaxed and there was a feeling of trust that built up within the team as it shifted into “performing”. But it strongly highlighted how effective communication and influence skills are becoming increasingly central to everything we do.

The power of showing

The talk about effective ways of communication and “highlighting some examples” diverted to talk about “the power of showing”.

Persuasion is always easier when you’re close and when you’ve worked together long enough to build trust in each other as a team. A charming story around this, is one of our testers just has to go gasp at something on her screen, and the rest of the team turns around and wants to know “what have you found”. Teams like this only form with time and trust from all parties, moving from that “forming” stage to building trust to “perform”.

For many, seeing is believing. It can be easy to get trapped into a conversation when you describe behaviour you’ve witnessed of “well, it shouldn’t do that”. A demonstration can simply and effectively show “well it does”.

One example came from a distributed team with some members in different countries. A remote team lead passed a link for the web page to be tested in an email.

The tester responded that it was the wrong link. The team lead replied back that it definitely was the correct one. This followed a few iterations.

Eventually the tester did a screen share with the team lead over Skype For Business, showing them clicking the link, and showing how the link was not valid and went nowhere. The team lead then responded immediately with the correct link the tester needed.

There is no substitute for colocation, but Skype For Business and other such screensharing tools are the next best thing. They are used extensively by our internal helpdesk to log and rectify issues with our machines, and so are logical tools for testers.

When this isn’t possible, recording your session with a commentary as a video can be really useful, and as a last resort, the tester favourite of taking screenshots for an email can be deployed.

However, sometimes email is not always the best way to interact. Along with the choice of channel, it was picked up how important it was to keep communication simple – you ideally want to take up as little time as possible. If you do it face-to-face or in real time in some form, you can be certain it won’t be ignored.

A multi-page email might contain all the detail that might be required, but you can’t be sure it’ll be read or that care will be taken with its salient points. This was highlighted to me recently with a joke I sent to a colleague, which they asked me about afterwards. My colleague didn’t get the joke because the actual punchline was in the second line of the email … but he didn’t read that far. When we send an email, even one with a read receipt, how can we really be sure that it’s been read? And much deeper, how can we be sure it’s been understood?

The joke in question

There also came out from conversations some of the basics that testers have been talking about for decades and which apply to communications – avoid being pedantic and recognise that anything you find issue with, represents someone’s hard work and effort. As such, there is always potential to upset and hurt people.

It’s been noticed that without colocation, it’s much easier for egos to be hurt in such interactions. A collocated team can build up a depth of social interaction to allow for a closer, deeper teambuilding. Typically, people in collocated teams feel they move from “forming” to “performing” much quicker.

Absence

As the sole tester, you’re a key resource in a sprint. When you’re not there, there is no tester.

This initially created some feeling of guilt when a member of staff had to take a sick day or even worse, planned leave!

Ultimately, responsibility with coping with your absence does not solely rest on your shoulders. There is some expectation on the managers and team to cope with this.

Within the team realm, members should have enough cross-skills, that any task on the board can be picked up by more than one team member. Of particular note from testers, was the close relationship between testers and business analysts on a team, and how a good business analyst should be able to pick up a testing task and vice versa.

Solid teams learned to focus on making sure stories were done, and that meant people going “this task needs finishing … I could do it” rather than sticking solely to their own discipline tasks.

The mind maps we talked about earlier were invaluable for the team – if they had been created, they formed a road map of planned testing which the rest of the team could cover, and which would allow them to continue in the case of short, unexpected absences.

For longer absences, being able to get another team member to cover for you, and coaching them through the basics of what you do, also turned to be invaluable.
A team needs an understanding of tools/product. It’s often been much easier to get a member of the team to cover vs bringing in a new tester onto the team, unless the absence is significant.

Conclusion

Our workshops were run using a modification of the Lean Coffee model. We all collected about five post-it notes of ideas, and put them on the board, clustering common items as topics. We then could use five dots to vote up the topics which were important to us.

At the end of our final workshop, there was a lot of satisfaction on the stories and learning we’d shared. We didn’t talk about every topic, but we’d talked about the most important topics applying the prioritisation that we spoke of being so important.

In you reading through our combined wisdom there is perhaps a slight sense of “déjà vu”. If you read enough about agile testing, you’ll come across items like “it’s not done until it’s tested”, “the WHOLE team is responsible for quality”, “everybody helps testing”.

These terms can be very validating to us as individuals. However, what we sometimes don’t appreciate in reading this material is that as testers, it often falls to us to educate and build trust with the larger team who to get to this “performing team” Utopia.

Achieving this is all about sharing our role and our struggles. Good communication and influence skills are pivotal because we need to get others on side, and yes, we may have to make a few compromises and understandings of others’ roles on the way.

In looking at the future of what skills the tester of five years in the future will need, we often find ourselves focusing on automation and technical skills. And these skills do have their place. But as our workshop highlighted, soft-skills are a core part of being a tester moving forward.

The sole tester is required to be the ambassador for their craft and their viewpoint. To be effective, they need an ability to influence, persuade, and teach. From a training perspective, the challenge going forward in my company is how do we as a discipline practice and build these skills.

That will be a focus of a future session, but feel free to share your experiences in the comments.

Selenium Interview Questions You Should be Prepared to Answer

September 5, 2018 By Alex McPeak Leave a Comment

Software development teams are constantly shifting, which means testers’ roles have to evolve along with them. As we’ve seen more teams go towards Agile, DevOps, and CI/CD, testers are encouraged to concurrently take on skills that better support these workflows and help with faster testing.

One of those skills is Selenium. As more software testers take on test automation and begin to shift left, Selenium testing is one of the most popular strategies, and a comprehensive knowledge of the tool is foundational to creating tests that provide the team with productive feedback and insight. While Selenium is a tool that can benefit from continuous learning, there are certain things that interviewers will want to make sure you know when you’re applying for you’re next role in automation.

To better prepare for that upcoming interview, here are the top Selenium interview questions you can expect to be asked.

Explain the different Selenium components.

It’s important to be able to prove you know the full breadth of Selenium beyond writing a script. Selenium has an entire suite of tools that are used for a different capability. Review the history of Selenium to fully understand the differences between IDE, RC, WebDriver, and Grid.

Do you know the difference between Selenium 1 and 2? How about the programming languages and browsers it supports? Knowing the tool and rounding out your understanding of how it works will be something that employers value.

What’s your experience with Selenium?

Be able to talk about the different test cases that you use Selenium for. What worked well in your last role, and how do you identify a good case for automation? Is there a time that you automated something you shouldn’t have, and what did you learn from it? What automated tests have you cut out of your suite because they weren’t valuable? How many tests did you automate? Do you have experience using a tool to run Selenium in the cloud?

Giving an in-depth look into the breadth of your experience will help your interviewer understand your qualifications.

What are the advantages of Selenium? What are the most common Selenium challenges?

There are plenty of reasons that people gravitate towards Selenium. Have an answer for why people would choose it over other test automation tools and what it can help accomplish.

However, Selenium also has its challenges. Flakiness, dynamic content, reporting, mobile testing, and captcha are a few things that may come to mind when you think of these limitations. Know which ones have solutions and how to implement them, and know which mean testing is better of done manually.

What should Selenium be used for? What shouldn’t it be used for?

Going off of the last question, you should have a comprehensive knowledge of which tests should be automated and which are better off done manually.

This is not necessarily subject to Selenium as you should be able to give an overview of the right times to automate, how to decide by creating a risk analysis, and how to prioritize your automation.

When do you use Waits and Sleeps?

Knowing the proper use of explicit wait, implicit waits, and sleeps is key to reducing flakiness in your tests, which is undoubtedly something your future employer will care about — no one wants false positives and negatives.

Be able to talk about the difference between implicit wait and explicit wait and sleep, when to use each one, and how.

How do you handle common commands?

To verify your basic knowledge of Selenium commands, it’s expected an interviewer might ask you how to:

Launch a browser
Click
Scroll
Drag and drop
Send keys
Find an element
Close and quit
Handle pop-ups
Navigate back and forth
Switching tabs
Minimize and maximize windows

Check out our Selenium toolkit for tutorials on these commands and more.

What is Page Object Model and Page Factory?

To get more readable maintainable Selenium tests, you need to have a structure that likely involves design patterns including Page Object Model and Page Factory Object.

You should be able to speak to what they are, the differences between them, why they’re important, and how you use them.

What’s your process for reporting?

Because Selenium has limited reporting capabilities, testers’ method of reporting will differ, and your future employer will likely be interested in getting a better understanding of how you report your Selenium tests.

How would you automate this?

In order to prove your technical capabilities, you may be asked to write a test case. This not only helps the interviewer understand whether you’re able to accurately use Selenium, it also helps them understand your process.

Straight From the Manager’s Mouth

These previous questions are common in the test automation space, but to give you a more well-rounded look at what you could be asked, we also asked a few others might ask for Selenium interview questions:

“There are a few questions I ask our candidates. In most cases, they are a bit general, however it will give us an overall picture of what to expect from your future employee. I value the logical and intuitive thinking of a candidate, as I understand that in tandem with their knowledge, this will make them perfect team players and valuable members of our company.

These questions help me assume if a candidate follows the best practices in work with Selenium. It also helps me to understand how much expertise the candidate has and if he or she is ready for further learning and improving as a QA professional.

In your opinion, why we should we choose this tool, and what are its advantages among others similar tools? What is the fundamental benefit of Selenium?
What are the limitations of Selenium? For what purpose it is not the best choice?
What type of tests have you automated?
What are some differences between CSS selector and XPath based locators? Explain how one is better than another and why. What are the testing process particularities while working with CSS selectors and Selenium?
What is a hybrid framework in Selenium?
What is Page Object Model and Page Factory in Selenium? Provide an example in which case we can use them.”

–QArea Software Development Company

“1. How many tools are there in the Selenium tool suite? Tell me more about each tool.
2. We use WebDriver’s APIs to write our tests. What do you think about calling WebDriver’s APIs directly inside the test case?

These first two questions are easy questions to start the interview. There is no need to continue if they fail these.

3. What are the solutions to eliminate bad test design problems when using WebDriver? Compare and contrast your solutions.

I expect the interviewee to show deep knowledge on industry best practices on test designs such as Page Object or BDD.

4. What is your preferred control locating technique? And Why? Compare.

Interviewee should have good knowledge about capturing UI controls, and ability to determine what is the best locator for different situation, knowledge about different locator techniques.

5. Do you have any experience integrating your Selenium tests with any CI/CD tools? If Yes, Explain your experience.

This question helps to identify the superstar candidates. One element is that in order to successfully answer this, they will have to have had to have some real experience working with CI/CD tools to explain the answer in detail.”

-Thanh Viet Le, Automation Project Lead & Framework Architect, LogiGear

“1. What are some common exceptions you have encountered in Selenium and what are the steps you took to handle those exceptions?

This question is asked to understand the candidate’s knowledge and experience of Selenium exceptions and handling. This enables maintainable automation and we want to ensure these basics are understood by the software engineers.

2. What would the optimal Selenium automation framework look like?

This question is asked to understand the candidate’s in-depth knowledge of automation testing and the automation process, as well as gauge the candidate’s technical knowledge of how Selenium frameworks are architected.

3. Can you provide a few examples of major challenges you have experienced when automating test cases with Selenium and how you went about resolving them?

This question is asked to get an understanding of the types of complex test cases the candidate has automated with Selenium and the strategies the candidate used to resolve the testing challenges.

4. What are some major innovations or enhancements you have created in your Selenium automation framework?

This question is asked to identify the candidates out of the box thinking and assess the candidate’s self-driven qualities. It allows us to gain insight into how a Selenium automation engineer thinks in regards to ensuring the frameworks are maintainable, robust, and portable.

5. Can you describe some changes that have been made in Selenium 3.0 from Selenium 2 and the reason for those changes?

This question is asked to learn about candidate’s complete knowledge of the Selenium testing framework and the candidate’s enthusiasm/curiosity about product updates/new versions. It also gives us a clear idea if they stay up-to-date on enhancements and best practices.”

-Vivek Kumar, Head of Automation, QASource Automation Testing Services

What are we missing? Leave the Selenium interview questions you’ve asked or answered in the comments.

What Robotic Restaurants Can Teach Us About Test Automation

August 28, 2018 By Alex McPeak Leave a Comment

To most people, going out to eat doesn’t evoke the image of robots. But to the four founders of this restaurant, robot chefs are the only viable solution.

In this automated kitchen, robots begin cooking your meal in temperature controlled pans as soon as you submit your order. The bots a programmed to spin the ingredients and spices around until everything’s perfectly cooked. Then, it dumps your meal into a compostable bowl.

While the restaurant may seem overrun by restaurants when you look around, there are more humans involved than you think.

Once in the bowl, a person takes it and adds yogurts, sauces, and dry toppings, then hands you your finished product. Behind the scenes, the Culinary Director and Executive Chef are the masterminds behind the delicious recipes, making sure the finished meals have the right balance of flavor. Additionally, there are people prepping the ingredients. Of course, there are also the four engineers behind the robots making sure that everything runs smoothly.

So what does this have to do with software testing?

In the shift from manual to automated testing, we often get ahead of ourselves in trying to automate as much as possible, or even attempting to automate everything.

But just like a fast-casual restaurant cannot operate on robotics alone, neither can we rely on automation to fulfill 100 percent of our testing. There are certain qualities that testers possess that we can’t automate.

So as we continue to explore the ways that automation can aid in our testing efforts, here are a few takeaways we can gain from robotic restaurants:

A Good Recipe is Key – Just as we need a chef to create the recipes for the robotic sous chef to follow, there needs to be the first test for automation to follow. Whether this is done through record & replay or by writing out a script, the quality of the first test is crucial to designing a suite that will provide insightful results. Just like a robot can’t follow a recipe it isn’t yet programmed to make and can’t create one on its own, a test automation tool can only test what it is told. Additionally, every bowl is only as good as the recipe, just as every regression test is only as good as the first test or script.
Robots Don’t Have an Eye for Detail – Just like humans are the final step in garnishing your meal at a robotic restaurant, human testers have an eye for detail that is critical in testing. There are many soft skills that software testers have — empathy, creativity, curiosity, and knowledge of the software, for example — that are impossible to teach to an automation system. This is why it’s important to have real people that can perform exploratory tests on new features, layouts, or other noticeable application updates, so they can bring unique perception and understanding to provide valuable insight. Not to mention the customer service aspect — communication is important that there’s someone on the team advocating for testing as well as the end users and who’s able to talk to other stakeholders.
There is Risk Associated with Automation – Automation usually accounts for “normal” situations but doesn’t consider edge cases or stress. What happens if a customer has an issue with their bowl? What happens if a concert gets out, and the influx of customers is 10 times what it normally is? What happens if you run out of utensils? These are the instances where automation can no longer solve the problem, and a human needs to step in. Automation can’t go outside of what it’s told, which means there will always be risk involved when expected situations arise.
Automation is Best for Repeated Actions – A McKinsey report said the food industry is most susceptible to automation because there are so many predictable physical activities involved. Similarly, during code changes and new integrations, it’s important to repeat some of the same tests run previously to make sure they work. This is where automation becomes a good candidate for testing — regressions that have programmable repetition that helps to reduce human error. Rather than have someone perform the same actions over and over, having a machine take over the tester has more time to add new tests is what will result faster productivity in less time.
Cut Costs Without Cutting Jobs – Because of reduced labor costs, the robotic kitchen’s bowls start at $7.50 — around $3 less than comparable options. If you’re getting to the point where your solo tester doesn’t have time to run all the regression tests they need across browsers in addition to new exploratory tests, introducing an automation solution can alleviate some of those time-consuming tasks and give you a higher ROI. However, if you implement automation, there needs to be a human behind that as well. It’s also important to have someone who understands automation and the tools in order to properly execute and maintain tests.

The Biggest Takeaway

You can’t automate everything. Though you can add significant automation to a project, there will always be a place for human testers. Additionally, automation should be used as a tool to support manual work rather than replace it.

Interested in hearing more about how test automation is like a robot restaurant. Listen to Episode 6, Manual vs Automated…Restaurants? of SmartBear’s podcast series “The Good, the Bad, and the Buggy” with an interview with Paul Grizzaffi.

How to Choose a Test Automation Tool

August 21, 2018 By Alex McPeak Leave a Comment

Once you or your team decide you’re ready to get started with automated testing, then comes time to choose your tool.

You’ll quickly find that there are a lot of different options to make your decision difficult, but there are going to be many differences between them as well. Looking a little bit closer will help you find the right fit, but first, you have to ask a few questions during the evaluation.

How long will it take to get started? – Depending on the tool, you may have to do some downloading, installation, and setup. You may have the time and patience for this, but if you want a tool that’ll take minimal effort to set up, then it’s something to take into consideration when looking. You also want to think about what the onboarding process is like. Some platforms will be fairly straightforward, while others take a while to get the hang of. This is something you can usually evaluate during the free trial. Determine how quickly you can get into the tool and start building out tests and if the interface is simple and clear, or if it will take some time to get familiar with the tool. Many tools will be intuitive and easy to dive into, but not all are. If it’s something that will take weeks of setup and training during onboarding, it may be worthwhile to consider another option.

How is customer support? – The trial is also a good time to evaluate some of the intangibles, such as customer support. Try contacting the support team to see how responsive they are if you run into issues during trial, whether that be setting up and getting started or running an automated test. Consider how helpful they when getting to know the tool, since you will likely rely on them as a regular resource. Check out the documentation and resources they provide in addition — tutorials, webinars, and other learning materials may be your best friend as you get started with automation. Open source options will be lacking in providing actual human support engineers, but it also helps to have a robust user community, or rely on the support team from a third-party integration tool.

Who will be using it? – How many people will be using the tool? Will it mainly be used by automation engineers and developers who are familiar with programming, or will you need something a bit more flexible for different technical levels? Some automation tools will require programming knowledge and other technical expertise, while others may be better suited for beginners or those with little automation experience. Has your team done automation before? If you’re moving from manual to automated, it might be worth it to look at which tools will help with the on-ramp. Look at your options with a Record & Replay tool versus a scripting tool, and decide which option makes more sense. Keep in mind that some tools offer both, which may be helpful if you have a wide range of technical skill sets on your team.

What systems does your organization already have in place? – Your choice may be confined to a few pre-determined details in your organization and the way it already runs, so it’s important to look at the integrations. If your team works in specific programming languages, you want something that can support those and has documentation for them. You also want to think about unit and end to end frameworks, CI/CD platforms, open source tools, and even your methods of communication like Slack and Jira. If your new tool doesn’t work with your former tools, you’re going to have to find different workarounds. Instead, make life easier on yourself by scoping it out first.

What will you be testing? – Are you testing desktop, web, mobile applications, or all of the above? Some tools are better suited for testing different environments or for certain kinds of applications, but others will provide an all-in-one solution. How many browsers and devices will you want to test on, and which ones do you need? It may be a good idea to look at the most popular browsers and devices your customers using, so you can evaluate whether the tool you’re looking at that offers them for automation. Additionally, consider how many test cases you have that you want to automate and how expansive your application and test coverage is. Thinking about the need for the tool and knowing what you’re specifically looking to test is paramount to finding a solution that will help you achieve your business goals.

How is it to maintain tests? Like your application, your tests will change over time. In addition to building tests, automation also demands that you spend time on maintenance for reusability and editing. Does the tool keep a comprehensive history or archive of your test cases and suites? Is it simple reuse or rerun previous test cases for regression testing? How easy is it to go in and edit a test that may need to be changed? With automation, you have to think beyond the first execution of the test to determine longevity, which means you want the tool to support this as well.

What does reporting look like? – Perhaps the only thing as important as the execution of your tests is the reporting of your tests, which means that paying special attention to reporting features should be one of your considerations when choosing a test automation tool. How is reporting handled and displayed in the tool you’re looking at? Is it easy to share results with different team members, both within and outside the tool? Screenshots, notes, and tracking features can make all the difference when it comes to making results are properly communicated.

What are the requirements and nice-to-haves? – You should be going into your tool research with a checklist of requirements. Look at some of the previous touchpoints to decide what you need in a tool, and make sure that your choice checks all the boxes. However, some tools might overlap and you might find you have more than one option. In that case, it’s also a good idea to consider what value-add features would be nice to have. Keep in mind the complexity of the application, though — just because it’s large and popular doesn’t mean it’s the right choice. Make sure that features that are included are ones you use and that you’re not spending extra money or time on things your team doesn’t need.

Finding the Right Test Automation Tool

There is no one answer when it comes to finding the best test automation tool, and everyone will have a different opinion. At the end of the day, your choice should be what’s right for your team. Spending the time coming up with the most important criteria will ensure the tool you choose now and into the future as you continue to expand automation across your organization.

How to Prove the ROI of Your Test Automation Efforts

August 16, 2018 By Alex McPeak Leave a Comment

The reason for test automation usually boils down to the fact we all want to get more testing done, faster. But when it comes to proving the return on investment, or the ROI, of your automation efforts, things can become a bit blurry.

Whether you’re trying to convince management, your teammates, or whoever’s in charge of your budget, prescribing an exact value can be difficult. But this doesn’t mean you should give up on the dream of implementing automation across your organization.

The Trifecta

When we talk about software testing and development, we want to address three major elements: speed, quality, and cost. As we evolve our processes to encompass Agile, CI/CD, and DevOps in our workflows, this becomes especially true.

We want to create and test more features in less time, rather than releasing on a monthly, quarterly, or yearly basis. We want those integrations to be innovative and functional at the same time, rather than releasing updates that cause the software to break. We also want to do all of this in a way that’s cost-efficient.

Often we get stuck and are only able to focus on one or two out of these three priorities. But the true return on investment of adopting automation and doing it successfully is addressing all at once.

So how does automation help answer speed, cost, and quality all at the same time?

Speed

We know that automation is supposed to save us time, but how does it do that? Especially when it takes additional time to write the code or create the test, how can we be sure the investment of time is worth the time we save?

Where automation really come into play is with repetitive tasks and repeatable scripts. Say you have a test case that takes about an hour total. Realistically, this isn’t that much time — if you’re only doing it once.

But if this test is at the top of your risk analysis, it means you should be performing it during regressions. In addition, it means that you want to perform it in multiple browsers and devices.

If your team is integrating new code twice a week and you identify 5 configurations that are necessary to run as part of your regression suite, this means this test is taking 10 hours of your work week.

And that’s only one regression test. If you have 5 of these, that’s 50 hours just to check that everything is still working as intentioned; it doesn’t include testing any new features or areas that should be explored.

Of course, you could skip regression testing completely. But there’s always the possibility that when the new integration is pushed live, it will break something that will cost the entire team even more time to fix.

Automated testing takes care of time-consuming tasks so you can focus on testing new areas and building new tests, rather than manually executing the same tests over and over.

In this example, if you were able to automate those 5 tests, you would be saving the time it takes to repeat them in every browser and during every regression. While the initial time investment is there, the return on that investment is greater because you can reuse the scripts instead of manually repeating them.

Not only does this save time, gives you more time to perform new tests and create more automated tests. It also decreases the time it takes to get the test results back to the development team and thus helps to speed up the entire SDLC.

The best way to prove the ROI of automated testing for speed if it’s part of your current strategy is to add up the time it takes to do manual regression testing, then compare it to the time it takes to automate those regression suites.

Cost

It can be hard to determine the true cost-benefit of automation, but there are a few ways you can understand ROI when it comes to budget.

A good place to start by thinking about cost in relation to time — as they say, time is money. Quite simply, if you’re paying one or more software testers, you’re getting more for your money with an automation strategy in place to alleviate time-consuming work by replacing it with new tests of value, as we previously discussed.

However, another way to consider cost is to look at how much money you’re saving every time you find a bug before it reaches your customers.

The earlier on you find each bug in the software development life cycle (SDLC), the better according to IBM research, which found that cost to fix a bug increases as it moves through the SDLC.

In fact, the cost to fix an error found after release was about 15 times higher than if it’s uncovered during the design phase. It’s up to 100 times more expensive if it’s identified in the maintenance phase.

For example, if a bug costs $100 to fix in Gathering Requirements phase, it would be $1,500 in the testing phase and $10,000 once in Production.

The idea here is that it’s harder to fix an issue and more expensive as it approaches the end of the development cycle. Once it reaches the customer, it’s the most costly — not to mention intangible costs to your reputation and trust as a company when customers find bugs that you missed.

By getting more testing done sooner and verifying that old and new code is functioning, you avoid the cost that comes with finding a bug as you move down the SDLC.

Quality

Improved application quality comes with finding more bugs before new integrations reach customers as we just discussed, but it also accounts for the fact that time allows for more exploratory testing ideally being conducted during each development cycle.

The job of the tester isn’t simply to find bugs. An often overlooked element of the tester’s role is to provide feedback and insight when going through the application to make suggestions and affect changes for future changes and additions to the application. It’s impossible to do this if the tester is spending all their time checking whether the code is stable and functional.

By making more time exploratory testing and leaving automated scripts to check regressions, testers can focus on suggesting improvements, collaborating with programmers, and helping the software evolve, rather than spending the bulk of their time manually going through the same actions on different browsers every time they need to check core functionalities.

Additionally, automation takes out the factor of human error. While manual testing is essential to make observations about the application, tests that are merely for conducting the same steps over and over is left up to risk when done by a person. Automation is the answer to ensuring that your results are consistent and will provide a “Pass” or “Fail” when run.

Automation should never replace manual testing but support it in regression checking to allow for more time for manual tests that require human observation. Not only will this go towards confirming the quality of the current application, it will also aid in making changes to support improved quality in the future.

Additionally, by allowing more time for these manual tests and regression checks, automation helps address broader code coverage, meaning that the application that reaches the customer is more likely to work the way it should.

How to Prove ROI

You can see where speed, cost, and quality come together to help create a more productive development workflow and a better application.

But, if you’re still confused on how to prove ROI, here’s a few actionable items you can take:

Relate it Back to Your Business – It’s impossible to give one blanket statement for ROI that works for every company in terms of effectiveness. If you’re finding a challenge in convincing members of your team that automation is the right option, do your best to add up the manual hours that could be solved with automation. Identify which tests could make a good case for automation and where automation could help you could save time in your day-to-day.
Offer a Success Story – There’s a reason why everyone’s talking about test automation. Plenty of individuals, teams, and organizations have found success in adopting automation practices. Take America’s Test Kitchen. Their team found that they weren’t able to cover all the tests they wanted on all the configurations they needed to and keep up with the continuous iterations of the development team. By adopting a Selenium testing strategy and bringing on an automation engineer, they were able to find a solution to some of these pain points.
Leverage Parallel Testing – Parallel testing allows you to run the same test in multiple browsers at once, which means it’s extremely easy to prove the ROI. With every test you run in parallel, you’re increasing your test coverage that many times. For example, if you run two tests in parallel, you’re testing twice as much in the same amount of time. If you up it to 10 tests in parallel, you’re testing 10 times faster than running them on after another. Those are numbers that anyone can agree to.
Do Your Homework – Take the time to learn tools and automation practices on your own time. Understand what will make automation successful and what skills you can learn to make the strategy effective, so once you’re given the green light you can start to see the benefits. Learning any test automation tool does take onboarding and time to learn, but staying ahead of the curve means you’ll be more ready to jump in and start making tests. If you’re already automating to an extent, share the results and insights with your team. Record the amount of time it takes to create tests and how much time it saves, so you can have tangible proof of the ROI.
Look at the Browser Gap – A major benefit of automation is that it takes out the manual work of repeating a test in different browsers. If you’re not currently testing more than a few browsers, it’s worth it to investigate and see where you could be compromising quality. Check out which are the most popular browsers and devices your customers are using — you may have a gap that automation could help fill.
Do the Math – If you’re looking for hard numbers, this guide on measuring the ROI of automated testing may be a good place to start. By calculating costs that relate to automation of new tests, automation of prior tests, coverage across environments, reduction of defects, test redundancy and reusability, you can back up your argument for test automation with real data.

The ROI of Test Automation

Yes, automation does take some investments to get started. It’s easy to see why teams can be hesitant to hop on board with automated testing when looking at these factors.

The value comes with the ROI. Why it may not be immediate, you will see the ways that automation aids with time, costs, and quality over time.

As the team becomes more well-acquainted with the ways automation can help move the business forward, it will continue to be more of an asset. In fact, you might begin to question how you ever managed without the help of test automation.

Webinar: How to Increase Test Coverage Over Time with Automation

July 30, 2018 By Alex McPeak Leave a Comment

When your application has endless possible test cases, do you know how much coverage is enough?

Test coverage gives us ideas of what to test and helps us know the extent of our tests. We often use automation a tool to increase test coverage by checking more parts of the application under test, faster.

But complete coverage is nearly impossible to achieve even with automation. And sometimes, more coverage isn’t always better — it’s a great measuring stick, but iti doesn’t guarentee that our application is bug-free.

The ideal amount of test coverage should be based on your priorities and analysis of risk. The good news is once we know which test cases to prioritize, we can apply long-term coverage criteria to optimize test coverage over time.

If you want to know more about how to get the most our of your test coverage, join us on August 7th at 2 PM ET.

CrossBrowserTesting joins Abstracta Co-founder and Director Federico Toledo in a webinar to discuss strategies for measuring test coverage and maximizing it for the long-term.

In this webinar, you will learn:

What test coverage is and how to measure it
Laying out a plan to optimize test coverage in the long-term
Demonstration with CrossBrowserTesting
Q&A

Save your seat

The Top 8 Things We Learned From the QA Team at America’s Test Kitchen

June 28, 2018 By Alex McPeak Leave a Comment

With three core websites, America’s Test Kitchen has a lot of content to cover. Yet when Megan Collins came on as QA manager, she was the only tester. Since then, America’s Test Kitchen has exponentially scaled their testing, adopted an automation strategy, and expand the QA team to three members, including Automation Engineer Michael Ng.

We recently had Megan and Michael join us for a webinar to talk a little bit about the web testing strategy behind America’s Test Kitchen and how they handled their shift to automation with the help of CrossBrowserTesting.

Here are the top 8 things we learned in the webinar:

Fast deployment schedule demand testing — Most organizations dream of achieving the agility that America’s Test Kitchen has reached, deploying code to production every two days. But when you’re integrating as often as they are, you can’t just rely on developers to test each others code or you’ll never achieve consistent test coverage. By bringing in Megan Collins as QA Manager, she was able to spend her time on something that had been severely lacking — testing. This way, proper exploratory tests could be performed when there are any changes to code, and the development team can continue working doing their jobs instead of worrying about the intricacies of whether a new feature works or not.
Real devices matter – When discussing browser simulators, mobile emulators and real devices, there’s a time and place for each one. In fact, Megan mentions that the developers on her team use emulators. However, the only way to see exactly what your users are seeing is to use real devices, which is why they are so important to the QA team. In fact, they even ran into a bug where membership details weren’t displaying for a user that wasn’t found on the emulator they were using — they noticed it once they performed the test on a real tablet device. While emulators are highly accurate, they do not replace real devices, and this is a large reason why they need a tool like CrossBrowserTesting that includes all three in order to prioritize both speed and accuracy during mobile testing.
It’s important to understand different user journeys – The success of the America’s Test Kitchen QA team has a lot to do with their ability to identify the different users they have and how it affects their testing. Because a lot of the web content they have is for members only, there are paywalls that occur when someone accesses from a non-paid account, for example. By separating user states into two focused areas for their automated tests, static and dynamic, Michael is able to create more precise tests for different parts of the website. Understanding that they need to test different based on whether users are new, registered, on unsubscribed helps them build out different test cases and scenarios.
Capture tools can help prove a point – Sometimes seeing is believing, and it can be hard to convince other team members of a bug they haven’t seen for themselves. Tools like CrossBrowerTesting’s screenshots and video capture help the America’s Test Kitchen QA team build out their test reports to include visual evidence of different occurrences during tests. This way, Megan doesn’t have to repeat the test case or find it on the right browser when she shows it to development, she can just show the screenshot or video instead so bugs get fixed faster. Additionally, by using the screenshot comparison engine, Michael has been able to create an archive of each homepage over time so it’s easy to show what changes in each regression.
Use automation to make your life easier – America’s Test Kitchen isn’t automating every single test on their website. In fact, manual testing is still a large part of their strategy. Automation comes in mostly with repetitive tasks that take up a lot of time. For example, they use automation to sign in as a member in order to see whether content can be viewed instead of going through the actions of logging in in every time. This saves Michael valuable time in performing different test cases under different user scenarios — instead of replicating the same actions every time, he can focus on how each test differs.
Collaboration is key – We say it over and over again, but communication with your team is important to productive testing. By leveraging tools for organizing your tests, taking notes, and tagging, it’s easier to remember which bugs you’ve encountered before and which are new. Additionally, keeping open communication with the development and design teams has been essential to America’s Test Kitchen for understanding priorities, risks, and hot spots for more insightful testing. Megan and Michael have found collaboration with the development team to be especially crucial in the shift to automation — learning new skills, choosing what to automate, and being notified when new code is added.
Test while you sleep – Many of the automated tests that the America’s Test Kitchen QA crew create run at night, meaning that you literally can test in your sleep. Overnight testing is at the crux of their Selenium strategy, meaning they can check tests in the morning to see if they pass or fail. Because they have so many tests that run for an extensive amount of time, they rotate their browsers each night to get coverage of all the different configuration their users are on to see which tests are failing on which browsers. By running these regression tests, they’re able to test a subset of test cases across different pieces of functionality that they need to make sure are working day to day.
Tests are only as good as how much time you put into them – Michael runs a lot of automated tests, but he also spends a lot of time maintaining them. The America’s Test Kitchen QA team emphasizes that maintenance is crucial to the success of the project. This is one of the main reasons they have a dedicated QA automation engineer. In order to ensure that Selenium tests aren’t just been created left and right, Michael spends a lot of time on each one to make sure it is accomplishing his goals (and may even be a few steps ahead of Elon Musk in this way).

Megan and Michael gave us a great look at how they use automated and manual testing to further the goals of America’s Test Kitchen. By taking the time to learn from other successful QA teams such as this one, you can apply some of their practices to your own testing strategy.

If you want to view the full webinar, you can watch below.

Testers: Beware of Automating Yourself In

June 27, 2018 By Amy Phillips Leave a Comment

Automated testing is increasingly becoming the future of testing in DevOps and Continuous Delivery environments. It’s assumed that simply training testers to move into purely technical roles where they focus almost entirely on test automation is the solution. In this article, we explore some of the pitfalls that can arise from this assumption.

I once quit a job over automated testing. Not because of something philosophical like the test strategy or the team’s perception of testing. No, I quit because I wasn’t allowed to contribute to the automated tests. The reason given: I was a performance tester, not an automation tester.

All around us, we’re trying to put people into boxes. We talk about manual testers, security testers, agile testers and act like we have a hierarchy of skills. Almost every week I hear of someone, usually not a tester, discussing how to “up skill” manual testers to become automation engineers. We expect all of our testing needs to fit neatly into predetermined scripts that can be executed time after time without any deep testing expertise.

The rise in DevOps, Continuous Delivery, and Lean development practices has been influential in moving testing from an afterthought to being seen as an integral part of successful software delivery. We’re aware that batching up large changes can increase risk and there’s growing demand for repeatable and reliable, build, test, and release pipelines. Unfortunately, the desire to break down siloed test teams and replace manual test phases with automated test suites to keep up with the fast-paced delivery cycles can lead to unintended consequences.

Lesson #1

One of my first experiences of streamlined release processes resulted in the test team being broken up to form cross-functional teams along with developers, designers, and product managers. It can be hard to break a team up, and there was pressure for every tester to act as an expert in all types of testing. Previously the test team had been diverse and our skills complementary. Maybe we should have been “up-skilling” to spread the skills to every tester, or maybe we should have recognized that testing is a broad role requiring different approaches and specialties.

The goal of creating cross-functional teams to develop and release every two weeks had another hitch. The first ten days of the iteration were a mix of developing and testing new features. The final two days of the iteration were dedicated to release testing and bug fixing. Sometimes things were delayed and we’d end up with untested features in the release candidate. Already tight, the release testing time would now need to include feature testing as well as release testing. The responsibility — and extra work — to get the release testing completed fell exclusively to the testers.

During the days leading up to and immediately after the release, the testers were obviously rather busy. Rather than sitting around waiting for testing to complete, the developers started working on the next round of features because there were deadlines to meet. It sounded sensible, but this subtle separation of the team led to the testers becoming more and more behind with their normal “feature testing”.

Eventually, someone suggested that testing shouldn’t even be in the same sprint as the development, but should intentionally take place in the following iteration to try and reduce the amount of rework on the automated tests. Despite being a single “team,” splitting responsibility for work created a new type of silo.

If we had owned a single vision of achieving fully released features, encompassing both development and testing, we might have had better success at collaborating together to meet the goal.

Lesson Learned – Own the vision but share the work.

Lesson #2

On my next project, the testers had been hired for cross-functional teams and our skills were complementary to the entire Scrum team. We followed a process that allowed new features to be exploratory tested before automated tests were added to the regression test suite. More importantly, the entire team was expected to contribute to testing. On paper, we had everything we needed to succeed.

Releases took place once a month and the entire team of developers and testers performed the release testing to check for regressions as well as make sure the new features were working as expected. One of the test engineers was responsible for maintaining and executing the automated regression test suite. Everyone else used ad-hoc scripts or exploratory testing to check high-risk areas.

One day, the release testing coincided with the automated regression test engineer’s holiday. The automated tests failed.

As we investigated the failures it became obvious that there was some incredible duplication going on with our release testing. We had failed to share the test scenarios with the entire team. Automated regression tests were being duplicated by ad-hoc scripts, and by exploratory testing too. No one in the team felt like they had the authority to disregard the automated testing results to make the release so we were forced to delay the release date to give us time to fix all the broken regression tests.

Lesson Learned – Share the test scenarios with the entire team. New automated tests should be replacing testing rather than duplicating effort.

Lesson #3

On another project, we’d managed to avoid most of the testing politics. Developers and testers were bought into the cause and actively contributing to the test suites. As we built new features, we exploratory tested the features and automated the tests that we believed we’d want to run against future release candidates. We had tests running in different browsers and on multiple devices. The test suites were well respected and formed the essential part of the automated pipeline that allowed us to deploy to production as frequently as we needed.

For a while, at least.

As time passed and we continued to diligently add tests we built up such a suite of tests that were taking a long time to execute. Developers were becoming more reluctant to run the tests locally before committing to the build because of the time it took to execute all the tests.

We’d already worked out which tests were most likely to fail and were executing these first. Now there was talk of running the tests overnight instead of with each commit. The team started to question whether we’d finally outgrown our release approach and looked at moving to scheduled releases to give us more time to execute the vast number of automated tests.

Fortunately, we decided to analyze what we were actually testing before doing anything drastic. As we dug deeper we discovered that we had a number of problems — we were running automated tests on features that were fine to break as long as we fixed them soon after (a.k.a. non-critical features); we also had a significant number of tests for each feature when, in, fact a simple sanity test would often have sufficed.

We’d fallen into the biggest automation trap: believing that more automation would make us safer.

As we switched our test approach to only automate the things that really mattered, we were able to get our release cycles back to a reasonable duration and avoided having to change our release cadence.

The lesson learned – Only automate the things you actually care about.

Lesson #4

Test automation can be a powerful tool in helping to create fast and flexible release processes. As the rise of DevOps and Continuous Delivery continues we should embrace the fact that testing is such an integral piece of the puzzle. Building true cross-functional teams consisting of the skills we really need rather than just job titles as well as creating and sharing a testing vision avoids so many of the test automation pitfalls. Teams that agree on the end goal are less likely to fall into the “automate everything” trap that can be the undoing of so many projects.

As you go through your testing journey, you’ll make mistakes and discover unexpected opportunities. The most important lesson of all is to use other people’s stories and lessons as a starting point for your own, rather than treating them as a direct set of instructions.

I remember listening to a talk of how Google had achieved incredible things with their test automation. It was inspiring, with so many technical and cultural issues solved by a deep and fully featured test suite. Finally, someone in the audience asked how long all the tests took to run. The answer: six minutes. We were amazed, until we discovered that Google used whole data centers to run their tests.

Most of us can’t ever hope to replicate Google’s test execution power, but that doesn’t mean the talk was wasted. Taking the inspiration of what can be done and then using it to pragmatically approach your own situation will help you create your own success.

Lesson Learned – Your team is unique.

About the Author: Amy Phillips is an Engineering Manager at MOO. She manages the Platform team and supports cross-functional product teams in their quest to build awesome products. To learn more about Amy, visit her website or follow her on Twitter.