agile Archives | CrossBrowserTesting.com

Running Automated Testing Software in an Agile Environment

August 14, 2017 By Claire Mackerras Leave a Comment

There is no doubt that automated software testing is critical for maintaining agility and establishing continuous improvement. As more teams adopt both automated testing and Agile development, it becomes clear they are best practiced together.

Any testing that is being performed for more than two or three times in a short span of time can be automated with paid and/or open source tools. Execution of unit tests, functional tests, continuous integration, and continuous deployment are common ways of utilizing automation for the greatest benefit.

Automating software testing is especially preferred in an Agile environment because it accomplishes the following tasks:

Identifies test productivity problems
Swiftly responds to the testers
Helps testers discuss and examine solutions
Implements technology to improve test processes
Assess automation possibilities for upcoming projects

Benefits of Automated Testing in an Agile Environment

Automation technology accounts for repeated testing and delivers extensive functionality and coverage through the software testing life cycle. If automation software is correctly executed, it provides the following benefits:

Improved Software Quality – Consistent, repetitive and extensive automation tests helps to deliver high-quality software
Improved Documentation – Automated tests produces instant documented quality proofs, comprises extensive flop/pass consequences, brings down testing manpower and time
Bring down testing manpower and time – As automated testing is quicker than manual testing, as these are labor-intensive, and can do verification of thousands to millions of test permutations from minutes to hours.
Cheaper – It also helps bring a remarkable reduction in the expenses related to the production of best-in-class software.

Automated Testing Trends

Organizations have largely merged automated testing and Agile development due to major trends that have arisen in the last few years.

Let’s take a brief look at some of the trends as follows:

1. Mobile Testing

Mobile testing will continue to remain at the top of the list for automation trends, owing to the remarkable shift in the adoption of mobile technology. Mobile application testing will indisputably have a major impact on the software testing industry with functionality, compatibility, performance, usability and security testing as major areas of focus.

In the next few years, mobile test automation will match web test automation if the market for mobile continues to rise at the same pace. There’s a great possibility in the open-source framework, mobile development, cross-browser testing, and app markets to expand for the mobile test automation landscape.

2. Big Data Testing

With an increase in Big Data analytics, quality management for Big Data implementations will emerge. Applications, networks, and devices will generate a huge source of unstructured data, ultimately leading to faulty and costly decisions.

Testing quality ahead of product release will continue to remain critical. With this type of testing, businesses will aim to get hard numbers and statistical indicators on whether the application behavior will meet the demand of customers or not.

3. Testing with TDD & BDD

Enterprises are likely to adopt TDD and BDD to deliver applications as per the customer needs, without any defects or delay. By engaging in this technology, teams can get an actual idea of what is required.

As a result, enterprises are better off in terms of cost and time since the longer it takes to identify bugs, the more it costs to fix them.

4. DevOps

Alliance among IT professionals, developers and operation engineers has increased since the emergence of DevOps. More and more organizations are merging the role of testers with operational engineers and developers due to overlapping competencies in the continuous Agile cycles.

Experts predict that testers, developers, and operation engineers will similarly need to utilize the same tools to increase the collaboration. This technique will promote better communication and feedback among teams to achieve Agile development, testing, and delivery.

6. Micro Testing

Enterprises financing in Micro Services make it easier to test an independent piece of functionality without altering other applications.

This trend will inevitably rise in the coming years, as it certifies testing complex applications which comprise small, independent processes and architectures.

The Verdict

Automated Testing is considered by many software testers as one of the most forthcoming trends practiced in the overall software development life cycle today. With the help of automated testing, one can get more effective and efficient testing results in an Agile environment.

About the author: Claire Mackerras is a Senior Quality Assurance Engineer & Editor at BugRaptors. BugRaptors is a CMMi5 certified company with extensive experience as a third party testing vendor for tools in Software Testing & QA. Claire enjoys creating, uncovering, and disseminating new and interesting perspectives on technology and software testing.

How Much Does a Software Bug Cost?

August 8, 2017 By Alex McPeak Leave a Comment

Test early and test often — that’s what everyone says will help you avoid the high cost of deploying a software bug. But a bug might not seem like a big deal without a dollar sign attributed to it. Why invest in testing if you can just fix your mistake after?

While there’s no set cost you can ascribe to a software bug found after the product release because it’s highly dependant on the organization’s size, customers, employees, and debugging resources, we can look at a few statistics and examples that show just how damaging it can be.

Starting with the big picture, software failures cost the worldwide economy $1.1 trillion in 2016. These failures were found at 363 companies, affected 4.4 billion customers, and caused more than 315 years of lost time. Most of these incidents were avoidable, but the software was simply pushed to production without proper QA.

Those may be stunning numbers, but you could still be thinking it doesn’t mean much to your specific organization, so let’s look at it on a smaller scale.

IBM found that the cost to fix an error found after product release was 4 to 5 times higher than if it’s uncovered during the design phase, and up to 100 more expensive than if it’s identified in the maintenance phase.

The costs go up as the bug moves through the SDLC. For example, IBM estimates that if a bug costs $100 to fix in Gathering Requirements phase, it would be $1,500 in QA testing phase, and $10,000 once in Production

And software bug expenses aren’t just related to the impact on internal operations. When a company announces a software failure, they lose an average of $2.3 B of shareholder value just on the first day.

The fact of the matter is that a software bug can affect everything from indirect costs like customer loyalty and brand reputation to more direct losses in business revenue and wasted time.

Companies Who Learned the Hard Way

You might think a software bug can only be so bad, or that any bug big enough to cause significant financial distress will be a complex mathematical error. In reality, some of the most detrimental bugs had minor mistakes that developers just didn’t catch because they weren’t used to putting the software through proper testing.

The following are companies who suffered monumental losses — some that almost put them out of business — due to easy errors, new software, or a bad line of code.

NASA – The Mars Climate Orbiter was a robotic space probe that NASA Launched in 1998 to study the climate and atmosphere of Mars. All was well until communication got cut-off a year into the mission when the Orbiter got lost in space. When dealing with spacecraft engineering, there’s a lot that could go wrong — it’s literally rocket science. However, the error could have been an easy fix if caught on time. The $125 million spacecraft presumably orbited too close to Mars’ surface and disintegrated because the engineering team failed to convert their measurements from U.S. units to metric. However, that’s not the only time a programming blunder destroyed a NASA rocket.

Knight Capital Group – As one of the biggest American financial services firms, Knight had a lot to lose, and in 2012 they lost $440 million of it. All it took was thirty minutes to drop 75 percent of their shares after an automated set of orders was sent out all at once to nearly 150 stocks instead of over a period of days like it was supposed to. The function, known as “power peg,” made stocks move up and down, lowering the stock price of all the affected companies. The loss was almost four times the company’s profit and would have put them into bankruptcy if a group of investors hadn’t come to the rescue.

AT&T – In January of 1990, AT&T failed to meet the basic needs of their customers as thousands attempted to call their friends, family, and airports. Long distance calling was down for nine hours, and customers sparked outrage after experiencing a total of 75 million missed phone calls and 200,000 lost airline reservations. Initially, AT&T thought they were being hacked, but the issue was due to a software bug. AT&T had just recently “updated” code, but the change of speed ended up being detrimental to a system that couldn’t keep up. AT&T lost an estimated $60 million from long distances charges that didn’t go through after being down less than a full day in an error that was the result of one line of buggy code.

EDS Child Support – In 2004, a computer system managed to overpay 1.9 million people, underpay 700,000, accumulate $7 billion in uncollected child support, backlog 239,000 cases, and get 36,000 new cases “stuck” in the system. This happened after a software company introduced a complex new IT system to the UK’s Child Support Agency. Meanwhile, the Department for Work and Pensions was restructuring their entire agency. The software was incompatible as the systems clashed, and as a result, the oversight cost taxpayers $1 billion.

Apple – For many consumers, Apple can do no wrong, but when they replaced Google Maps with Apple Maps on iOS 6 in 2012, they managed to put a pause on customer’s endless adoration. The update was nothing more than a product of corporate politics — Apple no longer wanted to be tied to Google’s application, so they made their own version. However, in Apple’s haste to release the latest and greatest map app, they made some unforgivable lapses in navigation warranting the #ios6apocalypse hashtag. Just a few of the many Apple Maps fails include erased cities, buildings disappearing, flattened landmarks, duplicated islands, warped graphics, and false location data. While an exact financial loss was never attributed to this lack of judgment, it made customers question whether Apple prioritized their needs over their money as they considered switching to the very competitor Apple was trying to combat.

An Argument for Agile and Continuous Testing

As technology evolves, so do our development methodologies and knowledge of testing practices. Getting stuck in the old age of software development means that crucial processes go ignored.

For example, while a Waterfall methodology used to be the go-to back in the day, high functioning software teams have realized that following Agile practices better support changes made throughout development. Agile’s incremental approach allows more feedback, flexibility, and of course, testing so that every time a feature, fix, or function is added or changed in code, it’s checked for bugs.

Since teams have adopted Agile and Continuous Integration, software is higher quality after a product release because bugs and inconsistencies are caught earlier and fixed easier. In turn, organizations have found that this helps avoid preventable bugs, save time and money, and retain their customers’ trust.

How to Prevent Your Next Software Bug

While we can’t expect to test everything and go our entire lives deploying a product that’s 100% error free, we can make strides to safeguard software as best we can. Many of the worst bugs that we encounter are simple mistakes that could have been caught with a more thorough quality analysis.

The moral of the story is to test your software. If you don’t, it’ll cost you.

Establishing Your Scrum Team’s Definition of Done

June 1, 2017 By Alex McPeak Leave a Comment

When talking about building culture in the software development process, deciding on a team’s Definition of “Done” has become a major component in the shift to Agile — most commonly in Scrum — that ensures every player works together toward the same end goals.

For example, if a developer is working on a certain user story and tells the product owner they’re done but it hasn’t been tested, this will backup product release once the mistake has been realized, or potentially affect reputation if a buggy app is deployed without being caught.

While one person may be finished with their tasks, that does not mean the project is done as a whole, and saying so can limit progress if it’s not done in all parts and stages. Teams that do not consider this groundwork often find that it severely impacts productivity.

The Definition of Done for Scrum Teams

When Agile strives for iterative development, fast feedback, and adapting requirements, having limited communication will delay a team’s acceleration and pace when pushing a project or feature set to market.

Maybe you feel factors like testing, reviewing, documenting, releasing, or even collaboration should go without saying, but many team members will consider tasks done based on whether or not they fulfilled their individual role.

Because Scrum has lightweight management overhead and bi-weekly check-ins (Sprints), having a Definition of Done becomes even more important to your team’s efficiency. Nothing kills team chemistry and collaboration quicker than having tasks done “half-way” only because another teammate thought they only had to scope out a project, and not actually supply mocks or a rough draft.

Coming up with a Definition of Done is about the bigger picture organizational process of your team. By looking at projects as a collaborative effort instead of dividing responsibilities among different departments or one leader, you encourage the communication and transparency that’s necessary for Agile.

This is why creating a Definition of Done should be a starting point for Scrum teams. Your organization should essentially have a checklist of specific criteria and requirements that need to be fulfilled before the user story is done. However, the definition will likely different from organization to organization based on the size of the teams, different fulfilled roles, and different project requirements.

Criteria to Consider When Defining Done

Prioritizing tasks – Having an order and process in place should be a prerequisite for defining done. This means having a set expectation for every time a task moves throughout your Kanban flow, whether something is selected for development or in code review. Also, this includes deciding whether a Definition of Done should just be attributed to user stories, or features and epics as well.
Including testing – Testing is often the most overlooked step. Is more manual or automated testing needed for this particular project? Are you using regression tests throughout development or just at the very end? Have you tested across multiple browsers? What processes do you have in place for analyzing risk and evaluating security?
Involving every role – If your developers have a different Definition of Done than your testers, for example, projects aren’t going to be done by everyone’s standard. This also includes product owners, project managers, scrum masters, designers, automation engineers, QA, or any other role. Furthermore, the definition should not be prescribed by one team member, but instead agreed upon by everyone.
Coming up with a timeline – How many iterations are you striving to make throughout the day? How many are you committing to at the end of the day? What are the goals during your sprints? How do you keep track of this and what happens if a task is not completed in time?
Ask what makes the process complete – Is it done when the product goes through a last round of testing, is it done after a last once-over by the product owner, or is it when it’s released to the end user? Having an end point is just as important as having a designated place to start.
Document the definition – Have the information easily accessible for people to reference, even if it’s printed out as a checklist. This way, everyone can double-check before they move a task into “done.” Additionally, in the same way, that Agile is adaptable, your Definition of Done should be, too. Make sure to return to the document and reassess processes that need to be adjusted according to changing requirements.

How do you decide when a project is “done”? Share your thoughts in our comments!

What is DevOps, Really?

May 15, 2017 By Alex McPeak Leave a Comment

DevOps: fundamentally, it’s the collaboration between Development and IT Operations. Okay, but what does that mean exactly? What is DevOps, really? Well, it depends on who you ask.

While https://crossbrowsertesting.com/devopsDevOps is certainly the buzzword of the software industry, with popularity comes criticism, and many people seem to disagree on what its definition truly is. After extensive investigation, these are a few of the themes we’ve found most people seem to be able to agree upon.

The Age of Agile

Maybe a few years ago siloed contributors could get away with creating high-performing deployments when they were only releasing features every few months, but amid today’s frequent and faced-paced release cycles, having isolated departments just doesn’t fly. This, of course, is because of the massive shift toward Agile development.

Luckily, if you know a thing or two about Agile, you have a basic foundation of what DevOps strives for. In fact, in the Agile Admin, Ernest Mueller says that the best way to understand DevOps is to equate its definition in reference to the philosophies of the Agile Manifesto.

That is to say, if you want to build new features, get fast feedback, ship often, and ensure quality, you have to have a certain infrastructure to make that happen. This includes having values, methods, tools, and teams working in harmony.

We’d like to think that DevOps is pretty just a common sense approach to successful Agile —
“Since the Agile methodology was only implemented during the development phase, the operations department was left behind, leading to deployments piling up faster than it was possible to release them,” noted DevOpsGuys.

By bridging the gap between two departments that instinctively should work together but weren’t, deploying rapidly and efficiently became a more achievable goal.

The Dev + Ops Alliance

Maybe you haven’t noticed, but culture is really big deal in the software development and IT landscape.

It seems like the most successful organizations are putting culture at the forefront, and it makes sense. With so many moving parts and processes, having everyone on the same page is the only way to be productive and proficient.

When we say culture is this scenario, we’re not talking about open offices, holiday parties, and company perks. In DevOps, the talk of culture is more of an encompassment of communication, transparency, collaboration, and respect.

This also means ending the stereotype of Development vs Operations. While the skill sets and day-to-day focus of those in Development and Operations may differ, they should complement each other to act as a united front. There should be an investment in innovation and problem-solving across the board.

Additionally, the more developers are willing to improve their own operational knowledge and the more Ops is willing to learn about best practices in development, the more cohesive the whole workflow will be.

At the end of the day, DevOps is all about being able to work well together. The effect of that, of course, is more frequent integrations, faster deployments, and better results.

Defining What is DevOps

We always come back to this description:

“DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.”

Like DevOps, Mueller’s explanation is broad. It leaves room for interpretation without limiting teams or defining what a role does or does not do, because that really depends on a lot of things.

People are never going to agree on one version of DevOps because it’s not consistent. You can think about it as a culture shift, an answer to Agile and Continuous Delivery, or simply just an effective approach to the software development life cycle.

Some may say there’s no such thing as a DevOps team, while some may agree it exists and be tempted to leave the term behind forever because of ambiguity and popularity. However, if a team calls itself DevOps and produces great software, there’s probably something we can learn from it.

Maybe the more we try to define DevOps, the more complicated and controversial it’s going to become, and we should simply focus on the principles and the processes that are working — unifying siloed departments, collaborating often, and leveraging a diverse array of strengths.

What’s your definition of “DevOps”? Tell us what you think in the comments!

The Difference Between Agile vs Waterfall Development

April 20, 2017 By Alex McPeak Leave a Comment

Waterfall and Agile are the two central methods of organizing software development. While Waterfall used to be the reigning method, Agile has become increasingly popular since “The Agile Manifesto” to keep up with consumers’ standards in today’s face-paced technological environment by ensuring continuous integration and continuous delivery.

To obtain a better understanding of Agile vs Waterfall, we’ll look at the differences between the two methodologies as well as which one is best for your development and testing teams.

Waterfall

In the most basic sense of the definition, Waterfall is a non-iterative or sequential design process.

With Waterfall, there are several distinct stages. These include requirements and analysis, design, development, testing, deployment, and maintenance that are completed in a linear flow where each stage must be finished before the next one can begin.

Those who still use waterfall might prefer the method because it requires a clear and concise plan from the beginning, ensuring that it’s easier to estimate timetables and associated costs.

Additionally, since the project has a more defined design, there’s usually a clear vision between customers and developers. To someone using a Waterfall methodology, a successful product always matches the initial plan without any surprises.

However, it’s rare that a customer won’t change their mind about a single feature or function throughout the entire development process. Agile supporters argue that Waterfall is extremely limiting in this way because it can’t keep up with changing customer needs.

The issue with Waterfall is that once a stage is completed, it’s almost impossible to go back and make changes. Consequently, it’s inflexibility does not serve as a practical option for teams that strive for continuous integration and delivery.

Not to mention, the high dependency on initial requirements leaves little room for error. If a requirement needs to be adjusted for any reason, you’ll likely have to start over from the beginning.

Agile

Agile is certainly a newer methodology that came to being by seventeen developers who created the “Manifesto for Agile Software Development” during a trip to a Utah ski resort in 2011.

Ultimately, they agreed that Waterfall wasn’t meeting their needs and came up with this alternative methodology that is known today as Agile to better support change throughout the development process.

Like Waterfall, Agile also follows a process, but its incremental approach allows more flexibility, feedback, and testing. The process includes a sprint planning session with smaller tasks and testing at every increment rather than just at the end before product launch.

This makes Agile the ideal environment for Continuous Integration, which requires team members to integrate their work frequently. In fact, CI was created for Agile development so teams could solve issues as they come instead of trying to predict and solve every issue up front.

Compared to Waterfall, developers find this often generates higher quality software because bugs and inconsistencies are caught earlier and fixed easier.

Agile also allows more communication between customers, developers, managers, testers throughout the entire process to ensure the product is moving in the right direction and adhering to the customer’s specifications.

Additionally, Agile can better accommodate the evolving requirements of customers because feedback is given simultaneously with development so that additions, changes, or complete feature removals can be easily adapted and tested without having to start over.

This ensures that as a customer’s vision changes over the time of development, it can be easily accounted for to build a truly successful product.

Tips for Transitioning from Waterfall to Agile

Perhaps the Waterfall methodology you’ve been using is working exceptionally well for your team and customers. If that’s the case, then keep doing what you’re doing. However, if you feel Agile may better suit the needs of your stakeholders, consider these points during your transition.

Culture and communication – One of the best things about an Agile process is that it enforces more departmental collaboration. However, this won’t come naturally, and the transition will likely require a culture change. It’s important to have that all the necessary team members to accommodate an Agile process, while transparent communication will become increasingly crucial for successful deployment.
Digging deeper into agile – As Agile is more of an umbrella term, deciding which specific Agile process you’d like to delve into is essential. The Agile methodologies include scrum, lean and kanban, extreme programming (XP), crystal, FDD, and DSDM. Learning more about the methods of each will allow you to tailor Agile to your projects.
Transition your customers, too – Your internal team isn’t the only one who needs to be updated. One of the benefits of Agile is constant customer feedback, but this means you’ll need to keep them in the loop. Make sure they are invested and committed to a more agile process and know what it requires, as they’ll probably need to be more involved in development.
Prioritize your tests – You can’t testing everything, especially in Agile. Executing a risk analysis will help your team understand the likelihood of bugs occurring and the impact features have on customers to determine how tests should be written and which ones should be included in the test suite. Assessing the priority of individual tests will ensure that the Agile process is an effective and efficient use of your team’s time.
Know that Agile is not a silver bullet – While Agile is becoming popular among software engineers being that it’s largely considered a more efficient organizational process, it will not solve all your problems and shouldn’t be expected to. Transitioning will likely require new hires and a shift in culture that may not always be smooth sailing, even if the ROI is optimal. Like anything else in the software industry, you need to examine the pros and cons of each method to determine which is best for your needs.

Do you follow a Waterfall or Agile methodology? Share your thoughts about them in the comments!