This is an edited chapter from Rob Lambert’s eBook, How to thrive as a Web Tester. Lambert’s book includes over 30 ideas to guide your web testing and some thoughts on how to thrive in the industry. You can read the full version by downloading it from Amazon and find more related insights at webtestingbasics.com. Neither Rob Lambert nor the eBook is affiliated with SmartBear.
The internet is made useful by the way we interact with it, and that’s usually through a web browser of some description.
There are plenty of browsers on the market, but there are five big players that most people use to access the web.
- Internet Explorer
A few niche and secure browsers are also entering the marketplace and worth keeping an eye on as people become more conscientious about their browsing data.
Most browsers support what are called “developer tools” that give you extra capabilities and tooling. Be careful though. When using browser add-ons, developer tools, and extensions, it’s not uncommon for them to interfere with your website and how pages are rendered.
Browsers are designed for different operating systems also and the growing array of mobile platforms too. Some browsers are available on all operating systems. Some aren’t.
Add to this the sheer number of different versions of each browser and it can quickly become practically impossible to test against every browser, version, and platform.
Your customers and end-users of your website will most likely be using a browser to access your site. Your goal is to work out what browsers they use, what’s supported, and your own company’s opinions on supporting old versions.
What is supported?
Check what your business wishes to support.
Most companies are going down the route of only supporting the latest version of each browser. Others don’t specify any browser, which presumably means their customers can use any browser they wish.
Even if you support the “latest” version, what does that mean? How much of a grace period do customers get when new versions are launched? After all, you cannot expect all customer to upgrade immediately. For some companies, a company-wide browser update might be a lengthy change request process.
Building and designing websites to work on old browsers is hard and not a path I would suggest unless your customers really do use old browsers. It can be expensive to build, test, and maintain software for old browsers.
Start by working out which browsers are supported and used and move on from there.
For example, you may decide you only support the latest of each browser. If this is the case, what happens if a customer doesn’t use one of those browsers? What happens if you find out that many customers are using older versions?
Testing is about asking these questions.
What do your customers use?
If you already have a website up and running, then install web/app analytic tools to give you insights in to browsers, user behavior, and much more. These tools will tell you which browsers your customers are using to connect to your site. They are very powerful.
Compare what is being used with what is supported and you may be surprised. Just because you don’t “support” an old browser, doesn’t mean that a large portion of your user base isn’t using one.
Whatever the outcome, you will have more information to use in deciding which browsers to test against.
How to test across browsers
Assuming you’ve narrowed down the browsers you are supporting, there are two main questions to ask when cross-browser testing:
- Does the business logic and flow work in each browser?
- Does the site look correct and is it consistent in each browser?
The first approach is something you may consider writing automated checks for using one of the many cross-browser automation tools. Once you have an automated check, your automated check may then tell you if it fails in the future. Or it may not. Automation isn’t always accurate.
The second approach is much harder to write an automated check for. Checking that the site looks OK is possible through code, but I’ve yet to justify the coding costs associated with this kind of approach – I prefer to rely on people to check it as they go about their every-day testing.
It’s also possible to use a visual testing tool to assess how the site looks. While no coding is required to run the tests, automated screenshots will allow you to compare multiple browsers at once. This is a good method for looking at design elements but probably won’t tell you much in terms of functionality.
Run tests against a mixture of browser environments
One of the simplest ways to test across browsers is to simply use different supported browsers as you test something else.
For example, you may be testing that a user can log in to a website, generate some reports, send the reports, and then log out. The system also has a simple “management” system where an admin or manager can view audit logs of activity on the site.
To get a wider coverage you could test your login functionality in one browser, test the send report functionality in another browser, and then audit trail functionality using a third browser.
This is an effective way of covering different combinations of browsers at the same time as doing your day-to-day testing. The above example won’t highlight a problem with the audit trail functionality in the first or second browser though. There are plenty of gaps for bugs to slip through, however, this approach can often reveal obvious issues and give you some confidence that the basics are working.
This approach also assumes that the site is not very big and hence won’t take a huge amount of time to test.
Parallel testing is another method that can be used to perform these tests. By using grid technology with an automated testing tool, you can run the same test against multiple browsers at once, instead of running them sequentially, to save time.
Let someone else do it
If you are using your own software internally as a company, then encourage those teams using it to use different browsers too. Your support, sales, or professional services teams may be using your software and will be putting it through every-day use; it would be great if they could do that across different platforms, operating systems, and browsers too.
Check against standards
HTML standards are supposed to ensure all browsers render and use HTML in the same way, and that all programmers and web designers follow the standards.
Saying that, checking against HTML standards will give you information about how compliant your own code is. It won’t tell you whether you have bugs, or whether pages will render differently, or whether customers will have problems, but it should give you enough to start a conversation with the relevant people in the team. By having compliant HTML, you should cut down the number of issues a customer may see due to cross-browser issues.
You could automate your cross-browser checks using a browser testing tool. This can be effective, but consider the problems you’re trading off against each other and weigh it up.
You’re taking away the challenge of manually checking your site in many browsers, but you’re adding in the challenge of writing re-usable checks and the maintenance costs that come with that. Someone must write the automated checks, maintain them, investigate failing checks, and improve them.
There are plenty of web tools with browser grid technology that also provide a whole host of browsers online. I’ve managed these approaches myself, but when your site gets big, gnarly, and complex, it brings its own set of challenges.
Test It with humans
You could use people to test every customer flow, in every browser, on every version, across every operating system — but you can see where this is going.
To make this less of a chore, use Virtual Machines or an online cross-browser service who provide you with browsers online for a cost.
If you are using your own virtual machines, then you can clone them, and other people can use them. This approach relies on infrastructure, hardware, and lots of time. Plus, someone needs to keep all these machines up-to-date.
This is very time consuming. And tedious. And boring. Hence many online services take this pain away from you for a cost. It’s also a human drain when your site gets really big.
You could emulate the different browsers. There are several tools on the market, as well as some inbuilt tools within the browsers themselves, that emulate older versions.
A word of caution though, they are not wholly accurate in my experience, but they may provide insights that are useful.
I’ve not touched on different operating systems or mobile devices, but as you can see, there are several ways to approach cross-browser testing.
Cross-browser support may be your biggest pain-point as a web tester, especially with the rapid rate of release that most browser companies have adopted.
In my experience, the best way to address cross-browser testing is to find out what your customers are using and focus your attention there. Work out a hardline company stance on the support of old browsers. This depends a lot on who your customers are. Believe it or not, some organizations are still using browsers from the late 90’s and to upgrade would cost millions of pounds in project management and internal tool rebuilding costs!
Rob Lambert is the voice behind Cultivated Management. Rob has spent two decades working in the tech industry as a tester, scrum master, agile coach, and manager. He has built epic teams, epic software, and helped others do the same. He’s also an author, the organizer of an award-winning workshop, keynote speaker, and consultant. To learn more about Rob, check out his website or follow him on Twitter.