Using Mixpanel for Data-Driven Cross-Browser Testing

Most testers know there’s much more to testing than going into an application blindly and clicking around. In fact, planning test cases and collecting data beforehand is the best way to understand who you’re testing for, what you should test, and why.

When it comes to browser testing, this data becomes even more valuable. For example, what good is it to waste time testing Opera if none of your traffic is visiting your web app through that browser?

While there are multiple ways to decide which browsers and devices to test, you’ll find the most accurate is through an analytics tool. We’ve already explored how to find this data in Google Analytics, so let’s see how to find it in Mixpanel.

1. Go to “Analysis” and “Segmentation”, which will both be at the head of your dashboard.
2. Go to the first drop-down menu you see on the top left and pick an event, which is an action correlated with a page on your web application. You probably want to pick a page that’s either high converting or high traffic to get the best glimpse of your customer data. For example, it might be a good idea to pick your homepage because that’s probably one of the first pages people see when they come to your website. You might also want to look at a few of these events to get a more accurate glimpse of how the data changes for different user journeys.
3. For the second drop-down menu, you want to pick a browser. As you can see, this gives you a good look at which desktop and mobile browsers hold the majority of your users. From our data, we can see that we’d want to focus on Chrome, Firefox, Safari, Mobile Safari, Internet Explorer, and Microsoft Edge. You can refine these results by time duration and view them as a line graph, bar graph, or pie chart.
4. You could stop there, but why not get as detailed as possible? Add a segment by another dimension and set it to Operating System. This will tell you which configurations your users are on, so you can get more accurate information about what machines you should be testing. Looking at our example, you can see we would want to specifically test Chrome on Windows, Mac, Chrome OS, Android, and Linux as well as Firefox on Windows, and Safari on Mac.
5. Another data point you may want to look at is devices. Mobile testing is taking center stage, and you want to have an idea of what mobile devices people are on. By picking “Device” as your main dimension, Mixpanel will give you a good idea of what you should test for whether it be Android, iPhone, iPad, Windows Phone, or BlackBerry. Once you see how many users on each device, you can decide which you want to test for.

Depending on what resources you have at hand (in-house device lab or cloud testing tool), how divided your Mixpanel data is, and the amount of cross-browser coverage your team has allocated for, you’ll want to use this information to form a risk analysis of your application across different configurations and determine the minimum number of browsers, operating systems, and devices you should be testing on. You may find that you have only five major players that need to be accounted for, or it could be fifteen. Either way, you should check this data on a semi-regular basis to maintain accurate information about where your customers are coming from.