One of the initial choices you have to make when you begin to build a cross browser testing strategy is what environment those tests are going to be run against. When it comes to choosing the mobile devices used for testing, you have three options — simulators, emulators, or real devices.

While developers evaluate the pros and cons of these choices, the answer to a classic testing dilemma may be more straightforward than you think.

Emulator vs Simulator

Emulators and simulators are both virtual devices that are used for the same basic purpose — they mimic the functionality of real devices and imitate a user’s actions to recreate operational behaviors.

The difference, respectively, is that one emulates (replicates or reproduces) real mobile device software, hardware, and the OS in order to test and debug applications within another software/hardware platform, while the other simulates (imitates or mimics) the internal behavior of a device, but does not emulate hardware or work on the OS.

The best way to exemplify this is thinking about a flight simulation. By using a simulator, you are mimicking the operation of the plane’s mechanics, journey, and surrounding environment through a computer-generated program that shows you what the procedure might look like. If there were such thing as a flight emulator, on the other hand, the device could theoretically take you to a destination the way a plane could.

In this instance, if you are deciding between emulation and simulation, emulation is your best bet as it’s more accurate and more suitable for debugging. However, emulators and simulators are often used and referenced interchangeably since they perform similar functions.

Additionally, when you think about the flight simulator example, you can see that both instances have degrees of variation from boarding a real plane, just as using simulators and emulators varies from using a real device. In real life, there are many factors that can affect the results that would not be considered in a simulation or emulation.

We’ve found that the use of emulators and simulators is most appropriate in the earlier stages of testing, where you may not require 100 percent accuracy but rather want a fast test that gives you a general idea of how a web application is performing.

As you continue in your testing process, you will likely find that real devices give you more precise observation and allow you to observe more particular aspects of a web application.

Real Devices

The advantage of real devices is in the name. Testing on real, physical devices means that you’re testing the same way that your users are operating their devices.

While simulators and emulators will come close to imitation, it’s impossible to perform exactly the same as a real device. This is because emulators can’t account for every single environmental factor, feature or user action performed on a real device. For example, emulators cannot simulate high traffic volume in the way that a real device will experience, which would possibly affect the results of performance or functional testing.

They also are known to give false negative/and positive testing results, which can be problematic in an advanced testing process and can negatively impact developers’ progress, ROI and bottom line goals. For this reason, testing on real devices more accurate, concise and user-specific, and they become essential for performance, reliability, operability, sanity, and regression testing.

Testing on real devices comes at a higher cost, however. Oftentimes, testers may resort to emulators and simulators because they’re a less expensive option compared to building a device lab for all the platforms you require. However, hosting real devices in a third-party cloud is a simple answer to cutting costs while obtaining access to a broad range of devices.

This will also allow you to perform faster testing, since you can run them in parallel. At the end of the day, real devices are necessary because testing on them will show a larger range of issues that your users are experiencing.

In Conclusion…

There’s a place and a time for emulators and simulators as well as real devices. Deciding between the two is a choice that is highly dependent on where you are in your development process. Additionally, because both options have their pros and cons, many developers will decide to us both practices for testing.

These are the main points to consider when choosing between emulators/simulators and real devices:

• Most developers agree that some combination of simulation/emulation and real devices is best for testing.
• Emulators are a good option in the development process because they’re a faster option, but often are less accurate due to lack of human observation that is critical in testing
• Real device testing can be done through a physical device lab, VMs or a third-party cloud
• Hosting real device testing in a secure cloud is often less expensive, while it allows you to test more devices at the same time