Manual Testing Archives | CrossBrowserTesting.com

The Unexpected Value of a Professional Developer Drunk Testing Your Web Design

May 31, 2017 By Alex McPeak Leave a Comment

It’s not often you stumble upon a service where you’re willing to pay a professional to work while intoxicated. In fact, getting drunk on the job is usually frowned upon. But when Richard Littauer founded The User is Drunk, he seemed to have found just the right niche where this was not only found acceptable, but people were willing to pay a little extra for it.

Operating under the tagline “Your website should be so simple, a drunk person could use it.” Littauer’s homegrown side-hustle took off as designers and businesses invested in a UX pro that would review their sites with honest commentary and a new perspective.

At CrossBrowserTesting, we believe web design should be optimized for every user, no matter what browser, device, or operating system. Littauer’s project suggests that it’s important to consider the universal user in another way — drunk or sober.

Though Littauer no longer regularly drunk reviews websites (for money, anyway), he still remains opinionated on what separates a good web design from a bad one. In order to answer some of our burning questions, Littauer gave us an exclusive on some of the things he learned and experienced while testing under the influence.

What’s your background in development and UI/UX?

I grew up with computers and had a half dozen dumpster-dived desktops in my basement. So, naturally, I did a university degree in Linguistics. After enjoying coding for my senior thesis and being the web developer for some societies in uni and for my own personal sites, I took a Masters in Computational Linguistics, with some UX courses which lead through a long, tangled route to me working as a front end developer for a couple of years in New York and San Francisco.

When I started the User Is Drunk, I mainly had front end experience, not UX, and was working as a freelancer on a project with the MIT Media Lab as a designer and developer. There’s no easy path that shows how I got from A to B, and I’m not entirely sure where B is, anyway.

So, how did you decide to start getting drunk and testing people’s websites for them?

Personal experience. Basically, I had come back from a nice night (I assume, I’ve since forgotten. I think it was a date.) and my friend asked me to jump on Google Hangouts and review his site for him. The feedback was brutally honest, and kind of useful because of that, and I remember saying “I should make a thing out of this.”

Drunk advice is different than normal advice; it’s not caged, it’s emotional bull**** but not normally buzzword bull****, and it is not good at error recognition or correction at all. It’s niche. And niche things are good.

I let the idea stew for a couple of months, and then, at a hackathon, decided to just run with it one day. The rest is history – it went viral, and there I was, reviewing people’s sites for money.

When and where have you primarily done this?

I did the majority of the reviews while traveling with Hacker Paradise through in Bali and Thailand. But I’ve done reviews all over the world – I’m a digital nomad, so it’s anyone’s guess where I am going to be at any given moment. I remember doing reviews in Tokyo, Berlin, Boston…

I don’t drink alone. That’s a quick way to hate yourself. So, normally, I invite some friends out for some drinks, and then slip away for twenty minutes, and come back and buy everyone a round. It seems to work pretty well. I’ve gotten better at planning to do little work the next day, too.

What do you do when you aren’t critiquing people’s websites?

Live my life. Recently, that means I run a company I founded – maintainer.io – which helps people maintain their open source software, and consults with businesses on their open source strategy. I am a community manager as well as a developer, so this came out of that, realizing that a lot of what maintainers do can be done as a service.

Sometimes I debate getting a lot of buzz words I’ve used to describe my work printed on hats, and hanging them all on my wall.

What’s your approach — do you wing it or are you usually looking at something specific?

I wing it. Sometimes clients have questions for me, but mostly I just go. It’s very hard to plan drunk. There have been many times where I’ve failed to turn on my microphone, for instance, so I am largely better off just clicking around and following my gut.

How do you think drinking allows you to look at websites differently? How do you think web designers benefit from getting a drunk user to look at their site from a UX perspective?

Basically, it means that I am honest, or at least that there is no filter. There’s what alcohol does; it inhibits your own ability to self-reflect and realize what you are saying and doing. This kind of reaction can’t be faked, and it’s one of the reasons I’ve never pretended to be drunk (the other reason being because that just wouldn’t be right).

The advice or complaints you give while drunk haven’t gone through some review board, haven’t been cleaned up – it’s what me, a real person, actually thinks about your site at a given time. That’s invaluable.

Couldn’t just anyone get drunk and click through their own website?

Sure. But they don’t have the experience I do, and they may not be used to developing and designing websites. Also, at this point, I’m the most experienced drunk UX auditor in the world. I think that counts for something. But if you’re up for it, go have a good time. Drink responsibly.

What are the top 3 worst things websites tend to do in the eyes of an inebriated user?

Dark UX patterns are very, very clear when you’re drunk. A lot of websites are jerks – no, I don’t want to sign up for your newsletter. No, I don’t want this pop-up to happen every time.
Another major thing would be too much text. Reading walls of texts isn’t really my thing, and probably isn’t other people’s either. On the other hand, Pam, the mom from The User Is My Mom, loves reading walls of texts. She has a longer attention span. Age matters.
The final worst thing would be red and green colors everywhere. I’m a bit colorblind; not horribly, but if you point to a cardinal in a Christmas tree, it’ll take me a while to find it. A lot of sites don’t cater to people with this disability, although it is fairly common. That gets annoying, fast.

Have you ever drunk tested in different browsers or devices? Do you think there would be any value in testing responsive design under the influence?

I’ve done a few apps, and a few private sessions. There is some value, there. Frankly, though, I’m not an app developer, and I’ve always felt that my advice on websites is more pertinent. There is definitely value in responsive design for drunk people. You have no idea where your users will be.

How did people respond to the results? Do a lot of people change their web design after your reviews?

It’s all across the board. I’ve gotten a few people who were full of praise, and changed a lot of things, working my tips into their sites. I’ve had people change their onboarding process after, especially.

I’ve also had a fair amount of people who never got back to me. Those are not fun. Drunk testing is very hard to standardize – sometimes I’m nice, sometimes I’m not. That’s how it goes.

What’s your drink of choice?

Uisge beatha. Talisker, 12 year, or a Laphroaig, quarter-cask. But normally it’s just a local beer. I’ve gone off stouts, wheat beers, and red wine with cigars as a result of some bad nights. That’s OK with me.

What advice do you have for designers who want to cater to the user experience, whether those users are drunk or sober?

Get on Twitter and Designer News. This is the golden age of learning – never stop.

And please don’t use red and green colors too much.

Any last thoughts you’d like to add?

While the User is Drunk is fun and stupid, it’s also fun and stupid. This is not my main job, at all. Currently, I’m working on consulting for companies wishing to do better open source, and helping individual coders and teams who have too much code to maintain – check out my business at maintainer.io.

I say this because I think a lot of people are under the illusion I only do the User Is Drunk, which couldn’t be further from the truth. I limit my reviews to around once a month now. Livers can’t be fixed by resetting the router. Drink responsibly, folks.

Choosing Between Simulators, Emulators, or Real Devices for Testing

March 20, 2017 By Alex McPeak Leave a Comment

Should you use simulators, emulators, or real devices for testing?

One of the initial choices you have to make when you begin to build a browser testing strategy is what environment those tests are going to be run against. When it comes to choosing the mobile devices used for testing, you have three options — simulators, emulators, or real devices.

While developers evaluate the pros and cons of these choices, the answer to a classic testing dilemma may be more straightforward than you think.

Emulators and Simulators

Emulators and simulators are both virtual devices that are used for the same basic purpose — they mimic the functionality of real devices and imitate a user’s actions to recreate operational behaviors.

The difference, respectively, is that one emulates (replicates or reproduces) real mobile device software, hardware, and the OS in order to test and debug applications within another software/hardware platform, while the other simulates (imitates or mimics) the internal behavior of a device, but does not emulate hardware or work on the OS.

The best way to exemplify this is thinking about a flight simulation. By using a simulator, you are mimicking the operation of the plane’s mechanics, journey, and surrounding environment through a computer-generated program that shows you what the procedure might look like. If there were such thing as a flight emulator, on the other hand, the device could theoretically take you to a destination the way a plane could.

In this instance, if you are deciding between emulation and simulation, emulation is your best bet as it’s more accurate and more suitable for debugging. However, emulators and simulators are often used and referenced interchangeably since they perform similar functions.

Additionally, when you think about the flight simulator example, you can see that both instances have degrees of variation from boarding a real plane, just as using simulators and emulators varies from using a real device. In real life, there are many factors that can affect the results that would not be considered in a simulation or emulation.

We’ve found that the use of emulators and simulators is most appropriate in the earlier stages of testing, where you may not require 100 percent accuracy but rather want a fast test that gives you a general idea of how a web application is performing.

As you continue in your testing process, you will likely find that real devices give you more precise application and allow you to observe more particular aspects of a web application.

Real Devices

The advantage of real devices is in the name. Testing on real, physical devices means that you’re testing the same way that your users are operating their devices.

While simulators and emulators will come close to imitation, it’s impossible to perform exactly the same as a real device. This is because emulators can’t account for every single environmental factor, feature or user action performed on a real device. For example, emulators cannot simulate high traffic volume in the way that a real device will experience, which would possibly affect the results of performance or functional testing.

They also are known to give false negative/and positive testing results, which can be problematic in an advanced testing process and can negatively impact developers’ progress, ROI and bottom line goals. For this reason, testing on real devices more accurate, concise and user-specific, and they become essential for performance, reliability, operability, sanity, and regression testing.

Testing on real devices comes at a higher cost, however. Oftentimes, testers may resort to emulators and simulators because they’re a less expensive option compared to building a device lab for all the platforms you require. However, hosting real devices in a third-party cloud is a simple answer to cutting costs while obtaining access to a broad range of devices.

This will also allow you to perform faster testing, since you can run them in parallel. In comparison, owning and operating a device lab means that you can only run one test at a time as you go through different phones, desktops, and operating systems.

At the end of the day, real devices are necessary because testing on them will show a larger range of issues that your users are experiencing.

In Conclusion…

There’s a place and a time for emulators and simulators as well as real devices. Deciding between the two is a choice that is highly dependent on where you are in your development process. Additionally, because both options have their pros and cons, many developers will decide to us both practices for testing.

These are the main points to consider when choosing between emulators/simulators and real devices:

Most developers agree that some combination of simulation/emulation and real devices is best for testing.
Emulators are a good option in the development process because they’re a faster option, but often are less accurate due to lack of human observation that is critical in testing
Real device testing can be done through a physical device lab, VMs or a third-party cloud
Hosting real device testing in a secure cloud is often less expensive, while it allows you to test more devices at the same time

Getting Started with Manual Testing

March 17, 2017 By Alex McPeak Leave a Comment

Manual testing is a must

What exactly is Manual Testing?

When any tester, developer or QA team begins a functional or performance testing process for the first time, they’ll likely start with manual testing. Regardless of the size of the company or web application, manual testing is an essential component of any testing strategy because it gives deep insight into the user experience.

This is because manual testing is done completely by human hand, without the assistance of automation tools or scripts of any kind. For this reason, manual testing is the ideal way to assess the details that are relevant to the end-user including how the application functions, how well visual elements render, and other GUI qualities.

Manual testing is specifically valuable in exploratory testing or other test cases that are only run once or twice and ensures that you discover bugs and other issues early on in the development cycle. These tests are an optimal environment for incorporating human observation without an urgent need for the speed and in-depth accuracy that’s characteristic of automated testing.

Manual testing also allows testers to build test cases, which are needed for automation so that you can repeat those actions. No matter how much time you’re willing to put into manual testing, it’s the first step to creating a script that can be run in parallel in other browsers, or many times over.

For this reason, it’s crucial to understand some of the stages and techniques before executing manual tests.

Types of Software Tests

These stages of testing are sequential, starting with the smallest component of the application and progressing to make sure that all parts are working together.

Unit

Unit testing is at the bottom of the testing pyramid because, as the smallest testable part of an application, it focuses on one individual function within the entire application. Although this type of test is better performed in an automated manner – it’s still the base of our pyramid and most plentiful type of tests out there.

Clicking a button to make sure it takes you to the correct destination would be a unit test. For example, making sure that the “Next page” button will actually take to you the next page.

Integration

Integration testing is the next step after unit testing and is the process of integrating multiple units to test them together. This lets testers observes how different components of the software harmonize to ensure functionality.

An example of an integrated test could be your applications log-in process. Now, you’re testing whether entering a username and password and clicking login will bring you to the Welcome screen. These types of tests match two or more unit tests together.

It is important to have an inverse amount of tests as you go down the testing pyramid. This allows for easy debugging and error locating, speeding up both testing and development. Have unit tests along with integration and other test lets you diagnose the exact location of bugs.

System

A system test refers to testing completely integrated software in order to evaluate that system meets the requirements of the other hardware or software systems that it’ll come in contact with.

This is actually a series of multiple tests including end-to-end scenario testing, verifying inputs and output and testing the user experience. There are also multiple types of system testing such as stress, regression and functional testing, the uses of which will differ from tester to tester depending on available time and resources.

Again, all the components that would be tested in a system test have already passed integration testing. System testing after is necessary for testing the design and behavior of the software.

Acceptance

Acceptance testing or User Acceptance testing is the beta testing of the software performed by actual end users. This, of course, is to bring the product into a real life environment to see how well it measures up to a user’s standards in case the tester misses an essential user function, specification or request.

As you can see, manual testing requires a thorough and sequential completion of these stages in order to properly measure a software’s functionality at different points in the testing process in order to target or fix various areas.

Black Box vs White Box Testing

Black box testing and white box testing are two techniques of testing that can be applied to the above stages.

Black box testing is where the internal structure of the software being tested isn’t known to the test, while the internal structure is known within white box testing.

Black box testing is the most common for testers because they don’t have to be expert in coding, making it easier and simpler to execute. It’s also quicker to do because testers only have to look at the GUIs that users are interacting with and looking at.

However, black box testing has restricted coverage in this way and can only evaluate a percentage of every possible test case or scenario, rather it just considers inputs and outputs.

In comparison, white box testing is a little more in-depth and looks at the programmatic structure of the system. In this way, it allows the developer or tester to spot errors and bugs quicker and more accurately and provides a more thorough insight.

Keeping Manual in Your Testing Strategy

While the testing environment is largely moving to automated testing as today’s developers focus on a strategy that embodies agile CI, there will always be a very specific and important place for manual testing. Those hoping to move towards 100 percent automation will find that their results will fall short from complete and efficient web development.

Instead, getting to know your code through the methods and techniques of manual testing will prove to be invaluable when it comes to present and future web testing practices.

Launching A Instant Test With The Bookmarklet

February 2, 2017 By Daniel Giordano Leave a Comment

Our Instant Test Bookmarklet allows you to quickly jump into a Live Testing session or take screenshots of the web page you are currently on at the time.

A bookmarklet is like a bookmark, but instead of loading a specific page, it injects JavaScript into the current page in your browser for added functionality. Our bookmarklet can be found at the Test Center, and quickly installed by being dragged into your bookmark bar.

Starting The Bookmarklet

To begin using the bookmarklet, simply click the bookmark while on the page you’d like to test. You’ll be presented with the following app on your web page:

From here you can jump right into a Live Testing session by clicking one of the browsers presented as a “Quick Launch Browser”.

Adding A Browser

To add a new browser, simply click Add and select the operating system, browser, and resolution. When you click Add, we’ll include that as a “Quick Launch Browser” as long as you are logged in.

Removing A Browser

To remove a browser, click Edit, and the X in the upper right-hand corner. In this view you can also change the resolutions of each configuration.

Running Screenshots

To run Automated Screenshots, select Shots on the left side navigation menu, right below Live. You can choose from popular or saved lists of browsers and quickly launch into our app’s UI.

Installing the CBT Bookmarklet

Known Issues:

Bookmarklet layout issues: The bookmarklet injects javascript into the current page you are viewing. CSS styles on these pages can impact the appearance of the bookmarklet. If you notice a site that it is not correct on, contact us and we will try to correct it.