I studied every user testing platform and here’s what I found

A startling figure i’ve seen banded about the internet as of late is 80% of digital product owners or teams don’t undertake user testing prior to releasing their initial product.

I don’t need to spell out to you why this is a bad idea, suffice to say it’s a ridiculous figure and it doesn’t have to be that way.

A recent post by Aytekin Tank (founder & CEO of Jotform) discussed in the early days of building Jotform he and his team toiled away behind the curtains building their new platform only to unleashed their new creation to a very disappointed audience.

Working in a vacuum prevented us from gathering important feedback. The release was messy and littered with bugs. Customers got mad, and many asked to revert back to the old version. Some even left us.

Personally, I will put my hand up and admit I was part of that statistic. I was naive, thinking I didn’t need to speak to users, that I had it all figured out and I knew exactly what my target user wanted.

Suffice it to say I was wrong.

After a few failed personal side projects and countless months wasted I eventually came to discover the lean startup methodologies and the build, measure, learn framework.

Flash forward 5 years and I’m now a Senior Product Manager at a Startup Studio in London working with some incredibly exciting new startups whilst still running a few of my own projects on the side.

As a Product Manager user testing is a weekly if not daily ritual and has become almost obsessive trait amongst the PM’s at ucreate. Trust me when I say, once you experience the insight from undertaking user testing sessions you will quickly understand what I’m talking about. This insight from testing early wireframes and prototypes will save you weeks if not months in wasted time and money.

Research

Through this obsession with user testing, I and our team of Product Managers at ucreate have tested every user testing platform we could get our hands on over the years.

One of the best things about being a Product Manager, is we’re at the intersection of the product and it’s users, so we’re in a great position to objectively assess the value of user testing platforms.

To validate my assumptions of why user testing isn’t undertaken I issued a survey to my network and received 98 responses. Here’s the top reasons why user testing isn’t undertaken:

• I don’t think it’s required (40%)
• It’s too expensive (22%)
• I don’t have the time (20%)
• I don’t know how (18%)

Some of these reasons are valid, perhaps…. however, I feel there is one fundamental reason missing from that list that could take the number one spot if we are all honest with ourselves.

We don’t want to hear the truth. We don’t want someone to prove our assumptions are wrong. We’re scared of the truth.

If this resonates with you, I would urge you to suck up your pride and get feedback as early as possible in your products early development phase. The sooner the better. It doesn’t need to be expensive, timely or complicated. Trust me!

The next step during my research was to map out all the current user testing platforms out there. I did this through a mixture of speaking to our Product Team, Google searches, IndieHackers forum, Quora, Product Hunt and Reddit to name a few. I then identified the key features of each platform and mapped them out in a spreadsheet and assigned a general offering level against each feature. This essentially allowed me to benchmark each product and their associated features against each other. From the 40 or so products I identified, I narrowed the list down to 5 key players.

Example of my product offering benchmark analysis

My Findings

Cost

The majority of user testing solutions either fall into one of two categories with their pricing.

1. Freemium tier with extremely limited offerings optimised to convert you to a paid user after very limited experience in using the product.
2. Paid tier, usually subscription based and a steep step from the previous tiers. It’s largely unaffordable to most makers and small teams. The typical subscription model doesn’t work for smaller teams and solo makers who don’t ship as frequently as larger teams or need to consistently test on a monthly basis.

The bottom line

Little to no value from the freemium tier and lack of flexibility in the paid subscription tiers. We need flexible pricing tiers so you can incrementally scale up when your ready.

Ease of use

As i touched upon earlier, there are a lot of user testing platforms out there. During my research i found at least 40. In such a crowded space comes competition, and what do product teams typically do to stand out from their competition? They built a ton of features with the hope it makes their product stickier and better than the competition. If only it was that easy.

The sheer amount of features available can be overwhelming and often not required. They are also typically aimed at specialist teams. For most part they are irrelevant for smaller teams and individuals.

The bottom line

Mature products tend to suffer from feature bloat and unfortunately progressively become more complex in their nature. Therefore they have sacrificed UX in the rat race with their competitors and ironically opened up the doors to new competition in doing so!

Recruiting candidates

Recruiting candidates can be one of the trickiest parts of undertaking user testing. You typically have two options:

1. Recruit your own test candidates. This is the obvious option if you have a large network to tap into or deep pockets to acquire candidates by offering some form of financial incentive.
2. Pay the platform to recruit candidates for you (more on that below).

Assuming you don’t have a large network and a line of people waiting to get their hands on your product, you would instinctively be drawn to using the user testing platform’s pool of candidates. The issue with relying on the platform to recruit the test candidates is you don’t know who they are, are they really who they say they are and what is their incentive to actually test your product?

The bottom line

A lot of the current platforms offering to recruit test candidates for a fee typically outsource this work to a third party and take a small cut themselves. This leads to lack of control over the candidates authenticity.

And what’s the candidates incentive, my guess is it’s typically financial. Although there isn’t necessary anything wrong with a financial incentive, I personally wouldn’t want all my candidates lead by this one incentive. Why?

Lets take Usability Hub for example, they charge $1 per tester recruited. The tester gets $0.10 per test and can only withdraw the money until they reach $100. That means they need to complete 1000 tests before they can withdraw their $100. You can imagine the tester is incentivised to get through as many tests as possible to get to that minuscule payout. Not ideal. They just want to get through your test as quickly as possible.

The solution

Through this frustration with the current offerings i’m re-building BetaTesta. A platform that simply pairs makers of digital products with test candidates.

Were on a mission to simplify user testing so makers can spend their time building great products with better insight into how their product satisfies the users needs. We want to demystify user testing and bring it to the masses, making it as simple and effortless as composing a message.

It doesn’t need to be complex! It doesn’t need to take ages! It doesn’t need to be expensive! So don’t let it! Join us today.New homepage design: www.betatesta.xyz

The BetaTesta MVP was originally launched in January 18 on Product Hunt and was extremely well received with 340 registered testers signing up from over 40 countries.

My weekend MVP: using zero code or cash and acquiring hundreds of users

As a company that’s run by passionate makers, we are well aware of the need for a simple and affordable solution. That’s why we created BetaTesta. To serve solo makers, designers, Product Managers, small and medium-sized startups, it’s in our DNA, everything from the features, support, and pricing is designed with you in mind.

We have removed all the unnecessary features and complexities and stripped it back to a simple yet powerful user testing tool for a fraction of the cost.

I can’t wait to show you what we’re building.

For this next phase I will be teaming up with ucreate (the company I work for) to build the next phase of BetaTesta. We will be running the MVP through our new Flash programme, because what better way to test it by running an internal project through it. Staying true to our mission we will be sharing our journey throughout the Flash programme.

Did you enjoy this post? If so I only ask one favour 🙏🏻. In the time it took you to read this post you could provide me with some incredibly helpful feedback by 👉 clicking here 👈

I studied every user testing platform and here’s what I found was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.