After you’ve applied your human centered design skills to understand your customer’s jobs-to-be done, identify their unmet needs, and develop a blueprint for a hypothetical service that meets those needs, it’s time to up the fidelity of your research by creating service experiences that live in the market. A simulated experience allows you to compare what your customers have said they will do to what they actually do. Alpha tests are a way of making sure those two things align before you begin to build and scale a concept.
What’s an Alpha Test?
Simply put, it’s a small start-up. An alpha test is a minimum viable experience (MVX) that delivers just enough elements of a service to make it feel real. You can also think of alpha testing as in-market experimentation. It should be scrappy on the back-end, but look polished to your users; you are still trying to minimize your investment while maximizing your learnings. It allows you to really get into the weeds and understand how customers behave with a proposed concept. Peer Insight uses the risk-investment curve to visualize how we can de-risk opportunities quickly through in-market research. Affordable, off-platform testing provides strategic learning that informs concept development. Alpha tests sit at the end of the research phase where prototype fidelity and investment increases, but ventures are still off-platform and validated learning around the customer need and business model is the highest priority.”
10 Tips For Launching A Successful Alpha Test
There’s no one-size-fits-all procedure for designing an alpha test. However, through our work with clients on launching successful alphas, we’ve identified 10 tips to navigate internal operational obstacles and optimize your learnings! These are some key strategies that every innovation team needs to keep in mind as they prototype experiences with real customers.
1. Consult with legal ASAP.
Your legal team will determine much of what you can and cannot do during the alpha test, so it’s essential to talk to them up-front before you dive into the experience design. Will the experience be branded or unbranded? What channels can you leverage to recruit participants? What agreements need to be drafted? All of these things will significantly impact your alpha strategy, so get that figured out up front.
2. Focus on what you still need to learn.
Assumption testing applies to alpha testing too! Determine what assumptions you still have and think about how they can be confirmed or disconfirmed via the test. This will directly affect the user experience design. You don’t necessarily want to design what you believe to be the ideal experience. Instead, it’s more important to test the key assumptions and aspects of the concept that you’re still uncertain about so you can see how users respond. The alpha experience should be relatively simple in terms of choices for the user. You’ll add complexity later in beta tests, once you’ve zeroed in on the core elements of your MVX.
3. Partner up.
Don’t feel the need to build it all yourself! Partnering with others allows you to move quickly, and therefore, save money. Partners also bring skills, resources, and insights that you may not have on your team. Often, small partners optimize speed and minimize IP risk.
4. Think about your test metrics early.
If you have specific metrics you want to capture, think about how you will measure them. You don’t want to get to the end of your test and realize you haven’t been tracking a variable needed to measure something. Consider ways to automatically capture or calculate data. While alpha tests will be inherently manual due to their level of fidelity, try to make your life as easy as possible by designing in efficiencies where possible.
5. Allow enough time to test your prototype with actual users before launch.
Websites, processes, and prototypes will inevitably have bugs that need to be worked out. It’s best not to let alpha participants feel the pain of error messages, delays, and gaps in processes because it will cloud the core experience you want to deliver and get feedback on. Run the alpha experience with several users (no voiceover!) and find out where they stumble, ask questions, or become confused. When it comes to websites or apps, a full quality assurance check is a must-have. When we design a digital experience we often have the ideal user flow in mind, yet its rare that users actually take that route…assume people will do everything out of order (because they very well may). With that in mind, it’s also important to try to “break” the site. Meaning, go through every possible user flow, across multiple devices and browsers, and discover if something breaks down. Embody your most impatient participant!
6. Have backup plans.
No matter how much quality testing you do, assume that there will be malfunctions, and have plans to overcome them. You want to avoid as much disruption in the experience as possible once the alpha goes live, so it’s important to be able to act quickly if something does go awry.
7. The devil is in the details.
Ensuring things happen as promised…within reasonable timing…without spelling errors or broken hyperlinks…is very important. It gives credibility to the service. Mistakes pull your user’s attention away from the core experience towards the mechanisms that are delivering it, which has the potential to bias their feedback and desire to continue engaging. That said, if something goes wrong, put your customer service hat on. Most of us have been on the other side of the line and know that patience, pleasantry and personalization can go a long way to make a customer happy. Customer service moments are also a great way to solicit direct feedback from a participant.
8. Transparency is key.
Alpha tests don’t start and stop according to our personal schedules, so creating channels to communicate with your team members when you’re not in the office is critical! Your alpha journal is a great start, but also leave notes for yourself and others to keep the process as transparent as possible. Ideally, anyone should be able to pick it up and drive. For example, if you’re using a spreadsheet to document user progress, include fields for initials after critical steps have been completed. If something goes awry or a mistake is discovered, it’ll be much easier to nail down what may have happened by working with the people closest to it. We’ve also been experimenting with Slack, a messaging app for teams, which has turned out to be a great way to communicate with a larger project team, efficiently.
9. Keep an alpha journal.
Keep an analog or digital journal of key activities, events, milestones, and/or revisions that occur on each day of the alpha. Without it, days will quickly blur together making it difficult to recount the experience to those who weren’t intimately involved. When it comes time to analyze the alpha data and you see changes in behavior or unusual trends, it’s important to cross check them against revisions or activities that occurred around the same time to help determine potential influencers.
10. Document everything.
Document any and all feedback that trickles in throughout the test! After all, feedback is what we’re really after in in-market tests. This means capturing all questions and observations participants provide. During the test, feedback may signal whether something needs to be revised and changed, or it just may confirm or disconfirm those make-or-break assumptions you drafted prior to the test. Either way, it’s critical to get it down so it can be shared with the entire project team.
