First thing to get out of the way… your users “don’t” understand BETA. This means that when they visit your business, if it doesn’t solve their problem they won’t come back. They don’t care if its Beta or Gamma. That is why BETA must not be approached as a way to get some feedback while work is in progress, but as a way of having a controlled situation where your objective is to learn as much as possible about the not-so-obvious motivations of your clients.
We have been “in Beta” since May and now we approach BETA in a different way. Mainly dividing it in two phases. This is what we’ve learned:
- Phase 1 or “the pat in the back”: the objective is to fix the most obvious problems. The idea is to get to a point where you have got the absolute minimum in order to “solve their problem”. We must use this phase to understand what aspects of our design work the way intended or not. For example, Giftybit is based in social interactions. One of our key objectives during this phase is to understand if the messages posted on the social networks or the mechanics, first of all work correctly and secondly, people are comfortable using them. The result of this analysis would be, for example, to change the format of the messages that go out (ie. we send a private message instead of posting it on the wall) or even how the text is written. We also make sure the payment platform works correctly under any circumstance or that the navigation makes sense and isn’t confusing.
* Whom should you get to participate in this phase? Go for the easy solution. Send some invitations to your friends and family. But don’t forget two things:
a) Try to match as much as possible your costumer segment. If you aim towards the young adult segment, don’t get your granddad to check your BETA. Even if he is surprisingly computer savvy he’s point of view will probably make you take decisions that maybe aren’t the best. If he insists, for gods sake let him in, just take his feedback with a pinch of salt.
b) Remember that this phase is NOT a proof of concept. Your objective is to make sure everything works as intended, not if it’s a good or bad idea. You should solve that variable with REAL costumers, not family and friends that will first, say everything is wonderful and secondly, in an extreme willingness to help, give you many more ideas to make it work “better”.
Once we are comfortable about having a solution that is understood by the client and “works” correctly on every aspect is time to work on phase two, with REAL strangers.
- Phase two or “the stranger who loved me” is the true costumer development project. What do I mean by costumer development? Getting information that will help you narrow as much as possible your costumer segment and understand what is the “real” problem to be solved.
During this phase the critical aspect is to get as much information as possible. And by this I mean not only quantitative information (google analytics) but, most of all, qualitative information.
There are many ways to get this information, but with Giftybit we are going to use three methods:
- Online Help – We are implementing a live chat in order to help costumer exactly when they have a problem and get to know them better. (http://www.olark.com/)
- Real-time feedback – Also with the chat online we are implementing a tool in order to make context surveys along the navigational process of the web. (https://qualaroo.com/)
- Key players follow-up – And finally, what should be the most valuable, we are establishing a procedure in order to make direct contact with those clients that decide to sign-up for the first time, understanding their reasons, and also almost as important we are contacting those that leave to understand why.
The important thing to understand is that you are looking for the motivational aspects that make the costumer do one thing or another. Most probably if you ask directly you will only get a cookie-cutter answer so you have to structure your research in a special way. Again, you have to reach the underlying motivation.
I know, this sounds a bit abstract so here you have a couple of posts of much more experienced people than me that may help you:
* Whom should you get to participate in this phase? For us Google Adwords has worked great. It has helped us, not only to get “strangers” that could be potential clients but also helped us to A/B test different messages, wording, tone, etc. (go for 20 or 30 people, even less if you see that its taking longer than you wish. HINT: It could be your landing page design or selling pitch)
Hope this helps and Happy New Year!
UPDATE: As noted by @diegomarino and @david_bonilla, as important as qualitative information is, it is also crucial to get as much quantitative information as possible to understand true client behavior. For that, there are tools like http://www.kissmetrics.com that make implementation a breeze. (In our case we are using self recorded metrics, but will also implement kissmetrics to make user we aren’t missing anything)