Connect, Share, Participate and Learn

Template / tips for Annual Product Review

I am doing a round of interviews with existing customers of one of my products. I am trying to answer the following:

what should I spend my 2012 roadmap budget on to keep these customers happy and increase their chance of up-sell / cross-sell
what should I spend my 2012 roadmap budget on to win new customers (in the same segment as these existing)

I have done this informally - but now have opportunity to speak to about 10 different customers over the course of a week or two - which I am very excited about! So I want to maximize my use of their time.

I have a bunch of questions I want to ask but what I am lacking is structure / agenda - I am hoping other people on here have done this before and might have templates, tips, blog posts etc that they can share? This is the first time I have done this on this scale so all help appreciated!

I want to cover all bases e.g. features, integration, reporting, pricing, maintenance, sales engagement etc

Tags: research, tools

▶ Reply to This

Replies to This Discussion

Permalink Reply by scott willeke on November 9, 2011 at 8:42pm

I don't have a good agenda/template for this, but I'm looking forward to the responses from others here. I know we have some members who can help!

I did some looking around and while I didn't find anything with a template or agenda, I found a couple things that might be useful for others:

▶ Reply

Permalink Reply by Jon White on November 10, 2011 at 9:13pm

Thanks Scott.

The first article there is interesting, I will have to try that next time I am trying potentials, sounds a lot simpler than my current method of LinkedIn InMail (with it's 5% response rate!).

This time around however these are scheduled meetings with existing customers. First one Monday, I will post my learnings and then the template I ended up sticking with.

▶ Reply

Permalink Reply by Jon White on January 9, 2012 at 6:00pm

So here are the things that worked for me that you might find useful. I have a few more in the back pocket so am going to expand on this and turn it into a blog post - but for now..... This is what I learned over the course of 15 or so interviews - your mileage my vary.

No Surveys

A mantra I learned from ProductCamp - ‘Qualitative research is for discovery, quantitative is for validation’. Very applicable here. I am interested in discovering new features, applications or even markets. Interviewing (either in person or on the phone) is a must. By all means structure the interviews and use a consistent format - this will help with analysis, but make sure your questions are open ended. Two of the participants did choose to answer the request by email - this feedback was very difficult to interpret. There was very little discovery and I was unable to ask why?. One I ended up disregarding altogether.

Be comfortable with uncomfortable silence

It is natural to fill gaps in conversation when the words run out. I learned that by leaving this to your participants, instead of filling the gaps yourself, you will get more out of the time. Give 5 or so seconds after a person has ‘finished’ their point before you follow up - I often found there was more to come. This applied especially to group conversations - often people will expect others to say the first/last word, when it doesn’t come they will fill the gaps themselves.

Second hand information is biased

For existing customers, typically you will go through a sales person or account rep in order to schedule a meeting. Sometimes you will get pushback - Sales are very protective over their accounts and the last thing they want is for you to put your foot in your mouth with them not around to cover. Often they will offer to answer your questions on behalf of the customer. Here is the problem - you will get a very different set of answers. Try it. This is not necessarily because they do not understand their customers (although this is sometimes the case).

Everyone has personal bias (including you). It is very hard even for me to try and minimize this and be objective about your research - sales are not even trying to do this when they talk to their customer. They are (in most cases) coin-operated to land short-term revenue within their assigned accounts. You are tasked with creating long-term value in a market. The conversations they have with a customer will be very different from the conversations you have. Speak to sales you will end up talking about the last conversation they had with the customer, or the feature that will bring them the quickest revenue uplift. This is OK - the reason salespeople are paid like that is because it works. They are optimized to maximise revenue, and without revenue there is no business. The best performers are the lifeblood of your company.

However for the purpose of research this does not work and may skew your results. I would not include second-hand information for analysis. This is not to say it doesn’t have value, but a different type of value.

When I first went from development to product, very few sales people trusted me with their customers (why should they? I had not track record). Over time this has improved. I like to think that if you do your job right, eventually the sales team will see your research as an asset - both for long-term growth and also customer relationship.

Make sure you can walk the walk

Before you embark on this exercise ask yourself what you are going to do with the feedback. If your development team is booked solid on other projects for the next 6 months, is now the right time? You are speaking to real customers who pay you money - they will be happy to talk to you but they will expect something to be done with their feedback. I set the scene with each customer by explaining to them that while I will listen to all suggestions, we cannot build all of them. Take notes and follow up, customers will really appreciate it when you tell them you are acting on something they suggested.

Set the expectation of ‘why’?

I find one of the most rewarding things about my job is that moment when you truly understand why a customer has a particular opinion, as in the true underlying cause. In one case a customer was dead set on a video version of some content we provide - this seemed strange to me - after some digging it turns out they had purchased a new CMS and video was particulalrly easy to integrate. But it re-framed the problem from ‘you should create videos’ to ‘I want something that is cheap and easy to integrate with my existing infrastructure’. The latter allows for creative problem solving - the former does not.

The problem is that asking ‘why’ can be interpreted in many different ways - including ‘I disagree with and want you to justify your position’. This is not a good place to be on a customer call. I find setting the expectation upfront solves this. Here is part of the intro script I used:

‘my aim in this meeting is to listen. I will probably ask ‘why’ a lot – that is not because I am questioning your opinion or feedback, I am genuinely trying to understand why things are important and discover underlying problems that can be solved. Please be honest’