Imagine conducting the following market research project:

You create a 20-minute survey and field it to a database of your customers.  You receive approximately 1,000 responses.  You perform all types of analysis on the data, from cross-tabulation to regression, even multivariate procedures.

What if I were to tell you that the most insightful learning from the project wasn’t contained in the data set at all?

When survey invitations are sent out via email, the recipient can often hit “reply” and send a message that is delivered to an actual person  (as opposed replies being disallowed by the occasional “do-not-reply” indicator).  Frequently, I’ve been involved in studies where the survey recipient chooses to reply to the invitation and offer all sorts of unsolicited feedback.  In the instances where this type of activity has occurred, I have found that unsolicited information to be extremely insightful.

Why would someone choose this method of feedback delivery?  Potential reasons:

  • The survey didn’t offer any built-in options for offering open-ended feedback.
  • The survey didn’t hit on the appropriate pain points experienced by the customer. Alternatively, it didn’t hit on the appropriate drivers of satisfaction.
  • The survey was too long. In these cases, respondents will sometimes drop out and just choose to tell you what they want you to know in the email, rather than complete the survey itself.
  • The respondent feels so passionate about the topic that they want to reinforce their points directly.

Here are suggestions to avoid a flood of “reply” responses next time you launch a survey via email.

  1. Include questions that allow participants to enter open-ended text responses, and include them early in the survey. Usually, the first item a respondent brings up is the most important to him/her.
  2. Shorten your surveys, OR be honest and upfront in the invitation about how long the survey should take the respondent.
  3. Conduct screening research first. Create your survey, and send it to a handful of people (maybe 50 or so) to see what type of response comes in.  If 10 people take the survey and 5 of them complain about it, you know you need to adjust something.

A final option is to not allow the respondent to reply to the email.  I do not advise this going this route. If your customer can’t get his/her point across in the survey, and then is denied the opportunity through an email, s/he may look instead to social media or review sites.  Then, more people learn of the customer’s complaint (which may be presented with more frustration than it otherwise would have been), potentially forcing public corrective action on your part.

#####