In Person Concept Testing: Quant + Qual Sample Optimization

While much of concept testing is appropriately done remotely online, certain products require sensory interaction – taste, feel, perhaps minor use, or a close look at design elements — to be properly tested.  In these situations, some form of central location test is required.  In partnership with various companies, MMR has conducted “in person concept testing” (IPCOT) that begins with quantitative research with the needed sample, followed immediately by qualitative research with a subset of that sample based on individual quantitative results.  While this is not “new” in theory with the long-ago introduction of auditorium-style research with keypads/dials (and now smartphones), in some situations our approach provides distinct advantages over the auditorium approach.

Quantitative concept testing has long been a staple of marketing research and by and large does a good job of separating potential marketplace winners from would-be losers.  Recently, a client came to MMR wanting to uncover the “why” as well as the “what” for product change preference.  So, our client asked us to follow up quantitative concept testing research with qualitative research.  An auditorium-style research setting can be efficient as clients can pick from the data patterns seen and invite a small group to a post-quant research discussion (in online concept testing, follow up chats with selected individuals fill this need as well.)  There are, however, instances when this is not ideal, especially when there is a need for sub-group analysis or when larger numbers of people need to view/consider products or prototypes.  This mirrored our client’s situation.

Develop a point of view on which product configuration was most likely to help rejuvenate the brand among both current and prospective users who were in-category buyers.

Recruiting was accomplished as it would be for a qualitative research project, but participants were recruited for seven separate sessions during the day.  Importantly, all participants were told there was the possibility they would be chosen for the second phase of the research (a one-hour discussion group following the product feedback session) and would be paid the additional agreed upon incentive if so chosen.  The quant phase of the research was set up as in the picture below:

In Person Concept Testing Set-up

We have found this “work-station” set-up has advantages over the auditorium-style approach by improving flexibility of testing, creating a better choice set for qualitative follow-up, and helping to reduce production costs.  In this set-up, clients in the back room do not “see” results fly by as products are evaluated by participants; however, a program that generates a quick Excel-based synopsis of key results can deliver near real-time summaries to be used for choosing participants for the qualitative phase.

While the data collection takes a full day as opposed to a few hours, we believe the quality of the results is improved for several reasons:

– More available times for participants: offering more times than just a 5pm or 7pm for auditorium approaches helps limit the impact of self-selection.

– More considered and focused choosing of participants for qualitative follow up: with only 20-25 people to look at each time, a wider set of data for each can be quickly scanned on the Excel-based synopsis.  We find this can be done with confidence in about five minutes and participants do not mind the wait – in fact it gives them time for a quick break/phone check etc.

– Clients have their own respite while the quantitative survey is going on: no need to “follow” the data because it is summarized quickly at the end.  Thus, things like conference calls, emails, and the “work back at the office” can easily be attended to as needed.

– More conducive setting for qualitative follow up: the research generally takes place at a focus group facility with all the amenities instead of requiring a hotel or theater setting.

Actionable Results
Product winners were clearly identified via the concept testing.  While samples of 250+ are not usually generated with central location testing, they CAN be accommodated whenever timing and budget associated with recruiting allows.  In this case, we sampled over 200 customers and non-customers across two cities who shared targetable characteristics.  We tested 11 different product types/configurations and 2 were declared the winners based on quantitative results and the qualitative follow-up.  This was important because multiple product versions could be produced. Qualitative follow-up told us why particular products were or were not appealing or performed/did not perform well.  The follow-up helped break ties among quantitative winners and generated specific consumer language that helped in the development of messaging for the product roll out.

Ultimately, the IPCOT approach proves to be cost effective and offers lots of flexibility for product development prototype research that requires sensory experience.