Research on Research at IIeX

As always, IIeX was a great experience for the entire Recollective team and for me, personally. We made some great connections, learned a lot and participated in collaborative problem solving. Part of the experience for us was hosting a roundtable with people who participate in marketing research studies. We did this to further our ongoing conversation about what our industry is like for them. Our goal is to create a more open dialogue between our side of the industry and theirs; the hope being that we can work towards some foundational shifts in our thinking and find some solutions to our shared problems with quality and trust. First, as a reminder, this wouldn’t have been possible without the help and support of our industry. I’d like to give a personal thank you to:

• Jason Horine with Schlesinger Associates
• Dennis Meng with User Interviews
• Scotty Greenberg with Tango Card
• J.R. Venza from Critical Mix
• Chris Stephenson from Prodege MR
• Tom Anderson with Odin Text Analytics
• Gabrielle Ferdman and Roger Mendez with Propeller Insights
• And the entire Cross Tabs team.

While Jessica Broome and I conceptualized and brought this to life we never would have been able to do this without help… lots and lots of help! To bring you up to speed - we first conducted an online community among people who participated in multiple kinds of research. To read the initial blog posting about those findings, please follow this link. Next, we ran a 1,500 person quant study to further explore and validate what we learned in Phase 1, but also to explore a few theories we came up with. Our thinking was that we might want to be considering or profiling things like creativity levels, empathy levels and learning styles of the participants as we are developing our questionnaires and think through how we recruit for different kinds of methodologies. We were fortunate enough to be able to present these findings at IIeX. If you missed our workshop, but would like to see the presentation you can read through our findings here. The roundtable discussion, with actual research participants, proved to be as fruitful as the first two phases. We had six participants recruited through a variety of sources and incentivized by Tango Card. Each of the participants was, again, recruited to have participated in multiple kinds of research. We learned so much from them, but there are definitely themes that have emerged across these three phases. Some of them are:

• They are all what we consider to be professional respondents. There is definite cause and effect going on, though. They do not feel informed or respected by the way we screen or the information we provide them around how we select them.
• The screening takes up too much of their time - so, they feel misled and therefore they try to ‘game’ the system.
• Additionally, they often feel in the dark about why we ask what we do, so they try to give us the answers they think we want. This creates an environment where, at the point of screening, they fudge… not lie, but ‘fudge’. Sometimes, though, this is because they can’t come up with accurate answers to the ‘impossible’ questions we ask.

It’s my fundamental belief that we, as an industry, already know this and just don’t know what to do about it. So, I throw out a few thoughts and hopefully a foundation for some solutions.. If we can’t beat them, join them

• Instead of trying to trick them through complicated and impossible to answer screening questions, can we be more transparent about what we are looking for upfront?
• We know they don’t like to participate in exercises where they feel like they don’t have a lot to add, so if we told them what we need and why (as much as we can), maybe they wouldn’t feel like screening was a chess match and it was more of a conversation. We could, potentially, also avoid scenarios where they think we want an expert on a topic so they educate themselves before the research to be helpful, when we actually wanted a novice.
• It’s also obvious that these aren’t inherently manipulative or dishonest people, they want to help - so let’s tell them how they can in a straightforward fashion.
• While there are few things more important to them than the monetary incentives, there are some things that are equally as important - they love seeing the results, it doesn't have to be the final results, but just enough to even know where they sit in the group or in the survey. We definitely could be doing more of this.
• We need to talk to them more - about us; who we are, why we need them, why the truth is important… we can do this in an automated fashion - embedding video into screeners or surveys. Let’s entertain them and help them understand us.
• Ultimately - it comes down to respect, transparency and cooperation. These things are the foundation for all good relationships. This is no different.

Jessica Broome will have more on this and we’ve got Phase 4, 5 and 6 in the hopper. In the meantime, we’d love to hear from you about what you’d like to learn, so please reach out and let’s further the conversation.