BellaVia Research: Mozilla: When to use focus groups

To Focus Group,
or Not To Focus Group:
This is the Question

Julie Francis / BellaVia Research
July 2012 / Presentation to Mozilla
1

Who Am I? Julie Francis

 Founder and principal of BellaVia Research
 20 years experience
 Market Research Background & Usability/UX since 2000
• Creative Good (NYC & SF): online UX strategy consultant
• Before that: Socratic Technologies, Audits & Surveys Worldwide
 Range of work:
• Usability & User Experience
• Mobile / online / multi-channel
• Contextual inquiry / “ethnography-lite”
• Persona creation
 “Agnostic” when it comes to methodology
• What is the best way to answer the research question?
• What methodology best fits budget, need for observation, desired geographic reach, timing, etc.
• Huge range of methodologies / approaches

 Arizona Public Service  Diageo  Sony
 Arhaus.com  Groupon.com  Southern California Edison
 Art Institute of Chicago  Interbrand  StubHub.com
 ArtThatFits.com  Motorola  thebar.com
 Bank of America  OPOWER  Walmart.com
 Busch Gardens  Planned Parenthood  Wetpaint.com

BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 2

How we’ll spend our time today…

 60 minutes

 ½ me talking
• Level setting – What the heck is a focus group?
• When to use focus groups
• When not to use focus groups
• About recruiting….

 ½ us talking together
• What‟s on your mind?


Research Methodologies: Decisions, Decisions

 We’re inundated with methodology options…
 Qualitative Research approaches:

 In person
• Focus groups
• IDIs / one-on-one interviews
• Mini-groups / Dyads / Triads
• Contextual? In-lab?
 Online/Remote
• Bulletin boards (open or closed)
• Diary studies, including mobile phone / text diary studies
• Remote moderated phone/online
• Social media “listening”
 And more…

 Hybrid methodologies
• Infinite combinations of the above
• Hybrid is the current trend –
No methodology is perfect.
All have strengths & weaknesses…
How to choose?
• a hybrid approach lets us
capitalize on upside of various approaches


Baseline: What’s a Focus Group?

Focused Group Discussion
 Moderated (trained human being)

 1.5 – 2 hours (usually)

 6-10 people (usually)
• Historically groups were larger (10 – 12)

• Current trend: Smaller groups (6-8 )

- Why? Deeper inquiry (the whole point of qual)

• Also Dyads & Triads (go really deep, but still get the benefit
of dialogue you miss in a one-on-one interview…)

(M&Ms are the official food of focus groups)

BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com Photo credit Willie Lunchmeat 5

Baseline: What’s a Focus Group?

 Pre-recruited participants
• Professional recruiters (facility or independent)

• Client or facility-lists

• Strangers (usually)

• As with any kind of research, good recruiting is a must
(weed out “cheaters & repeaters”)
(be careful with Craig‟s List & Friends/Family recruiting)

• Recruited at least in part by phone (online surveys & email
blasts may also be involved) (definitely use the phone –
casual conversation helps ensure the person is a good fit
and helps identify cheaters)

• Over-recruit in anticipation of no-shows

 Paid to participate (almost always)
• Lures „em in, & thanks people for their time

• Helps with “show rate”

• Addresses participation bias  Tangent: for bulletin board and other online/written research,
(We want busy people not just broke people) use a written open-ended screening question for articulation.

• $75 & up Can the person express themselves well in writing with
minimal prodding?
• Better incentive  better show rate


Baseline: Logistics – Lab with Conference Table

 Most groups are held in research facilities using a conference table setup & a one-way mirror…

Got lots of written exercises or
paper to juggle?

Use a conference table setup…

watchLAB

One way mirror in facility 
lots of real time observation
(get the most bang for your buck by inviting team
to watch real-time)
Budget conscious? Research doesn‟t need to
be “blind”? Use your in-house lab.
In rural markets we sometimes set up in hotels. Amplify, Pleasanton


Baseline: Logistics – Lab with Living Room Setup

 There‟s a growing trend towards Living Room Setups at research facilities

watchLAB

A living room setup can feel more comfortable &
less foreign to people who aren‟t used to sitting
around conference tables.
GREAT for body language
(much of which is hidden by conference table)
watchLAB, San Francisco
Not good if there are lots of papers to juggle
(unless a clipboard will suffice)


Baseline: Logistics – in Context

 A lot of research (including focus groups) takes place in-
context, in the “real world”
• Street intercept -- a great way to find “fresh” participants
(beware of intercept bias)
• In-homes / Contextual – always good to see people in context –
Friend Groups, Mom‟s groups

• But… you lose the ability for your team to observe (one of the
greatest benefits of research)

TecEd, Inc. Ethnographic Field Study

Susan Hosking & Jon Littel
(photo from MomsSupport.org)


Overheard Recently…

“I don‟t believe in focus groups”
(followed by)

“I‟m thinking about getting
together a group of people
to talk about this topic.”

 (Um…call it what you will…
but that’s what I call a focus group)


Focus Groups are getting a bad rap

“Focus groups are
dead. The paradigm
has shifted.” “it‟s observational
vs. directed market
research”
“awkward tactic”

 Do they deserve this bad rap? IMHO,
most “problem groups” stem from
unrealistic expectations, improper use,
DIY moderators who lack training/
experience
 Clients tell researchers “I want to do some
focus groups” when what they mean is “I
want to understand my target” (and focus
groups aren’t always the right fit)

Let’s not bury focus groups

All approaches can work well
when used appropriately

There is a role for focus groups

It can be incredibly insightful to hear
a group of people banter about your
product/service/category

See Francesca Johnson, Research Arts:
http://www.researcharts.com/2011/01/in-praise-of-focus-groups-and-moderators/


When to use focus groups?

 IMHO, Focus Groups are best when you want/need to:

Explore needs / Explore
thoughts / feelings Explore brand consumer
perceptions language / issues
(fuel for innovation, not (yours, competitors)
iteration) (to inform/explain survey
research)

Understand Uncover what Explore use
decision process you don‟t cases
/ factors know… (why, where, when, &
how…)


Focus Groups are not my first choice for…

 IMHO, Focus Groups are not the best fit for…

Usability
Advertising impact /
(product / website / Design iteration
communication
mobile – don’t do it)

Understanding Situations where
what people really you worry about
do (instead of what people influencing
they say they do) each other

 Sometimes we use groups anyway. Why?
 Efficiency / Speed
(You can interview 24 people in groups much faster than you can in 1:1)
 Budget (if it’s faster, it’s often cheaper too)
 Client comfort (Clients ask for focus groups because it’s what they are used to)


Focus Groups are Best When…

They are open & exploratory

• You don’t know what you don’t know
• Use projective techniques to get below the surface (don‟t knock it until you‟ve tried it!)

Banter happens!

• You use the group dynamic to its full advantage! (and use a trained moderator, to help
manage the chaos)
• Let people interact, banter, build on each other‟s ideas, trigger each other with
experiences/questions, etc. That‟s where some of the richest insights come from…

The guide isn‟t too packed

• Allow conversation to happen! Give people time to talk, banter, build, interact – let the
magic unfold
• Litmus test: Aim to wonder “Uh oh, is this enough to fill 90 minutes?”
• (FYI, we’re seeing a trend towards clients packing guides…which diminishes our ability to
get good quality insights.)

There is no counting

• It‟s not quant. It‟s qual. It‟s all about what drives opinion (the WHY), & the range of
opinion/experience. It‟s not definitive. It‟s often followed by quant research, for good
reason.


What do I mean by projective technique?

 One example: picture selection
• i.e., “Which of these images best describes how
you feel about _______________”? How and why
does that fit? (then probe, probe, and probe some
more)
• This seemingly bizarre approach is brilliant for
cutting through all the superficial, high level
words down to the core issue. Trust me, it‟s
brilliant.
• Each pictures should have multiple interpretations.
Avoid literal images
• istockphoto is great for abstract searches (but the
pictures can be really expensive, and it drives me
CRAZY that their photo credits expire.
• Not just for focus groups; Works great in 1:1
interviews, too. Also, works both online & in-person.
I‟ve used them in all sorts of methodologies with great
success.

 There are lots more projective techniques, but
this is one of the most common and least
terrifying to use.


How to Choose a Methodology?

 Always start with your objectives: What are you trying to learn?

Corporate See behavior Consider
User Tactical
Usability Identity (logo, in action
Experience Feedback
name) (observational)
IDI (1:1)

Explore drivers /
To help you
Understand Consider
When you don‟t barriers / behavior/decision
Fuel for innovation know what you perceptions /
prepare for quant
drivers, longitudinal Focus
research (identify
don‟t know attitudes / brand behaviors Groups
issues & language)
perceptions (reflective)


More Methodology Considerations

 How important is team observation?
• In-lab can be more efficient (complete more sessions per day)
• You think going to a focus group is weird? Having 2 researchers & a videographer come into your home is even weirder!

Observation Less important – Observation More important –
do your research in-context do your research in a lab

• If you don‟t have observers, you can go in- • Labs are great for team observation. When else do
context! In-homes, street you get to spend a full day as a group in the presence
of your customers?
• Use a videographer
• Cram up to 20 observers in the back room. Great
• Consider a highlights reel (people have bang for your research buck. (facilitate the
good intentions to watch videos, but they observers!)
don‟t) (mic your participant)

Best practice:
Hybrid! (capitalize on the upside / strength of each approach)

 Other considerations: Timing / budget / need for geographic reach


Focus Group Moderating Toolbox

 What‟s in my toolbox?

Moderating to the Max: A full-tilt guide to Dominators, Cynics & Wallflowers:
creative, insightful focus groups and depth Practical Strategies for Moderating
interviews (Bystedt, Lynn and Potts) Meaningful Focus Groups (Kahle)
• My favorite reference book on “how to” for projective • Moderating groups is really, really hard.
techniques • This book is a lifesaver for dealing with group
dynamics.

Secrets of a Master Moderator Riva Training Institute: Hands-on Training
(Henderson) • Moderate in front of masters & your peers, get
• Naomi Henderson is a brilliant moderator, & the critiqued (rinse, repeat)
founder of the RIVA Training Institute • Rockville, MD (unless you organize a custom course)
• Fundamentals of Moderating course: 3 days:
$3600, worth every penny (really)

QRCA (Qualitative Research Consultants Impulse Survey of Focus Facilities
Association) • Find the best facilities in every market! Rated by
• Webinars recruit, location, value, etc.
• Free QRCA Views Magazine • Aim for an overall rating of 85 with lots of reviewers.
• Find a Researcher • Online version is $65
• Great annual conference (research suppliers only)


A word on recruiting…

 You didn‟t ask me about recruiting, but I can‟t resist a chance to talk about it
 Getting harder & harder to find quality participants who truly qualify (and aren‟t oddballs)
 Lots of “friends and family” research – is that really your target?
 Craigslist can be a problematic source! Easy to find warm bodies, but hard to find quality recruits
• Tend to be in it for the money (why else would you be trolling for focus groups on CraigsList?)
• Lots of dishonesty -- i.e., Woman in SF with >75 aliases, fake Facebook profiles, etc.
• According to one recruiter, 50% of her leads from Craigslist try to fake their way into research.
• Be careful! Many professional recruiters have database they build in a variety of ways – use them, use your own customer
lists, and use Craigslist sparingly & carefully (i.e., for niche recruits)

It‟s very easy to “lie” your way into
research when the posting or email
blast says exactly the type of
participant they are looking for…


Tips for Finding Great Research Participants

Open 2-way
Use carefully created Find cheaters / Ensure qualifications &
communication with the
materials! repeaters engagement
recruiter
• Use a professional • Hide your topic in the • Does your recruiter • Maximize open-ended
(it‟s worth it. Budget email blast (approve have a “wall of screening (engage in
$75-$150 per or write it yourself) shame”? Shared with conversation, don‟t
consumer recruit, more • Hide your topic in the other recruiters? give them a list of
for specialized B2B) pre-screener survey • Use “dummy” brands/behaviors to
• Bring the recruiter onto (approve or write it questions to sniff out click on)
the project team yourself) dishonesty (ask about • Make it easy for
before you start work • (don’t bias your flavors / features that people to be honest
on the screener research before it don‟t exist (i.e., “we‟re looking for
even starts. People • Require ID & write a mix, so please be
• Explain the type of do, and will, cheat check to that exact honest”)
person you are looking their way into name • Re-screen: confirm
for, & agree upon
strategies to find
research) • Verify identities key questions (pretend
• B2B: Call employer it‟s important to “get it
these people exactly right,” i.e., “I‟m
• (In the dog example: and ask for them by
• Agree what “research with pet title (esp. if they sorry I didn‟t take good
sources/approaches owners” should be won‟t share their notes on this”)
will & will not be enough, especially if company email • Homework! (send me
used (i.e., when/how you make people address) a picture of your music
to use Craig‟s List, collection, see
prove they have a • Look at their
email blasts, Survey dog via lots of Pinterest page, see
LinkedIn Profile (but
Monkey) pictures, & bring their your kitchen, a
beware of fake
• Include original treats into the profiles). For senior screenshot of your
participant source on group… pictures of execs, look at the iPhone apps…)
recruiting profiles dog’s gnarly teeth?) corporate website.


Great Recruiters

Hagen Sinclair

•Cynthia Hagen-Cross, President & CEO
•National recruiting, general pop & hard to find niche populations, online & in-person methodologies
•www.hagensinclair.com/ / hagenc@hagensinclair.com / (530) 899-3140

Blarry House Research

•Annaliese Furnas, CEO
•National recruiting, general pop & hard to find niche populations, online & in-person methodologies
•www.blarry.com / annaliese@blarry.com / 415-648-2196

Satellite Research

•Claudia Page, Owner & Research Specialist
•National recruiting, general pop & hard to find niche populations, medical, online & in-person methodologies
•www.satelliteresearch.net/ / claudia@satelliteresearch.net / (888) 639-5312

Intact Qualitative

•Richard Ngo, Partner
•SF recruiting only (& SOMA loft facility on 3rd Street)
•Great for SF Bay Area in-home or contextual interviews
•www.intactqualitativeresearch.com/ / richard@intactqualitativeresearch.com / 415.400.5945

Your Research Facility

•Doing research in a hired research facility? Use their team
•First, check Impulse Survey to make sure they are highly reviewed on recruiting


Your Turn!


Want to talk methodology?
Need a recommendation on a recruiter,
videographer, transcriptionist, facility…?
Ping me!

Julie Francis
Principal & Founder, BellaVia Research
www.bellaviaresearch.com
@BellaVia
831.454.8217

BellaVia Research ©2011 24

BellaVia Research: Mozilla: When to use focus groups

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (11)

Último

Último (20)

BellaVia Research: Mozilla: When to use focus groups

Notas del editor