[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

BUILDING A GROWTH MACHINE
THE METHOD BEHIND THE MACHINE
!
Brian Balfour :: @bbalfour :: http://www.coelevate.com

WANT TO KNOW THE #1 QUESTION I’M
ASKED ABOUT GROWTH?

WHAT TACTIC WOULD YOU USE TO GROW
MY COMPANY?

4

ONE…
TIP…
SECRET…
HACK…
TACTIC…

THE ANSWER TO…
IS NOT WHAT MOST THINK IT IS
WHAT TACTIC WOULD YOU USE TO GROW
MY COMPANY?

IT HAS NOTHING TO DO WITH TACTICS

IT HAS EVERYTHING TO DO WITH PROCESS

PROCESS?!?!?!
WHAT CAN BE SO SPECIAL ABOUT PROCESS?

4 REASONS TO FOCUS ON
PROCESS FIRST, TACTICS SECOND

12
1. WHAT WORKS FOR OTHERS, WON’T
WORK FOR YOU

YOUR AUDIENCE IS DIFFERENT

YOUR PRODUCT IS DIFFERENT

YOUR BUSINESS MODEL IS DIFFERENT

YOUR CUSTOMER JOURNEY IS DIFFERENT

YOUR BUSINESS IS DIFFERENT
PLAIN AND SIMPLE

18
2. GROWTH = SUM(1, 2, 3, 4….X)

What happened here?

20
SILVER BULLETS DON’T EXIST

21
3. RATE OF CHANGE IS ACCELERATING

SEO No Keyword Data, Author Rank, Content
Freshness, Social Shares, Search Result Format
DISPLAY RTB, No Cookies, Mobile
EMAIL Tabbed Inboxes, Inbox Actions, Unsubscribe,
Mobile
FACEBOOK
Newsfeed Ads (4 formats), Video Ads,
Optimized CPM, CTA Format, Custom Audiences,
etc, etc

CHART CREDIT: JAMES CURRIER

24
4. YOU NEED A MACHINE

A MACHINE IS…
1
2 PREDICTABLE
SCALABLE
3 REPEATABLE

THE MACHINE PRODUCES THE TACTICS
THE PROCESS MAKES THE MACHINE

SO LET’S SEE THE PROCESS…

THE GOALS OF THE PROCESS
1 RHYTHM
Momentum is powerful. Establish a cadence to ﬁght
through failures, get to successes, and ﬁnd
momentum
2 LEARNING Constant learning of our customer, product, channels
and feeding that learning into the process to improve.
3 AUTONOMY Individuals decide what they work on to achieve the
team goals.
4 ACCOUNTABILITY
With autonomy, comes accountability. You
don’t have to be right all the time, but there
is an expectation to improve.

THE FIRST STEP: KNOW WHERE
WE ARE GOING

HOW WE SET GOALS
OBJECTIVE: Qualitative statement
!
TIMEFRAME: 30 - 90 Days
!
KR1: Measurable Goal 1 (Hit 90% Of Time)
!

ONCE WE KNOW WHERE WE ARE GOING WE
HIT THE ROAD AS FAST AS POSSIBLE

6. Systemize
(Playbooks)
1. Brainstorm
(Backlog)
2. Prioritize
(Experiment Doc + Backlog)
3. Test
(Experiment Doc)
4. Implement
5. Analyze
(Experiment Doc)

FOUR KEY DOCUMENTS
1 BACKLOG The backlog of experiment ideas.
2 PIPELINE Experiments that were ran or are currently running
and the highlights of their results.
3 EXPERIMENT DOC Objective, Hypothesis, Experiment Design,
Results, Analysis, and Learnings
4 PLAYBOOKS Step by step guides for successful
experiments that should be repeated.

STEP ONE: BRAINSTORM
BRAINSTORM ON THE INPUTS
NOT THE OUTPUTS…

ACTIVATION RATE = STEP 1 CONV * STEP 2 CONV * STEP 3 CONV
IF WE WANT TO
IMPROVE THIS
WE BRAINSTORM ON THESE

4 WAYS TO GENERATE GROWTH IDEAS
(Read The Innovators Solution)
1 OBSERVE How are others doing it? Look outside of your
immediate product space. Walk through together.
2 QUESTION Question brainstorming. Why? What is… What if…
What about… How do we do more of…
3 ASSOCIATE Connect the dots between unrelated things. i.e What
if our activation process was like closing a deal?
4 NETWORK Growth mastermind groups. Build your network of
growth people. I’m on the phone every day for ideas

CREATE A BACKLOG

STEP TWO: PRIORITIZE
1
2 IMPACT
PROBABILITY
3 RESOURCES
LOW - 20%, MEDIUM - 50%, HIGH - 80%
Comes from your hypothesis. Take into account
long lasting eﬀects vs one hit wonders.
Marketing, Design, Engineering
1 Hour, 1/2 Day, 1 Day, 1 Week

CREATING A HYPOTHESIS
IF SUCCESSFUL, [VARIABLE] WILL
INCREASE BY [IMPACT], BECAUSE
[ASSUMPTIONS].

JUSTIFYING YOUR ASSUMPTIONS
1
2 QUALITATIVE
QUANTITATIVE
3 SECONDARY
Previous Experiments, Surrounding Data,
Funnel Data
Surveys, Support Emails, User Testing
Recordings
Networking, Blogs, Competitor Observation,
Case Studies

CREATE EXPERIMENT DOC

STEP THREE: TEST
WHAT IS THE MVT
(MINIMUM VIABLE TEST)
TO UNDERSTAND OUR HYPOTHESIS?

STEP THREE: TEST
WRITE THE EXPERIMENT DESIGN IN EXPERIMENT DOC

STEP FOUR: IMPLEMENT
GET SHIT DONE

STEP FIVE: ANALYZE
1
2 ACCURACY
IMPACT
3 WHY?
How close to your hypothesis?
The most important question you can ask. Why did
you see the result that you did?
What were the results of the experiment?

STEP FIVE: ANALYZE
LEARNINGS + ACTION ITEMS IN EXPERIMENT DOC

STEP SIX: SYSTEMIZE
1 PRODUCTIZE Productize as much as you can with technology
and engineering.
2 PLAYBOOKS For the things you can’t productize, build into step
by step playbooks to make them repeatable.

6. Systemize
(Playbooks)
1. Brainstorm
(Backlog)
2. Prioritize
(Experiment Doc + Backlog)
3. Test
(Experiment Doc)
4. Implement
5. Analyze
(Experiment Doc)
REPEAT

WHAT A TYPICAL WEEK LOOKS LIKE
Monday Tuesday Wednesday Thursday Friday
Growth Meeting
- Personal Learnings
- Goal Review
- Experiment Review
- This Week’s
Experiments
Predicting, Prioritizing, Implementing,
Analyzing, Systemizing

ZOOM OUT (set OKR’s)
ZOOM IN (4 - 12 weeks)
ZOOM OUT
ZOOM IN
ZOOM OUT….

MACRO OPTIMIZATION
QUARTERLY REVIEW
1
2 ACCURACY
BATTING AVG
3 THROUGHPUT
How many successes to failures? Is it improving
over time?
Are your hypotheses getting more accurate?
How many experiments are you running in a
given time period? How do you do more?

THANKS!
MORE ON MY SITE: HTTP://WWW.COELEVATE.COM
BRIAN BALFOUR
VP Growth @ HubSpot
Previously EIR @ Trinity Ventures, Co-Founder @
Boundless, Co-Founder @ Viximo (acq by Tapjoy)

[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to [500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments

Similar to [500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments (20)

More from 500 Startups

More from 500 Startups (20)

Recently uploaded

Recently uploaded (9)

[500DISTRO] The Scientific Method: How to Design & Track Viral Growth Experiments