SlideShare una empresa de Scribd logo
1 de 169
Descargar para leer sin conexión
Lean Analytics
Lean Startup conference
San Francisco, November 16 2015
@acroll
Don’t sell what you can make. Make what you can sell.
Kevin Costner is a lousy entrepreneur.
The core of Lean
is iteration.
Waterfall approach
You know the problem and the solution.
Known set of
requirements
Known ways to satisfy
them
Spec Build Test Launch
Agile methodologies
Know the problem, find the solution
Known set of
requirements
Unclear how to satisfy
them
Build Test LaunchViable?Problem

statement
Adjust
Sprints
Unknown set of
requirements
Lean approach
First, know that you don’t know.
Possible problem
space
Product/
market
hypothesis
Trial startup
Product/
market
hypothesis
Trial startup
Product/market
hypothesis
Trialstartup
Product/market
hypothesis
Trialstartup
You are
herePIVOT
Unfortunately,
we’re all liars.
Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
No data, no
learning.
Most startups don’t know what they’ll
be when they grow up.
Hotmail

was a
database
company
Flickr

was going to
be an MMO
Twitter

was a
podcasting
company
Autodesk

made
desktop
automation
Paypal

first built for
Palmpilots
Freshbooks

was invoicing
for a web
design firm
Wikipedia

was to be
written by
experts only
Mitel

was a
lawnmower
company
Gottfried von Leibniz
was a geek just like you.
First calculator

(stepped reckoner)
One of the first
to recognize the
importance of
binary.
“I thought again
about my early plan
of a new language or
writing-system of
reason, which could
serve as a
communication tool
for all different
nations..”
The best of all possible
worlds is the one in
which the fewest
starting conditions
produce the greatest
variety of outcomes.
Rethinking the fundamentals is how
you discover new models.
What’s the
biggest
problem

in a hair
salon?
The empty chair.
Knowing this, how
might you change
the client/customer
relationship?
Finding the tipping points of markets.
“Using Cue, you can tell if someone
has the ‘flu in 10 minutes.”
http://tiltthewindmill.com/when-can-becomes-must/
“Using Cue, you must tell if someone
has the ‘flu in 10 minutes…
…or Johnny can’t come on the school trip.”
Quantifying the riskiest parts.
The Attention Economy
“What information consumes
is rather obvious: it consumes
the attention of its recipients.
Hence a wealth of information
creates a poverty of attention, and a
need to allocate that attention efficiently
among the overabundance of
information sources that might
consume it.”
(Computers, Communications and the Public Interest, pages 40-41,
Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon
Lit motors tests the risky part
But how?
Analytics can help.
Analytics is the measurement of
movement towards your business
goals.
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
Some fundamentals.
Analytics, performance, aggregation, and the right metrics
Fundamental:

What makes a good metric?
A good metric is:
Understandable
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparative
Comparison is
context.
A ratio or rate
The only way to
measure
change and roll
up the tension
between two
metrics (MPH)
Behavior

changing
What will you
do differently
based on the
results you
collect?
The
simplest
rule
bad

metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%

of retailers
20%

of retailers
10%

of retailers
You are
just like
Customers that
buy >1x in 90d
Once
2-2.5

per year
>2.5

per year
Your customers
will buy from you
Then you are
in this mode
1-15%
15-30%
>30%
Low acquisition
cost, high checkout
Increasing return
rates, market share
Loyalty, selection,
inventory size
Focus on
(Thanks to Kevin Hillstrom for this.)
Qualitative
Unstructured, anecdotal,
revealing, hard to
aggregate, often too
positive & reassuring.
Warm and fuzzy.
Quantitative
Numbers and stats.
Hard facts, less insight,
easier to analyze; often
sour and disappointing.
Cold and hard.
Exploratory
Speculative. Tries to find
unexpected or
interesting insights.
Source of unfair
advantages.
Cool.
Reporting
Predictable. Keeps you
abreast of the normal,
day-to-day operations.
Can be managed by
exception.
Necessary.
Rumsfeld on Analytics
(Or rather, Avinash Kaushik channeling Rumsfeld)
Things we
know
don’t

know
we know Are facts which may be wrong and
should be checked against data.
we don’t

know
Are questions we can answer by
reporting, which we should baseline
& automate.
we know
Are intuition which we should
quantify and teach to improve
effectiveness, efficiency.
we don’t

know
Are exploration which is where
unfair advantage and interesting
epiphanies live.
MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this is the April cohort)
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
Multivariate

analysis
Changing several
things at once to
see which correlates
with a result.
☀
☁
☀
☁
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
☀
☁
Which of these two companies
is doing better?
  January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April   $8 $7
May       $9
How about
this one?
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1  
March $7 $6 $5    
April $8 $7      
May $9        
Averages $7 $5 $3 $1 $0.5
Look at the
same data
in cohorts
Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.
Explaining the
past.
Leading
Forward-looking.
Number today that
predicts tomorrow;
reports the news.
Example: pipeline.
Predicting the
future.
Fundamental:

Correlation
1
10
100
1000
10000
Ice cream consumption Drownings
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Correlated
Two variables that are
related (but may be
dependent on
something else.)
Ice cream &
drowning.
Causal
An independent variable
that directly impacts a
dependent one.
Summertime &
drowning.
A leading, causal metric
is a superpower.
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one file in one folder on one device
(ChenLi Wang)
Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
Some examples
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Why is Nigerian spam so badly
written?
Aunshul Rege of Rutgers University, USA in 2009
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages
emailed; they expect to land 2 or 3 “Mugu” (fools) each week.
One scammer boasted “When you get a reply it’s 70% sure you’ll get the money”
“By sending an email that repels all but the most gullible,” says [Microsoft Researcher
Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts
the true to false positive ratio in his favor.”
1000 emails
1-2 responses
1 fool and their money, parted.
Bad language (0.1% conversion)
Gullible (70% conversion)
1000 emails
100 responses
1 fool and their money, parted.
Good language (10% conversion)
Not-gullible (.07% conversion)
This would be horribly
inefficient since
humans are involved.
Turns out the word “Nigeria” is the best
way to identify promising prospects.
Nigerian spammers
really understand their target market.
They see past vanity metrics.
Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spend money to
get customers.
Customers are
worth more than
they cost.
Stickiness
Keep people
coming back.
Approach
Get customers
faster than you
lose them.
Math that
matters
Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Retention
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Revenue
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Referral
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
Gate
Thefivestages
Six business model archetypes.
E-commerce SaaS Media
Mobile

app
User-gen

content
2-sided

market
The business you’re in
(Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium
churn
Engaged User
Free user
disengagement
Reactivate
Cancel
Trial abandonment
rate
Invite Others
Paying Customer
Reactivation

rate
Paid
conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficient

Viral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling
rate Upselling
Disengaged DissatisfiedTrial Over
Model + Stage = One Metric That Matters.
One Metric

That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric
addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics
SaaS-based SEO toolkit in the scale stage. Focused on net adds.
Was a marketing campaign successful?
Were customer complaints lowered?
Was a product upgrade valuable?
Net adds up:
Can we acquire more valuable customers?
What product features can increase engagement?
Can we improve customer support?
Net adds flat:
Are the new customers not the right segment?
Did a marketing campaign fail?
Did a product upgrade fail somehow?
Is customer support falling apart?
Net adds down:
Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile

app
User-gen

content
2-sided

market
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
CAC, shares,
reactivation
Transaction,
CLV
Affiliates,
white-label
Engagement,
churn
Inherent
virality, CAC
Upselling,
CAC, CLV
API, magic #,
mktplace
Content,
spam
Invites,
sharing
Ads,
donations
Analytics,
user data
Inventory,
listings
SEM, sharing
Transactions,
commission
Other
verticals
(Money from transactions)
Downloads,
churn, virality
WoM, app
ratings, CAC
CLV,
ARPDAU
Spinoffs,
publishers
(Money from active users)
Traffic, visits,
returns
Content
virality, SEM
CPE, affiliate
%, eyeballs
Syndication,
licenses
(Money from ad clicks)
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its
customers every year.
Is this good or bad?
Not knowing what normal is
makes you do stupid things.
Baseline:
5-7% growth a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doing exceptionally well. If you
can only manage 1%, it's a sign
you haven't yet figured out
what you're doing.” At revenue
stage, measure growth in
revenue. Before that, measure
growth in active users.
Paul Graham, Y Combinator
• Are there enough people who really care
enough to sustain a 5% growth rate?

• Don’t strive for a 5% growth at the expense
of really understanding your customers
and building a meaningful solution

• Once you’re a pre-revenue startup at or
near product/market fit, you should have
5% growth of active users each week
• Once you’re generating revenues, they
should grow at 5% a week
Baseline:
10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day use web or mobile app
1% of users/day use it concurrently
Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.
Baseline:
Calculating customer lifetime
25%

monthly churn
100/25=4

The average
customer lasts
4 months
5%

monthly churn
100/5=20

The average
customer lasts
20 months
2%

monthly churn
100/2=50

The average
customer lasts
50 months
Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time to find
out whether your churn and revenue projections
are right
• Cashflow: You’re basically “loaning” the
customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a
CAC of only a third of your CLV will forces you
to verify costs sooner.
Lifetime of 20 mo.
$30/mo. per
customer
$600 CLV
$200 CAC
Now segment
those users!
1/3 spend
Who is worth more?
Today
A
Lifetime:
$200
Roberto Medri, Etsy
B
Lifetime:
$200
Visits
<Break>
</Break>
The Lean Analytics cycle
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:

find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
Do AirBnB hosts
get more business
if their property is
professionally
photographed?
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts
5,000 shoots per month
by February 2012
Hang on a second.
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
SRSLY?
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:

find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
“Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses have in
common?”
Camera model.
With data:

find a commonality
Without data: make a
good guess
Circle of Moms: Not enough engagement
• Too few people were
actually using the
product
• Less than 20% of any
circles had any activity
after their initial creation
• A few million monthly
uniques from 10M
registered users, but no
sustained traction
• They found moms were far more engaged
• Their messages to one another were on average 50% longer
• They were 115% more likely to attach a picture to a post they wrote
• They were 110% more likely to engage in a threaded (i.e. deep)
conversation
• Circle owners’ friends were 50% more likely to engage with the circle
• They were 75% more likely to click on Facebook notifications
• They were 180% more likely to click on Facebook news feed items
• They were 60% more likely to accept invitations to the app
• Pivoted to the new market, including a name change
• By late 2009, 4.5M users and strong engagement
• Sold to Sugar, inc. in early 2012
Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publisher analytics
SaaS analytics
Gaming analytics
User interaction Customer satisfaction KPI dashboardsUser segmentation
User analytics Spying on users
The mobile app!
customer lifecycle!
Ratings
Reviews
Search
Leaderboards
Purchases
Downloads
Installs
Play
Disengagement
Reactivation
Uninstallation
Disengagement
Account"
creation
Virality
Downloads,"
Gross revenue
ARPU
App sales
Activation
Churn, CLV
In-app"
purchases
Appstore!
Incentivized
Legitimate
Fraudulent
Ratings!
Intrapreneurship in large
organizations
(http://csinvesting.org/2012/01/06/fortune-500-extinction/)
F500 Life
Expectancy
Growth by entering
a new business 95

% fail
Corporate
Strategy Board
99

% fail
Clay
Christensen
75
years
15
years
1950 2010...
When you’re a startup
your goal is to find a sustainable,
repeatable business model.
When you’re a big company
your goal is to perpetuate one.
Intrapreneur:
Someone working to produce
disruptive change in an organization
that has already found a sustainable,
repeatable business model.
Intrapreneurship:
The difference between a rogue agent
and a special operative is permission.
The job of an intrapreneur is to
identify an adjacent market, product,
or method that conforms to
organizational filters.
It is not to improve the current
product, market, or method.
Also: a pariah.
Successful innovators share certain attributes.
Bad listener: Wilfully ignore feedback from your best customers.
Cannibal: If successful, destroying existing revenue streams.
Job killer: Automation & lower margins are your favorite tools.
Security risk: Advocate of transparency, open data, communities.
Narcissist: Worry constantly about how you’ll get attention.
Slum lord: Sell to those with less money, deviants, and weirdos.
In other words, if your job is change you
have your work cut out for you.
This isn’t about a lack of resources.
http://www.flickr.com/photos/maladjusted/5207565912
Blockbuster had a lot going for it.
Plenty of inventory, of course.
But that matters less than...
...market intelligence,
customers, existing
payment approval,
and customer
history.
The problem was framing:
Blockbuster thought it was in the video
store management business. Netflix
realized it was in the entertainment
delivery business.
YOU ARE
HERE
YOU ARE
HERE
LOCAL

MAXIMUM
OPTIMIZATION

OF CURRENT
METRICS
YOU ARE
HERE
GLOBAL

MAXIMUMINNOVATION

WITH NEW

RULES
YOU ARE
HERE
SHORT-TERM INVESTORS

HATE GOING DOWNHILL
In a big company,
analytics replaces opinion with fact.
Companies that use data-driven
analytics instead of intuition have
5%-6% higher productivity and
profits than competitors.
Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven
Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).
2011 MIT study of 179 large publicly traded firms
Arbitron and radio data
Times a song in “heavy
rotation” is played daily
2007 2012
266
Current

state
Business
optimization
(five mores)

Product,

market,

method
innovation

Business
model
innovation

You can convince
executives of this
because some of it
is familiar.
This terrifies them
because it eats the
current business.
A three-maxima model
of enterprise innovation
Improvement Adjacency Remodeling
Do the same,

only better.
Explore what’s

nearby quickly
Try out new

business models
Lean approaches apply, but the metrics vary widely.
Sustain/

core
Innovate/

adjacent
Disrupt/

transformative
Sustaining Adjacent Disruptive
Next year’s car Electric car,

same dealer
On-demand, app-based

car service
Sustaining
innovation
is about
more of
the same.
(says Sergio Zyman)
More things
To more people
For more money
More often
More efficiently
Supply chain optimization
Per-transaction cost reduction
Loyal customer base that returns
Demand prediction, notification
Maximum shopping cart
Price skimming/tiering
Highly viral offering
Low incremental order costs
Inventory increase
Gifting, wish lists
Blizzard extends the
lifespan of WOW
Early

adopters
Rapid

growth
Market

saturation
The infamous S-curve
(Product lifecycle, Bass diffusion curve, etc.)
Blizzard extends the
lifespan of WOW
Blizzard extends the
lifespan of WOW
Fixing this: sustaining growth with novelty
Product & market innovation
(“New & improved!”)
Blizzard extends the
lifespan of WOW
WOW
Burning

Crusade
Wrath of

the Lich King
Mists of

PandariaCataclysm
Warlords of
Draenor
Adjacent innovation is about changing
one part of the model in a way that
alters the value network.
Experiment with product, market,
and method.
Product
(new “what”)
Market
(new “who”)
Method
(new “how”)
3 kinds of
innovation
Adjacent product
to the same
market in the
same way
Transformative innovation is about
taking a leap, changing more than one
dimension simultaneously in search of
a new business model.
If sustaining, incremental innovation
produces linear growth, then
disruptive, transformative innovation
produces exponential growth.
Transformative isolation:
Skunkworks
Transformative
incubation:
Metlife Infinity
Some street-smart

tactics that work
“How quickly can you test this”

should be a criteria.
You’re not building a product,

you’re figuring out what product to build.
How easily can you test these out?
Take baby steps
How you get there matters as much as where you’re going.
Netflix
Tesla
http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character
limit isn’t arbitrary. It’s
constrained by the size
of SMS (160
characters) and
username (20
characters.)
http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/
sms_screen_twitter_activity_stream_270x405.png
Black car Uberpool
Let them ask for an out
Detecting SaaS churn early without hurting cashflow
The tradeoff
Charge a monthly fee Charge annual fee up front
Find out if they hate it sooner, when
they cancel on first billing cycle.
No need to pay back CAC; cash you
can use right away.
Takes months to recoup the money
you spent acquiring them.
May be a zombie user who vanishes
when the year is up.
They’re happy, you keep your
money, goodwill for offering.
They’re unsatisfied, you made it
right, they tell you why, you learn.
The solution—because the goal is to
learn, not to trap customers.
Annual fee, with an out
1. Offer an annual, discounted fee.
2. After a couple of weeks, ask if
they’d like a refund.
Protip: Campaigns that don’t suck.
Targeting
matters.
How to build a marketing campaign
http://www.yearonelabs.com/three-questions-all-marketers-must-answer/
When will you decide if it worked, and adjust?
Who are you
targeting?
Size, reachability,

homogeneity.
What do you
want them to
do?
Specific,
measurable

call to action.
Why should
they do it?
Laid, paid, made,

or afraid;
message

fits their mindset.
How will you
know if they
did?
Analytics,

instrumentation.
Waterslides, not funnels
The whole point of digital is personal
Segment 1
User segment

(who)
Segment 2
Segment 3
Goal

(what)
Goal 1
Goal 2
Goal 3
Motivation

(why)
Goal 1
Goal 2
Goal 3
http://www.thirdwunder.com/funnels-are-a-horrible-metaphor/Photo Credit: Patrick McGarvey
More like this.
Pitchers not buckets
Bounces Non-

creators
Non-

payers
Churn
Signup

rate
Engagement

level
Conversion

rate
Visitors
Subscribers
Engaged users
Paying

customers
Leaky buckets are a
bad metaphor
Bounces Non-

creators
Non-

payers
Churn
Improve stickiness; call-to-action optimization;

A/B page testing; picking better traffic sources
Notifications; updates; reactivating users;

segmenting those who are engaged.
Usage caps, natural upselling;

premium features
Better support; credit card renewal;
naturally sustained features
Hardcore interviews
Maybe they don’t love you

like they said they do.
N
Your offering doesn’t make them want to

brag or their contact isn’t really a friend
N
Advocates can’t learn & convey

your message easily
N
They don’t trust you entirely
N
Woohoo! Scalable, viral,

explainable product!
Y
Get a meeting
Y
Grab the phone
Y
They pitch it
Y
Call them now?
Y
Intro to a friend?
Interview
Use outliers and missed searches to
hunt for good ideas & adjacencies
(Multi-billion-dollar hygiene product company)
1/8 men have an incontinence issue. 1/3
women do.
When search results show a significant
number of men searching, this suggests the
adjacent (male) market is underserved.
Frame it like a study
Product creation is almost
accidental.
Unlike a VC or startup, when
the initiative fails the
organization still learns.
http://www.flickr.com/photos/creative_tools/8544475139
Use data to create a taste for
data
Sitting on Billions of rows of
transactional data
David Boyle ran 1M online surveys
Once the value was obvious to
management, got license to dig.
Focus on the desired behavior, not just
the information.
http://www.psychologytoday.com/blog/yes/
200808/changing-minds-and-changing-towels
26% increase in towel
re-use with an appeal
to social norms; 33%
increase when tied to
the specific room.
Energy Conservation “Nudges” and Environmentalist
Ideology: Evidence from a Randomized Residential Electricity
Field Experiment - Costa & Kahn 2011
The effectiveness of energy
conservation “nudges” depends on
an individual’s political ideology ...
Conservatives who learn that their
consumption is less than their
neighbors’ “boomerang” whereas
liberals reduce their consumption.
http://www.flickr.com/photos/bootbearwdc/1243690099/
Think subversively.
But don’t break promises.
Conclusions
“The most important figures that one
needs for management are unknown
or unknowable, but successful
management must nevertheless take
account of them.”
Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES
HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others
in the absence of data.
Now, a leader knows
what questions to ask.
Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko

Más contenido relacionado

La actualidad más candente

Introduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFIntroduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFLean Analytics
 
Growth hacking secrets
Growth hacking secrets Growth hacking secrets
Growth hacking secrets alig12
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Lean Analytics
 
Digital Strategy Workbook
Digital Strategy WorkbookDigital Strategy Workbook
Digital Strategy Workbooktim richards
 
Growth Hacking: A Crash Course
Growth Hacking: A Crash CourseGrowth Hacking: A Crash Course
Growth Hacking: A Crash CourseDavid Fallarme
 
Growth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about processGrowth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about processRuben Hamilius
 
Business model innovation
Business model innovationBusiness model innovation
Business model innovationDavid Skok
 
Customer Segmentation for Retention Strategy
Customer Segmentation for Retention StrategyCustomer Segmentation for Retention Strategy
Customer Segmentation for Retention StrategyMelody Ucros
 
Lean Stack - A Story Of Continuous Improvement
Lean Stack - A Story Of Continuous ImprovementLean Stack - A Story Of Continuous Improvement
Lean Stack - A Story Of Continuous ImprovementLukas Fittl
 
The Next Wave in Sales & Marketing
The Next Wave in Sales & MarketingThe Next Wave in Sales & Marketing
The Next Wave in Sales & MarketingDrift
 
Outbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flowOutbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flowDavid Skok
 
The Customer Journey Funnel: The Key to Sustainable Growth
The Customer Journey Funnel: The Key to Sustainable GrowthThe Customer Journey Funnel: The Key to Sustainable Growth
The Customer Journey Funnel: The Key to Sustainable GrowthRocketSource
 
Startup Metrics for Pirates (SF, Jan 2010)
Startup Metrics for Pirates (SF, Jan 2010)Startup Metrics for Pirates (SF, Jan 2010)
Startup Metrics for Pirates (SF, Jan 2010)Dave McClure
 
12 Key Levers of SaaS Success
12 Key Levers of SaaS Success12 Key Levers of SaaS Success
12 Key Levers of SaaS Successsaastr
 
Zero to 100 - Part 7: The Role of the CEO
Zero to 100 - Part 7: The Role of the CEOZero to 100 - Part 7: The Role of the CEO
Zero to 100 - Part 7: The Role of the CEODavid Skok
 
29 Growth Hacking Quick Wins
29 Growth Hacking Quick Wins29 Growth Hacking Quick Wins
29 Growth Hacking Quick WinsMattan Griffel
 
Growth Hacking Power Session at PIM Online 2017
Growth Hacking Power Session at PIM Online 2017 Growth Hacking Power Session at PIM Online 2017
Growth Hacking Power Session at PIM Online 2017 Growth Tribe
 
Startup Metrics for Pirates
Startup Metrics for PiratesStartup Metrics for Pirates
Startup Metrics for PiratesDave McClure
 

La actualidad más candente (20)

Introduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFIntroduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SF
 
Growth hacking secrets
Growth hacking secrets Growth hacking secrets
Growth hacking secrets
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)
 
Zero to 50m
Zero to 50m Zero to 50m
Zero to 50m
 
Digital Strategy Workbook
Digital Strategy WorkbookDigital Strategy Workbook
Digital Strategy Workbook
 
Growth Hacking: A Crash Course
Growth Hacking: A Crash CourseGrowth Hacking: A Crash Course
Growth Hacking: A Crash Course
 
Growth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about processGrowth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about process
 
Business model innovation
Business model innovationBusiness model innovation
Business model innovation
 
Growth Hacking
Growth HackingGrowth Hacking
Growth Hacking
 
Customer Segmentation for Retention Strategy
Customer Segmentation for Retention StrategyCustomer Segmentation for Retention Strategy
Customer Segmentation for Retention Strategy
 
Lean Stack - A Story Of Continuous Improvement
Lean Stack - A Story Of Continuous ImprovementLean Stack - A Story Of Continuous Improvement
Lean Stack - A Story Of Continuous Improvement
 
The Next Wave in Sales & Marketing
The Next Wave in Sales & MarketingThe Next Wave in Sales & Marketing
The Next Wave in Sales & Marketing
 
Outbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flowOutbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flow
 
The Customer Journey Funnel: The Key to Sustainable Growth
The Customer Journey Funnel: The Key to Sustainable GrowthThe Customer Journey Funnel: The Key to Sustainable Growth
The Customer Journey Funnel: The Key to Sustainable Growth
 
Startup Metrics for Pirates (SF, Jan 2010)
Startup Metrics for Pirates (SF, Jan 2010)Startup Metrics for Pirates (SF, Jan 2010)
Startup Metrics for Pirates (SF, Jan 2010)
 
12 Key Levers of SaaS Success
12 Key Levers of SaaS Success12 Key Levers of SaaS Success
12 Key Levers of SaaS Success
 
Zero to 100 - Part 7: The Role of the CEO
Zero to 100 - Part 7: The Role of the CEOZero to 100 - Part 7: The Role of the CEO
Zero to 100 - Part 7: The Role of the CEO
 
29 Growth Hacking Quick Wins
29 Growth Hacking Quick Wins29 Growth Hacking Quick Wins
29 Growth Hacking Quick Wins
 
Growth Hacking Power Session at PIM Online 2017
Growth Hacking Power Session at PIM Online 2017 Growth Hacking Power Session at PIM Online 2017
Growth Hacking Power Session at PIM Online 2017
 
Startup Metrics for Pirates
Startup Metrics for PiratesStartup Metrics for Pirates
Startup Metrics for Pirates
 

Similar a Lean Analytics: Using Data to Build a Better Business Faster

Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014Lean Analytics
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics
 
online startups metrics
online startups metrics online startups metrics
online startups metrics Hatem Kameli
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Startup Co.
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Lean Analytics
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Lean Startup Co.
 
SBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsMeasureWorks
 
Help Employees Socialize Your Brand
Help Employees Socialize Your BrandHelp Employees Socialize Your Brand
Help Employees Socialize Your BrandThe Safdar Group
 
Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungMattan Griffel
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
 
Introduction to lean analytics
Introduction to lean analyticsIntroduction to lean analytics
Introduction to lean analyticsKartik Narayanan
 
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Kate O'Neill
 
Social Media Dashboarding (reporting)
Social Media Dashboarding (reporting)Social Media Dashboarding (reporting)
Social Media Dashboarding (reporting)Scott K. Wilder
 
Social Media Dashboarding by Scott Wilder and semphonic
Social Media Dashboarding by Scott Wilder and semphonicSocial Media Dashboarding by Scott Wilder and semphonic
Social Media Dashboarding by Scott Wilder and semphonicEdelman Digital
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking jeroentjepkema
 
Analytics & Product development
Analytics & Product developmentAnalytics & Product development
Analytics & Product developmentMuthoniWanyoike
 
The World of Measurement and Analytics
The World of Measurement and AnalyticsThe World of Measurement and Analytics
The World of Measurement and AnalyticsJarred Cinman
 
Start-up Metrics (Smetrics )
Start-up Metrics (Smetrics )Start-up Metrics (Smetrics )
Start-up Metrics (Smetrics )James Alexander
 

Similar a Lean Analytics: Using Data to Build a Better Business Faster (20)

Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
 
online startups metrics
online startups metrics online startups metrics
online startups metrics
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair Croll
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
 
SBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean Analytics
 
Help Employees Socialize Your Brand
Help Employees Socialize Your BrandHelp Employees Socialize Your Brand
Help Employees Socialize Your Brand
 
Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-Young
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
 
Introduction to lean analytics
Introduction to lean analyticsIntroduction to lean analytics
Introduction to lean analytics
 
Startup analytics
Startup analyticsStartup analytics
Startup analytics
 
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
 
Social Media Dashboarding (reporting)
Social Media Dashboarding (reporting)Social Media Dashboarding (reporting)
Social Media Dashboarding (reporting)
 
Social Media Dashboarding by Scott Wilder and semphonic
Social Media Dashboarding by Scott Wilder and semphonicSocial Media Dashboarding by Scott Wilder and semphonic
Social Media Dashboarding by Scott Wilder and semphonic
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
 
Analytics & Product development
Analytics & Product developmentAnalytics & Product development
Analytics & Product development
 
The World of Measurement and Analytics
The World of Measurement and AnalyticsThe World of Measurement and Analytics
The World of Measurement and Analytics
 
Start-up Metrics (Smetrics )
Start-up Metrics (Smetrics )Start-up Metrics (Smetrics )
Start-up Metrics (Smetrics )
 

Más de Lean Startup Co.

A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...Lean Startup Co.
 
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...Lean Startup Co.
 
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...Lean Startup Co.
 
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...Lean Startup Co.
 
Keynote: Innovation is a Habit, not a Mindset, Diana Kander
Keynote: Innovation is a Habit, not a Mindset, Diana KanderKeynote: Innovation is a Habit, not a Mindset, Diana Kander
Keynote: Innovation is a Habit, not a Mindset, Diana KanderLean Startup Co.
 
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...Lean Startup Co.
 
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil LavingiaG3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil LavingiaLean Startup Co.
 
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...Lean Startup Co.
 
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...Lean Startup Co.
 
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric TachibanaE3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric TachibanaLean Startup Co.
 
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...Lean Startup Co.
 
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina MedoffD5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina MedoffLean Startup Co.
 
F1: Mastering the Art of Telling Your Story, Jamie Lazzeri
F1: Mastering the Art of Telling Your Story, Jamie LazzeriF1: Mastering the Art of Telling Your Story, Jamie Lazzeri
F1: Mastering the Art of Telling Your Story, Jamie LazzeriLean Startup Co.
 
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...Lean Startup Co.
 
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...Lean Startup Co.
 
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh PhillipsG5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh PhillipsLean Startup Co.
 
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...Lean Startup Co.
 
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...Lean Startup Co.
 
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas OberhuberB5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas OberhuberLean Startup Co.
 
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish Lean Startup Co.
 

Más de Lean Startup Co. (20)

A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
A5: Designing Gamified Experiences to Gain Customer Insights, Richardo Chen &...
 
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
Keynote: The Subscription Economy: How to Survive the End of Ownership, Tien ...
 
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
 
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
A3: Lean For Social Impact: Innovating for Greater Impact and Scale, Steve Na...
 
Keynote: Innovation is a Habit, not a Mindset, Diana Kander
Keynote: Innovation is a Habit, not a Mindset, Diana KanderKeynote: Innovation is a Habit, not a Mindset, Diana Kander
Keynote: Innovation is a Habit, not a Mindset, Diana Kander
 
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
F3: Employee = Founder: Building a Startup inside a Fortune 500 Company, Max-...
 
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil LavingiaG3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
G3: Reflecting on My Failure to Build a Billion Dollar Company, Sahil Lavingia
 
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
C3: Optimize, Grow or Catapult: Where to Target Your Organization’s Innovatio...
 
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
G6: Getting Leaders On Board: Don’t Let Your Innovation Program be Pronounced...
 
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric TachibanaE3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
E3: Amazon’s Approach to Culture Change, Wen Huang, Eric Tachibana
 
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
G1: Brackitz Toy Story: Learning From Mistakes and Mental Models for Success,...
 
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina MedoffD5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
D5: Be the Solution: Use Lean to Solve Community Problems, Katrina Medoff
 
F1: Mastering the Art of Telling Your Story, Jamie Lazzeri
F1: Mastering the Art of Telling Your Story, Jamie LazzeriF1: Mastering the Art of Telling Your Story, Jamie Lazzeri
F1: Mastering the Art of Telling Your Story, Jamie Lazzeri
 
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
Keynote: Intelligent Growth in Startups: Building More Than a Product to Get ...
 
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
Keynote: From Founder to Investor: Stories From the Trenches of Entrepreneurs...
 
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh PhillipsG5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
G5: Big Problems Require Big Solutions: the Story of Earn, Leigh Phillips
 
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
 
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
B6: Leading Innovation and Building Better Lives: Intercorp Latin America, Ed...
 
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas OberhuberB5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
B5: The 8 Deadly Maladies of Rapid Experimentation, Lukas Oberhuber
 
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
B4: The Road to Startup Success is Paved in Pivots, Kate Skavish
 

Último

Church Building Grants To Assist With New Construction, Additions, And Restor...
Church Building Grants To Assist With New Construction, Additions, And Restor...Church Building Grants To Assist With New Construction, Additions, And Restor...
Church Building Grants To Assist With New Construction, Additions, And Restor...Americas Got Grants
 
Annual General Meeting Presentation Slides
Annual General Meeting Presentation SlidesAnnual General Meeting Presentation Slides
Annual General Meeting Presentation SlidesKeppelCorporation
 
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...ictsugar
 
FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607dollysharma2066
 
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCRashishs7044
 
International Business Environments and Operations 16th Global Edition test b...
International Business Environments and Operations 16th Global Edition test b...International Business Environments and Operations 16th Global Edition test b...
International Business Environments and Operations 16th Global Edition test b...ssuserf63bd7
 
Organizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessOrganizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessSeta Wicaksana
 
Investment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy CheruiyotInvestment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy Cheruiyotictsugar
 
Buy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail AccountsBuy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail AccountsBuy Verified Accounts
 
Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03DallasHaselhorst
 
Financial-Statement-Analysis-of-Coca-cola-Company.pptx
Financial-Statement-Analysis-of-Coca-cola-Company.pptxFinancial-Statement-Analysis-of-Coca-cola-Company.pptx
Financial-Statement-Analysis-of-Coca-cola-Company.pptxsaniyaimamuddin
 
Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737Riya Pathan
 
Cyber Security Training in Office Environment
Cyber Security Training in Office EnvironmentCyber Security Training in Office Environment
Cyber Security Training in Office Environmentelijahj01012
 
Pitch Deck Teardown: Geodesic.Life's $500k Pre-seed deck
Pitch Deck Teardown: Geodesic.Life's $500k Pre-seed deckPitch Deck Teardown: Geodesic.Life's $500k Pre-seed deck
Pitch Deck Teardown: Geodesic.Life's $500k Pre-seed deckHajeJanKamps
 
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCRashishs7044
 
Kenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby AfricaKenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby Africaictsugar
 
PSCC - Capability Statement Presentation
PSCC - Capability Statement PresentationPSCC - Capability Statement Presentation
PSCC - Capability Statement PresentationAnamaria Contreras
 
Innovation Conference 5th March 2024.pdf
Innovation Conference 5th March 2024.pdfInnovation Conference 5th March 2024.pdf
Innovation Conference 5th March 2024.pdfrichard876048
 

Último (20)

Church Building Grants To Assist With New Construction, Additions, And Restor...
Church Building Grants To Assist With New Construction, Additions, And Restor...Church Building Grants To Assist With New Construction, Additions, And Restor...
Church Building Grants To Assist With New Construction, Additions, And Restor...
 
Annual General Meeting Presentation Slides
Annual General Meeting Presentation SlidesAnnual General Meeting Presentation Slides
Annual General Meeting Presentation Slides
 
No-1 Call Girls In Goa 93193 VIP 73153 Escort service In North Goa Panaji, Ca...
No-1 Call Girls In Goa 93193 VIP 73153 Escort service In North Goa Panaji, Ca...No-1 Call Girls In Goa 93193 VIP 73153 Escort service In North Goa Panaji, Ca...
No-1 Call Girls In Goa 93193 VIP 73153 Escort service In North Goa Panaji, Ca...
 
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...
 
FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607
 
Enjoy ➥8448380779▻ Call Girls In Sector 18 Noida Escorts Delhi NCR
Enjoy ➥8448380779▻ Call Girls In Sector 18 Noida Escorts Delhi NCREnjoy ➥8448380779▻ Call Girls In Sector 18 Noida Escorts Delhi NCR
Enjoy ➥8448380779▻ Call Girls In Sector 18 Noida Escorts Delhi NCR
 
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
 
International Business Environments and Operations 16th Global Edition test b...
International Business Environments and Operations 16th Global Edition test b...International Business Environments and Operations 16th Global Edition test b...
International Business Environments and Operations 16th Global Edition test b...
 
Organizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessOrganizational Structure Running A Successful Business
Organizational Structure Running A Successful Business
 
Investment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy CheruiyotInvestment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy Cheruiyot
 
Buy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail AccountsBuy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail Accounts
 
Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03
 
Financial-Statement-Analysis-of-Coca-cola-Company.pptx
Financial-Statement-Analysis-of-Coca-cola-Company.pptxFinancial-Statement-Analysis-of-Coca-cola-Company.pptx
Financial-Statement-Analysis-of-Coca-cola-Company.pptx
 
Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737
 
Cyber Security Training in Office Environment
Cyber Security Training in Office EnvironmentCyber Security Training in Office Environment
Cyber Security Training in Office Environment
 
Pitch Deck Teardown: Geodesic.Life's $500k Pre-seed deck
Pitch Deck Teardown: Geodesic.Life's $500k Pre-seed deckPitch Deck Teardown: Geodesic.Life's $500k Pre-seed deck
Pitch Deck Teardown: Geodesic.Life's $500k Pre-seed deck
 
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
 
Kenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby AfricaKenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby Africa
 
PSCC - Capability Statement Presentation
PSCC - Capability Statement PresentationPSCC - Capability Statement Presentation
PSCC - Capability Statement Presentation
 
Innovation Conference 5th March 2024.pdf
Innovation Conference 5th March 2024.pdfInnovation Conference 5th March 2024.pdf
Innovation Conference 5th March 2024.pdf
 

Lean Analytics: Using Data to Build a Better Business Faster

  • 1. Lean Analytics Lean Startup conference San Francisco, November 16 2015 @acroll
  • 2. Don’t sell what you can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
  • 3. The core of Lean is iteration.
  • 4. Waterfall approach You know the problem and the solution.
  • 5.
  • 6. Known set of requirements Known ways to satisfy them Spec Build Test Launch
  • 7. Agile methodologies Know the problem, find the solution
  • 8.
  • 9. Known set of requirements Unclear how to satisfy them Build Test LaunchViable?Problem
 statement Adjust Sprints Unknown set of requirements
  • 10. Lean approach First, know that you don’t know.
  • 11. Possible problem space Product/ market hypothesis Trial startup Product/ market hypothesis Trial startup Product/market hypothesis Trialstartup Product/market hypothesis Trialstartup You are herePIVOT
  • 13. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  • 14. Most startups don’t know what they’ll be when they grow up. Hotmail
 was a database company Flickr
 was going to be an MMO Twitter
 was a podcasting company Autodesk
 made desktop automation Paypal
 first built for Palmpilots Freshbooks
 was invoicing for a web design firm Wikipedia
 was to be written by experts only Mitel
 was a lawnmower company
  • 15. Gottfried von Leibniz was a geek just like you.
  • 16. First calculator (stepped reckoner) One of the first to recognize the importance of binary. “I thought again about my early plan of a new language or writing-system of reason, which could serve as a communication tool for all different nations..”
  • 17. The best of all possible worlds is the one in which the fewest starting conditions produce the greatest variety of outcomes.
  • 18. Rethinking the fundamentals is how you discover new models.
  • 20. The empty chair. Knowing this, how might you change the client/customer relationship?
  • 21. Finding the tipping points of markets.
  • 22. “Using Cue, you can tell if someone has the ‘flu in 10 minutes.” http://tiltthewindmill.com/when-can-becomes-must/
  • 23. “Using Cue, you must tell if someone has the ‘flu in 10 minutes… …or Johnny can’t come on the school trip.”
  • 25. The Attention Economy “What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” (Computers, Communications and the Public Interest, pages 40-41, Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon
  • 26. Lit motors tests the risky part
  • 29. Analytics is the measurement of movement towards your business goals.
  • 30. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 31. Some fundamentals. Analytics, performance, aggregation, and the right metrics
  • 33. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior
 changing What will you do differently based on the results you collect?
  • 34. The simplest rule bad
 metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
  • 35. Metrics help you know yourself. Acquisition Hybrid Loyalty 70%
 of retailers 20%
 of retailers 10%
 of retailers You are just like Customers that buy >1x in 90d Once 2-2.5
 per year >2.5
 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
  • 36. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  • 37. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  • 38. Rumsfeld on Analytics (Or rather, Avinash Kaushik channeling Rumsfeld) Things we know don’t
 know we know Are facts which may be wrong and should be checked against data. we don’t
 know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t
 know Are exploration which is where unfair advantage and interesting epiphanies live.
  • 39. MayAprMarFeb Slicing and dicing data Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate
 analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  • 40. Which of these two companies is doing better?
  • 41.   January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April   $8 $7 May       $9 How about this one?
  • 42. Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1   March $7 $6 $5     April $8 $7       May $9         Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 43. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 45. 1 10 100 1000 10000 Ice cream consumption Drownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
  • 46. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 47. A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 48. A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 49. Why is Nigerian spam so badly written?
  • 50. Aunshul Rege of Rutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
  • 51. Turns out the word “Nigeria” is the best way to identify promising prospects.
  • 52. Nigerian spammers really understand their target market. They see past vanity metrics.
  • 53. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  • 54. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 55. Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
  • 56. Six business model archetypes. E-commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market The business you’re in
  • 57. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation
 rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
  • 58. Model + Stage = One Metric That Matters. One Metric
 That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  • 61. In a startup, focus is hard to achieve.
  • 62. Having only one metric addresses this problem.
  • 64. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  • 65. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 66. Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  • 68. Drawing some lines in the sand.
  • 69. A company loses a quarter of its customers every year. Is this good or bad?
  • 70. Not knowing what normal is makes you do stupid things.
  • 71. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  • 72. Baseline: 10% visitor engagement/day Fred Wilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  • 73. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 74. Baseline: Calculating customer lifetime 25%
 monthly churn 100/25=4
 The average customer lasts 4 months 5%
 monthly churn 100/5=20
 The average customer lasts 20 months 2%
 monthly churn 100/2=50
 The average customer lasts 50 months
  • 75. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
  • 76. Who is worth more? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
  • 80. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 81. Do AirBnB hosts get more business if their property is professionally photographed?
  • 82. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 83. 5,000 shoots per month by February 2012
  • 84. Hang on a second.
  • 85. Gut instinct (hypothesis) Professional photography helps AirBnB’s business SRSLY?
  • 86. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 87. “Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data:
 find a commonality Without data: make a good guess
  • 88. Circle of Moms: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
  • 89. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
  • 90.
  • 91.
  • 92. The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU App sales Activation Churn, CLV In-app" purchases Appstore! Incentivized Legitimate Fraudulent Ratings!
  • 93.
  • 94.
  • 95.
  • 97.
  • 98. (http://csinvesting.org/2012/01/06/fortune-500-extinction/) F500 Life Expectancy Growth by entering a new business 95
 % fail Corporate Strategy Board 99
 % fail Clay Christensen 75 years 15 years 1950 2010...
  • 99. When you’re a startup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
  • 100. Intrapreneur: Someone working to produce disruptive change in an organization that has already found a sustainable, repeatable business model.
  • 101. Intrapreneurship: The difference between a rogue agent and a special operative is permission.
  • 102. The job of an intrapreneur is to identify an adjacent market, product, or method that conforms to organizational filters. It is not to improve the current product, market, or method.
  • 103. Also: a pariah. Successful innovators share certain attributes. Bad listener: Wilfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.
  • 104. In other words, if your job is change you have your work cut out for you.
  • 105. This isn’t about a lack of resources.
  • 107. Blockbuster had a lot going for it.
  • 108. Plenty of inventory, of course. But that matters less than...
  • 109. ...market intelligence, customers, existing payment approval, and customer history.
  • 110. The problem was framing: Blockbuster thought it was in the video store management business. Netflix realized it was in the entertainment delivery business.
  • 115. In a big company, analytics replaces opinion with fact.
  • 116. Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and profits than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011). 2011 MIT study of 179 large publicly traded firms
  • 118. Times a song in “heavy rotation” is played daily 2007 2012 266
  • 119. Current
 state Business optimization (five mores)
 Product,
 market,
 method innovation
 Business model innovation
 You can convince executives of this because some of it is familiar. This terrifies them because it eats the current business. A three-maxima model of enterprise innovation
  • 120. Improvement Adjacency Remodeling Do the same,
 only better. Explore what’s
 nearby quickly Try out new
 business models Lean approaches apply, but the metrics vary widely. Sustain/
 core Innovate/
 adjacent Disrupt/
 transformative
  • 121. Sustaining Adjacent Disruptive Next year’s car Electric car,
 same dealer On-demand, app-based
 car service
  • 122. Sustaining innovation is about more of the same. (says Sergio Zyman) More things To more people For more money More often More efficiently Supply chain optimization Per-transaction cost reduction Loyal customer base that returns Demand prediction, notification Maximum shopping cart Price skimming/tiering Highly viral offering Low incremental order costs Inventory increase Gifting, wish lists
  • 123. Blizzard extends the lifespan of WOW Early
 adopters Rapid
 growth Market
 saturation The infamous S-curve (Product lifecycle, Bass diffusion curve, etc.)
  • 125. Blizzard extends the lifespan of WOW Fixing this: sustaining growth with novelty Product & market innovation (“New & improved!”)
  • 126. Blizzard extends the lifespan of WOW WOW Burning
 Crusade Wrath of
 the Lich King Mists of
 PandariaCataclysm Warlords of Draenor
  • 127. Adjacent innovation is about changing one part of the model in a way that alters the value network.
  • 128. Experiment with product, market, and method.
  • 130. Adjacent product to the same market in the same way
  • 131. Transformative innovation is about taking a leap, changing more than one dimension simultaneously in search of a new business model.
  • 132. If sustaining, incremental innovation produces linear growth, then disruptive, transformative innovation produces exponential growth.
  • 136. “How quickly can you test this”
 should be a criteria. You’re not building a product,
 you’re figuring out what product to build.
  • 137. How easily can you test these out?
  • 138. Take baby steps How you get there matters as much as where you’re going.
  • 141. Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size of SMS (160 characters) and username (20 characters.) http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
  • 143. Let them ask for an out Detecting SaaS churn early without hurting cashflow
  • 144. The tradeoff Charge a monthly fee Charge annual fee up front Find out if they hate it sooner, when they cancel on first billing cycle. No need to pay back CAC; cash you can use right away. Takes months to recoup the money you spent acquiring them. May be a zombie user who vanishes when the year is up.
  • 145. They’re happy, you keep your money, goodwill for offering. They’re unsatisfied, you made it right, they tell you why, you learn. The solution—because the goal is to learn, not to trap customers. Annual fee, with an out 1. Offer an annual, discounted fee. 2. After a couple of weeks, ask if they’d like a refund.
  • 146. Protip: Campaigns that don’t suck.
  • 148. How to build a marketing campaign http://www.yearonelabs.com/three-questions-all-marketers-must-answer/ When will you decide if it worked, and adjust? Who are you targeting? Size, reachability,
 homogeneity. What do you want them to do? Specific, measurable
 call to action. Why should they do it? Laid, paid, made,
 or afraid; message
 fits their mindset. How will you know if they did? Analytics,
 instrumentation.
  • 150. The whole point of digital is personal Segment 1 User segment
 (who) Segment 2 Segment 3 Goal
 (what) Goal 1 Goal 2 Goal 3 Motivation
 (why) Goal 1 Goal 2 Goal 3
  • 154. Bounces Non-
 creators Non-
 payers Churn Improve stickiness; call-to-action optimization;
 A/B page testing; picking better traffic sources Notifications; updates; reactivating users;
 segmenting those who are engaged. Usage caps, natural upselling;
 premium features Better support; credit card renewal; naturally sustained features
  • 156. Maybe they don’t love you
 like they said they do. N Your offering doesn’t make them want to
 brag or their contact isn’t really a friend N Advocates can’t learn & convey
 your message easily N They don’t trust you entirely N Woohoo! Scalable, viral,
 explainable product! Y Get a meeting Y Grab the phone Y They pitch it Y Call them now? Y Intro to a friend? Interview
  • 157. Use outliers and missed searches to hunt for good ideas & adjacencies (Multi-billion-dollar hygiene product company) 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved.
  • 158. Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
  • 159. Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
  • 160. Focus on the desired behavior, not just the information. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011 The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption.
  • 162. But don’t break promises.
  • 164. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 165. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 167. Once, a leader convinced others in the absence of data.
  • 168. Now, a leader knows what questions to ask.