Museums and the Web 2015 workshop includes the analytics process, case studies and a social media framework. Presented by Brian Alpert, Erin Blasco, Effie Kapsalis and Sarah Banks, Smithsonian Institution.
2. #mwmetrics 2
Our Workshop Today
Brian’s Stuff
Puppet Invasion! – Erin Blasco
Activity!
Coffee Break (10:30-11:00am)
Smithsonian Archives: Fast, Cheap & In
Control – Effie Kapsalis
Creating a Social Media Metrics
Reporting Framework You Can Live with
& Setting up Analytics for Native Apps –
Sarah Banks
Activity!
3. #mwmetrics 3
My Stuff Today
The analytics process
Choosing social media metrics
Choosing website metrics
Google Analytics update
4. #mwmetrics 4
Once upon a time…
We just had websites…
Website measurement
tools were getting better
and better…
Some of the best ones
were even free!
We thought we had it all
figured out…
And then…
Source: Seattle Municipal Archives
5. #mwmetrics 5
Social media disrupted everything
Today's landscape is a splintered
collection of
New channels
Sublimely-named yet inscrutable metrics
A dizzying array of tools both free and paid
Breathing new life into old questions
"Why is this important?
“How do we know it’s working?“
"What do I measure?“
"What does that have to do with our program?“
The good news is…
Source: http://rosemia.wordpress.com/2012/02/
6. #mwmetrics
There is a systematic, step-by-step process
Articulate your program’s goals.
Decide strategies to achieve
those goals.
Decide tactics to pursue the
strategies.
Decide what and how to measure.
Benchmark to get a sense of
what’s normal.
6
Source: http://www.homedit.com
7. #mwmetrics 7
Your goal: storyteller
Use data to tell a story.
Management loves stories:
What was happening.
What it meant.
What you did.
What’s happening now.
Source: http://www.squidoo.com
8. #mwmetrics
Articulate specific goals
Not too many!
Express what you’re trying to
accomplish.
Make high-level goals more
specific:
“Increase influence” is too broad.
“Become the definitive source on
Smithsonian history” is more specific.
Specificity makes it easier to identify
strategies and tactics.
8
Reuters: Toru Hanai
Start the conversation! Articulate
goals & next steps on your own;
work with management to finalize.
9. #mwmetrics 9
Determine strategies & tactics
Strategies – the plans you make to achieve the goals.
Employing social media is a strategy.
Tactics – the things you do to advance the strategy.
Producing a specific type of content is a tactic.
Individual channels (facebook, twitter) are tactics.
Per the example:
Goal: “Become the definitive source on Smithsonian history.”
Strategy: Increase engagement with history of the Smithsonian
content.
Tactic: Make SI-history content more findable and measureable.
10. #mwmetrics 10
Decide how to measure your tactics
Choose measurements to learn
if your tactics are succeeding.
Choose a few measurements.
Trend them over time.
Per the example:
Strategy: increase engagement with
SI history website content.
Tactic: make website history
content more findable /
measureable.
History-content segment
Measure that segment’s
engagement metrics:
Visit frequency
Visit depth
Bounce rate
History-
related visits
All
visits
“Deep history visits” were 94% higher!
11. #mwmetrics 11
"I got 20 retweets! Wait – is that good?"
You can’t set targets w/o benchmarks
You need at least six months of data.
Data is seasonal.
Depends on how much traffic your site gets.
Pull data regularly, or you may be out of luck.
Twitter and Flickr API’s limits 3rd party tools to 28
days of data.
Balance targets with factors beyond your control:
Are the improvements you’re seeking known to be difficult?
How much resources will you have to implement tactics?
Google Analytics Benchmarking Reports!
‘Museum & Libraries’ category under ‘Reference’
Source: news.com.au
13. #mwmetrics 13
“Quantity of Stuff” metrics
No actionable data
Establish scope / context
Measure growth / acquisition
Number of Followers
FB
TW
Instagram
Pinterest
Number of ‘Likes’
FB Pages
FB Content
Instagram
Pinterest
FB post views
Source:http://janicesyearinsunderland.blogspot.com
14. #mwmetrics 14
“Quantity-Plus” metrics
Still about quantity, but more meaningful.
Show the type of content your audience
responds to.
Basic
Reach (FB)
Post-Clicks (FB)
Website visits referred by social
properties
Better - “mini-conversions”
Retweets (TW)
Favorites (TW)
Comments (FB)
Shares (FB)
Source:http://socialmediatoday.com
15. #mwmetrics
A classic blog post…
Avinash Kaushik’s Best Social Media Metrics
Conversation Rate
# of Audience Comments (or Replies) Per Post
Amplification Rate
# of Retweets Per Tweet
# of Shares Per Post
# of “Share Clicks” Per Post (or Video)
Applause Rate
# of Favorite Clicks Per Post (TW)
# of Likes Per Post (FB)
# of #1s Per Post (Goog+)
# of +1s and Likes Per Post (or video) (Blog / YouTube)
Economic Value
For revenue-driven businesses
Sum of Short and Long Term Revenue and Cost Savings
Goal is to identify macro and micro conversions and then compute economic
value.
A manual spreadsheet is available, here, or…
Source: Occam's Razor
Avinash Kaushik
16. #mwmetrics 16
Trendable social metrics – YAY!
TrueSocialMetrics offers an automated solution.
http://www.truesocialmetrics.com/
Free / $30 per month / $100 per month / $350/month plans
17. #mwmetrics 17
Here is the bottom line!
Your measurements validate
your tactics (or not).
On a campaign by campaign
basis, you can use “quantity-
plus” metrics to tell your story.
“Here was the goal. We did
this. That happened. It was
the best EVER!
But to improve your overall
program, you need more
refined, trendable metrics.
Source: NY Daily News
19. #mwmetrics 19
The inevitability of “Quantity of Stuff”
No actionable data
Visits (Sessions)
Visitors (Users)
Pageviews
Establish scope / context.
Measure growth / acquisition.
You can’t improve your site by
measuring these.
Reporting them out of context can be
misleading.
Source: Occam's Razor
“All data in
aggregate is crap.”
20. #mwmetrics 20
Engagement metrics are more meaningful
“Quantity-Plus” for websites.
Use as proxies for visitor
engagement.
Under Audience >> Behavior
Frequency
Recency
Depth (“Engagement”)
New vs. Returning users
Use with advanced segments.
21. #mwmetrics 21
Segmentation: GA’s most powerful feature?
Analyze subsets of traffic.
Search engine visits
Social media visits
Demographics
Import ready-made
segments and custom
reports from the
“Gallery”!
Google Blog
Kissmetrics Overview
Examples (Cutroni)
Segments are accessed
from this pull-down arrow.
22. #mwmetrics
All Visits data tells a nice story...
22
Minimal loyalty
group (purple)
downward trend
indicates
improving content
engagement
High loyalty
group (blue)
upward trend
indicates same
Impact of this Data on the Site or Program
• This good-looking chart may indicate high content engagement and/or perceived value
• This data may correlate to increasing conversion behaviors
Acting on this Data
• Identify moderate and high loyalty pages as a means of duplicating, or improving others
• Examining conversion behaviors of these segments may yield add'l insights
• Correlating high bounce rate pages to one-time visits may yield add'l insights
• Test different content types in an attempt to move 'minimal' visitors into 'moderate' group
Key Trends
and Insights
23. #mwmetrics 23
This Impact of this Data on the Site or Program
• Organic search listings are driving poorly-targeted traffic
• Will result in decreased organic search performance over time
Acting on this Data
• Refocus title tags, meta-description tags and page content for important pages
• Perform link analysis to see where other SEO improvements can be made
Minimal
frequency group
upward trend
indicates organic
listings are not
appropriately
targeted
Moderate
frequency group
downward trend
indicates same
High frequency
group trending
slightly downward,
in contrast to
previous chart’s
upward slope
Key Trends
and Insights
…But applying segmentation tells a different story
24. #mwmetrics 24
Deeper understanding with Conversion Goals
A conversion is any measureable behavior
with an implicitly (or explicitly) higher value.
You learn more by studying conversion rates.
Some are super-easy to program:
Destination (ex: thanks.html)
Duration (ex: 5 minutes or more)
Pages/Screens per session (ex: 3 pages)
Event (ex: downloaded a PDF, played a video)
(REQUIRES CODE)
Even if you’re not selling stuff, you can – and
should – add a “value” to a completed goal.
25. #mwmetrics 25
‘Event Tracking’ is super-important
More sophisticated Goals typically involve Events.
A little extra javascript gives you data on:
External links
Sign-ups, form submissions
Downloads
Many types of conversion goals
To use Events:
Define and categorize your events.
Configure and add the code, usually right in the link (not always).
Many social-share widgets automatically add Events.
Events are critical for Mobile App Google Analytics!
The Complete Google Analytics Event Tracking Guide Plus 10 Amazing
Examples
26. #mwmetrics 26
Benchmarking Reports!
Compare your site to others in the same
category (or across categories).
Compare by:
Channels
Location
Devices
How to find it:
Audience Benchmarking
Click ‘Business and Industrial’ (top left) for pull-down
Scroll down to ‘Reference’ ‘Libraries & Museums’
Benchmarking Reports
27. #mwmetrics 27
Here is the (website) bottom line!
Your measurements validate your
tactics (or not).
If your goal is purely audience
acquisition, you can use “quantity-
of-stuff” metrics to tell your story.
To work the process and improve
your site, you need more
meaningful data:
Engagement Metrics
Segments
Goal Completion / Conversion rates
Flow (landing pages / exit pages)
A-B tests
Qualitative data (surveys)
Source: NY Daily News
29. #mwmetrics 29
Universal Analytics means all new code
We are in Phase three of a multi-year
rollout – “all Universal features supported.”
In Phase two, all GA accounts were
migrated to Universal. (Surprise!)
In Phase four, legacy GA code will be
deprecated (date TBD).
“Data collected from the deprecated
features will be processed for a minimum of
2 years.”
You should upgrade your code SOON, if
you haven’t already done so.
You'll also need to upgrade custom code,
e.g., events, virtual pageviews, etc.
Universal Analytics Upgrade Center Source: Vampyre Fangs
30. #mwmetrics 30
Universal Analytics means user-centric data
GA can now display data in terms of users or
sessions.
“User ID Feature” enables tracking individual users
across devices.
Other network-connected devices can send data
to GA:
Kiosks
Game consoles
Point-of-sale systems
It’s all based on custom coding…
But it can be done - now!
About the User ID feature
Custom Dimensions and Metrics
Tracking even
anonymous users has
significant privacy
implications, esp. if your
site caters to children.
31. #mwmetrics 31
Google’s “Analytics Academy”
Free video-based courses
Digital Analytics
Fundamentals
Google Analytics Platform
Principles
Ecommerce Analytics: From
Data to Decisions
Mobile App Analytics
Fundamentals
analyticsacademy.withgoogle.com
32. #mwmetrics 32
Resources
Google Analytics Blog
http://analytics.blogspot.com/
Universal Analytics Upgrade Guide
https://developers.google.com/analytics/devguides/collection/upgrade/
Google ‘User ID’ Guide
https://support.google.com/analytics/answer/3123662/
Avinash Kaushik’s “Occam’s Razor”
http://kaushik.net/avinash
Analytics Talk (Justin Cutroni)
http://cutroni.com/blog/
Supermetrics Data Grabber
http://supermetrics.com/
Automate Analytics Google Group
http://groups.google.com/group/automateanalytics/topics
Lunametrics blog
http://www.lunametrics.com/blog
Kissmetrics
http://blog.kissmetrics.com/
33. Case Study: Puppet Invasion
@erinblasco
#mwmetrics
Muppets copyright Disney. Fraggles copyright The
Jim Henson Company. Sesame Street characters
copyright Sesame Workshop.
34. Case Study: Puppet Invasion
• Opportunities: Muppets! Educational and
promotional goals should fit together nicely.
• Challenges: Not on display. Embargoed.
Needed a fresh angle.
35. Before writing goals: Who will I need
to report to about this?
• Must tell a complete story with a mix of
metrics, benchmarks, photos, and anecdotes.
• Various audiences: Data fiends, visual
learners. New Media, Education, Public
Affairs, Curatorial, and the Director.
36. Identify goals
• Highlight the diversity of
puppets (not just Miss Piggy)
• Promote the new donation
but manage expectations:
they won’t be on display
• Increase knowledge of and
interest in puppet
conservation, a fresh angle
37. Do one thing (strategy) to make each
goal happen
• Highlight diversity of
puppets (sorry, Piggy)
• Promote donation
while managing
expectations
• Increase knowledge
about puppet
conservation
• Tell stories of non-Piggy
puppets
• Show cool visuals of
puppets but explain
they aren’t on display
• Tell stories from
conservation lab,
highlight staff expertise
38. Select tactics that fit goals & strategies
• Stories of non-Piggy
puppets
• Pics of puppets but
captions explain they
aren’t on display
• Stories from
conservation lab,
highlight voices from
behind-the-scenes
• Guessing game with
Fraggle fuzz
• Flickr set with juicy
captions, blog post,
series of tweets
• Blog post highlighting
conservation process
with lots of quotes
39. Before implementing tactics: which
metrics will you gather and report?
• Tell stories beyond Miss
Piggy
• Guessing game with
Fraggle Rock puppet
• Participation level in
guessing game
• Metrics:
– Clicks to view photo
– Number of guesses on
Twitter, FB, Instagram
– Tone of responses (word
cloud?)
40. Before implementing tactics: which
metrics will you report?
• Increase puppet
conservation knowledge &
interest
• Tell stories from
conservation lab, highlight
staff expertise
• Blog post highlighting
conservation process with
lots of quotes
• Metrics
• Word cloud of hashtagged
tweets reveals high use of
conservationy keywords
• Anecdotal evidence from
tweets, quotes, comments
on blog post
• Impressions and
engagements on
conservationy posts higher
than benchmark posts
• Survey for followers
• Learning = linger time?
41. Before campaign starts: Implement
metrics gathering system
• Save and gather what’s
being said
• Storify best tweets
• IFTT recipes
• Make a Bitly bundle of
customized links for
easy stats
• Do friends have access
to enterprise metrics
systems?
42. Metrics gathering system: Tweet
Archivist
• Start TweetArchivist archive for hashtag
• Mentions by influencers
• Impressions
• “Top users”
• Volume over time
• Hootsuite also archives
43. Metrics gathering system
• Fine-tuned listening and
gathering:
• Hootsuite or TweetDeck search
columns
– With keywords and location filter
– Include “?” to track questions
– Search for people tweeting links
to your content
• Search on Twitter: are you in
“top” or “all” categories?
• Capture free Tweet Reach report
at intervals
44. Metrics gathering system
• Plan where participants
can put their photos
(Flickr Group); these are
gold in reports later!
• Compare hashtag volume
on Topsy trends
• Topsy’s most-shared
images
• Pre-write survey
questions and schedule a
tweet linking to it
45. Benchmarks gathering system
• Develop a benchmark:
what feels like “average”
activity the week before
the big campaign?
• Swap stats with a
colleague at a similar
institution to put your
campaign in context.
• Dig out your own stats
from past campaigns.
Export past tweets.
46. OMG ELEVATOR AMBUSH! Boss’s boss: How was that
puppet campaign you worked on?
• Our Fraggles game got 100 guesses—
twice our usual number of comments!
Some even shared Fraggly memories
with us.
• Content highlighting our conservators
was really popular—the pageview
record was broken in just a few hours.
• And one person actually tweeted that
she wanted to quit her job and become
a conservator!
47. Select a balanced blend of metrics for the report
What was the best part of the tweetup? “Bringing
‘behind the scenes’ look at museum exhibition to
people outside DC/around the world. I learned about
an exhibit I didn't know about, and now I want to go
see it in person.” – survey response by a tweeter who
followed online
1) Survey quote: testimonial
2) Word cloud of tweets shows themes of
conversation: relevant & on-topic
3) Storify of best tweets: endorsement
4) High traffic to blog post vs. last
month: exciting campaign content
Take home message: The project was popular, with almost 8,000 views, which is
higher than our usual average. Conversation was a major focus and key themes rose
to the surface. Participants raved about it, both publicly and in a survey.
48. Select a balanced blend of metrics for the report
On an average day in January, @amhistorymuseum is
mentioned in about 45 tweets. During #docsocial, we were
mentioned over 160 times.
This is a chart of the top people that tweeted with
the hashtag. NMAH talked a lot (56 tweets) but so
did many others. Most participants sent more than
10 tweets.
1) List of influencers: VIPs
2) A variety of voices participated
3) More mentions than usual:
conversation!
Take home message: The program enticed
many people to participate, including fancy
influencers. We reached more people than
usual and were a hot topic of conversation.
49. Usable metrics to make next time better
• Survey participants to identify what to do
differently next time.
• Keep some of the same questions in every
survey to compare across campaigns.
• Ask tweetup participants to tweet to their
own followers, too. Valuable feedback!
50. Usable metrics to make next time better
• Today’s campaign always
looks better than last year’s
because your base of social
media followers has grown.
• Pick one metric that still
compares despite growth,
such as the pace of growth
in pageviews.
• Did audiences clamor for
this content? How viral did
it feel?
51. Usable metrics to make next time better
• Collect and analyze fewer
metrics.
• Which report components
spoke to your stakeholders
most?
• Which were most
meaningful to you?
• Keep those. Jettison the
rest.
53. Activity #1
Help us fix our jumbled up report. Sort the
jumbled pieces of the report into categories
(goals, metrics, etc.) and arrange them
logically.
If you have time, critique the metrics. Do you
think they are meaningful? If not, write a new
one you think would be better.
54. AMBUSH!
You’re on the elevator and the museum director jumps
on. He’s read your report and has a few questions. No
escape! How might you answer him?
a) Which of these metrics is most and least meaningful?
b) I saw 3 negative comments on Facebook. Are you sure
the campaign was successful?
c) What questions do you still have about the campaign?
d) So did we “go viral” yet?
59. Target Audiences
Researchers (interns, fellows, academic
researchers)
Serious online researchers (Wikipedians, Collections
mystery solvers on Flickr Commons etc)
General public enthusiasts (Lifelong Learners, DIYer
looking for information and help)
Smithsonian record creators
60. Broad Organizational
Mission
“The Archives’ mission is to document the goals and
activities of the whole Smithsonian in its pursuit of
increasing and diffusing knowledge, and exciting
learning in everyone. The Archives is also responsible
for ensuring institutional accountability, and for
enhancing access to the rich and diverse resources in
its care. ”
64. Strategy: Tell Stories Highlighting Smithsonian
as Research & Education Catalyst
Blog KPI – Low Frequency Visits are trending down + High Frequency Visits are trending up!
77. Foresee Open Ended: Did not find looking
for
• Collections (30)
• Of 30 responses, only 4 were searching for
items we DO have.
• “list of architectural styles of the Smithsonian buildings”
• “Mary Henry's complete diary”
• “Electronic version of the Smithsonian Annual Report 1895.”
• “http:// siarchives.si.edu/history/exhibits/historicpicturessmithsonian but got
AFRICAN MUSEUM”
• Photos/Digitized Collections (3)
• In general, people were looking for more.
78. Foresee Open Ended: What ways can we
improve search?
Want “Google” results
Many of the titles are unclear about what they
contain
I wish audio were available
Need greater clarity on what can be accessed
electronically.
Access to records, pictures
Who to contact for help, live chat
83. Keep Learning
Know your bone (Colleen Dilenschneider) -
http://colleendilen.com/
Beth Kanter – How Connected Non-Profits Leverage
Networks & Data for Social Change-
http://www.bethkanter.org/
Avinash Kaushik - http://www.kaushik.net/avinash/
@digitaleffie
84. Creating a Social Media Metrics
Reporting Framework & Setting up
Analytics for Native Apps
Sarah Banks
Manager of Online Engagement
85. Goals for Air and Space’s
Metrics Reporting
1) Ties social media
reporting to museum
and institutional goals
2) Enables the gathering
of meaningful,
actionable data
3) Lets us know whether
or not we are meeting
our goals
4) Remains manageable,
and flexible as the
available metrics and
tools change
86. Creating the Framework: Step 1
Align Goals and Priorities, Select Metrics: Start
with strategic priorities for your institution.
Work down from there to your goals for social
media and select metrics.
Institutional Strategic Plan
Museum Goals
Social Media Goals
Quantitative & Qualitative Data
87. Example: Mapping Goals to
Data & Stories
1. Smithsonian Strategic Priority – Revitalizing education
2. Air and Space Museum Goal – Enhancing outreach efforts
3. Social Media Goal – Increase engagement with online
communities
1. Quantitative & Qualitative Data – Average numbers of
likes, comments, and shares per post; average number of
comments per day on Twitter; examples of comments
showing engagement and learning; most engaging
content on each platform
88. Creating the Framework: Step 2
Decide on Frequency of
Data Collection: How
often do you want to
collect and analyze data?
How do those reports
relate to each other?
89. Example: Reporting Frequencies
Frequency Purpose Distribution
Monthly Capture highlights, stories, and data
to create a snapshot of the month.
Web & New Media Dept,
Intranet
Quarterly Summarize highlights and stories,
and add analysis about what’s
working and what could be
improved.
Web & New Media Dept,
Web Advisory Group,
Intranet
Yearly Encapsulate the year’s activities,
successes, and areas for
improvement. Make
recommendations for changes to
goals, strategies, and tactics for the
next year.
Web & New Media Dept,
Web Advisory Group,
Intranet
Special
Projects/Events/
Campaigns
Provide a more in-depth look at a
story covered at a high-level in a
monthly report.
Participants, Web & New
Media Dept, Web Advisory
Group, Intranet
90. Creating the Framework: Step 3
Create Templates: Set up
your mapped priorities,
goals, and metrics and
think about how to fit in
the stories.
91. Example: Monthly Report
• Case Studies and Highlights -> stories that
capture the month’s activities and examples
that illustrate impacts
• Broadening Access -> growth in followers and
number of people reached by our content
• Revitalizing Education -> engagement metrics
92. Creating the Framework: Step 4
Test and Refine: Expect
bumps in the road as you
try to collect and analyze
data. Adjust as needed!
93. Lessons Learned from Initial
Implementation
Trouble Spots Successes
Had created something time-consuming
and therefore difficult to sustain.
Tying goals to metrics was useful for
positioning social media with institutional
priorities.
Found discrepancies between data
collected by 3rd-part tools and data seen
on posts.
Templates are flexible enough for our
needs as those change.
Discovered that a different setup of tools
was needed to make data collection
easier.
94. Refining & Adjusting
• Revised approach – start small and build
• Conducted further research on tools –
combination of Twitter analytics, Facebook
Insights, and TrueSocialMetrics
• Simplified data collection – focused on
essentials that are easy to track over time
95. Next Steps
• Keep experimenting
with how to convey
qualitative data.
• Continue looking at
how best to measure
whether goals are being
met.
97. What’s Possible?
• As with websites, Google Analytics is a powerful
tool to help you know if you’re meeting your
goals.
• You have to ask your developer to add it in. Can’t
be added later without an update to the app.
• Basic GA only give you so much. Event-based
analytics allow for more nuanced data about
usage.
98. How to Start?
1. Write down your goals for the app and think about the
concerns/questions that are arising during development.
2. Use the app to see where users take action – make a decision, click
something, etc.
3. Label each point of action, as well as things with duration. Those
are your event-based metrics.
4. Think ahead and label each event with “appname_label” in a way
that will make sense to someone who doesn’t know your system.
5. Put all labels in a spreadsheet and give to the developer.
100. Pilot Pals - Lots of Choices to Make!
• Enter name or not
• Mute or unmute sound
• Access parents section and
its tabs
• Select 1 of 4 games
• Select aircraft within games
• Try again
• Play again or return to main
menu
• Reset personalization
• Parents accessing different
parts of the parents section
101. Using Data to Address Concerns:
“Kids won’t pick the autogiro.”
103. #mwmetrics
Discuss today’s workshop “takeaways”: What are your Chicago-style
#mwmetrics ingredients?
103
Food for thought (to go)!
Crust: 1 way in which
you will try to implement
this metrics model
Chunky tomato sauce:
1 tool you’ll give a try
Toppings: 1 way you’ll
improve & jazz up reports
Cheese: 1 new way you’ll
communicate metrics to
colleagues or partners
Chicago style pizza photo by
HarlanH on Flickr. CC BY-NC 2.0.