Practical Agile Analytics: REDUCE UNCERTAINTY AND IMPROVE ESTIMATION ACCURACY

Practical Agile Analytics:
REDUCE UNCERTAINTY AND STOP MAKING
SUCH A BIG DEAL OUT OF STORY SIZING
SagePath Technologies, LLC Copyright 2017

 These slides focus on analyzing user story size estimates and actual
task hours to gauge the uncertainty around those estimates.
 Based on that analysis we will see that it is the uncertainty about the
work that needs to be done that makes accurate estimation difficult.
 An approach is suggested for reducing uncertainty and improving
estimation accuracy.
SagePath Technologies, LLC Copyright 2017 2
What’s in these slides

• Context and assumptions
• A few words about story points
• Uncertainty and story size
• User story size data
• Analysis – What the data tells us
• In a Nutshell: Key points and improving estimates
Outline

 In this presentation the following assumptions are made:
• The teams follow an Agile/scrum development process.
• Estimating the work to be done each sprint uses the concept of story points.
 The goal of this set of slides is to expand our understanding of the outcome
of the impact of uncertainty on the user story estimation process.
 We will analyze real Agile data with the idea of gaining some perspective on:
• How much effort should really be put into story size estimating.
• Why some estimates, either too high or too low, are so far off the mark.
Some Context

A Few Words about Story Points

 Story points aren’t a measure of the time needed to complete a feature, but
rather a measure of a feature’s size relative to other features. [ref 7]
 Using story points allows team members with different skill levels, and who may
perform at different speeds, to communicate about and agree on an estimate.
[ref 5]
 There are a number of approaches that are used to estimate story points.
Examples include T-shirt Sizing, Planning Poker, and Fibonacci numbers to name
a few. (see e.g. [ref 8])
A few words about Story Points

 Collaborative estimation helps ensure that teammates have a say in the
process and are committed to completing stories on time. [ref 2]
 These estimations encourage open discussion that may uncover potential
issues, unforeseen dependencies, and possible shortcuts. [ref 2]
 The estimation process moves the team to a state of shared understanding.
[ref 3]
Benefits of the Story Pointing Process

Story size estimation is the focus of a lot of discussion and effort. Just look at the
many articles, blogs, seminars, and training classes that cover this topic.
Though the estimation process is important, it is the accuracy of those estimates
that causes the most consternation and angst.
Why is this?
• Because in spite of the Agile manifesto, people† still need some idea about “How long is
this going to take?”
• More importantly, if we don’t have at least some idea about how much work there is and
how long it will take, how do we make reliable business commitments?
Story Sizing: What’s the Big Deal?
†People here refers to all stakeholders such as project, product, and functional managers, senior leadership, marketing, sales, and customers.

Uncertainty and Story Size

An Axiom of Software Development†
Software development is knowledge-based work that
by its nature always contains some level of uncertainty.
Estimating time and effort in software development projects
is a challenge. The primary reason for this is because:
†For other “axioms” of SW development see references 9, 10, and 11.

Story Size Estimating
Though we may view story size as an estimate of the effort needed to
complete the work, the accuracy of that estimate hinges on uncertainty.
The idea here is to understand where story size uncertainty comes from
and see if we can reduce it to the point where it does not contribute to
the inherent uncertainty in the software development process.

A common approach to estimating story size is to
use a Fibonacci sequence such as 1, 2, 3, 5, 8, 13.
The diagram on the left uses surface area to show
the relative size of each number in our Fibonacci
sequence.
Though Fibonacci sequences abound in nature,
there is no particular reason why user story sizes
have to line up with these numbers.
12
Estimating Story Size with Fibonacci

If the numbers represented just the relative level of effort to get to done on a user
story, then a linear sequence of sizes would do the trick.
Instead, we us a Fibonacci Sequence because:
13
So Why use a Fibonacci Sequence?
The increasing gaps between the numbers correspond to an
increasing level of our uncertainty about the amount of
work that needs to get done.

User story size estimates are typically relative estimates whereby the current story
is compared to a previously completed story of similar complexity.
These estimates are colored by each team member’s
 past experience and intuition
 knowledge base and skillset
 understanding of the nature of the work to be done
This last part, an understanding of the work, is where we find most of the
uncertainty.
To get a clearer picture of this, let’s look at what some data can tell us.
14
Story Point Uncertainty

User Story Size Data

The data we will look at is from a mix of new and well-established Agile
teams and consists of the following user story information:
 Estimated story size
 Actual task hours
Story sizing was done using a Fibonacci series of 1, 2, 3, 5, 8, 13.
About the Data

1. Plot the story size versus actual task hours for each user story.
2. Generate a histogram of actual task hours for each story size.
3. Calculate some basic statistics such as mean and standard deviation.
4. Examine the plots and note our observations.
5. Based on the observations and simple statistics, see what insights we can gain.
17
Data Analysis Approach

User Story Size, Task Hours Comparison
About the chart:
• This chart plots user story size against task hours.
• x-axis: actual task hours logged for each user story.
• y-axis: story size assigned during sprint planning.
• Each circle corresponds to a specific user story.
• Darker areas indicate a higher density of stories.
• Green markers indicate the mean actual task hours
for a given story size.

 There is significant overlap of actual task hours for all story sizes.
 The distribution of task hours for each story size appears to be skewed to the left.
 For small story sizes (1 and 2) there are only a few stories with task hours greater
than the mean.
 For medium and larger stories (3, 5, and 8) there are many stories with task hours
exceeding the mean.
 There seems to be a linear relationship between story size and the corresponding
mean task hours.
For a more quantitative understanding of the spread of
actual task hours, we look at histograms for each story size.
19
Observations about Story Size vs. Actuals

Story Size Histogram Matrix
About the charts:
• Each chart is a histogram of actual task hours
for a specific story size.
• x-axis: task hours logged for all stories of a
given size.
• y-axis: number of stories of a given size.
• The mean and standard deviation is
indicated on each chart by μ and σ.
• The coefficient of variation is indicated on
each chart by CV.
• CV is the standard deviation divided by the
mean (i.e. σ/μ).

 Except for story size 1, the CV values are roughly equal.
 For story sizes 1, 2, 3, and 5 the histograms are skewed to the left. This is
less clear for size 8 and 13.
 There appears to be a linear relationship between the story size and mean
task hours. For example, the mean task hours for size 8 stories is about
four times that of size 2.
o By plotting the μ and σ values, and using linear regression, we can verify this
observation.
21
Observations about the Histograms

22
Linear Regression of Task Hour Means
About the chart:
• This chart is a straight line fit (i.e. linear
regression) of the story size estimates as a
function of the mean task hours.
• The green diamonds are the mean task hours for
each story size.
• The blue and red squares correspond to +/- one
standard deviation from the mean.
• r2 is an indicator of the strength of the linear
relationship between two variables.

What the Data Tells Us

 The significant overlap of task hours seen in the scatterplot indicates a wide
variation in task hours for each story size and a corresponding wide variation in
story size estimates.
 Many stories were sized several sizes too large or too small. E.g. there were
many size 3 and 5 stories, and some size 8 stories that have actual task hours
below the mean of the size 2 stories.
 Since story size is an estimate of the relative effort needed to complete a story,
as compared to that of previously completed stories, the large over or under
sizing indicates the work to be completed is not well understood.
24
Overlap of Task Hours

 The skew in distribution of task hours is implied in the scatterplot and
verified by the histograms.
 This skew indicates that the sizing of the stories tended to favor
overestimation rather than underestimation.
 Though the data tends towards overestimation, there are a significant
number of stories underestimated. This is another indicator of the work to
be done not being well understood.
25
Left-Leaning Skew in Task Hours

 Except for story size 1, the CV values are roughly equal with values of about 0.6.
o This indicates the relative spread of task hours are comparable across story sizes larger than 1.
o This implies that variability in estimates is somewhat independent of story size.
o For story size estimates, a CV value of 0.6 indicates a large spread of actual task hours.
 The linear regression plot of the mean task hours has an r2 value of 0.997. This indicates a
very strong linear relationship between the mean task hours and story size.
 The implication is that story size is a strong indicator of the mean task hours.
 Unfortunately, knowing the mean task hours is not very helpful when there is such a large
variation in estimates.
26
Linearity of Story Size and Mean Task Hours

In a Nutshell: Key Points

 These Agile teams have accurate means, but large variation in
task hours.
 The cause of this large variation is uncertainty about the work
that needs to be done. This is consistent with the uncertainty
gap concept of the Fibonacci numbers used for story sizing.
 A reduction of this variation would result in improved estimates
of story sizes.
28
Key Points

To get to better estimates of user story size, reduce the uncertainty…
Reduce the uncertainty by gaining a better understanding of the work
that needs to get done…
To increase your understanding of the work…
1. Split stories into smaller and smaller pieces until the team is sure they understand what is
really entailed to complete the work.
2. Anticipate future sprints by setting aside time to do the “get ready” work during the
current sprint. Ensure you account for this when making your sprint commitment.
3. Actually do the “get ready” work.
Improve Story Size Estimates

Even if you forget everything else in this slide deck, keep this in mind:
1. When it comes to estimating work, get comfortable with ambiguity, you will be
happier and your friends will thank you for it.
2. Understand what an estimate is and do not over-commit:
• Recognize that estimates are not foretelling the future.
• Estimates are educated guesses.
• Estimates are not negotiable.
Reduce Story Size Estimating Angst

 The data analyzed in this presentation is from a mix of new and well-
established Agile teams.
 The results from other teams or teams of teams, e.g. your organization,
may differ from those presented here.
 None the less, such an analysis is straightforward. You will likely gain
useful insights about your own organization and story size estimating
through a similar analysis.
Your Mileage May Vary

About the Data Visuals

The Data Plots:
• The data was analyzed using Python 3.6 and pandas from the Python Data Analysis Library .
• The data was plotted using the Python Matplotlib plotting library.
Background Images:
All background images are licensed under the Creative Commons Zero (CC0) license from the following sources:
• unsplash.com : slides 10, 11, 32
• pixabay.com : slides 1, 2, 5, 9, 27, 30
• pexel.com : slides 15, 16, 23, 31
Data Visuals

1. Sizing and Estimates Overview, https://help.rallydev.com/sizing-and-estimates-overview.
2. Agile Estimation: Sizing Stories, https://www.productmanagerhq.com/2015/03/agile-estimation-sizing-stories/.
3. The 5 Stages of User Story Sizing, http://illustratedagile.com/2012/11/13/the-5-stages-of-user-story-sizing/.
4. What is a story point, https://agilefaq.wordpress.com/2007/11/13/what-is-a-story-point/.
5. The Main Benefit of Story Points, https://www.mountaingoatsoftware.com/blog/the-main-benefit-of-story-points.
6. Success Story: How to Estimate Quickly and Efficiently, https://www.scrumalliance.org/community/articles/2013/november/success-
story-how-to-estimate-quickly-and-efficien.
7. Estimating with Story Points, https://msdn.microsoft.com/en-us/library/hh273055(v=vs.88).aspx.
8. 7 Agile Estimation Techniques – beyond Planning Poker, https://technology.amis.nl/2016/03/23/8-agile-estimation-techniques-
beyond-planning-poker/.
9. Joe’s Axioms of Software Development, http://www.softwareops.com/blog/2014/5/23/joes-axioms-of-software-development.
10. Software Development Axioms, John Gmutza, leanpub.com, 2012.
11. Two axioms of lean software development, http://leansoftwareengineering.com/2008/08/25/two-axioms-of-lean-software-
development/.
References

Practical Agile Analytics: REDUCE UNCERTAINTY AND IMPROVE ESTIMATION ACCURACY

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Practical Agile Analytics: REDUCE UNCERTAINTY AND IMPROVE ESTIMATION ACCURACY

Similar a Practical Agile Analytics: REDUCE UNCERTAINTY AND IMPROVE ESTIMATION ACCURACY (20)

Último

Último (20)

Practical Agile Analytics: REDUCE UNCERTAINTY AND IMPROVE ESTIMATION ACCURACY