SlideShare una empresa de Scribd logo
1 de 38
Descargar para leer sin conexión
SagePath Technologies, LLC Copyright 2017
Practical Agile Analytics:
MEASURE PREDICTABILITY AND QUANTIFY RISK
WITH CYCLE TIME
1
One of the promises of Agile is that SW development
teams, and teams of teams, will be more predictable.
Ok, but...
 Why does predictability matter?
 How do I quantify it?
SagePath Technologies, LLC Copyright 2017 2
What’s in these slides
 These slides focus on using user story cycle time data to measure the
predictability of Agile software development teams.
 The same data that gives us our predictability measure is then used
to quantify the likelihood of user story completion.
 Once we know the likelihood of user story completion, we can put a
number on the risk of missing software delivery commitments.
SagePath Technologies, LLC Copyright 2017 3
What’s in these slides
• Predictability in software development
• Why predictability matters
• Uncertainty and Agile team metrics
• Cycle time
• Cycle time data analysis
• In a nutshell: A few essentials to remember
• About the data visuals
SagePath Technologies, LLC Copyright 2017 4
Outline
SagePath Technologies, LLC Copyright 2017 5
Predictability and Software Development
When we say someone is predictable typically we mean:
 We have a pretty good idea of what we are going to get and when.
 We expect to see similar outcomes in the future.
Predictable people make promises they can keep:
 They do what they said they were going to do.
 They complete things when they said they would.
6
What do we mean by Predictability?
SagePath Technologies, LLC Copyright 2017
Predictable Agile teams:
 Consistently deliver on sprint commitments
 Typically deliver code that is working, tested, and maintainable
 As a team of teams, consistently meet program objectives
 There is high confidence this behavior will continue
SagePath Technologies, LLC Copyright 2017 7
Characteristics of Predictable Agile Teams
SagePath Technologies, LLC Copyright 2017 8
Why Does Predictability Matter?
 Are more teams needed? If so, how big should the teams be?
 Is there a risk of missing delivery commitments?
 Should outside vendors be considered?
 What delivery timeframe should be communicated to customers so they can
effectively plan out their testing, maintenance, and upgrades?
 What training is needed and when should that be scheduled?
SagePath Technologies, LLC Copyright 2017 9
Predictability and Software Deliveries
Knowing how much software is likely to be delivered in a
given timeframe helps answer key planning questions such as:
SagePath Technologies, LLC Copyright 2017 10
Predictability and Resource Allotment
The ability to
 Accurately estimate delivery dates
 Properly resource projects
 And with confidence, make commitments to customers
Directly impacts funding and investment decisions
Predictability is a key element to achieving a successful
scaled Agile process.
• Effective and efficient collaboration between teams requires trust that the
other teams will complete their work when they said they would.
• Large complex SW projects and systems often have multiple, far-reaching
dependencies.
• Predictability is fundamental to effective dependency management.
SagePath Technologies, LLC Copyright 2017 11
Predictability and Scaling Agile
Poor predictability of software delivery teams leads to:
 Missed delivery dates and missed revenue targets
 Low confidence in delivery commitments and loss of customer trust
 Erosion of trust between development teams and senior management
 Increased schedule pressure and a corresponding drop in quality
 Schedule delays and the associated financial consequences
 Re-planning and priority shifts that disrupt development and exacerbate
project risk
SagePath Technologies, LLC Copyright 2017 12
Poor Predictability is Insidious
SagePath Technologies, LLC Copyright 2017 13
Uncertainty and Agile Team Metrics
Software development is knowledge-based work that by its
nature always contains some level of uncertainty.
A few examples of uncertainties that can arise in a software project:
• Gaps in development team capabilities and skillsets
• The challenge of estimating work effort when innovation is required to get the job done –
How do you schedule a breakthrough?
• Ambiguity in requirements and unclear technical paths
• Test planning and execution – How much testing is enough?
SagePath Technologies, LLC Copyright 2017 14
Uncertainty Makes it Harder to be Predictable
 Typical Agile team metrics include things such as story points, velocity,
capacity, and task hours.
 These metrics can help teams understand how well they are doing with
respect to their past performance.
 At the program level these metrics can be difficult to interpret.
 It is often unclear how these metrics can be used to assess value delivery
and program level productivity.
15
Agile Team Metrics
SagePath Technologies, LLC Copyright 2017
One way to address this is to use a metric that
reflects actual work delivered at a macroscopic level.
Such a metric is Cycle Time.
16
Agile Team Metrics
SagePath Technologies, LLC Copyright 2017
Additional challenges with Agile team metrics can include:
- Unstable team velocities
- Inconsistent tracking of actual task hours
- Committing to user stories that are too big
- Story point estimation not being normalized across all teams
SagePath Technologies, LLC Copyright 2017 17
Cycle Time
SagePath Technologies, LLC Copyright 2017 18
What is Cycle Time?
Cycle Time:
The amount of elapsed time it takes for a given work
activity to be completed.
 It’s easy to measure
 It’s easy to understand
 It measures something real and important
 It’s a difficult metric to “game”
SagePath Technologies, LLC Copyright 2017 19
Cycle Time as a Metric
SagePath Technologies, LLC Copyright 2017 20
Cycle Times in an Agile Process
There are a number of different cycle time measures that can be
applied to an Agile/scrum process. A few examples include:
• Backlog-to-Ready
• Ready-to-In Process
• Backlog-to-Deployed
In these slides our interest is in the predictability of SW teams so we
will look at the amount of time spent on story implementation.
This is referred to as the time In-Process.
In subsequent slides the term cycle time refers to:
The number of days between when a team pulls a user story into a sprint to begin
work on it and when the story is accepted by the Product Owner.
In the parlance of work flow:
This is the elapsed time between the date a user story goes into the In-Process state
and the date it goes into the Accepted state.
SagePath Technologies, LLC Copyright 2017 21
In-Process Cycle Time
 It’s a direct measure of the time it takes to get something done.
 It doesn’t require detailed information from individual
developers such as actual task hours.
 It mitigates the variabilities of unstable velocities and non-
normalized story points.
 It applies to both scrum and Kanban teams.
SagePath Technologies, LLC Copyright 2017 22
Why Use the In-Process Cycle Time?
SagePath Technologies, LLC Copyright 2017 23
Cycle Time Data Analysis
• The data used for analysis is from an Agile team of teams.
• This team of teams uses an electronic tool to track workflow. Cycle time data is
easily extracted from this tool using an API.
• The cycle time data is for user stories started and completed in 2015 and user
stories started and completed in 2016.
• This team of teams made two significant process changes in January of 2016:
 They moved from 3 week sprints to 2 week sprints
 They adopted a scaled Agile process
SagePath Technologies, LLC Copyright 2017 24
About the Data
Some data scrubbing is often required before proceeding with
analysis. A few of the reasons for this include:
• Missing data points
• Data that needs reformatting. E.g. string data that should be numeric.
• Data that is suspect and may be in error
• Inconsistencies in the processes for gathering and input of some data points
The following user story categories were removed from analysis:
• User stories where the in-process date and the accepted date are the same.
• System test user stories.
• Stories flagged as obsolete.
SagePath Technologies, LLC Copyright 2017 25
Data Scrubbing
When it comes to data analysis, one of the first things a
person should do is plot the data and look at it.
 The idea is to get a general sense of what we are dealing with and see if there
are any obvious trends or unusual features.
 One good tool for doing this is a scatter plot.
 Sometimes it is also useful to include a corresponding histogram to help gauge
the density of the data.
SagePath Technologies, LLC Copyright 2017 26
Looking at the Data
About the Chart:
• The chart on the left is a scatter plot of user stories
started and accepted in 2015.
• Each circle corresponds to a specific user story.
• The x-axis is the date a user story was accepted.
• The y-axis is user story cycle time in days.
• To the right of the scatter plot is a cycle time histogram
of the 2015 user stories.
What does this scatter plot tell us?
• The first thing to note is how spread out the data is in
along the y-axis (i.e. cycle times are all over the place).
• Since most of the teams are using scrum, the expectation
is that story acceptance would be clumped near or
below the sprint duration of 21 days.
• The histogram confirms what we are seeing. A significant
number of stories took 20 or more days to complete.
SagePath Technologies, LLC Copyright 2017 27
User Story Scatter Plot
About the Chart:
• The chart on the left is a scatter plot of user stories
started and accepted in 2016.
• Each circle corresponds to a specific user story.
• The x-axis is the date a user story was accepted.
• The y-axis is user story cycle time in days.
• To the right of the scatter plot is a cycle time histogram
of the 2016 user stories.
What does this scatter plot tell us?
• Contrary to the 2015 scatter plot, in this scatter plot the
user stories are more closely grouped along the lower
part of the chart.
• The histogram shows a strong peak between 10 and 15
days. This correlates well with the change to 2 week
sprints in 2016.
SagePath Technologies, LLC Copyright 2017 28
User Story Scatter Plot
About the Chart:
• In this chart the 2015 and 2016 scatter plots have been overlaid.
This plot is referred to as a “splatter plot†”
• The diameter of the marker circle for each user story is scaled
according to the size of the story it represents.
What does this plot tell us?
• The first thing to note is that the maximum story size is much
greater in the 2015 data than the 2016 data.
• The larger user stories have cycle times significantly longer than
one or two sprints.
• Somewhat unexpected: there are a number of user stories that are
small in size, but also have cycle times significantly longer than one
or two sprints.
SagePath Technologies, LLC Copyright 2017 29
Spatter Plot
†The earliest reference to the term “splatter plot” that I am aware of is by Dennis
Sweitzer in his presentation When Projects go Splat: Introducing Splatter Plots.
SagePath Technologies, LLC Copyright 2017 30
Quantifying Predictability
Predictability is the likelihood that any particular user story will
be completed within a given time frame such as one sprint.
If we can measure this likelihood, we can quantify predictability.
This likelihood can be determined from the cycle time data by
calculating the cumulative percentage of story completion over
time.
SagePath Technologies, LLC Copyright 2017 31
Measuring Predictability
About the Chart:
• This chart shows the cumulative percentage of 2015 stories that
were accepted over time.
• Note that the x-axis is in units of sprints rather than days.
• In the ideal, all stories that were committed to in a given sprint are
completed within that sprint.
• At the 1 sprint line, the larger the percentage of completed stories
the closer the teams are to the ideal.
Of Note:
• Only about 43% of the user stories were completed within 1 sprint.
• After 2 sprints the completion percentage is at 74%.
— This means that over one quarter of the user stories took longer than 2
sprints to complete
• Based on this data, the likelihood that any particular story would be
completed within one sprint is well short of 50-50.
2015
SagePath Technologies, LLC Copyright 2017 32
Measuring Predictability
About the Chart:
• This chart overlays the cumulative percentage of accepted user
stories from 2015 and 2016.
A look at the numbers:
• In 2016 the acceptance percentage within 1 sprint is 73%. This is a
significant improvement from the 43% seen in 2015.
• After 2 sprints the 2016 completion percentage is above 90%.
• Between 2015 and 2016 the likelihood that a user story will be
completed within 1 sprint increased from 43% to 73%.
• In 2016 user stories were 67% more likely to be completed within
1 sprint than in 2015. This means SW delivery risk decreased in
2016.
• Between 2015 and 2016 the aggregate team of teams
predictability increased by a factor of about 1.7.
Improvement in
Predictability
2016
2015
SagePath Technologies, LLC Copyright 2017 33
In a Nutshell: A few essentials to remember
SagePath Technologies, LLC Copyright 2017 34
Predictability can be Measured and Quantified
By tracking cycle time we can quantify predictability.
In other words:
We can put a number on the likelihood that user story
commitments and program objectives will be met.
SagePath Technologies, LLC Copyright 2017 35
Quantifying Software Delivery Risk
Timely completion of user stories is a highly desirable objective.
Therefore, we can define SW delivery risk as:
Risk = 1 – (likelihood of on-time completion)
Once we quantify the likelihood of user story completion, we
can also quantify the corresponding project and program risk.
 When you want to help teams understand their own work trends,
focus on team metrics such as velocity.
 For forecasting elapsed time of software deliverables focus on work
delivered at a macroscopic level.
 Forecast the likelihood of timely Agile team deliveries by looking at in-
process cycle time.
 Cycle time allows us to quantify software project delivery risk.
SagePath Technologies, LLC Copyright 2017 36
Key Takeaways
SagePath Technologies, LLC Copyright 2017 37
About the Data Visuals
The Data Plots:
• The data was analyzed using Python 3.6 and pandas from the Python Data Analysis Library .
• The data was plotted using the Python Matplotlib plotting library.
Background Images:
All background images are licensed under the Creative Commons Zero (CC0) license from the following sources:
unsplash.com : slides 17, 18, 21, 22, 26, 30, 34-37
pixabay.com : slides 1, 3, 5, 8, 13, 33
pexel.com : slides 23, 24
SagePath Technologies, LLC Copyright 2017 38
Data Visuals

Más contenido relacionado

La actualidad más candente

Scrum retrospective
Scrum retrospective Scrum retrospective
Scrum retrospective Priyanka Rana
 
[HCM Scrum Breakfast] Agile estimation - Story points
[HCM Scrum Breakfast] Agile estimation - Story points[HCM Scrum Breakfast] Agile estimation - Story points
[HCM Scrum Breakfast] Agile estimation - Story pointsScrum Breakfast Vietnam
 
A. Kamran's DoD and DoR: Definition of Done and Definition of Ready in Scrum
A. Kamran's DoD and DoR: Definition of Done and Definition of Ready in ScrumA. Kamran's DoD and DoR: Definition of Done and Definition of Ready in Scrum
A. Kamran's DoD and DoR: Definition of Done and Definition of Ready in ScrumArman Kamran
 
Agile stories, estimating and planning
Agile stories, estimating and planningAgile stories, estimating and planning
Agile stories, estimating and planningDimitri Ponomareff
 
The Agile Adoption Roadmap (Keynote by Tim Abbott)
The Agile Adoption Roadmap  (Keynote by Tim Abbott)The Agile Adoption Roadmap  (Keynote by Tim Abbott)
The Agile Adoption Roadmap (Keynote by Tim Abbott)Agile Days Middle East
 
Scaling Agile With SAFe (Scaled Agile Framework)
Scaling Agile With SAFe (Scaled Agile Framework)Scaling Agile With SAFe (Scaled Agile Framework)
Scaling Agile With SAFe (Scaled Agile Framework)Andreano Lanusse
 
How to estimate in scrum
How to estimate in scrumHow to estimate in scrum
How to estimate in scrumGloria Stoilova
 
SAFe® - scaled agile framework in practice
SAFe® - scaled agile framework in practiceSAFe® - scaled agile framework in practice
SAFe® - scaled agile framework in practiceIntland Software GmbH
 
Understanding the Agile Release and Sprint Planning Process
Understanding the Agile Release and Sprint Planning Process Understanding the Agile Release and Sprint Planning Process
Understanding the Agile Release and Sprint Planning Process John Derrico
 
Backlog Refinement 101 & 202
Backlog Refinement 101 & 202Backlog Refinement 101 & 202
Backlog Refinement 101 & 202David Hanson
 
Agile Estimation Techniques.pptx
Agile Estimation Techniques.pptxAgile Estimation Techniques.pptx
Agile Estimation Techniques.pptxPriyanka Gurnani
 
Agile 101
Agile 101Agile 101
Agile 101beLithe
 
Agile effort estimation
Agile effort estimation Agile effort estimation
Agile effort estimation Elad Sofer
 
Agile-overview: Agile Manifesto, Agile principles and Agile Methodologies
Agile-overview: Agile Manifesto, Agile principles and Agile MethodologiesAgile-overview: Agile Manifesto, Agile principles and Agile Methodologies
Agile-overview: Agile Manifesto, Agile principles and Agile MethodologiesBalaji Sathram
 

La actualidad más candente (20)

Scrum retrospective
Scrum retrospective Scrum retrospective
Scrum retrospective
 
[HCM Scrum Breakfast] Agile estimation - Story points
[HCM Scrum Breakfast] Agile estimation - Story points[HCM Scrum Breakfast] Agile estimation - Story points
[HCM Scrum Breakfast] Agile estimation - Story points
 
Agile Retrospectives
Agile RetrospectivesAgile Retrospectives
Agile Retrospectives
 
Agile estimation
Agile estimationAgile estimation
Agile estimation
 
A. Kamran's DoD and DoR: Definition of Done and Definition of Ready in Scrum
A. Kamran's DoD and DoR: Definition of Done and Definition of Ready in ScrumA. Kamran's DoD and DoR: Definition of Done and Definition of Ready in Scrum
A. Kamran's DoD and DoR: Definition of Done and Definition of Ready in Scrum
 
Agile stories, estimating and planning
Agile stories, estimating and planningAgile stories, estimating and planning
Agile stories, estimating and planning
 
The Agile Adoption Roadmap (Keynote by Tim Abbott)
The Agile Adoption Roadmap  (Keynote by Tim Abbott)The Agile Adoption Roadmap  (Keynote by Tim Abbott)
The Agile Adoption Roadmap (Keynote by Tim Abbott)
 
SCRUM Estimation
SCRUM EstimationSCRUM Estimation
SCRUM Estimation
 
Scaling Agile With SAFe (Scaled Agile Framework)
Scaling Agile With SAFe (Scaled Agile Framework)Scaling Agile With SAFe (Scaled Agile Framework)
Scaling Agile With SAFe (Scaled Agile Framework)
 
Agile introduction
Agile introductionAgile introduction
Agile introduction
 
How to estimate in scrum
How to estimate in scrumHow to estimate in scrum
How to estimate in scrum
 
SAFe® - scaled agile framework in practice
SAFe® - scaled agile framework in practiceSAFe® - scaled agile framework in practice
SAFe® - scaled agile framework in practice
 
Understanding the Agile Release and Sprint Planning Process
Understanding the Agile Release and Sprint Planning Process Understanding the Agile Release and Sprint Planning Process
Understanding the Agile Release and Sprint Planning Process
 
Backlog Refinement 101 & 202
Backlog Refinement 101 & 202Backlog Refinement 101 & 202
Backlog Refinement 101 & 202
 
Agile Estimation Techniques.pptx
Agile Estimation Techniques.pptxAgile Estimation Techniques.pptx
Agile Estimation Techniques.pptx
 
Scrum Process
Scrum ProcessScrum Process
Scrum Process
 
Agile 101
Agile 101Agile 101
Agile 101
 
Agile effort estimation
Agile effort estimation Agile effort estimation
Agile effort estimation
 
Agile Methodology
Agile MethodologyAgile Methodology
Agile Methodology
 
Agile-overview: Agile Manifesto, Agile principles and Agile Methodologies
Agile-overview: Agile Manifesto, Agile principles and Agile MethodologiesAgile-overview: Agile Manifesto, Agile principles and Agile Methodologies
Agile-overview: Agile Manifesto, Agile principles and Agile Methodologies
 

Similar a Practical agile analytics: Measure predictability and quantify risk with cycle time

Agile Governance for Hybrid Programs
Agile Governance for Hybrid ProgramsAgile Governance for Hybrid Programs
Agile Governance for Hybrid ProgramsCprime
 
Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...
Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...
Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...Laszlo Szalvay
 
Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...
Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...
Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...Hannah Flynn
 
How and Why: Embedded Analytics Interfaces For Your SaaS Product
How and Why: Embedded Analytics Interfaces For Your SaaS ProductHow and Why: Embedded Analytics Interfaces For Your SaaS Product
How and Why: Embedded Analytics Interfaces For Your SaaS ProductAggregage
 
Microsoft DevOps - Fast track
Microsoft DevOps - Fast track Microsoft DevOps - Fast track
Microsoft DevOps - Fast track girish goudar
 
significance_of_test_estimating_in_the_software_development.pptx
significance_of_test_estimating_in_the_software_development.pptxsignificance_of_test_estimating_in_the_software_development.pptx
significance_of_test_estimating_in_the_software_development.pptxsarah david
 
significance_of_test_estimating_in_the_software_development.pdf
significance_of_test_estimating_in_the_software_development.pdfsignificance_of_test_estimating_in_the_software_development.pdf
significance_of_test_estimating_in_the_software_development.pdfsarah david
 
6 Steps to Confirm Successful Workday Deployment
6 Steps to Confirm Successful Workday Deployment6 Steps to Confirm Successful Workday Deployment
6 Steps to Confirm Successful Workday DeploymentZaranTech LLC
 
Allstate-T&M for ITSM-Kirch Final ipad
Allstate-T&M for ITSM-Kirch Final ipadAllstate-T&M for ITSM-Kirch Final ipad
Allstate-T&M for ITSM-Kirch Final ipadCathy Kirch
 
Agile Methods to Develop Tangible Products Quickly
Agile Methods to Develop Tangible Products QuicklyAgile Methods to Develop Tangible Products Quickly
Agile Methods to Develop Tangible Products QuicklyJohn Carter
 
IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...
IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...
IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...Quantitative Software Management, Inc.
 
Capacity Planning and Demand Management
Capacity Planning and Demand ManagementCapacity Planning and Demand Management
Capacity Planning and Demand ManagementLawrence Putnam Jr
 
How to build a data analytics strategy in a digital world
How to build a data analytics strategy in a digital worldHow to build a data analytics strategy in a digital world
How to build a data analytics strategy in a digital worldCaseWare IDEA
 
CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...
CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...
CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...TheConnectedCause
 

Similar a Practical agile analytics: Measure predictability and quantify risk with cycle time (20)

English digital business 2.1.pptx
English digital business 2.1.pptxEnglish digital business 2.1.pptx
English digital business 2.1.pptx
 
Agile Governance for Hybrid Programs
Agile Governance for Hybrid ProgramsAgile Governance for Hybrid Programs
Agile Governance for Hybrid Programs
 
Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...
Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...
Proposed Title Fear and Loathing in Agility: Long Live the Accounting Departm...
 
Innovate session-2333
Innovate session-2333Innovate session-2333
Innovate session-2333
 
Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...
Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...
Modern Product Data Workflows: How and Why: Embedded Analytics Interfaces For...
 
How and Why: Embedded Analytics Interfaces For Your SaaS Product
How and Why: Embedded Analytics Interfaces For Your SaaS ProductHow and Why: Embedded Analytics Interfaces For Your SaaS Product
How and Why: Embedded Analytics Interfaces For Your SaaS Product
 
Microsoft DevOps - Fast track
Microsoft DevOps - Fast track Microsoft DevOps - Fast track
Microsoft DevOps - Fast track
 
Agile at scale
Agile at scaleAgile at scale
Agile at scale
 
significance_of_test_estimating_in_the_software_development.pptx
significance_of_test_estimating_in_the_software_development.pptxsignificance_of_test_estimating_in_the_software_development.pptx
significance_of_test_estimating_in_the_software_development.pptx
 
significance_of_test_estimating_in_the_software_development.pdf
significance_of_test_estimating_in_the_software_development.pdfsignificance_of_test_estimating_in_the_software_development.pdf
significance_of_test_estimating_in_the_software_development.pdf
 
Resume - Anil Kumar Krishna
Resume - Anil Kumar KrishnaResume - Anil Kumar Krishna
Resume - Anil Kumar Krishna
 
6 Steps to Confirm Successful Workday Deployment
6 Steps to Confirm Successful Workday Deployment6 Steps to Confirm Successful Workday Deployment
6 Steps to Confirm Successful Workday Deployment
 
Allstate-T&M for ITSM-Kirch Final ipad
Allstate-T&M for ITSM-Kirch Final ipadAllstate-T&M for ITSM-Kirch Final ipad
Allstate-T&M for ITSM-Kirch Final ipad
 
Agile in a nutshell
Agile in a nutshellAgile in a nutshell
Agile in a nutshell
 
Agile in a nutshell
Agile in a nutshellAgile in a nutshell
Agile in a nutshell
 
Agile Methods to Develop Tangible Products Quickly
Agile Methods to Develop Tangible Products QuicklyAgile Methods to Develop Tangible Products Quickly
Agile Methods to Develop Tangible Products Quickly
 
IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...
IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...
IT Demand Management and Capacity Planning: Why Estimation Is Vital to Balanc...
 
Capacity Planning and Demand Management
Capacity Planning and Demand ManagementCapacity Planning and Demand Management
Capacity Planning and Demand Management
 
How to build a data analytics strategy in a digital world
How to build a data analytics strategy in a digital worldHow to build a data analytics strategy in a digital world
How to build a data analytics strategy in a digital world
 
CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...
CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...
CRMready Webinar Series - Part 3 - How to Make Your Nonprofit’s CRM Implement...
 

Último

Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
How to Improve the Employee Experience? - HRMS Software
How to Improve the Employee Experience? - HRMS SoftwareHow to Improve the Employee Experience? - HRMS Software
How to Improve the Employee Experience? - HRMS SoftwareNYGGS Automation Suite
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
React 19: Revolutionizing Web Development
React 19: Revolutionizing Web DevelopmentReact 19: Revolutionizing Web Development
React 19: Revolutionizing Web DevelopmentBOSC Tech Labs
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
Understanding Native Mobile App Development
Understanding Native Mobile App DevelopmentUnderstanding Native Mobile App Development
Understanding Native Mobile App DevelopmentMobulous Technologies
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 

Último (20)

Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
How to Improve the Employee Experience? - HRMS Software
How to Improve the Employee Experience? - HRMS SoftwareHow to Improve the Employee Experience? - HRMS Software
How to Improve the Employee Experience? - HRMS Software
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
React 19: Revolutionizing Web Development
React 19: Revolutionizing Web DevelopmentReact 19: Revolutionizing Web Development
React 19: Revolutionizing Web Development
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
Understanding Native Mobile App Development
Understanding Native Mobile App DevelopmentUnderstanding Native Mobile App Development
Understanding Native Mobile App Development
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 

Practical agile analytics: Measure predictability and quantify risk with cycle time

  • 1. SagePath Technologies, LLC Copyright 2017 Practical Agile Analytics: MEASURE PREDICTABILITY AND QUANTIFY RISK WITH CYCLE TIME 1
  • 2. One of the promises of Agile is that SW development teams, and teams of teams, will be more predictable. Ok, but...  Why does predictability matter?  How do I quantify it? SagePath Technologies, LLC Copyright 2017 2 What’s in these slides
  • 3.  These slides focus on using user story cycle time data to measure the predictability of Agile software development teams.  The same data that gives us our predictability measure is then used to quantify the likelihood of user story completion.  Once we know the likelihood of user story completion, we can put a number on the risk of missing software delivery commitments. SagePath Technologies, LLC Copyright 2017 3 What’s in these slides
  • 4. • Predictability in software development • Why predictability matters • Uncertainty and Agile team metrics • Cycle time • Cycle time data analysis • In a nutshell: A few essentials to remember • About the data visuals SagePath Technologies, LLC Copyright 2017 4 Outline
  • 5. SagePath Technologies, LLC Copyright 2017 5 Predictability and Software Development
  • 6. When we say someone is predictable typically we mean:  We have a pretty good idea of what we are going to get and when.  We expect to see similar outcomes in the future. Predictable people make promises they can keep:  They do what they said they were going to do.  They complete things when they said they would. 6 What do we mean by Predictability? SagePath Technologies, LLC Copyright 2017
  • 7. Predictable Agile teams:  Consistently deliver on sprint commitments  Typically deliver code that is working, tested, and maintainable  As a team of teams, consistently meet program objectives  There is high confidence this behavior will continue SagePath Technologies, LLC Copyright 2017 7 Characteristics of Predictable Agile Teams
  • 8. SagePath Technologies, LLC Copyright 2017 8 Why Does Predictability Matter?
  • 9.  Are more teams needed? If so, how big should the teams be?  Is there a risk of missing delivery commitments?  Should outside vendors be considered?  What delivery timeframe should be communicated to customers so they can effectively plan out their testing, maintenance, and upgrades?  What training is needed and when should that be scheduled? SagePath Technologies, LLC Copyright 2017 9 Predictability and Software Deliveries Knowing how much software is likely to be delivered in a given timeframe helps answer key planning questions such as:
  • 10. SagePath Technologies, LLC Copyright 2017 10 Predictability and Resource Allotment The ability to  Accurately estimate delivery dates  Properly resource projects  And with confidence, make commitments to customers Directly impacts funding and investment decisions
  • 11. Predictability is a key element to achieving a successful scaled Agile process. • Effective and efficient collaboration between teams requires trust that the other teams will complete their work when they said they would. • Large complex SW projects and systems often have multiple, far-reaching dependencies. • Predictability is fundamental to effective dependency management. SagePath Technologies, LLC Copyright 2017 11 Predictability and Scaling Agile
  • 12. Poor predictability of software delivery teams leads to:  Missed delivery dates and missed revenue targets  Low confidence in delivery commitments and loss of customer trust  Erosion of trust between development teams and senior management  Increased schedule pressure and a corresponding drop in quality  Schedule delays and the associated financial consequences  Re-planning and priority shifts that disrupt development and exacerbate project risk SagePath Technologies, LLC Copyright 2017 12 Poor Predictability is Insidious
  • 13. SagePath Technologies, LLC Copyright 2017 13 Uncertainty and Agile Team Metrics
  • 14. Software development is knowledge-based work that by its nature always contains some level of uncertainty. A few examples of uncertainties that can arise in a software project: • Gaps in development team capabilities and skillsets • The challenge of estimating work effort when innovation is required to get the job done – How do you schedule a breakthrough? • Ambiguity in requirements and unclear technical paths • Test planning and execution – How much testing is enough? SagePath Technologies, LLC Copyright 2017 14 Uncertainty Makes it Harder to be Predictable
  • 15.  Typical Agile team metrics include things such as story points, velocity, capacity, and task hours.  These metrics can help teams understand how well they are doing with respect to their past performance.  At the program level these metrics can be difficult to interpret.  It is often unclear how these metrics can be used to assess value delivery and program level productivity. 15 Agile Team Metrics SagePath Technologies, LLC Copyright 2017
  • 16. One way to address this is to use a metric that reflects actual work delivered at a macroscopic level. Such a metric is Cycle Time. 16 Agile Team Metrics SagePath Technologies, LLC Copyright 2017 Additional challenges with Agile team metrics can include: - Unstable team velocities - Inconsistent tracking of actual task hours - Committing to user stories that are too big - Story point estimation not being normalized across all teams
  • 17. SagePath Technologies, LLC Copyright 2017 17 Cycle Time
  • 18. SagePath Technologies, LLC Copyright 2017 18 What is Cycle Time? Cycle Time: The amount of elapsed time it takes for a given work activity to be completed.
  • 19.  It’s easy to measure  It’s easy to understand  It measures something real and important  It’s a difficult metric to “game” SagePath Technologies, LLC Copyright 2017 19 Cycle Time as a Metric
  • 20. SagePath Technologies, LLC Copyright 2017 20 Cycle Times in an Agile Process There are a number of different cycle time measures that can be applied to an Agile/scrum process. A few examples include: • Backlog-to-Ready • Ready-to-In Process • Backlog-to-Deployed In these slides our interest is in the predictability of SW teams so we will look at the amount of time spent on story implementation. This is referred to as the time In-Process.
  • 21. In subsequent slides the term cycle time refers to: The number of days between when a team pulls a user story into a sprint to begin work on it and when the story is accepted by the Product Owner. In the parlance of work flow: This is the elapsed time between the date a user story goes into the In-Process state and the date it goes into the Accepted state. SagePath Technologies, LLC Copyright 2017 21 In-Process Cycle Time
  • 22.  It’s a direct measure of the time it takes to get something done.  It doesn’t require detailed information from individual developers such as actual task hours.  It mitigates the variabilities of unstable velocities and non- normalized story points.  It applies to both scrum and Kanban teams. SagePath Technologies, LLC Copyright 2017 22 Why Use the In-Process Cycle Time?
  • 23. SagePath Technologies, LLC Copyright 2017 23 Cycle Time Data Analysis
  • 24. • The data used for analysis is from an Agile team of teams. • This team of teams uses an electronic tool to track workflow. Cycle time data is easily extracted from this tool using an API. • The cycle time data is for user stories started and completed in 2015 and user stories started and completed in 2016. • This team of teams made two significant process changes in January of 2016:  They moved from 3 week sprints to 2 week sprints  They adopted a scaled Agile process SagePath Technologies, LLC Copyright 2017 24 About the Data
  • 25. Some data scrubbing is often required before proceeding with analysis. A few of the reasons for this include: • Missing data points • Data that needs reformatting. E.g. string data that should be numeric. • Data that is suspect and may be in error • Inconsistencies in the processes for gathering and input of some data points The following user story categories were removed from analysis: • User stories where the in-process date and the accepted date are the same. • System test user stories. • Stories flagged as obsolete. SagePath Technologies, LLC Copyright 2017 25 Data Scrubbing
  • 26. When it comes to data analysis, one of the first things a person should do is plot the data and look at it.  The idea is to get a general sense of what we are dealing with and see if there are any obvious trends or unusual features.  One good tool for doing this is a scatter plot.  Sometimes it is also useful to include a corresponding histogram to help gauge the density of the data. SagePath Technologies, LLC Copyright 2017 26 Looking at the Data
  • 27. About the Chart: • The chart on the left is a scatter plot of user stories started and accepted in 2015. • Each circle corresponds to a specific user story. • The x-axis is the date a user story was accepted. • The y-axis is user story cycle time in days. • To the right of the scatter plot is a cycle time histogram of the 2015 user stories. What does this scatter plot tell us? • The first thing to note is how spread out the data is in along the y-axis (i.e. cycle times are all over the place). • Since most of the teams are using scrum, the expectation is that story acceptance would be clumped near or below the sprint duration of 21 days. • The histogram confirms what we are seeing. A significant number of stories took 20 or more days to complete. SagePath Technologies, LLC Copyright 2017 27 User Story Scatter Plot
  • 28. About the Chart: • The chart on the left is a scatter plot of user stories started and accepted in 2016. • Each circle corresponds to a specific user story. • The x-axis is the date a user story was accepted. • The y-axis is user story cycle time in days. • To the right of the scatter plot is a cycle time histogram of the 2016 user stories. What does this scatter plot tell us? • Contrary to the 2015 scatter plot, in this scatter plot the user stories are more closely grouped along the lower part of the chart. • The histogram shows a strong peak between 10 and 15 days. This correlates well with the change to 2 week sprints in 2016. SagePath Technologies, LLC Copyright 2017 28 User Story Scatter Plot
  • 29. About the Chart: • In this chart the 2015 and 2016 scatter plots have been overlaid. This plot is referred to as a “splatter plot†” • The diameter of the marker circle for each user story is scaled according to the size of the story it represents. What does this plot tell us? • The first thing to note is that the maximum story size is much greater in the 2015 data than the 2016 data. • The larger user stories have cycle times significantly longer than one or two sprints. • Somewhat unexpected: there are a number of user stories that are small in size, but also have cycle times significantly longer than one or two sprints. SagePath Technologies, LLC Copyright 2017 29 Spatter Plot †The earliest reference to the term “splatter plot” that I am aware of is by Dennis Sweitzer in his presentation When Projects go Splat: Introducing Splatter Plots.
  • 30. SagePath Technologies, LLC Copyright 2017 30 Quantifying Predictability Predictability is the likelihood that any particular user story will be completed within a given time frame such as one sprint. If we can measure this likelihood, we can quantify predictability. This likelihood can be determined from the cycle time data by calculating the cumulative percentage of story completion over time.
  • 31. SagePath Technologies, LLC Copyright 2017 31 Measuring Predictability About the Chart: • This chart shows the cumulative percentage of 2015 stories that were accepted over time. • Note that the x-axis is in units of sprints rather than days. • In the ideal, all stories that were committed to in a given sprint are completed within that sprint. • At the 1 sprint line, the larger the percentage of completed stories the closer the teams are to the ideal. Of Note: • Only about 43% of the user stories were completed within 1 sprint. • After 2 sprints the completion percentage is at 74%. — This means that over one quarter of the user stories took longer than 2 sprints to complete • Based on this data, the likelihood that any particular story would be completed within one sprint is well short of 50-50. 2015
  • 32. SagePath Technologies, LLC Copyright 2017 32 Measuring Predictability About the Chart: • This chart overlays the cumulative percentage of accepted user stories from 2015 and 2016. A look at the numbers: • In 2016 the acceptance percentage within 1 sprint is 73%. This is a significant improvement from the 43% seen in 2015. • After 2 sprints the 2016 completion percentage is above 90%. • Between 2015 and 2016 the likelihood that a user story will be completed within 1 sprint increased from 43% to 73%. • In 2016 user stories were 67% more likely to be completed within 1 sprint than in 2015. This means SW delivery risk decreased in 2016. • Between 2015 and 2016 the aggregate team of teams predictability increased by a factor of about 1.7. Improvement in Predictability 2016 2015
  • 33. SagePath Technologies, LLC Copyright 2017 33 In a Nutshell: A few essentials to remember
  • 34. SagePath Technologies, LLC Copyright 2017 34 Predictability can be Measured and Quantified By tracking cycle time we can quantify predictability. In other words: We can put a number on the likelihood that user story commitments and program objectives will be met.
  • 35. SagePath Technologies, LLC Copyright 2017 35 Quantifying Software Delivery Risk Timely completion of user stories is a highly desirable objective. Therefore, we can define SW delivery risk as: Risk = 1 – (likelihood of on-time completion) Once we quantify the likelihood of user story completion, we can also quantify the corresponding project and program risk.
  • 36.  When you want to help teams understand their own work trends, focus on team metrics such as velocity.  For forecasting elapsed time of software deliverables focus on work delivered at a macroscopic level.  Forecast the likelihood of timely Agile team deliveries by looking at in- process cycle time.  Cycle time allows us to quantify software project delivery risk. SagePath Technologies, LLC Copyright 2017 36 Key Takeaways
  • 37. SagePath Technologies, LLC Copyright 2017 37 About the Data Visuals
  • 38. The Data Plots: • The data was analyzed using Python 3.6 and pandas from the Python Data Analysis Library . • The data was plotted using the Python Matplotlib plotting library. Background Images: All background images are licensed under the Creative Commons Zero (CC0) license from the following sources: unsplash.com : slides 17, 18, 21, 22, 26, 30, 34-37 pixabay.com : slides 1, 3, 5, 8, 13, 33 pexel.com : slides 23, 24 SagePath Technologies, LLC Copyright 2017 38 Data Visuals