Learn how TUI UK&I selected qTest as their preferred replacement Test Tool to meet their current and future Enterprise Testing challenges. How they successfully implemented qTest and where they hope to be in 12 months’ time.
1. TUI & qTest:
Why, How and Where Next
Phil Thomas: General Manager of QA & Test | 11th October 2017
2. About TUI UK& Ireland
TUI UK & Ireland is
the UK’s largest tour
operator with key
brands including
Thomson, First
Choice, and
Thomson Cruises
as well as Crystal
Ski and Thomson
Lakes & Mountains.
TUI UK & Ireland
has a team of more
than 10,000
employees and
serves over six
million customers
each year. TUI UK
and Ireland is a
member of TUI
Group.
TUI UK & Ireland’s
airline, Thomson
Airways is the UK’s
third largest airline
with 62 aircraft
operating to over 88
destinations in 30
countries.
UK’s largest
tour operator
UK 3rd largest
airline
10k employees
& 6m customers
5. System &
Integration
Automation &
Performance
Exploratory
& Smoke Testing
Regression &
OAT
UAT & Production
Projects,
Releases, BAU &
CR’s
Agile & Waterfall
In-house, 3rd Party
& Offshore
UK&I, Northern
Region & Group
Test Scope Delivery
DevOps & AWS….
6. Why Bother?
1.Lack of real time Test reporting
2.No integration with Rally & Jira
3.Consolidate on a single Tool
4.Lacked traceability
5.Not sufficiently intuitive
6.Not future proofed for DevOps & Cloud
7.TUI Test Mgrs lacked visibility of Test planning & execution
8. Set-up a Team for the
DurationTUI Test Managers
UAT Managers
Sonata Test Managers
Delivery Manager
Scrum Master
Project Manager
9. Within budget
Handle multiple Business Portfolios
Test Tool as cloud solution
Real time MI dashboard reporting
Real-time integration with Rally, Jira & CI/CD tools
Capture and manage Business Priority & Risk
Top Requirements
10. Steps to choosing qTest
Business, UK IT & CIO
Approval
Evaluated the 7 against
Requirements
Resulted in remaining 3
Detailed POC & Commercials
Unanimous agreement
Agreed Requirements & Weighting
Suggestions shortlisted to 7
11. Scoring the Seven
Criteria
Cost (H) Average 3 Poor 1 Average 3 Average 2
Test Design &
Planning (H)
Good 4 Good 5 Good 5 Good 5
Traceability (H) Good 4 Good 5 Good 4 Good 5
Real Time MI (H) Average 3 Good 5 Good 5 Good 4
Integration
Capability (H)
Average 3 Average 3 Good 5 Good 3
Defect Mgt (H) Average 3 Good 5 Good 4 Good 4
Useability/User
Friendliness (H)
Good 5 Good 5 Good 5 Average 3
Result / Score Result / Score Result / Score Result / Score
PractiTest HP ALM/QC qTest QAComplete
JIRA/Zephyr qTest SpiraTest QA Complete PractiTest HP ALM/QC
89.5 88.1 86.5 86.3 69.6 N/A
13. Implementation Overview
Identified Stakeholders & Comms Plan
Regular promotion across TUI & 3rd Parties
Weekly calls with Matt, our QAS Success Champion
Ensured Sonata fully engaged
Issued TUI standards for qTest
TUI training materials & training sessions
Targeted training for the initial Projects Teams
Completed data cleanse and migration
14. Implementing qTest
Set up 5 sub teams covering
1. Requirements & Test Planning
2. Test Design & Scripts
3. Defect Management
4. Reporting & MI Dashboards
5. Migration & Integration
Cross team coordination for MVP
Used an early Gold Copy instance
Weekly meetings, including UAT and offshore
15. Implementing qTest: The
Plan Jan Feb March April May June July Aug Sept
Analysis
Initial Set-up
Pilot Training
Pilot 1
Pilot 2
Final Set-up
User Training
17.3 Release
Daily Standups
BAU
16. Cutover: Release 17.3 -
qTest
17.3:
Projects
Requiremen
ts Loaded
Initial
Scripts
Loaded
Scripts run
& defects
raised
Remaining
projects join
qTest
Regression
Pack & O/s
17.2 defects
Release 17.2: Bugzilla & TestOpia
On Going Training and Communications
Daily reviews and rapid issue resolution
19. qTest – Results to far
Phase 1: Targets all met & Benefits realised
Exceeded expectations:
qTest live a month early
Integration to Rally & Jira
Adopted by
o TUI Group projects
o E-Commerce scrums teams for SIT
o Airline Portfolio
Total active users 513
Logged once since 1st June 510
Logged once since 1st Aug 456
Logged once since last wk 320
Logged once since Monday 243
22. qTest – Next Steps
Further teams moving onto qTest
Session Based Testing & Automation
Reporting on Automation results
Using qTest data to drive improvements
UAT use of EggPlant
DevOps: CI/CD
Putting qTest front & centre of Nordics delivery