Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
www.sonata-software.comTh3 -Open Source Tools for Test Management 
Narayanan C. V., Vice President, Sonata Software Limited
Agenda 
•Introduction 
•Methodology 
•Architectural View 
•Test Management Best Practices 
•Take Away 
1
Test Management -Introduction 
Test Management is a method of organizing Test Assets –Test Requirements, Test Plans, Test ...
Introduction –Purpose Challenges 
Prioritization of requirements 
Extent of Test coverage 
Cost Vs Quality 
Aligning to Re...
Methodology –[Waterfall / Iterative] Requirements Management 
•Functional and NFR Base lining 
•E2E scenarios and Acceptan...
Test Tools Integration Architecture 
ENTERPRISE SERVICE BUS (ESB) 
Requirement Management FrameworkUnit and Build Testing ...
Deployment View 
ENTERPRISE SERVICE BUS 
Test Integration Framework (TIF) 
-Use case 
-ScenariosSONNET TEST DATABASE ENGIN...
Deployment ViewSonnetTESTTest ManagementRequirements ManagementTest Case ManagementTest Data ManagementUnit Testing and Bu...
Requirements and Test Case Management FrameworkNEEDSFEATURESUSE CASESSCENARIOSTEST CASESRequirement Management Process 
St...
Risk-Based TestingTransformation EngineRequirement Management SystemDefect Management SystemDashboardRelease Notes Managem...
Test Data Management Framework 
Test Data repositoryDevelopmentTesting 
StagingProductionApproach: 
-Data Profiling for al...
Unit Testing & Build Setup Framework 
Strategy 
Description 
Tool 
SuiteofOpenSourcetoolslikeHttpUnit,Jwebunit,Junit,Junit...
Unit Testing & Build Setup -BenefitsReusable Unit Test Case for Security, Covers security threats from OWASPDash Board vie...
SOA Testing FrameworkPresentation Layer / Application Level Business Layer 
Service Layer / Component Layer 
Database/ Ext...
SOA Testing Framework 
Strategy 
Description 
Tool 
SOAPUI 
Expected Output 
•Regression, Unit, Interoperability, Security...
Functional Automation Framework 
Strategy 
Description 
Tool 
Selenium 
Expected Output 
•Customized keywords for Function...
Security Testing FrameworkProduct Requirements 
Functional DesignTechnical Design 
Development / Implementation 
TestingRe...
Security Testing Framework 
Strategy 
Description 
Tool 
Paros Proxy 
Expected Output 
•Threat Modeling of Application 
•S...
Performance Testing 
Strategy 
Description 
Tool 
Jmeter 
Expected Output 
•Set Goals and System boundaries 
•Define Servi...
Sonnet Reporting Engine 
Centralized report for the following 
•Single view of the overall QA 
•Requirements Traceability ...
Test Management –Take Away 
•Tools Integration Architecture 
•Option for Open Source and branded tools 
•Functional and NF...
Thank you for attending this session. 
Please fill out an evaluation form and place it in the box located at the back of t...
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
Customized Test Automation Solution
Next
Download to read offline and view in fullscreen.

2

Share

C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010

Download to read offline

EuroSTAR Software Testing Conference 2010 presentation on Open Source Tools for Test Management by C.V, Narayanan. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010

  1. 1. www.sonata-software.comTh3 -Open Source Tools for Test Management Narayanan C. V., Vice President, Sonata Software Limited
  2. 2. Agenda •Introduction •Methodology •Architectural View •Test Management Best Practices •Take Away 1
  3. 3. Test Management -Introduction Test Management is a method of organizing Test Assets –Test Requirements, Test Plans, Test Cases, Test Scripts, Test Data, Requirements Traceability and Test Results to enable optimized Test Delivery TEST STRATEGY & GOVERNANCEInception DesignExecutionMaintenance TEST MANAGEMENT PROCESS 2
  4. 4. Introduction –Purpose Challenges Prioritization of requirements Extent of Test coverage Cost Vs Quality Aligning to Release Plan Environment control Solution Streamlining the processIntegrated tracking across life cycleOptimized Testing based on constraintsAdoption of Open Source toolsBenefits: •Helps in managing changing requirements •Repeatable process •Effective release cycles and bug tracking 3
  5. 5. Methodology –[Waterfall / Iterative] Requirements Management •Functional and NFR Base lining •E2E scenarios and Acceptance scenarios •Define Test StrategyPlanning •Baseline productivity •Effort estimation (FP, WBS, Test Points) •Identify dependencies of Test Cases, Identify critical path •Determine Acceptance and exit criteria •Define Roles and responsibilitiesDesign •Setup Test Environment •Test Cases, Test Data Preparation •Test Scripts mapping to Test Data •Requirements Traceability / Automation CoverageExecute •Smoke / Sanity Test •Execute Test Cases / Test Scripts for each Test Cycle •Defect Management / Root Cause Analysis •Generate Test report for each Test CycleMaintenance & Release •Versioning of Test Artifacts •Updates to Test Cases / Test Scripts for the changes / Enhancements / New Requirement •User Acceptance Test •Understanding document •Test Strategy •Test Plan •Test Artifacts •Traceability Matrix •Test Report •Defects List / Root cause analysis •Test Artifacts •UAT Report4
  6. 6. Test Tools Integration Architecture ENTERPRISE SERVICE BUS (ESB) Requirement Management FrameworkUnit and Build Testing FrameworkAutomation Testing FrameworkPerformance Testing FrameworkSecurity Testing FrameworkSOA Testing FrameworkDefect TrackingSONNET TEST DATABASE ENGINESONNET REPORTING ENGINE •Dashboard view of results •Result AnalysisSONNET RISK BASED ASSESSMENT FRAMEWORK SONNET TRANSFORMATION ENGINESonnet Test Data Repository 5
  7. 7. Deployment View ENTERPRISE SERVICE BUS Test Integration Framework (TIF) -Use case -ScenariosSONNET TEST DATABASE ENGINESONNET REPORTING ENGINE •Dashboard view of results •Result Analysis Transformation Engine Pk: Req ID Requirement ManagementTool: TestLinkTest ManagementTool: TestLink -Test case -Test Scenarios -Test Conditions -RTM Pk: Test Case IDFk: Req ID Unit & Build ManagementTool: Junit, HTTPUnitTest Data ManagementTool: GenerateData, Excel SOA Testing Tool: SoapUI Regression Testing Tool: SeleniumPerformance TestingTool: WebLOAD, JMeter Security Testing Tool: Paros Proxy -Test Conditions -Performance Pk: Test Case ID Defect ManagementTool: Bugzilla/ Mantis -Reviews -Functional / Non functional Pk: Defect IDFk: Req ID -Service Identification -Request construction -Response validation -Code Coverage -Method Signature -Build & Deployment -Smoke Test Pk: Test Suite IDFk: Req IDPk: Unit / Build ID -Functional Automation -Business Process Testing -Risk Based Testing Fk: Test Case ID -Critical Business Scenario / Transactions -Load Pattern Fk: Test Case ID -Thread Modeling -Security Scanning -Report Fk: Test Case ID Reporting Engine6
  8. 8. Deployment ViewSonnetTESTTest ManagementRequirements ManagementTest Case ManagementTest Data ManagementUnit Testing and Build SetupSOA TestingFunctional AutomationSecurity TestingPerformance Testing Defect managementDashboard Framework Components Open Source tools Requirements Management Test Link Test Case management Test Link Test Data management GenerateData, Excel (customized) Unit Testing and Build setup HttpUnit, Jwebunit, Junit, JunitPerf SOA Testing SOAP UI Automation Testing Selenium Security Testing Paros Proxy Performance Testing Jmeter Defect Tracking Bugzilla/ Mantis Dashboard Inbuilt 7
  9. 9. Requirements and Test Case Management FrameworkNEEDSFEATURESUSE CASESSCENARIOSTEST CASESRequirement Management Process Strategy Description Tool TestLink[Open Source] Expected Output •Centralized repository for Requirements and Test management •Requirement Hierarchy •Requirements mapping to Test Cases for better coverage Integration Process •Test Management for traceability •Defect Management for stability •Dashboard for monitoring [Requirement ID is the primary key for linking other test assets] Benefits •Collect and organize your requirements •Track specific information about individual tests •Customize Test Link to fit your requirements and processes. •Improved user management (editable) •Attachments •Enhanced reports •Import / Export through XML 8
  10. 10. Risk-Based TestingTransformation EngineRequirement Management SystemDefect Management SystemDashboardRelease Notes ManagementAlgorithmReports9
  11. 11. Test Data Management Framework Test Data repositoryDevelopmentTesting StagingProductionApproach: -Data Profiling for all environments -Masking of confidential information -Data sampling technique used to slice the production data -Standard template for data requests including for change requests Strategy Description Tool GeneratorData[Open Source] Excel [Customized] Expected Output •Centralized Test Data repository for the following Unit Testing, Functional / Regression Testing, Performance Testing etc. Test Environmentslike Development,Testing, UAT, Production etc. •Coverage includes new requirements, change requests / enhancements Integration Process •Requirements Management for traceability •Test management for mapping •Defect Management for defect simulation •Dashboard for Reporting [Requirement ID is the primary key for linking other testassets] Benefits •Test Data is availableon demand •Versioned enabling to roll back whenever required •Scripts available to refresh / create / update on need basis SOAUIPerformance 10
  12. 12. Unit Testing & Build Setup Framework Strategy Description Tool SuiteofOpenSourcetoolslikeHttpUnit,Jwebunit,Junit,JunitPerf Expected Output UnitTestingfordifferenttypesofTestinglikeFunctional,SecurityandPerformance. Integration Process TheTestResultsofthevariousUnitTestingframeworksaresenttotheSonnetTransformationEnginewhichinturngetsconsolidatedintheSonnetReportingEngineanddisplayedinDashBoardview. Benefits Reusableand efficient 11
  13. 13. Unit Testing & Build Setup -BenefitsReusable Unit Test Case for Security, Covers security threats from OWASPDash Board view providing information on Code Quality, Unit Testing coverage on-going basis, alert notification Unit Testing framework is extended to provide complete end-to-end Build Framework providing, code analysis, Unit Testing & DeploymentFramework provides various standards, Guideline, Checklists to implement Unit Testing in an optimum way. Framework provides feature to verify functionality at Unit level Support to Test Performance as part of Unit Testing, provides time taken at Method level and helps to identify bottlenecks early in phase of Development Security Testing Best Practices End-to-End Build FrameworkDash Board (reSonate) Functional Unit TestingPerformance Unit Testing Sonnet Unit Testing Framework –Benefits 12
  14. 14. SOA Testing FrameworkPresentation Layer / Application Level Business Layer Service Layer / Component Layer Database/ External Application / Internal ApplicationBrowser Web2.0 UIThick Client POSSonnet Automation Test Framework Based on (QTP, SOAPUI) Process / Integration Testing FrameworkComponent Level Testing FrameworkSystem Level Testing FrameworkKeyword-driven Functional & Regression Testing Framework SOA Governance QoS(Security, Performance and Availability) 13
  15. 15. SOA Testing Framework Strategy Description Tool SOAPUI Expected Output •Regression, Unit, Interoperability, Security, Governance and Integration Testing of Web services •Ensuring the quality of service of Web Services Integration Process The results of the SOA Testing will be exported to the Sonnet Transformation engine which in turn will be displayed in the dashboard of the Sonnet reporting engine Benefits Stabilize the Functionality Testing early 14
  16. 16. Functional Automation Framework Strategy Description Tool Selenium Expected Output •Customized keywords for Functional Automation •Regression Test Cases selection •Test scripts mapping to Test Casesand in turn to Requirements Integration Process •Scripts / component mapping to Test Cases for coverage •Defect Management for re-executing Test scripts •Test Results exported to SonnetReporting Engine for Risk- based Assessment / Testing Benefits •Reusability of Test Scripts for Smoke & Sanity Test •Execution of Test Scripts across browsers and version Includes executing scripts on different staging environment •Auto Configuration settings makes it drive independent. Hence easy to carry and maintain •Test Data file facilitates execution of same Test Case with different sets of Data (multiple Iteration) 15
  17. 17. Security Testing FrameworkProduct Requirements Functional DesignTechnical Design Development / Implementation TestingRelease / DeploymentSecurityRequirementsDocumentArchitectural Risk Analysis Guidelines Security GatesOWASP & PCI Threat ModelingManual and Tool-based testingTest Report Secure Coding Checklists / Guidelines Secure SDLCSonata Accelerator 500+ Pre-built Test CasesCovers OWASP Top 10 Threats 16
  18. 18. Security Testing Framework Strategy Description Tool Paros Proxy Expected Output •Threat Modeling of Application •Secure Code review •Security assessment of the application Integration Process •Requirement IDmapping to security requirements •Vulnerabilities identified get consolidated in Sonnet Reporting Engine and a Dash Board view of the threats is displayed Benefits •Security Testing is in compliance with well known standards like OWASP,PCI/DSS,WASC and CWE/SANS •Security Testing across SDLC •Hack resilient application 17
  19. 19. Performance Testing Strategy Description Tool Jmeter Expected Output •Set Goals and System boundaries •Define Services/Components and possible outcomes •Select Metrics •List Parameters •Select Factors to Study •Select Evaluation Technique •Select Workload •Design Tests •Run, Analyze and Interpret data •Present Results Integration Process •The results of the Performance Testing will be exported to the Sonnet Transformation engine which in turn will be displayed in the Dash Board of the Sonnet reporting engine. Benefits Help to mix and match the load 18
  20. 20. Sonnet Reporting Engine Centralized report for the following •Single view of the overall QA •Requirements Traceability to Test Cases and in turn to automation Test Scripts •Dash Board with functional and non-functional Testing details •Day / Build-wise Test Design and execution count •Weekly comparison / progress of Test Design and execution •Categorization of defects based on build, severity / priority, requirements •Defect Trends across multiple releases 19
  21. 21. Test Management –Take Away •Tools Integration Architecture •Option for Open Source and branded tools •Functional and NFR Testing Integration •Risk-based Testing to optimisethe COQ •Integrated view supporting Unit and Integrated testing •TCO Reduction 20
  22. 22. Thank you for attending this session. Please fill out an evaluation form and place it in the box located at the back of the room.
  • SRILAKSHMIK4

    Aug. 6, 2017
  • nlsiitb

    Nov. 3, 2016

EuroSTAR Software Testing Conference 2010 presentation on Open Source Tools for Test Management by C.V, Narayanan. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

Views

Total views

1,894

On Slideshare

0

From embeds

0

Number of embeds

319

Actions

Downloads

47

Shares

0

Comments

0

Likes

2

×