Best practices and tips on how to design and develop a Data Warehouse using Microsoft SQL Server BI products.
This presentation describes the inception and full lifecycle of the Carl Zeiss Vision corporate enterprise data warehouse.
Technologies covered include:
•Using SQL Server 2008 as your data warehouse DB
•SSIS as your ETL Tool
•SSAS as your data cube Tool
You will Learn:
•How to Architect a data warehouse system from End-to-End
•Components of the data warehouse and functionality
•How to Profile data and understand your source systems
•Whether to ODS or not to ODS (Determining if a operational Data Store is required)
•The staging area of the data warehouse
•How to Build the data warehouse – Designing Dimensions and Fact tables
•The Importance of using Conformed Dimensions
•ETL – Moving data through your data warehouse system
•Data Cubes - OLAP
•Lessons learned from Zeiss and other projects
What Are The Drone Anti-jamming Systems Technology?
Architecting a Data Warehouse: A Case Study
1. Architecting A Data Warehouse: A Case Study A Case Study Project: zBis Carl Zeiss Vision North America Mark Ginnebaugh, User Group Leader, Mark Ginnebaugh User Group Leader mark@designmind.com
2. The Journey Determined Need for Enterprise Data Warehouse Determined Need for Enterprise Data Warehouse Worked with Business Users to Understand Business RequirementsDDetermined Software Requirements i dS f R i MS SQL Server 2005 & 2008 MS SSIS (ETL Tool) MS SSIS (ETL Tool) MS SSAS (Analytic Cube Tool) MS SSRS & Excel (Reporting Tools) SharePoint for Deploying Reports over Company Intranet Designed and Developed zBis Data Warehouse g p
3. Z BIS = What We Will Deliver The DesignMind project team will deliver The DesignMind project team will deliver Consolidated reporting for Carl Zeiss Vision North America Reporting that is consistent and from one data warehouseR Reporting that is easy to use and easy to access ti th t i t d t Toolset will be flexible and able to grow and change with your business Phase I rock solid download from ERP/Manf – Providing ability to review lab information as a lab network – not individual silos – with accurate reporting across all products and servicesWe will deliver the best product possible based on the information we can place in our data warehouse!
5. • Reporting from cubes – off source systems only – No data warehouse N d t h• Disparate data systems with different results from p y each• Most systems not balanced to GL• Reporting for each business unit only• No reporting across all business units
6. Transactional Cube of Approach Sales Queries Other Reports Sales Reports Corporate Download D l d Data Mart Data Mart Data Mart Finance Inventory Sales & Marketing ETL Loads ETL Load ODS/Staging g g Operational Data Store ETL LoadERP Manufacturing Other
7. BI Tools/Analytics Active Excel Static Reports Reports PerformancePoint Server SharePoint SQL SQL Analytics Reporting Server (SSAS) ServerAggregated Finance Inventory Sales Data Mart Data Mart Data Mart Data Mart TBD ETL Load (SSIS) Data Warehouse ETL Load (SSIS) ODS/Staging O S/S Operational Data Store ETL Load (SSIS) ERP Manufacturing SW Other Data Sources
9. Data Warehouse Project Lifecycle Technical Product Architecture Selection & Design InstallationProject Business Data Staging TestingPlanning Dimensional Physical Requirement Design & ETL & Deployment Maintenance Modeling Design Definition Development DW/DM Report Report Report Specifications Development Testing Project Management
10. 4 + 1 – Steps4 + 1 Steps Dimensional Design Process Ralph Kimball’s Process for Developing Star Schemas1. Determine Business Process Model business Processes Model business Processes Each Process will determine 1 or more Facts Design DW by Business Process Not Business Unit2.2 Identify the Grain of the Fact Identify the Grain of the Fact • What does 1 row in Fact table represent • Transactional or Summary 3. Design the DW Dimensions D i h DW Di i4. Design the DW Facts+1 Determine Hierarchies Determine Hierarchies
13. The Business Executive InterviewThe Business Executive Interview• What are the objectives of your organization? • What Business goals do you want to accomplish with the development of zBis d t d l t f Bi data warehouse System? h S t ?• How do you measure success? How do you know you are doing How do you measure success? How do you know you are doing well? How often do you measure your corporate performance? • What are your key business issues that you are trying to solve from the zBis system? If these issues are not justified what is the impact to your department and organization? impact to your department and organization?
14. The Business Executive InterviewThe Business Executive Interview• How do you identify problems or know when you might be headed for trouble? • How do you spot exceptions in your business? What opportunities exist to dramatically impact your business based opportunities exist to dramatically impact your business based on improved access to information? What is the financial impact • If you could….., What would it mean to your business?• What is your vision to better leverage information within your What is your vision to better leverage information within your organization?•H How do you anticipate that your staff will interact directly with d ti i t th t t ff ill i t t di tl ith this information?
15. Th B i M I t iThe Business Manager Interview• What are the objectives of your department? What are the objectives of your department?• What are you trying to accomplish? How would do you go about achieving your objectives? about achieving your objectives?• What are your success metrics?• How do you know you are doing well?• How often do you measure your department/team? y y p• How do you anticipate that your staff will interact directly with this information?
18. Determine Hierarchies Product Hierarchy Manufacturer Brand Product Type – Each product type had own Hierarchy Lens Service Equipment etc… t Design Make/Model /
20. Conformed Dimensions Standardized dimensions across data warehouse St d di d di i d t h Dimensions are associated with multiple business processes Determine by using Bus Matrix & enforced in ETLC f Conformed Dimensions are shared and consistent d Di i h d d it t across fact tables
21. Use Data Warehouse BUS Matrix Use Data Warehouse BUS Matrix for Understanding & mapping of Business Processes and Dimensions Ongoing DW/BI planning efforts Team & Management Communications Team & Management Communications Understand Business Process unions across the enterprise
22. Data Warehouse BUS Matrix Date Company Customer Product Geo Dist Ctr PromoCompany X X X X X XSalesCustomer X X X X X XDiscountsProduct X X X X X X XCostCompany X X XInventoryDist Ctr X X XInventory
24. Sl Ch i Di iSlow Changing Dimensions Type 1 – Overwrite existing Dimension Row Type 1 Overwrite existing Dimension Row Use when don’t need to keep history data row Can be used to correct bad data Type 2 – Create a new Dimension Row Use date and/or active non‐active fields to identify current and inactive data rows Type 3 – Keep old and add new attributes in Dimension Row Allow Alternate realities to exist simultaneously in one Allow Alternate realities to exist simultaneously in one Dimension Row Slow Changing Dimensions are handled in the ETL
25. T f Di iType of Dimensions Mini‐Dimension Mini Dimension Junk Dimensions Outrigger Dimensions Outrigger Dimensions Small Static Dimensions Lookup tables Lookup tables
26. T fF tType of Facts Transaction Fact Tables Snapshot Fact Tables Accumulating Snapshot Fact Tables Consolidated or Aggregated Fact Tables
29. R d d R di liRecommended Reading list The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling (Second Edition) by Ralph Kimball and Margy Ross M d li (S d Edi i ) b R l h Ki b ll d M R The MicrosoftData Warehouse Toolkit: With SQL Server2005 and the MicrosoftBusiness Intelligence Toolset by Joy Mundy, Warren Thornthwaite, and Ralph Kimball Building a Data Warehouse: With Examples in SQL Server (Experts Voice) Building a Data Warehouse: With Examples in SQL Server (Expert s Voice) by Vincent Rainardi The Data Warehouse Lifecycle Toolkit by Ralph Kimball, Margy Ross, Warren Thornthwaite, and Joy Mundy The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleanin by Ralph Kimball and Joe Caserta by Ralph Kimball and Joe Caserta