The International Association of Emergency Managers Developing and Deploying Data Warehouse and Business Intelligence Solutions Kerr-McGee Information Management Group

Developing and Deploying Data Warehouse and
Business Intelligence Solutions
Kerr-McGee Information Management Group
Skye Brannon
DW Analyst

Jeff Bridgwater
DW Manager

Sarena Sherrard
Sr. DW Analyst


Who is Kerr-McGee?

•

Kerr-McGee is an Oklahoma City-based energy and
inorganic chemical company with worldwide operations
and assets of approximately $10 billion.

•

/>

Agenda
•

Introduction to DW/BI Concepts

•

Extract, Transform & Load (ETL)

•

Business Intelligence / Reporting

•

A Day in the Life


DW / BI
Concepts


Information Management Strategy

Structure the systems and data
relationships to provide user-friendly
customer access to data in order to
provide decision-making information.


Adding Value to Data


Information Pyramid

What is a Data Warehouse?
A copy of data from one or more On-line
Transaction Processing (OLTP) systems
specifically structured for Query,
Reporting and Analysis (QRA).
• Data is typically at a summarized level to limit the size and
complexity of the data warehouse
• Data is usually cleansed and merged to create an “apples to
apples” comparisons
End-User
Reporting

OLTP
Systems
Data
Warehouse


The Idea Behind Data Warehousing
Source
Systems
POS
INVENTORY
FINANCIALS

Extract
&
Transform

Data

Warehouse

Meaningful
&
Easy
Access

?

BILLING

Data

Information


Framework Architecture
Metadata Management

Enterprise
Reporting

External Data
Data
Extraction

Datamarts

Data
Cleansing

Data
Integration

OLAP
Query

Data
Improvement
Data
Mining

Source
Systems

Data
Transformation

Operational
Data Store

Data
Warehouse

Project Management & Quality Assurance
Operations & Systems Management

Information
Delivery

Business Intelligence
Business
Activity
Sales
Marketing
Service Customers
Manage Inventory
Maintain Accounts

Data

Business
Intelligence
Integrated
Meaningful
Consistent
Validated
Easy to Use
Leveragable
Timely

Product Performance
Sales Performance
Customer Behavior
Marketing Performance
Inventory Monitoring
Financial Performance

Information

Corp.
Oracle
Financials

Passport
InPower
Production
Operations
Financial
Operations

Chemical
Passport

Maximo

European
Oracle
Financials

Chemhouse

TOLAS

O&G
Tobin

Issues
• Multiple Versions of

the Truth
• Different definitions
for similar data
• Multiple Reporting
Tools and Reports
• Impacts Performance
of Oracle Financials

Domestic
Oracle
Financials

Adage

Data
Warehouses
Existing
Reporting
Systems

HR Data
Warehouse

DFW

Novistar
Energy
Financials

O&G

Data
Warehouse

PREMAS
P2000
Merak

Others

Aberdeen
Oracle
Financials
Intl.
Systems

Aberdeen
Data
Warehouse


HR Data
Warehouse

Corp.
Oracle
Financials

Passport?

Phased Out

Peoplesoft
Production
Operations

“Centralized”
Data Warehouse(s)

Chemical
Passport

Domestic
Oracle
Financials

Financial
Operations
Data
Warehouses
Existing
Reporting
Systems

Consolidated
Analysis &
Reporting
Solution
(Cognos
Business
Intelligence)

Adage
European
Oracle
Financials

Maximo

TOLAS

Consolidated
Reporting
Systems

Benefits

• Consolidated Tool
Enables Multiple
Economies of Scale
- Massaging & Distribution

- Desktop/Maintenance
- Transaction System Load’

Novistar
Energy

Tobin

• Single Version of the

Truth
• Business Intelligence
environment leads to
Timely Analysis &
Reporting

Dashboards
& KPI Mgmt

Chemhouse

O&G
DFW

PREMAS
P2000
Merak

Others

Budget &
Forecasting
Application

Aberdeen
Oracle
Financials
Intl.
Systems

O&G
Data
Warehouse

Aberdeen
Data
Warehouse

Possibly
Phased Out or Integrated

Possibly
Phased Out or Integrated

STANDARDIZE,
CONSOLIDATE,
MINIMIZE, & SIMPLIFY


Data Warehouse Roles
•

Manager Planning and management of entire product or project lifecycle;
May assist in ETL & BI Interface design and development

•

Data Warehouse Architect – Applies knowledge of technology options,
platforms, and design techniques across product and project lifecycle;
responsible for design of overall warehouse process

•

ETL Specialist – Analysis and design of extraction, transformation, and
loading strategy; development of ETL scripts and procedures

•

Business Intelligence Specialist – Design and development of
multidimensional-cubes & reports; performance and tuning of chosen
technologies

•

Web Interface Specialist – Design and development of application interface
elements; coordinates interfaces between application components


ETL


Data Extraction and Transformation
Metadata Management

•
External
Data

•

Data
Extract
Cleansing
and
Integration
Process

•
•
•

Data Extraction and Transformation
Plan/Forecast/
Applying business rules to turn data into useable
Analysis
information
Clean up and standardization OLAP
of consumers,
vendors,
Global / Dept/
Query
products, etc.
Business Unit
Summary
Integration of disparate internal and external
data and
Analysis
Can be 70% - 80% of effort
Issues
Data

Datamarts
- Can be difficult and time consuming toVisualization
define
Data
business rules
Mining

Source
Systems

Data
Transformation

- Extraction tools automate only the more simple
tasks
Executive
Operational
Data
Information
Information
Data Store
Warehouse
Analysis
Systems

Project Management & Quality Assurance
Operations & Systems Management


ETL Challenges

Data
Volume

+

Source
Inclusion

+

Extract
Timing

=

Warehouse
Complexity

• Warehouse Complexity
• Destructive vs. Incremental Loads
• Integration (mergers/acquisitions)
– Data integration (consolidation of information)
– Process integration (consolidation of processes)

• Smaller windows of opportunity
– Make decision in a shorter period of time due to
competitive, global market

• Global marketplace (DW timing updates)

• High-profile e-Business initiatives
– Satisfying requirements


ETL - The “Heavy Lifting”
•

Challenge to develop efficient, consistent methods
of gathering and cleansing heterogeneous data

– Capture and load of data from
multiple source systems (both
internal and external)
– Integrates data into a single source
– Cross-system mapping to standard
identifiers (surrogate keys)
– Aggregation for information
delivery and BI initiatives


ETL Tools - Only Half the Story
•
•
•
•

Half the story: ETL Tools Extract, Transform, and Load data
Transport data between sources and targets
Document data element changes (metadata)
Administer run-time processes and operations

–
–
–
–

Scheduling
Error management
Audit logs
Statistics


ETL Tools – Core Components
Databases/Files
Metadata
Import/Export

Target Adapters

Extract

Transform

Load

Source Adapters

Databases/Files/
Legacy Apps

Runtime
Metadata
Services

Metadata
Repository

Design
Manager


ETL - The Options
•

Software Products

– “Homegrown” Solutions
• SQL*LOADER, PL/SQL, GATEWAYS & LINKS
• Developer-bound
• Little cohesion between components

– Niche Players
(Oracle Warehouse Builder / DataJunction)
• Enhanced Scheduling & Logging
• Not Multi-Warehouse Oriented

– Informatica Powermart
•
•
•

•

Great UI
Powerful Scheduling & Logging
High Price
Proprietary Transform Language


ETL - The Reality

Flat File

O/S
Scripts

Mainfra
me
DBMS

COBOL
Code

ERP

3GL
Code

RDBMS

SQL

Scripts

Loader
Utility

Data
Repository

Oracle
RDBMS

PERL
Scripts

“New”
Sources

Interface
Apps


Informatica Powermart

Repository Manger

Workflow Manager

Designer

Workflow Monitor

Business Intelligence
/
Reporting


What is Business Intelligence?

Business Intelligence is the transformation of data into
information you can use to drive your business.
There are a number of vendors that have developed Business
Intelligence software. Kerr-McGee uses Cognos.