achieving extreme performance with oracle exadata elektronisk ressurs

Achieving Extreme
Performance with
Oracle Exadata
Rick Greenwald
Robert (Bob) Stackowiak
Maqsood Alam
Mans Bhuller
New York Chicago San Francisco
Lisbon London Madrid Mexico City
Milan New Delhi San Juan
Seoul Singapore Sydney Toronto
Copyright © 2011 by The McGraw-Hill Companies, Inc. (Publisher). All rights reserved. Except as permitted under the United States
Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or
retrieval system, without the prior written permission of the publisher.
ISBN: 978-0-07-175260-2
MHID: 0-07-175260-9
The material in this eBook also appears in the print version of this title: ISBN: 978-0-07-175259-6,
MHID: 0-07-175259-5.
All trademarks are trademarks of their respective owners. Rather than put a trademark symbol after every occurrence of a trademarked name,
we use names in an editorial fashion only, and to the benefi t of the trademark owner, with no intention of infringement of the trademark.
Where such designations appear in this book, they have been printed with initial caps.
McGraw-Hill eBooks are available at special quantity discounts to use as premiums and sales promotions, or for use in corporate training
programs. To contact a representative please e-mail us at
Information has been obtained by Publisher from sources believed to be reliable. However, because of the possibility of human or
mechanical error by our sources, Publisher, or others, Publisher does not guarantee to the accuracy, adequacy, or completeness of any
information included in this work and is not responsible for any errors or omissions or the results obtained from the use of such
information.
Oracle Corporation does not make any representations or warranties as to the accuracy, adequacy, or completeness of any information
contained in this Work, and is not responsible for any errors or omissions.
TERMS OF USE

This is a copyrighted work and The McGraw-Hill Companies, Inc. (“McGrawHill”) and its licensors reserve all rights in and to the work. Use
of this work is subject to these terms. Except as permitted under the Copyright Act of 1976 and the right to store and retrieve one copy of the
work, you may not decompile, disassemble, reverse engineer, reproduce, modify, create derivative works based upon, transmit, distribute,
disseminate, sell, publish or sublicense the work or any part of it without McGraw-Hill’s prior consent. You may use the work for your own
noncommercial and personal use; any other use of the work is strictly prohibited. Your right to use the work may be terminated if you fail to
comply with these terms.
THE WORK IS PROVIDED “AS IS.” McGRAW-HILL AND ITS LICENSORS MAKE NO GUARANTEES OR WARRANTIES AS
TO THE ACCURACY, ADEQUACY OR COMPLETENESS OF OR RESULTS TO BE OBTAINED FROM USING THE WORK,
INCLUDING ANY INFORMATION THAT CAN BE ACCESSED THROUGH THE WORK VIA HYPERLINK OR OTHERWISE,
AND EXPRESSLY DISCLAIM ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. McGraw-Hill and its licensors do not warrant
or guarantee that the functions contained in the work will meet your requirements or that its operation will be uninterrupted or error free.
Neither McGraw-Hill nor its licensors shall be liable to you or anyone else for any inaccuracy, error or omission, regardless of cause, in the
work or for any damages resulting therefrom. McGraw-Hill has no responsibility for the content of any information accessed through the
work. Under no circumstances shall McGraw-Hill and/or its licensors be liable for any indirect, incidental, special, punitive, consequential
or similar damages that result from the use of or inability to use the work, even if any of them has been advised of the possibility of such
damages. This limitation of liability shall apply to any claim or cause whatsoever whether such claim or cause arises in contract, tort or
otherwise.
For Elinor Greenwald, on her way to many successes; Josephine
Greenwald, author of the forthcoming book
Frog on a Log;
and Robin
Greenwald, who keeps things interesting along the way. For my wife,
LuAnn Greenwald, who has made this wonderful family possible. And for
my longtime best pal, Bailey Greenwald—we miss you.
—

Rick Greenwald
For Jodie, my wife of over 30 years, who has always been there
for me and has learned to say impressive technical phrases to my
friends without having a full understanding of how to translate them.
—
Robert Stackowiak
To my parents, who are the reason for my being here; to my father-in-law,
a professor and an author whose dedication and commitment inspired
me to write; to my wonderful wife, Suraiya, who fully supported me in
this ordeal and gave excellent input despite not knowing the Oracle jargon;
and finally, to my wonderful children, Zuha and Firas, who are the
inspiration of my life.
—
Maqsood Alam
For my parents, who taught me the value of education and hard work.
For my brother, who stood by my shoulder guiding me through life’s
tribulations (and without whom I would still be selling music and stacking
frozen chickens!). For my dear wife, Ladi, and for our bundle of joy, Pavan.
—
Mans Bhuller
About the Authors
Rick Greenwald has been active in the computer industry for more than
25 years. He is currently Director of Partner Enablement at Oracle, where
he has worked for more than 10 years. He is also the author or coauthor of
more than 15 books, including the best-selling
Oracle Essentials
for O’Reilly &
Associates, now in its fourth edition;
Beginning Oracle Application Express
from Wiley;

Professional Oracle Programming
from Wrox; and
Oracle Data
Warehousing and Business Intelligence Solutions
from Wiley.
Robert (Bob) Stackowiak, vice-president of Business Intelligence and Data
Warehousing in the Enterprise Solutions Group at Oracle, has worked for
more than 20 years in the IT industry. His roles have included software
development, management of software development, systems engineering,
sales and sales consulting, and new business innovation and development.
Bob has spoken at numerous computer-related conferences and has
conducted briefings with companies around the world. He has coauthored
several books, including
Oracle Essentials: Oracle Database 11g (4th Edition)
by O’Reilly and
Oracle Data Warehousing and Business Intelligence
Solutions
by Wiley. In addition, his papers regarding data warehousing and
computer and software technology have appeared in publications such as
The Data Warehousing Institute’s
Journal of Data Warehousing.
Maqsood Alam, senior manager of product development at Oracle, is a
software professional with more than 15 years of experience working with
Oracle technologies, specializing in databases, maximum availability
architectures, data warehousing, database migrations, and business intelligence.
His current initiatives are focused on evangelizing Oracle Exadata,
performing customer Proof-Of-Concepts and benchmarks, promoting
best practices for migrations to Oracle from Oracle and non-Oracle
databases, and also providing support to customers undergoing large Exadata
implementations. He is an Oracle Certified Professional and holds a master’s

and a bachelor’s degree in computer science.
Mans Bhuller, senior director of Database, Operating Systems and
Management, Enterprise Solutions Group, Oracle, has worked at the forefront
of emerging technologies at Oracle Corporation for the last 14 years. In this
role, he has been paving the way for the Oracle Exadata Database Machine
and other foundational technologies, such as Real Application Clusters (RAC),
Grid, and Private Cloud. Mans is active in the Oracle Enterprise Architect
community and has visited hundreds of customers. Recently, he has been
building internal teaching classes around Exadata and Private Cloud and
training others.
About the Technical Editors
Kevin Closson is a performance architect in Oracle’s Systems Technology
Group focused on Exadata and future platform solutions. His 24-year career
has included engineering, technical marketing, support and application
development positions specializing in Oracle, and clustered platforms. Prior
to his tenure with Oracle, Kevin held positions within HP, PolyServe, IBM,
Sequent Computer Systems, and Veritas where his main engineering focus
was throughput and scalability enhancements of the Oracle server on high-
end Unix SMP and clustered systems. Kevin holds patents in SMP-locking
algorithms and database caching methods. He is a frequent speaker at
industry trade shows such as Oracle OpenWorld, IBM PartnerWorld, IOUG-A,
and regional Oracle user groups. In addition to book collaborations, Kevin’s
written works have appeared in
IBM Redbook, Oracle Magazine, Oracle
Internals Magazine, SELECT
, and
CMG.
He is a charter member of the
OakTable Network Organization. Kevin maintains a popular technical blog
at .

Maria Colgan is a principal product manager at Oracle Corporation and has
been with the company since version 7.3 was released in 1996. Maria’s
core responsibilities are data warehouse best practices and the Oracle
Optimizer. Based on Maria’s extensive experience in Oracle’s Server
Technology Performance Group—where she conducted competitive
customer benchmarks and tuning sessions—Maria creates and lectures on
data warehouse best practices and in-depth insights into the Oracle Optimizer
and the statistics it relies on.
Tim Shetler leads product management for the Systems Technology Group
at Oracle. His product responsibilities include the Exadata Storage Server,
database high-availability, database performance, compression, backup,
and archiving. He has spent most of his career working with database
management products, initially developing applications with Andersen
Consulting (Accenture), and subsequently in product management and
marketing roles at Hewlett-Packard, Informix Software, TimesTen, InQuira,
and Oracle, with a primary focus on parallel processing and ultra-high-
performance systems.
This page intentionally left blank
Contents
FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
PART I
Features and Foundations
1 Oracle and Tightly Integrated Hardware and
Software Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
A History of Appliance-like Computing Solutions . . . . . . . . . . . . 5
Oracle’s Evolution Towards Integrated
Hardware and Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Oracle Exadata Database Machine Fundamental Concepts . . . . . 10

Software Integration and the Oracle Exadata
Database Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Impact of the Platform on Personnel . . . . . . . . . . . . . . . . . . . . . . 15
Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Oracle 11
g
Enterprise Edition Features . . . . . . . . . . . . . . . . . . . . 19
Data Integrity and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Locks and Lock Management . . . . . . . . . . . . . . . . . . . . . . . 22
MVRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Flashback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Real Application Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
What Is RAC? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
RAC and Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
RAC and Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
vii
viii
Achieving Extreme Performance with Oracle Exadata
Cache Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Allocating Resources and RAC . . . . . . . . . . . . . . . . . . . . . . 33
RAC One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
RAC and the Exadata Database Machine . . . . . . . . . . . . . . 36
Automatic Storage Management . . . . . . . . . . . . . . . . . . . . . . . . . 37
What Is ASM? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
ASM and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
ASM and Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
ASM and Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
What Is Partitioning? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Partitioning Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Other Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Partitioning and the Exadata Database Machine . . . . . . . . 47
Parallel Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
What Is Parallel Execution? . . . . . . . . . . . . . . . . . . . . . . . . 48
What Can Be Parallelized? . . . . . . . . . . . . . . . . . . . . . . . . . 48
How Parallelism Works . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Partition-wise Parallel Joins . . . . . . . . . . . . . . . . . . . . . . . . 52
How Do You Configure Parallel Execution? . . . . . . . . . . . . 54
Degree of Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Modifying DOP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Ensuring DOP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Parallelism and RAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
In-memory Parallel Execution . . . . . . . . . . . . . . . . . . . . . . 62
Parallelism and Exadata . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Data Guard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
What Is Data Guard? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
How Can Data Guard Be Implemented? . . . . . . . . . . . . . . 63
Data Guard and Exadata . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
What Types of Compression
Does Oracle Support? . . . . . . . . . . . . . . . . . . . . . . . . . . 67
SecureFiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Benefits of Compression . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Contents
ix
Database Resource Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
What Is Database Resource Manager? . . . . . . . . . . . . . . . . 69
How Does Database Resource Manager Work? . . . . . . . . . 70
What Can Database Resource Manager Affect? . . . . . . . . . 71

Other Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
How Does Database Resource Manager
Work with Exadata? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Analysis Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Analytic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Enterprise Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Data Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Operating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Solaris Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Oracle Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Other Oracle Database 11
g
Features . . . . . . . . . . . . . . . . . . . . . . 80
Materialized Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Oracle OLAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Star Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3 Exadata Software Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Smart Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
How Standard Queries Work . . . . . . . . . . . . . . . . . . . . . . . 89
How Smart Scan Queries Work . . . . . . . . . . . . . . . . . . . . . 90
Individual Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Join Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Monitoring Savings from Smart Scan . . . . . . . . . . . . . . . . . 94
Other Offloaded Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Fast File Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Incremental Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Data Mining Scoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
x
Achieving Extreme Performance with Oracle Exadata
Exadata Hybrid Columnar Compression . . . . . . . . . . . . . . . . . . . 99
What Is Exadata Hybrid Columnar Compression? . . . . . . . 100
How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Compression Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Decompression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
ILM and Oracle Compression . . . . . . . . . . . . . . . . . . . . . . 104
Storage Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
How Storage Indexes Work . . . . . . . . . . . . . . . . . . . . . . . . 105
Storage Indexes at Work . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Exadata Smart Flash Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
What Is the Exadata Smart Flash Cache? . . . . . . . . . . . . . . 107
How Can You Use the
Exadata Smart Flash Cache? . . . . . . . . . . . . . . . . . . . . . . 108
How Does Exadata Smart Flash Cache Determine
What Is Cached? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Exadata Smart Flash Cache Statistics . . . . . . . . . . . . . . . . . 111
Benefits from Exadata Smart Flash Cache . . . . . . . . . . . . . . 112
I/O Resource Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Benefits from I/O Resource Manager . . . . . . . . . . . . . . . . . 113
Architecture of an IORM Plan . . . . . . . . . . . . . . . . . . . . . . 113
IORM at Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Interacting with Exadata Storage Server Software . . . . . . . . . . . . . 118
Management Software Components . . . . . . . . . . . . . . . . . . 118
Command Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

CellCLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
dcli . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
ADRCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
CellCLI Command Overview . . . . . . . . . . . . . . . . . . . . . . . 120
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4 Oracle Exadata Database Machine Platform
Hardware Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Latency and Balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Processor Speeds, Memory Capacity,
and Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
How Hardware Components Work Together . . . . . . . . . . . 129
Contents
xi
Oracle Exadata Database Machine Packaging Basics . . . . . . . . . . 130
Installation and Initial Deployment Considerations . . . . . . . . . . . 137
Upgrade Choices for Existing Systems . . . . . . . . . . . . . . . . . . . . . 140
Connecting to the Database Machine . . . . . . . . . . . . . . . . . . . . . 141
Highly Available Hardware Considerations . . . . . . . . . . . . . . . . . 143
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
PART II
Best Practices
5 Managing the Exadata Database Machine . . . . . . . . . . . . . . . . . . 149
Exadata Storage Server Architecture . . . . . . . . . . . . . . . . . . . . . . . 151
Database Server Software Components . . . . . . . . . . . . . . . 151
Exadata Storage Server Software Components . . . . . . . . . . 153
Exadata Storage Server Administration . . . . . . . . . . . . . . . . . . . . . 162
Using CellCLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Exadata Storage Server OS
Users and Privileges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Using dcli . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Exadata Storage Server Setup . . . . . . . . . . . . . . . . . . . . . . . 168
Exadata Storage Server Security Configuration . . . . . . . . . . 175
Exadata Storage Server Monitoring . . . . . . . . . . . . . . . . . . . . . . . 179
Monitoring with Metrics and Alerts . . . . . . . . . . . . . . . . . . 179
Monitoring Active Requests . . . . . . . . . . . . . . . . . . . . . . . . 191
Monitor Using the Oracle Database . . . . . . . . . . . . . . . . . . 192
Monitoring with Oracle Enterprise Manager . . . . . . . . . . . 196
Oracle Integrated Lights Out Manager . . . . . . . . . . . . . . . . 205
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
6 High Availability and Backup Strategies . . . . . . . . . . . . . . . . . . . 209
Exadata Maximum Availability Architecture (MAA) . . . . . . . . . . . 210
High Availability with Oracle Data Guard . . . . . . . . . . . . . 212
Using Oracle GoldenGate
with Database Machine . . . . . . . . . . . . . . . . . . . . . . . . . 218
Database Machine Patches and Upgrades . . . . . . . . . . . . . 224
Exadata Storage Server High Availability . . . . . . . . . . . . . . 230
Preventing Data Corruption . . . . . . . . . . . . . . . . . . . . . . . . 232
xii
Achieving Extreme Performance with Oracle Exadata
Exadata Database Machine Backup and
Recovery Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
Backup Tools Best Practices . . . . . . . . . . . . . . . . . . . . . . . . 240
Oracle Database Backup Strategy . . . . . . . . . . . . . . . . . . . 246
Database Recovery Best Practices . . . . . . . . . . . . . . . . . . . 257
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
7 Deploying Data Warehouses on the Oracle Exadata
Database Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
Data Warehousing Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Generic Oracle Query Optimization . . . . . . . . . . . . . . . . . . . . . . 268
Embedded Analytics in Oracle . . . . . . . . . . . . . . . . . . . . . . . . . . 271

SQL Aggregation and Analytics Extensions . . . . . . . . . . . . 271
OLAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Unique Exadata Features for Optimal Query Response . . . . . . . . 275
Data Warehousing Compression Techniques . . . . . . . . . . . . . . . . 278
The Typical Life of a Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Best Practices for Data Loading . . . . . . . . . . . . . . . . . . . . . . . . . . 280
Partitioning, Backups, and High Availability
in Data Warehouses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Data Models, Business Intelligence Tools, and Security . . . . . . . . 283
Data Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Business Intelligence Tools . . . . . . . . . . . . . . . . . . . . . . . . 285
Security Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Sizing the Platform for Data Warehousing
and Justifying Purchase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
8 Exadata and OLTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
OLTP Workloads and Exadata Features . . . . . . . . . . . . . . . . . . . . 294
Exadata Hardware and OLTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
General Hardware and Infrastructure Considerations . . . . . 296
Exadata Smart Flash Cache . . . . . . . . . . . . . . . . . . . . . . . . 297
Oracle 11
g
and OLTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
Classic Oracle Features . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
Oracle and Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
Quality of Service Management . . . . . . . . . . . . . . . . . . . . . 306
Contents
xiii
Exadata Software and OLTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

Exadata Nodes and OLTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Exadata as a Complete System . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
9 Consolidating Databases with the Oracle Exadata
Database Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Why Consolidate? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
How the Oracle Exadata Database Machine
Helps Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
Database Server Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . 318
CPU Sizing Considerations . . . . . . . . . . . . . . . . . . . . . . . . 318
Memory Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
I/O Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
System Sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
Storage Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
Network Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
Workload Consolidation and Isolation . . . . . . . . . . . . . . . . . . . . . 325
Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
Database Server Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
Workload Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
Meeting and Exceeding SLAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
Instance Caging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
I/O Resource Manager (IORM) . . . . . . . . . . . . . . . . . . . . . . 333
Quality of Service Management . . . . . . . . . . . . . . . . . . . . . 338
Consolidation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
RAC Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
Workload-Based Consolidation . . . . . . . . . . . . . . . . . . . . . 339
Time Zone-Based Consolidation . . . . . . . . . . . . . . . . . . . . 340
Overprovisioning Consolidation . . . . . . . . . . . . . . . . . . . . 340
Tight SLA Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
10 Migrating to the Exadata Database Machine . . . . . . . . . . . . . . . . 343
Premigration Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
Discovering the Current Environment . . . . . . . . . . . . . . . . 346
Database Machine Capacity Planning and Sizing . . . . . . . 355
Choosing a Migration Strategy . . . . . . . . . . . . . . . . . . . . . . 360
Migration Steps for Non-Oracle Databases . . . . . . . . . . . . . . . . . 364
Database Schema Migration . . . . . . . . . . . . . . . . . . . . . . . 364
Server-side Scripts Migration . . . . . . . . . . . . . . . . . . . . . . . 365
Data Migration and Synchronization . . . . . . . . . . . . . . . . . 365
Using Automated Tools for Migration . . . . . . . . . . . . . . . . 369
Migration Steps for Oracle Databases . . . . . . . . . . . . . . . . . . . . . 374
Migrating Using Physical Methods . . . . . . . . . . . . . . . . . . . 375
Migrating Using Logical Methods . . . . . . . . . . . . . . . . . . . 381
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
A Exadata Capacity and Performance Specifications . . . . . . . . . . . 385
Exadata Database Machine Storage Capacity . . . . . . . . . . . . . . . . 386
Exadata Storage Server Performance . . . . . . . . . . . . . . . . . . . . . . 387
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
xiv
Achieving Extreme Performance with Oracle Exadata
Foreword
Exadata has consumed much of my attention and the attention of a significant
portion of my development team here at Oracle since the early 2000s. This
project began when we saw our customers struggling with Oracle database
implementations on hardware platforms that were inadequate to meet the
demands of growing data sizes and workloads. In particular we found that
Oracle performance was limited by the inability to move data from disks
into the database fast enough. There was plenty of throughput in the disks
themselves, and plenty of processing power in the servers, but the architecture

for moving data from disks to processors was severely limited. Since
inadequate throughput was clearly the cause of most performance bottlenecks,
we started an effort to develop software to move database processing
directly into intelligent storage. Later, it became clear to us that we needed
to focus all our efforts on a specific storage platform to achieve the best
performance and reliability.
When we first productized the Exadata platform, we offered two options.
One option was to attach intelligent storage to customer purchased and
configured database servers. The other option was to purchase a complete
optimized Database Machine from Oracle, including servers, storage, and
networking. We thought that many customers would find it more convenient
to add storage to their existing standard configurations than to adopt a
completely new platform. We could not have been more wrong. Almost no
customers bought the storage only solution. Instead, we found huge interest
in our customers for a completely integrated and optimized full database
solution from Oracle. Thus, Oracle entered the integrated systems business.
We have come a long way since Larry Ellison introduced the first version
of the Oracle Exadata Database Machine in 2008 at Oracle OpenWorld.
Today, Exadata is used in production to run an incredible variety of
xv
applications. Exadata runs everything from enormous telecom data warehouses
to response-time-critical financial trading applications. We have a growing
number of deployments of Exadata running the most sophisticated applications
in the world—Oracle E-Business Suite, Siebel, and PeopleSoft.
Many lessons were learned along the way as we worked with this fast-
growing customer base. This book takes you through some of the history of
the Oracle Exadata Database Machine and some of the best practices we
developed. After you read this book, you should have a better understanding
of fundamental Exadata concepts and how to leverage Exadata’s benefits in
your applications. My hope is that you find this book useful as you deploy

and manage your Oracle Exadata Database Machine. I sincerely hope that
the extreme performance you experience will take your business or
organization to a new level of success.
Juan Loaiza
Redwood Shores, CA
October 2010
Juan Loaiza is senior vice-president of systems technologies at Oracle and manages
the “systems” aspects of Oracle’s database development organization, including
Exadata, high availability and Data Guard, Oracle Secure Backup and RMAN,
Flashback, performance, data and index management, and platform-specific
development. He has worked in the database development organization since 1988
and contributed to every release since Oracle version 6. Juan has a bachelor’s and
master’s degree in computer science from M.I.T.
xvi
Achieving Extreme Performance with Oracle Exadata
Acknowledgments
Rick, Bob, and the entire author team would like to acknowledge the
excellent review process managed by our editorial director, Wendy Rinaldi,
and our acquisitions coordinator, Stephanie Evans. The technical editors,
Maria Colgan, Kevin Closson, and Tim Shetler, provided more thorough
reviews than any previous book we have been associated with—we feel
confident that this book is highly accurate, but accept sole responsibility for
any errors that may have slipped through the cracks. Rick and Bob would
like to especially call out the work of Kevin, who often provided direction
at a level that is not documented or widely known within Oracle.
Rick Greenwald would also like to thank his longtime collaborator, Bob
Stackowiak, for his contributions and friendship for more than 15 years—
quite an outcome from that initial meeting in line, and Maqsood Alam, who
really delivered on his first book. The process of writing this book has been
a process of almost continual learning and discovery, which would not have

been as rich and complete without the help of many (in no particular order),
including Robert Pastijin, Mark Townsend, Ron Weiss, Peter Wahl, Pete
Sharman, Andrew Babb, Kodi Umamageswaran, Miroslav Lorenc, Herman
Baer, Mahesh Subramaniam, Christian Craft, Mark Scardina, Kevin Jernigan,
George Spears, Phil Stephenson, Alex Tsukerman, Jenny Tsai-Smith, Dan
Norris, and all the many folks who contribute to internal Exadata mailing
lists. And a special thanks to both Maria Colgan and Kevin Closson, who
provided valuable insights over the course of the entire project.
Finally, a sincere shout-out to Nick Kritikos, my boss, who encouraged
the creation of this book in many ways.
xvii
Bob would also like to acknowledge Rick Greenwald’s extensive efforts
on this book, including where he filled in gaps in some chapters when
needed and his review of the book’s content for consistency. Juan Loaiza
provided important support for this project early in the process as the
authors considered whether the time was right for a book covering Exadata.
Of course, having the support of management at Oracle is critical, and Bob
would like to acknowledge the support of Mark Salser and David O’Neill in
the Oracle Enterprise Solutions Group and Kate Johnson in the Oracle
Exadata Program Management Office in North America. Bob would also
like to acknowledge the contributions and guidance offered over the years
by George Lumpkin, Hermann Baer, Ron Weiss, Tony Politano, Joe
Rayman, Rob Reynolds, Warren Dagenbach, Louis Nagode, and Alan
Manewitz.
Maqsood Alam would like to acknowledge the coauthors for making this
project a success, with a special thanks to Rick Greenwald for accepting this
project and being a mentor along the way. A sincere token of appreciation
goes out to all the reviewers, especially to Kevin Closson, who provided
invaluable technical insights. Lastly, to my colleagues at Oracle Platform
Technology Solutions, Henry Byorum, Aalok Muley, Milton Wan, and others,

who time and again provide exciting opportunities that allow me to work
with the latest and greatest technologies.
Mans Bhuller would like to send a big thank-you to the Oracle Exadata
development team and executive management, in particular, Ron Weiss,
Phil Stephenson, Mahesh Subramaniam, Kevin Jernigan, Tim Shetler, Juan
Loaiza, and Kevin Closson (more for his sardonic wit rather than his galactic
intellect). Mans would also like to recognize Andrew Holdsworth from the
Real World Performance Team and Maria Colgan from the DW/SQL team,
as well as his manager, David O’Neill, and SVP Mark Salser from the ESG
team. Over the years I have benefited from the warmth of true friendship
from Peter Sharman and Holger Kalinowski and would like to honor those
mentors who have guided me in the past and those that still continue to
guide me to this day: Mark Ashdown, Mark Large, Graham Wood, and Paul
Cross.
xviii
Achieving Extreme Performance with Oracle Exadata
Introduction
You are holding the first comprehensive book covering the Oracle Exadata
Database Machine. You might be holding this book because of your desire
to know more about exactly what this technology can do for you, but you
and your fellow readers have come to this place from different starting
points. You could be investigating the Exadata Database Machine because
you are a longtime Oracle user who is curious as to the new capabilities
you will get with this platform. Your curiosity may have been sparked by
the large amount of publicity around the Exadata Database Machine, from
Oracle, the press, and industry analysts. Or you might be approaching
Exadata with a fair amount of skepticism, since new platforms that can
deliver orders of magnitude of performance improvement are rare, to say
the least, in a mature market like the relational database market.
This book should help you understand Exadata technology in a way that

will extend your knowledge, validate the publicity (well, at least some of it),
and show you how these vaunted performance improvements are actually
delivered.
What Is Exadata?
This most basic question will require much of this book to fully explain, but
the Exadata Database Machine is a bit like an elephant being approached
by blind men. Is Exadata a new type of storage hardware? Is Exadata similar
to an appliance built with the latest and greatest hardware components? Is
Exadata primarily software that uses hardware components in some new
ways? Is Exadata simply an optimally configured Oracle-complete hardware
and software stack?
xix
The answer to the question “Which of these is Exadata?” has a simple
answer—“Yes.” The Exadata Database Machine has been described in all
of these ways, and more.
One facet of the Exadata Database Machine that we want to make sure
to recognize is its simplicity. The Exadata Database Machine comes
preconfigured and optimized, designed to implement best practices for
Oracle software and hardware that have been decades in the making.
For the most part, the Exadata Database Machine just works.
This simplicity has been the cause of much curiosity in the user community,
since the internal operations of Exadata technology—the
way
it just works—
have been somewhat shrouded in mystery. We believe that this book will
go a long way toward removing these shrouds, while still leaving you with
a real appreciation of just how innovative, powerful, and cool the Exadata
Database Machine is.
The remainder of this book is our attempt to deliver on this promise.
xx

Achieving Extreme Performance with Oracle Exadata