oracle database 11g release 2 high availability elektronisk ressurs maximize your availability with grid infrastructure, oracle real application clusters, and oracle data guard

Oracle Database 11g
Release 2 High
Availability


About the Authors

Scott Jesse (Colorado Springs, Colorado) has worked for Oracle since 1996 and is
currently Customer Support Senior Manager for the Oracle RAC Assurance Team.
He has worked with the Oracle RAC Assurance Team since 2007. Prior to this
assignment, Scott’s primary focus was on clustering technologies. He has served as
a Global Team Lead and as a member of the Advanced Resolution and Escalations
Team within Oracle Support, providing him with a unique insight into the needs of
Oracle technology customers. Scott is the coauthor of Oracle Database 10g High
Availability with RAC, Flashback, and Data Guard and Oracle9i for Windows 2000
Tips & Techniques, both by McGraw-Hill/Oracle Press.
Bill Burton (United Kingdom) joined Oracle from Computer Associates in 1998,
spending two years with the Oracle UK product support group for Database before
transferring to the U.S. Gold Support Team in October 2000. He spent time in the
Oracle HA Support group, dealing with Oracle RAC and Oracle Data Guard, before
joining the Bug Determination and Escalation group (BDE) for those products. He
has been a member of the Oracle RAC Assurance Development Team since 2007.
Together with Josh Ort, he wrote and presented the Upgrading to 11gR2 Session at
Oracle OpenWorld 2010.
Bryan Vongray (Beaver, Pennsylvania) has more than nine years of experience
implementing and supporting all aspects of the Oracle Database. He specialized
in Oracle High Availability with specific focus in Oracle Real Application Clusters
(Oracle RAC), Oracle Data Guard, Streams, backup and recovery, as well as Grid
Control. As a Senior Consultant for Oracle Consulting Services, Bryan had planned
and implemented Oracle Maximum Availability Architecture (MAA) Solutions for

numerous Oracle customers. Bryan is now putting his Oracle MAA expertise to use
as a member of the Oracle RAC Assurance Team within Oracle Global Customer
Support.

About the Contributor

Hagen Herbst (Germany) is an OCP-certified DBA with more than a decade of
experience in Oracle products. Prior to joining the Oracle RAC Assurance Team
within Oracle Global Customer Support, he spent years installing and upgrading
countless databases, mainly Oracle RAC and Failsafe Systems in Europe, the
Middle East, and Africa. Hagen has also served as the senior Oracle DBA in a data
center, utilizing all aspects of Oracle’s Maximum Available Architecture.

About the Technical Editors

Farouk Abushaban is a Senior Principal Engineer at Oracle. He is a founding
member of the Center of Excellence Team and is the Global Technical Lead for
Enterprise Manager High Availability implementations. He spent 22 years in
customer support and information systems. Farouk works directly with strategic


Oracle customers and partners to improve product utilization and performance in
large deployments. He has a bachelor’s degree in Electrical Engineering from the
University of Colorado in Colorado Springs and completed his master’s degree in
Management of Information Technologies with emphasis on Project Management
and Security from Colorado Technical University. Farouk has been with Oracle
since 1997.
Jiong Dai (Daedal) has worked with Oracle for more than ten years. He started in
Advanced Product Support, where he supported Oracle RDBMS, Oracle Developer,
Oracle Designer, and Oracle Application Server, delivering onsite OPS design,

Oracle backup/recovery strategy review, and database performance review. He
became an Oracle Certified Professional in 2000. In 2004, he worked with Intel
Server Performance Scalability Lab, publishing the world’s leading TPC-H results and
world leading MAPI Messaging Benchmark 3 (MMB3) results, and working for the
enterprise workload character analysis on TPC-C, SPECjAppServer (Java Application
Server), SJBB, and so on. In 2006, he moved to the Oracle HA Product Support Team
and eventually moved on to the Oracle RAC Assurance Team in 2009. Currently,
he provides support to critical Oracle RAC customers to assure their successful
implementation on Oracle RAC. He is the author of RAC Assurance Support
Team: RAC Starter Kit and Best Practices (AIX) (Linux).
Michael Smith is a principal member of the technical staff in Oracle’s Maximum
Availability Architecture (MAA) Team in Server Technologies. Mike has been
with Oracle for 12 years, previously serving as the Oracle Data Guard Global
Technical Lead within Oracle Global Support. Mike’s current focus is
developing, validating, and publishing HA best practices using Oracle Data
Guard in an integrated fashion across all Oracle Database HA features. His
Oracle Data Guard technical specialties focus on network transport, recovery,
role transitions, Oracle Active Data Guard, and client failover. He has published
a dozen MAA Best Practice Papers for Oracle 9i, 10g, and 11g. He has been a
contributing author to previous Oracle Press publications. Mike has also been
speaker at three Oracle OpenWorld events held in San Francisco. His “What
They Didn’t Print in the DOC” Best Practice Presentations covering Oracle Data
Guard and MAA are a favorite among Oracle users, with attendance at the top
of all Oracle Database technology presentations.
Paul Tjhang, Principal Technical Support Engineer, Oracle Corporation, worked
as a system administrator and support engineer before becoming an Oracle DBA.
He is responsible for maintaining the Oracle RAC Assurance Support Team’s
Starter Kit and Best Practices documentation. He has reviewed many Oracle RAC
configurations and provided feedback for best practices configuration. He is also
actively supporting Oracle customers with Oracle RAC-related issues.

This page intentionally left blank


Oracle Database 11g
Release 2 High
Availability
Maximize Your Availability with Grid
Infrastructure, Oracle Real Application
Clusters, and Oracle Data Guard,
Second Edition
Scott Jesse
Bill Burton
Bryan Vongray

New York  Chicago  San Francisco 
Lisbon  London  Madrid  Mexico City  Milan 
New Delhi  San Juan  Seoul  Singapore  Sydney  Toronto


Copyright © 2011 by The McGraw-Hill Companies, Inc. (Publisher). All rights reserved. Except as permitted under the United States Copyright
Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system,
without the prior written permission of the publisher.
ISBN: 978-0-07-175207-7
MHID: 0-07-175207-2
The material in this eBook also appears in the print version of this title: ISBN: 978-0-07-175208-4,
MHID: 0-07-175208-0.
All trademarks are trademarks of their respective owners. Rather than put a trademark symbol after every occurrence of a trademarked name, we
use names in an editorial fashion only, and to the benefit of the trademark owner, with no intention of infringement of the trademark. Where such

designations appear in this book, they have been printed with initial caps.
McGraw-Hill eBooks are available at special quantity discounts to use as premiums and sales promotions, or for use in corporate training programs.
To contact a representative please e-mail us at
Oracle is a registered trademark of Oracle Corporation and/or its affiliates. All other trademarks are the property of their respective owners, and
McGraw-Hill makes no claim of ownership by the mention of products that contain these marks.
Information has been obtained by Publisher from sources believed to be reliable. However, because of the possibility of human or mechanical error
by our sources, Publisher, or others, Publisher does not guarantee to the accuracy, adequacy, or completeness of any information included in this
work and is not responsible for any errors or omissions or the results obtained from the use of such information.
Oracle Corporation does not make any representations or warranties as to the accuracy, adequacy, or completeness of any information contained in
this Work, and is not responsible for any errors or omissions.
TERMS OF USE
This is a copyrighted work and The McGraw-Hill Companies, Inc. (“McGrawHill”) and its licensors reserve all rights in and to the work. Use of this
work is subject to these terms. Except as permitted under the Copyright Act of 1976 and the right to store and retrieve one copy of the work, you may
not decompile, disassemble, reverse engineer, reproduce, modify, create derivative works based upon, transmit, distribute, disseminate, sell, publish
or sublicense the work or any part of it without McGraw-Hill’s prior consent. You may use the work for your own noncommercial and personal use;
any other use of the work is strictly prohibited. Your right to use the work may be terminated if you fail to comply with these terms.
THE WORK IS PROVIDED “AS IS.” McGRAW-HILL AND ITS LICENSORS MAKE NO GUARANTEES OR WARRANTIES AS TO THE
ACCURACY, ADEQUACY OR COMPLETENESS OF OR RESULTS TO BE OBTAINED FROM USING THE WORK, INCLUDING ANY
INFORMATION THAT CAN BE ACCESSED THROUGH THE WORK VIA HYPERLINK OR OTHERWISE, AND EXPRESSLY DISCLAIM
ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR A PARTICULAR PURPOSE. McGraw-Hill and its licensors do not warrant or guarantee that the functions contained in the work
will meet your requirements or that its operation will be uninterrupted or error free. Neither McGraw-Hill nor its licensors shall be liable to you
or anyone else for any inaccuracy, error or omission, regardless of cause, in the work or for any damages resulting therefrom. McGraw-Hill has
no responsibility for the content of any information accessed through the work. Under no circumstances shall McGraw-Hill and/or its licensors be
liable for any indirect, incidental, special, punitive, consequential or similar damages that result from the use of or inability to use the work, even if
any of them has been advised of the possibility of such damages. This limitation of liability shall apply to any claim or cause whatsoever whether
such claim or cause arises in contract, tort or otherwise.

FREE SUBSCRIPTION
TO ORACLE MAGAZINE

GET
YOUR

Oracle Magazine is essential gear for today’s information technology professionals.
Stay informed and increase your productivity with every issue of Oracle Magazine.
Inside each free bimonthly issue you’ll get:

If there are other Oracle users at
your location who would like to
receive their own subscription to
Oracle Magazine, please photocopy this form and pass it along.

Three easy ways to subscribe:
1 Web

oracle.com/oraclemagazine

2 Fax

+1.847.763.9638
3 Mail

P.O. Box 1263, Skokie, IL 60076-8263

Copyright © 2008, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

Want your own FREE subscription?
To receive a free subscription to Oracle Magazine, you must fill out the entire card, sign it, and date
it (incomplete cards cannot be processed or acknowledged). You can also fax your application to
+1.847.763.9638. Or subscribe at our Web site at oracle.com/oraclemagazine
No.

Yes, please send me a FREE subscription Oracle Magazine.
From time to time, Oracle Publishing allows our partners
exclusive access to our e-mail addresses for special promotions and announcements. To be included in this program,
please check this circle. If you do not wish to be included, you
will only receive notices about your subscription via e-mail.

x
signature (required)

date

name

Oracle Publishing allows sharing of our postal mailing list with
selected third parties. If you prefer your mailing address not to
be included in this program, please check this circle.
If at any time you would like to be removed from either mailing list, please contact
Customer Service at +1.847
.763.9635 or send an e-mail to
If you opt in to the sharing of information, Oracle may also provide you with
e-mail related to Oracle products, services, and events. If you want to completely
unsubscribe from any e-mail communication from Oracle, please send an e-mail to:
with the following in the subject line: REMOVE [your

e-mail address]. For complete information on Oracle Publishing’s privacy practices,
please visit oracle.com/html/privacy/html

title

company

e-mail address

street/p.o. box
city/state/zip or postal code

telephone

country

fax

Would you like to receive your free subscription in digital format instead of print if it becomes available?

Yes

No

YOU MUST ANSWER ALL 10 QUESTIONS BELOW.
1

WHAT IS THE PRIMARY BUSINESS ACTIVITY
OF YOUR FIRM AT THIS LOCATION? (check
one only)

o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o

08014004

2

01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
98

Aerospace and Defense Manufacturing
Application Service Provider
Automotive Manufacturing
Chemicals

Media and Entertainment
Construction/Engineering
Consumer Sector/Consumer Packaged
Goods
Education
Financial Services/Insurance
Health Care
High Technology Manufacturing, OEM
Industrial Manufacturing
Independent Software Vendor
Life Sciences (biotech, pharmaceuticals)
Natural Resources
Oil and Gas
Professional Services
Public Sector (government)
Research
Retail/Wholesale/Distribution
Systems Integrator, VAR/VAD
Telecommunications
Travel and Transportation
Utilities (electric, gas, sanitation, water)
Other Business and Services _________

3

o
o
o
o
o

o
o
o
o
o
o
o
o
o
o
o
o

99
4

CORPORATE MANAGEMENT/STAFF
o 01 Executive Management (President, Chair,
CEO, CFO, Owner, Partner, Principal)
o 02 Finance/Administrative Management
(VP/Director/ Manager/Controller,
Purchasing, Administration)
o 03 Sales/Marketing Management
(VP/Director/Manager)
o 04 Computer Systems/Operations
Management
(CIO/VP/Director/Manager MIS/IS/IT, Ops)
IS/IT STAFF
o 05 Application Development/Programming
Management

o 06 Application Development/Programming
Staff
o 07 Consulting
o 08 DBA/Systems Administrator
o 09 Education/Training
o 10 Technical Support Director/Manager
o 11 Other Technical Management/Staff
o 98 Other

99
5

o

Digital Equipment Corp UNIX/VAX/VMS
HP UNIX
IBM AIX
IBM UNIX
Linux (Red Hat)
Linux (SUSE)
Linux (Oracle Enterprise)
Linux (other)
Macintosh
MVS
Netware
Network Computing
SCO UNIX
Sun Solaris/SunOS
Windows
Other UNIX

Other
None of the Above

01
02
03
04
05
06
07
o

Hardware
Business Applications (ERP, CRM, etc.)
Application Development Tools
Database Products
Internet or Intranet Products
Other Software
Middleware Products
None of the Above

6

HARDWARE
o 15 Macintosh
o 16 Mainframe
o 17 Massively Parallel Processing

o
o

7

More than 25,000 Employees
10,001 to 25,000 Employees
5,001 to 10,000 Employees
1,001 to 5,000 Employees
101 to 1,000 Employees
Fewer than 100 Employees

01
02
03
04
05
06

Less than $10,000
$10,000 to $49,999
$50,000 to $99,999
$100,000 to $499,999
$500,000 to $999,999
$1,000,000 and Over

WHAT IS YOUR COMPANY’S YEARLY SALES
REVENUE? (check one only)
o
o
o
o

o

9

01
02
03
04
05
06

DURING THE NEXT 12 MONTHS, HOW MUCH
DO YOU ANTICIPATE YOUR ORGANIZATION
WILL SPEND ON COMPUTER HARDWARE,
SOFTWARE, PERIPHERALS, AND SERVICES FOR
YOUR LOCATION? (check one only)
o
o
o
o
o
o

8

18
19
20
21
22

23

WHAT IS YOUR COMPANY’S SIZE?
(check one only)
o
o
o
o
o
o

IN YOUR JOB, DO YOU USE OR PLAN TO PURCHASE ANY OF THE FOLLOWING PRODUCTS?
(check all that apply)
SOFTWARE
o 01 CAD/CAE/CAM
o 02 Collaboration Software
o 03 Communications
o 04 Database Management
o 05 File Management
o 06 Finance
o 07 Java
o 08 Multimedia Authoring
o 09 Networking
o 10 Programming
o 11 Project Management
o 12 Scientific and Engineering
o 13 Systems Management
o 14 Workflow

o

o
o
o
o
o

o
o
o
o
o
o

DO YOU EVALUATE, SPECIFY, RECOMMEND,
OR AUTHORIZE THE PURCHASE OF ANY OF
THE FOLLOWING? (check all that apply)
o
o
o
o
o
o
o

WHICH OF THE FOLLOWING BEST DESCRIBES
YOUR PRIMARY JOB FUNCTION?
(check one only)

01
02

03
04
05
06
07
08
09
10
11
12
13
14
15
16
98

Minicomputer
Intel x86(32)
Intel x86(64)
Network Computer
Symmetric Multiprocessing
Workstation Services

SERVICES
o 24 Consulting
o 25 Education/Training
o 26 Maintenance
o 27 Online Database
o 28 Support
o 29 Technology-Based Training

o 30 Other
99 o None of the Above

WHAT IS YOUR CURRENT PRIMARY OPERATING
PLATFORM (check all that apply)

01
02
03
04
05

$500, 000, 000 and above
$100, 000, 000 to $500, 000, 000
$50, 000, 000 to $100, 000, 000
$5, 000, 000 to $50, 000, 000
$1, 000, 000 to $5, 000, 000

WHAT LANGUAGES AND FRAMEWORKS DO
YOU USE? (check all that apply)
o
o
o
o

01
02
03
04

Ajax
C
C++
C#

o
o
o
o

13
14
15
16

Python
Ruby/Rails
Spring
Struts

10

05 Hibernate
06 J++/J#
07 Java
08 JSP
09 .NET
10 Perl
11 PHP
12 PL/SQL

o 17 SQL
o 18 Visual Basic
o 98 Other

WHAT ORACLE PRODUCTS ARE IN USE AT YOUR
SITE? (check all that apply)
ORACLE DATABASE
o 01 Oracle Database 11g
o 02 Oracle Database 10 g
o 03 Oracle9 i Database
o 04 Oracle Embedded Database
(Oracle Lite, Times Ten, Berkeley DB)
o 05 Other Oracle Database Release
ORACLE FUSION MIDDLEWARE
o 06 Oracle Application Server
o 07 Oracle Portal
o 08 Oracle Enterprise Manager
o 09 Oracle BPEL Process Manager
o 10 Oracle Identity Management
o 11 Oracle SOA Suite
o 12 Oracle Data Hubs
ORACLE DEVELOPMENT TOOLS
o 13 Oracle JDeveloper
o 14 Oracle Forms
o 15 Oracle Reports
o 16 Oracle Designer
o 17 Oracle Discoverer
o 18 Oracle BI Beans
o 19 Oracle Warehouse Builder

o 20 Oracle WebCenter
o 21 Oracle Application Express
ORACLE APPLICATIONS
o 22 Oracle E-Business Suite
o 23 PeopleSoft Enterprise
o 24 JD Edwards EnterpriseOne
o 25 JD Edwards World
o 26 Oracle Fusion
o 27 Hyperion
o 28 Siebel CRM
ORACLE SERVICES
o 28 Oracle E-Business Suite On Demand
o 29 Oracle Technology On Demand
o 30 Siebel CRM On Demand
o 31 Oracle Consulting
o 32 Oracle Education
o 33 Oracle Support
o 98 Other
99 o None of the Above


I would like to dedicate this to my father, Larry Jesse, who has fought
for his entire adult life for what he has believed to be right and true.
I will never fully understand the struggles he has faced, but I have
learned to appreciate them more as I have grown older.
–Scott
For Mum and Dad, though sorely missed you remain an inspiration;
also for Molly (6) and Izzy (4), whose ability to adapt to change and
boundless imagination never cease to amaze me.
–Bill

This book is dedicated to my wife, Joleen, and my two children,
Hayden (8) and Delaney (1).
–Bryan


This page intentionally left blank


Contents at a Glance
part i

Oracle’s Grid Infrastructure


1 Architecting the Oracle Database Grid  . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3



2 Oracle VM  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
MAA Workshop: Installing the Oracle VM Manager  . . . . . . . . . . . . . .
MAA Workshop: Creating a Virtual Machine with the
Oracle VM Manager  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Adding Shared Disks and NICs with the
Oracle VM Manager  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Creating a VM from a Template
Without the Manager  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19
26



3 Grid Infrastructure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47



4 Grid Infrastructure Installation and Configuration  . . . . . . . . . . . . . . . . . . . . 77
MAA Workshop: Prepare a Linux/Oracle RAC Cluster  . . . . . . . . . . . . . 85
MAA Workshop: Delete a Node from a Cluster  . . . . . . . . . . . . . . . . . . 118



5 Oracle Automatic Storage Management  . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Setting ASM Disk String  . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Creating a Disk Group  . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Creating an ASM Dynamic Volume  . . . . . . . . . . . . . .
MAA Workshop: Creating an ACFS for DB Home  . . . . . . . . . . . . . . . .
MAA Workshop: Creating a General Purpose ACFS   . . . . . . . . . . . . . .
MAA Workshop: Set Up ACFS Replication  . . . . . . . . . . . . . . . . . . . . .

30
36
40

125

128
137
144
149
155
163

ix


Oracle-Regular / Oracle Database 11g Release 2 High Availability / Jesse et al / 208-0

x 

  Oracle Database 11g Release 2 High Availability

part iI

Oracle Real Application Clusters (Oracle RAC)


6 Oracle RAC Setup/Configuration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
MAA Workshop: RDBMS Software Install. . . . . . . . . . . . . . . . . . . . . . 180
MAA Workshop: Oracle RAC Database Creation. . . . . . . . . . . . . . . . 185



7 Oracle RAC Administration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
.
MAA Workshop: Apply a Patch with the Rolling

Patch Method  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
MAA Workshop: Convert an Administrator-Managed
Database into a Policy-Managed Database  . . . . . . . . . . . . . . . . . 237



8 Utility Computing: Applications as Services  . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Using OEM 11g Grid Control
to Create Services  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Create a Simple FAN Callout  . . . . . . . . . . . . . . . .
MAA Workshop: Configure and Test TAF  . . . . . . . . . . . . . . . . . . . . .

243
253
263
273

part iII

Disaster Planning




9 Oracle Data Guard  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Creating an Oracle RAC Physical
Standby Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Converting an Oracle RAC Physical Standby
to a Snapshot Standby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Creating an Oracle RAC Logical

Standby Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Changing the Protection Mode . . . . . . . . . . . . . . . .
MAA Workshop: Opening a Standby in Read-Only Mode
Including Oracle Active Data Guard. . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Switchover to a Physical Standby. . . . . . . . . . . . . . .
MAA Workshop: Failover to a Physical Standby . . . . . . . . . . . . . . . . .
MAA Workshop: Reinstating a Failed Primary as a Standby
After Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10 Backup and Recovery for MAA Environments  . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Configure the Flash Recovery Area. . . . . . . . . . . . . .
MAA Workshop: Setting Persistent Configuration Parameters . . . . . . .
MAA Workshop: Implementing a Recovery Catalog. . . . . . . . . . . . . .
MAA Workshop: Compressed Backup Sets. . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Using Incrementally Updated Backups . . . . . . . . . .
MAA Workshop: Recovering from a Complete Loss. . . . . . . . . . . . . .
MAA Workshop: Using the Data Recovery Advisor. . . . . . . . . . . . . . .
MAA Workshop: Back Up Directly to Tape from the Standby. . . . . . .

283
288
299
302
312
322
333
337
339
341
349
353

357
360
364
373
380
388

Oracle-Regular / Oracle Database 11g Releas


Contents 

11g Release 2 High Availability



11 Flashback Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Using Flashback Transaction  . . . . . . . . . . . . . . . .
MAA Workshop: Enabling Flashback Database  . . . . . . . . . . . . . . . .
MAA Workshop: Perform Flashback Database  . . . . . . . . . . . . . . . .

397
410
415
417

part iV

Enhancing Availability with Additional Features

12 Oracle Data Guard Broker  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Creating a Physical Standby Oracle Data
Guard Broker Configuration  . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Modifying the Oracle Data Guard Protection
Level with DGMGRL  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Performing a Switchover Using the Broker  . . . . . .
MAA Workshop: Performing a Failover Using the Broker  . . . . . . . . .
MAA Workshop: Reinstatement of a Broker-Managed
Database  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13 Oracle Grid Control  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Installing WebLogic for Use by
Grid Control 11g  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MAA Workshop: Installing Oracle Grid Control 11g Release 1  . . . .
MAA Workshop: Installing Management Agents Using
the Push Method  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

423
425
430
431
433
435
443
453
461
471

Index  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491

xi


This page intentionally left blank


Contents
Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   xxiii
Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
x
part I

Oracle’s Grid Infrastructure


1 Architecting the Oracle Database Grid  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
LunarTrax: To the Moon and Beyond!  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Planning the Grid  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
The Grid and Planned Maintenance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Oracle Technologies: Reducing the Impact of Maintenance  . . . . . . . 7
Recovering Quickly from Failures  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Oracle Technologies at Play  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Protecting Against and Recovering from User Errors  . . . . . . . . . . . . . . . . . . . 10
Oracle Technology Checkpoint: Flashback Query
and Flashback Table  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Again with the Flashback Database  . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Planning for Expansion and Future Growth  . . . . . . . . . . . . . . . . . . . . . . . . . 11
Oracle Technology: Automatic Storage Management  . . . . . . . . . . . . 12

Again with the Oracle Clusterware  . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Disaster Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Oracle Technology: Oracle Data Guard  . . . . . . . . . . . . . . . . . . . . . . 13
What Next?  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Test, Test, and Test Some More  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Go Forth and Conquer  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17



2 Oracle VM  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
.
Virtualization Basics  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Oracle VM  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Oracle VM Server and Dom-0  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Oracle VM Manager  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

xiii


Oracle-Regular / Oracle Database 11g Release 2 High Availability / Jesse et al / 208-0

xiv  

  Oracle Database 11g Release 2 High Availability

Using Oracle VM Manager  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Server Pools  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Servers  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Resources  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Virtual Machines  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Using Oracle VM Server  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Multiple VM Servers Using iSCSI for Shared Discs  . . . . . . . . . . . . . . . . . . . . 42
Supported Oracle VM Configurations for Grid Infrastructure
and Oracle RAC  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Using Prebuilt Oracle RAC Templates  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45


3 Grid Infrastructure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Cluster Ready Services  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
CRS Concepts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Voting Disk/File  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Oracle Cluster Registry (OCR)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Oracle Local Registry (OLR)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Grid Naming Service (GNS)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Single Client Access Name and Its Listener  . . . . . . . . . . . . . . . . . . . . 58
Virtual IP Addresses  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Cluster Time Synchronization Services  . . . . . . . . . . . . . . . . . . . . . . . 61
Server Pools and Policy-Based Cluster Management  . . . . . . . . . . . . . 62
Role-Separated Management  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Node Number Pinning and Leases  . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Agents  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Integrating Intelligent Platform Management Interface (IPMI)  . . . . . . . 66
CRS Architecture  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Oracle HA Services Daemon  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Cluster Ready Services Daemon (CRSD)  . . . . . . . . . . . . . . . . . . . . . . 73
Clusterware Trace Files  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

4 Grid Infrastructure Installation and Configuration  . . . . . . . . . . . . . . . . . . . 77
GI Installation Choices and Requirements  . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Shared GI Home vs. Local GI Home  . . . . . . . . . . . . . . . . . . . . . . . . . 78
Separate Users for GI Home and DB Home  . . . . . . . . . . . . . . . . . . . 78
Shared Storage Choices for Clusterware Files  . . . . . . . . . . . . . . . . . . 79
Networking Requirements for CRS and Oracle RAC  . . . . . . . . . . . . . 81
Network Interface Bonding  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Hardware Requirements  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Cluster Verification Utility  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Grid Infrastructure Install  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Troubleshooting a GI Installation  . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1
Recovering from Failure in root.sh  . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1

Oracle-Regular / Oracle Database 11g Releas


Contents 

e 11g Release 2 High Availability

Adding and Removing Nodes from a Cluster  . . . . . . . . . . . . . . . . . . . . . . . . 16
1
Adding a Node  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1
Deleting a Node  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1
Upgrading to GI  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1
Things to Note for ASM  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1
Upgrade Paths  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1
The Actual Upgrade  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1
What’s New for 11.2.0.2  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1


5 Oracle Automatic Storage Management  . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1
ASM Concepts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1
ASM for Single-Instance Databases  . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1
ASM Instances  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1
ASM Disks  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1
ASM Disk Groups  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1
ASM Files, Directories, and Aliases  . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1
ASM Metadata  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
1
ASM Dynamic Volumes  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

1
ASM Dynamic Volume Manager   . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
1
ASM Cluster File System  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
1
ACFS for the Database Home  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
1
ACFS for a General Purpose File System  . . . . . . . . . . . . . . . . . . . . . . 54
1
ACFS Read-Only Snapshots  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
1
ACFS Tagging  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
1
ACFS Replication  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
1
ACFS Security  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
1
ACFS Encryption  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
1
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
1
part II

Oracle Real Application Clusters (Oracle RAC)


6 Oracle RAC Setup/Configuration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
1
Oracle RAC/RDBMS Install Options  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
1

Shared Home vs. Private Drives for the RDBMS  . . . . . . . . . . . . . . . . 74
1
Owner of the DB Home  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
1
File System Choice for DB Files  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
1
cluvfy  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
1
Installing Oracle RAC  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
1
Patching the Environment  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
1
ASM Disk Group Creation  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
1
Database Creation with DBCA  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
1
Workload Management Considerations  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
1

xv


xvi  

  Oracle Database 11g Release 2 High Availability

Enabling Archiving  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
1
Extending Oracle RAC Databases to New Nodes  . . . . . . . . . . . . . . . . . . . . . 91
1

Extending the Oracle RDBMS/Oracle RAC Home  . . . . . . . . . . . . . . . 92
1
Add an Instance to a Policy-Managed Database  . . . . . . . . . . . . . . . . 92
1
Add an Instance to an Administrator-Managed Database  . . . . . . . . . 92
1
Deleting an Instance from the Oracle RAC Database  . . . . . . . . . . . . . . . . . . 93
1
Delete an Instance from a Policy-Managed Database  . . . . . . . . . . . . 93
1
Delete an Instance from an Administrator-Managed Database  . . . . . 93
1
Installing Earlier RDBMS Releases  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
1
Using Older Oracle Database Versions with GI  . . . . . . . . . . . . . . . . 94
1
Installing a Version 10.2 RDBMS on 11.2 GI  . . . . . . . . . . . . . . . . . . . 95
1
Creating a 10.2 Database Using DBCA  . . . . . . . . . . . . . . . . . . . . . . . 97
1
Oracle RAC on Extended Distance Clusters  . . . . . . . . . . . . . . . . . . . . . . . . . 98
1
Stretching a Cluster  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
1
Stretching Network Connections  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
1
Shared Storage  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
1
Voting Disks  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 01
2

Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 02
2


7 Oracle RAC Administration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 03
.
2
Oracle RAC vs. Single-Instance: Additional Processes  . . . . . . . . . . . . . . . . . 04
2
LCK: Lock Process  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 04
2
LMD: Lock Manager Daemon Process  . . . . . . . . . . . . . . . . . . . . . . . 04
2
LMON: Lock Monitor Process  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 05
2
LMS: Lock Manager Server Process  . . . . . . . . . . . . . . . . . . . . . . . . . . 05
2
ACFS: ASM Cluster File System CSS Process  . . . . . . . . . . . . . . . . . . . 05
2
ACMS: Atomic Control File to Memory Service Process  . . . . . . . . . . 05
2
GTXn: Global Transaction Process  . . . . . . . . . . . . . . . . . . . . . . . . . . 06
2
LMHB: Global Cache/Enqueue Service Heartbeat Monitor  . . . . . . . . 06
2
PING: Interconnect Latency Measurement Process  . . . . . . . . . . . . . . 06
2
RMSn: Oracle RAC Management Process  . . . . . . . . . . . . . . . . . . . . . 06
2
RSMN: Remote Slave Monitor Process  . . . . . . . . . . . . . . . . . . . . . . . 06

2
Oracle RAC vs. Single-Instance: The Basics  . . . . . . . . . . . . . . . . . . . . . . . . . 06
2
Cache Fusion: A Brief Intro  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 06
2
Dynamic Resource Mastering  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 07
2
Reconfiguration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 07
2
Cache Coherency in an Oracle RAC Environment  . . . . . . . . . . . . . . . 08
2
Redo and Rollback with Oracle RAC  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 09
2
Redo Logs and Instance Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . 09
2
Redo Logs and Media Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2
Parallelism in an Oracle RAC Environment  . . . . . . . . . . . . . . . . . . . . . . . . . 11
2
Types of Parallelism  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2


Contents 

e 11g Release 2 High Availability

Monitoring the Environment  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2
Database Control and Grid Control  . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2
OS Watcher (OSW)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2
Cluster Health Monitor (CHM)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2
ORION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2
Tuning with AWR and ADDM  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2
Intelligent Infrastructure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2
MMON Background Process  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2
Automatic Workload Repository  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2
Automatic Database Diagnostic Monitor  . . . . . . . . . . . . . . . . . . . . . . 20
2
What Drives ADDM?  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2
Interconnect Performance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2
Sequence Caches  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2
HugePages on Linux  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2
Archiving and Backing Up  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2
Archiving with NFS Mount Points for Your Archive Destination  . . . . 26
2
Archiving in a Cluster File System Environment  . . . . . . . . . . . . . . . . . 27

2
Archiving in an ASM Environment  . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2
Additional Notes on Archiving in an Oracle RAC Environment  . . . . . 29
2
Patches and Patchsets  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2
Rolling Patch Updates  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2
Online Patching (Hot Patching)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2
The Out-of-Place Patchset Model  . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2
Proactive Maintenance Strategy  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2
Recommended Patches  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2
Patchset Updates  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2
Managing the Oracle RAC Database with SRVCTL  . . . . . . . . . . . . . . . . . . . 33
2
The Server Control Utility (SRVCTL)  . . . . . . . . . . . . . . . . . . . . . . . . . 33
2
Management Through Policies  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2
Managing Diagnostic Data  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2
Automatic Diagnostic Repository  . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2
ADRCI Command-Line Utility  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2
ADR Structure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2
ADR in Oracle RAC  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2
Reporting and Resolving a Problem  . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2


8 Utility Computing: Applications as Services  . . . . . . . . . . . . . . . . . . . . . . . . 43
2
Services Concepts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2
Services as a Workload  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2
Services as the Application’s Interface to the Database  . . . . . . . . . . . 45
2
Services from the Database Perspective  . . . . . . . . . . . . . . . . . . . . . . . 45
2

xvii


xviii  

  Oracle Database 11g Release 2 High Availability

Distributing Work Among Oracle RAC Nodes for Performance  . . . . . . . . . . 48

2
Client-Side Load Balancing  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2
Server-Side Load Balancing  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2
Load-Balancing Configuration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2
Event Notification  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2
Notification Concepts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2
Oracle Notification Service   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2
FAN Callouts   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2
Creating Services and Callouts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2
Creating Services  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2
Viewing Services from Within the Database  . . . . . . . . . . . . . . . . . . . 58
2
Using SRVCTL to Manage Services and Node Applications  . . . . . . . . . . . . . 59
2
Node Applications  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2
Managing Services via SRVCTL  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2
Cluster Listener Configuration  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2
Grid Naming Service and Listeners  . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2
SCAN Listeners  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2
Local Listeners  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2
Listener Registration and PMON Discovery  . . . . . . . . . . . . . . . . . . . . 66
2
Why a Virtual IP? TCP Timeouts  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2
Why a SCAN Virtual IP?  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2
Connect-Time Failover  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2
Transparent Application Failover  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2
Server-Side vs. Client-Side TAF  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2
Implementing TAF  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2
Easy Connect  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2
Oracle RAC/Standby/Replication Environments  . . . . . . . . . . . . . . . . . 75
2
Integration with OEM  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2
part III

Disaster Planning

9 Oracle Data Guard  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
2
Making the Right Choice  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2
Physical Standby Databases  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
2
Snapshot Standby Databases  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
2
Logical Standby Databases  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
2
Creating a Physical Standby  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
2
Creating a Snapshot Standby  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
2
Creating a Logical Standby  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 00
3
Logical Standby Unsupported Objects  . . . . . . . . . . . . . . . . . . . . . . . 00
3


Contents 

ase 11g Release 2 High Availability

Log Transport Services  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 05
3
Defining Log Transport Services Destinations  . . . . . . . . . . . . . . . . . . 05
3

Log Transport Services Security  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 09
3
Standby Redo Logs  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 09
3
Protection Modes  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3
Maximum Protection  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3
Maximum Availability  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3
Maximum Performance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3
Handling Network Disconnects  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3
Gap Detection and Resolution  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3
Automatic Gap Resolution  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3
FAL Gap Resolution  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3
Managing a Physical Standby Database  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3
Starting a Physical Standby  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3
Starting and Stopping Managed Recovery  . . . . . . . . . . . . . . . . . . . . . 17
3
Accommodating Physical Changes on the Primary  . . . . . . . . . . . . . . 18
3
Monitoring a Physical Standby  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3

Read-Only Physical Standby Including Oracle
Active Data Guard  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3
Managing a Logical Standby Database  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3
Starting and Stopping SQL Apply  . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3
Monitoring SQL Apply Progress  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3
Deletion of Standby Archived Logs  . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3
Protecting the Logical Standby from User Modifications  . . . . . . . . . . 29
3
Recovering from Errors  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3
Changing the Default Behavior of the SQL Apply Engine  . . . . . . . . . 31
3
Performing a Role Transition Using Switchover  . . . . . . . . . . . . . . . . . . . . . . 32
3
Performing a Role Transition Using Failover  . . . . . . . . . . . . . . . . . . . . . . . . 35
3
Failover First Steps  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3
Reinstating a Database After a Failover  . . . . . . . . . . . . . . . . . . . . . . . 39
3
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3


10 Backup and Recovery for MAA Environments  . . . . . . . . . . . . . . . . . . . . . . . 41

3
The Importance of Media Backups  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3
RMAN: A Primer  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3
RMAN and the Controlfile  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3
RMAN and the Data Block  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3
RMAN Command-Line Usage  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3
Preparing an RMAN Backup Strategy  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3
The Flash Recovery Area  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3
RMAN Configuration Parameters  . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3
Caring for Your Controlfile  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3
The Snapshot Controlfile  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3
The Recovery Catalog  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3

xix


xx  

  Oracle Database 11g Release 2 High Availability

Backups for the MAA Database  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3
The High Availability Backup Strategy  . . . . . . . . . . . . . . . . . . . . . . . 58
3
Backing Up the Flash Recovery Area  . . . . . . . . . . . . . . . . . . . . . . . . . 66
3
Backup Housekeeping  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3
Performing Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3
Database Recovery: Restore and Recover  . . . . . . . . . . . . . . . . . . . . . 72
3
Block Media Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3
Database Recovery: Data Recovery Advisor  . . . . . . . . . . . . . . . . . . . 76
3
Media Management Considerations  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3
The SBT Interface  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3
Backing Up Directly to Tape  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3
Oracle Secure Backup and the OSB Cloud Module  . . . . . . . . . . . . . . 85
3
RMAN and Oracle Data Guard  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3
Using RMAN to Build the Standby Database  . . . . . . . . . . . . . . . . . . . 85
3
Using the Physical Standby Database to Create Backups  . . . . . . . . . . 87

3
RMAN and Oracle RAC  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3
RMAN Configuration for the Cluster  . . . . . . . . . . . . . . . . . . . . . . . . . 91
3
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3


11 Flashback Recovery  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3
Being Prepared for the Inevitable: Flashback Technology  . . . . . . . . . . . . . . . 98
3
Laying the Groundwork  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3
Flashback Query and Flashback Table  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 00
4
Configuring for Flashback Query and Flashback Table  . . . . . . . . . . . 01
4
Flashback Query  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 02
4
Flashback Versions Query  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 03
4
Flashback Transaction Query  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 04
4
Flashback Table  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 05
4
Flashback Drop  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 06
4
The Recycle Bin  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 06

4
Flashback Transaction  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 08
4
Flashback Database  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4
Flashback Logs  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4
Flashback Retention Target  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4
Flashback Database: Tuning and Tweaking  . . . . . . . . . . . . . . . . . . . . 16
4
Different Uses of Flashback Database  . . . . . . . . . . . . . . . . . . . . . . . . 18
4
Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4
part IV

Enhancing Availability with Additional Features


12 Oracle Data Guard Broker  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4
Oracle Data Guard Broker Architecture Overview  . . . . . . . . . . . . . . . . . . . . 24
4
Oracle Data Guard Broker Configuration  . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4