Troubleshooting and Management

10

Troubleshooting and Management

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Objectives
After completing this lesson, you should be able to:
• Work with Access Tester
• Identify connectivity issues
– Between agents and servers (impact of load balancers and
firewalls)

•
•

10 - 2

Explain OAM-specific WLST commands
Work with Oracle Enterprise Manager Fusion Middleware
Control

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Objectives
After completing this lesson, you should be able to:
• Describe the diagnostic capabilities within OAM 11g
– OAM Access Tester

•

Explain EM FMW Control integration
–
–
–
–
–

10 - 3

Server processes and charts
Topology viewer
Farm and domain
OAM server management
MBean browser

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Road Map
•
•
•
•
•

10 - 4

Working with Access Tester

WLS troubleshooting tips and agent and
server monitoring
Top problem areas
Working with WLST
Monitoring by using EM FMW Control

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Access Tester
•

Simulates interactions between registered OAM agents
and OAM 11g servers
– You can verify agent connection and test policy definitions.
– An administrator emulates the end user and the Access
Tester emulates agents.

•
•
•

10 - 5

Is a stand-alone Java application that ships with Oracle
Access Manager 11g
Can be run from any computer
Has both a GUI (manual testing) and command-line
interface (automated testing)

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Use Cases: Access Tester
•

Use Cases:
– Simulate interaction between OAM agents and the OAM
server
– Handle the response from the OAM server in the same
manner as a real agent
– Review the results of intended policy changes
– Troubleshoot issues with agent connections or access policy
definitions
– Track the latency of authentication and authorization
requests
– Stress-test the OAM server
– Establish performance metrics

10 - 6

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Access Tester Simulating Steps 1, 3, 5, 6
of Agent and OAM Server Interaction
User

WebGate
(agent)

2

7

Application

4
1.
2.
3.

4.

5.

6.

7.

10 - 8

Agent connects to OAM server - Connect
User accesses application resource
Agent makes IsProtected (Validate) request
3
1
• OAM server returns Yes/No and type of
credentials required
5
6

For protected resources, agent prompts user for
credentials
• User or user agent submits credentials
Agent makes IsAuthenticated request
• OAM server validates user credentials and
returns Y/N and additional responses
For authenticated users, agent makes IsAuthorized
request
• OAM server evaluates policies and returns Y/N
and additional responses
Oracle Access
Agent grants or denies access to application
Manager Server

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

User
Store

Policy
Store


Access Tester: Core Functionality
Testing
•
Connect to policy servers
•
Validate resource protection
•

Authenticate users
•
Authorize users
Automation and Analysis
•
Collect test cases
•
Generate test scripts
•
Run test scripts
•
Evaluate results and analyze differences
Usability
•
GUI (manual) and command-line (automated) testing modes
•
Scalable testing framework via separation of test cases from physical servers
•
Auto-import of resources to test
•
XML persistence

10 - 9

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Access Tester Architecture

10 - 10

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Output Files and Security Features
•

The following XML files are produced when you run the
Access Tester :
–
–
–
–
–

•

config.xml
script.xml
oamtest_target.xml
oamtest_stats.xml
oamtest_log.xml

Security:
– Supports Open and Simple modes
– Encrypts passwords

10 - 12

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

Starting Access Tester
•

Ensure that the computer from which the tester will be run
includes JDK/JRE 6
– Java –version

•

Copy the Access Tester JAR files:
– IDM_HOME/oam/server/tester/oamtest.jar
IDM_HOME/oam/server/tester/nap-api.jar

•

•

Ensure that the nap-api.jar is present in the same
directory as oamtest.jar on any computer from which
you want to run the Access Tester.
Start in Console mode:
– java –Dlog.traceconnfile=“d:\conn.txt” -jar
oamtest.jar

•

Start in command line mode:
– java -Dscript.scriptfile=“d:\tests\script.xml"

-Dcontrol.ignorecontent="true" -jar oamtest.jar

10 - 13

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


System Properties

10 - 15

Property

Mode

log.traceconnfile

Console and
Command Line

display.fontname

Console

display.fontsize

Console

display.usesystem

Console

script.scriptfile

Command Line

control.configfile

Command Line

control.testname

Command Line

control.testnumber

Command Line

control.ignorecontent

Command Line

control.loopback

Command Line

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Access Tester Console

10 - 18

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Test Cases and Test Scripts

10 - 20

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Road Map
•
•
•
•
•

10 - 24

Working with Access Tester
WLS troubleshooting tips and agent and
server monitoring
Top problem areas
Working with WLST
Monitoring by using EM FMW Control

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

Using weblogic.Admin Utility to Check the State
of Servers
•

weblogic.Admin utility is a command-line interface that
you can use to administrate, configure, and monitor
WebLogic Server.
– Run setWLSEnv.bat
– java weblogic.Admin -url t3://localhost:7001
-username weblogic –password <Password> GET
-pretty -type ServerRuntime
– java weblogic.Admin -url t3://localhost:7001
-username weblogic –password <Password> GETSTATE
– java weblogic.Admin -url t3://localhost:7001
-username weblogic –password <Password> GETSTATE
oam_server1

10 - 25

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Examining Admin Server and Managed Server
Logs
•

Default location for the WebLogic Server log files:
– DOMAIN_NAME\servers\SERVER_NAME\logs\SERVER_N

AME.log

•

Domain log resides in:
– DOMAIN_NAME\servers\ADMIN_SERVER_NAME\logs\DO
MAIN_NAME.log

•

HTTP subsystem keeps a log of all HTTP transactions in:
– DOMAIN_NAME\servers\SERVER_NAME\logs\SERVER_N
AME.out

•

Node manager writes its startup and status messages to:
– NM_HOME\nodemanager.log

–

WebLogic auditing provider saves auditing information to:
– WL_HOME\DOMAIN_NAME\servers\SERVER_NAME\logs\
DefaultAuditRecorder.log

10 - 26

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

WebLogic Admin Server and Managed Server
Thread Dump
•

•

Thread dumps are JVM reports that can be used to
analyze admin and managed servers, as well as JVM hang
situations, and determine the root cause of the issue.
To take a thread dump:
– Admin console > Server > <Server_Name> > Monitoring >
Threads > Dump Thread Stack
– connect(‘weblogic’,'weblogic’,'t3://localhost:7001
′)
cd (”Servers’)
ls()
cd (‘AdminServer’)
ls()
threadDump()

10 - 28

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Agent and Server Monitoring

10 - 30

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.

OAM Proxy Errors
•
•

Uses Apache log4j for logging
Writes logging information into a log file mentioned in
log4j.properties

•

The logger name used by OAM proxy components is
oracle.oam.proxy.oam

10 - 31

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Configuration Data
•

Stored in an XML file: oam-config.xml
– <Default Domain Directory>/config/fmwconfig

•

10 - 32

Only OAM admin console or WLST commands to be used
for changes; do not edit this file manually

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Road Map
•
•
•
•
•

10 - 33

Working with Access Tester
WLS troubleshooting tips and agent and
server monitoring
Top problem areas
Working with WLST
Monitoring by using EM FMW Control

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


Top Problem Areas
•
•
•
•

•
•

10 - 34

LDAP server and identity store
OAM run-time servers and hosts
Agent side configuration and load
Run-time database issues (audit and session data)
Admin change propagation and activation
Policy repository database issues

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


LDAP Server
Operational slowness:
• Non-OAM load impacting OAM operations
• Capacity problems due to gradual increase in peak load
• Consequences:
– Poor user experience
– Agent timeouts leading to retries

LDAP server availability
• Outage of all LDAP servers
• Load balancer timing out old connections
• Consequence:
– Total loss of service

10 - 35

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.


OAM Runtime Servers
Capacity problems
• CPU cycles
• Memory issues
• Consequence:
– Poor user experience due to slow operations
– Agent timeouts and retry may result in extra load

Interference with other services on host
• CPU cycle contention
• Memory contention
• File system full
• Consequence:
– Same as above
10 - 36

Copyright © 2010, Oracle and/or its affiliates. All rights reserved.