Tools

1 Tools
1.1 LoadRunner 6.5
LoadRunner is Mercury Interactive’s tool for testing the performance of client/server systems.
LoadRunner enables you to test your system under controlled and peak load conditions. To
generate load, LoadRunner runs thousands of Virtual Users that are distributed over a network.
Using a minimum of hardware resources, these Virtual users provide consistent. Repeatable
and measurable load to execute your client/server system just as real users would.
LoadRunner’s in depth reports and graphs provide the information that you need to evaluate
the performance of your client/server system.
1.2 WebLoad 4.5
Webload is a testing tool for testing the scalability, functionality and performance of Web-based
applications – both Internet and Intranet. It can measure the performance of your application
under any load conditions. Use WebLoad to test how well your web site will perform under real-
world conditions by combining performance, load and functional tests or by running them
individually.
Webload supports HTTP1.0 and 1.1, including cookies, proxies, SSL, TSL, client certificates,
authentifications, persistent connections and chunked transfer coding.
Webload generates load by creating virtual clients that emulate network traffic. You create test
scripts (called agendas) using Java Scripts that instruct those virtual clients about what to do.
When Webload runs the test, it gathers results at a per-client, per-transaction and per-instance
level from the computers that are generating the load. Webload can also gather information
server’s performance monitor. You can watch the results as they occur- Webload displays them
in graphs and tables in real-time and you can save and export the results when the test is
finished.

Performance Testing Tools - summary and comparison
This table lists several performance testing tools available on the market. For your convenience we
compared them based on cost and OS required.
Tool Name URL Cost OS Description
Web

Performance
Trainer
perf
center.com/loadtesting.
html
Price ($) per
number of
virtual users:
1400-100
2495-200
4995-300
7995-1000
11995-5000
Windows NT,
Windows 2000,
Linux Solaris
Load test tool
emphasizing ease-
of-use. Supports all
browsers and web
servers; simulates
up to 200 users per
playback machine
at various
connection speeds;
records and allows
viewing of exact
bytes flowing
between browser
and server. Modem

simulation allows
each virtual user to
be bandwidth
limited. Can
automatically
handle variations in
session-specific
items such as
cookies,
usernames,
passwords, and any
other parameter to
simulate multiple
virtual users.
Notes:
downloadable, will
emulate 25 users,
and will expire in 2
weeks (may be
extend)
Astra LoadTest ratryand
Price ($) SunOS, Mercury's

buy.com
per number
of virtual
users:
9995-50
17995-100
29995-250

HP-UX,
IBM AIX,
NCR,
Windows NT,
WIN2000
load/stress testing
tool; includes
record/playback
capabilities;
integrated
spreadsheet
parameterizes
recorded input to
exercise application
with a wide variety
of data. 'Scenario
Builder' visually
combines virtual
users and host
machines for tests
representing real
user traffic. 'Content
Check' checks for
failures under heavy
load; Real-time
monitors and
analysis
Notes:
downloadable,
evaluation version

Benchmark
Factory
chmark
factory.com
$ Windows NT,
Windows2000
E-commerce load
testing tool from
Client/Server
Solutions, Inc.
Includes
record/playback,
web form
processing, user
sessions, scripting,
cookies, SSL. Also
includes pre-
developed industry
standard
benchmarks such
as AS3AP, Set-
Query, Wisconsin,
WebStone, and
others. Includes
optimized database
drivers for vendor-
neutral comparisons

- MS SQL Server,
Oracle 7 and 8,

Sybase System 11,
ODBC, IBM's DB2
CLI, Informix.
Notes:
downloadable (?),
after submitting
information A Page
with suggestion to
apply for next infos
to closest dealers
appeared
Radview's
WebLoad

$
Win95/98
Windows NT,
Windows 2000
Solaris, AIX
Supports recording
of SSL sessions,
cookies, proxies,
password
authentication,
dynamic HTML;
multiple platforms
Notes:
downloadable,
Evaluation version
does not support

SSl
MS Web
Application
Stress Test
rosoft.
com
Free
Windows NT,
Windows2000
Microsoft stress test
tool created by
Microsoft's Internal
Tools Group (ITG)
and subsequently
made available for
external use.
Includes
record/playback,
script recording
from browser, SSL,
adjustable delay
between requests
Notes: one of the
advanced tools in
the listing…
Rational Suite
Performance
/>products
$ Windows NT,
Windows2000,

Unix
Rational's
client/server and
web performance

Studio,
Rational
SiteLoad
testing tool.
'LoadSmart
Scheduling'
capabilities allow
complex usage
scenarios and
randomized
transaction
sequences; handles
dynamic web
pages.
Notes: request a cd
only. Not
downloadable
Forecast
$ Unix
Load testing tool
from Facilita
Software for web,
client-server,
network, and
database systems

Notes: not
downloadable
Zeus
s.
co.uk/intro.html
Free Unix
Free web
benchmarking/load
testing tool
available as source
code; will compile
on any UNIX
platform
Notes:
unsupportable (?),
broken download
link.
E-Load software.
com/products/eload_
index.shtml
$ Win95/98/
Windows NT
Load test tool from
RSW geared to
testing web
applications under
load and testing
scalability of E-
commerce
applications. For

use in conjunction
with test scripts
from their e-Tester

functional test tool.
Allows on-the-fly
changes and has
real-time reporting
capabilities.
Notes:
downloadable, free
cd request,
evaluation copy
HTTP-Load
/>software/http_load
Free Unix
Free load test
application to
generate web
server loads
Notes: free and
easy.
QALoad
puware.
com/products/auto/
releases/QALoad.htm
$
Win95/NT
- manager;
Unix,

Windows NT -
load test
player
Compuware's
QALoad for
load/stress testing
of database, web,
and char-based
systems, works with
such middleware
as: SQLnet, DBLib
or CBLib, SQL
Server, ODBC,
Telnet, and Web
Notes : free cd
request
SilkPerformer
/>html/s_solutions/s_perf
ormer/s_performer.htm
$
Windows NT,
Windows 2000
Load and
performance testing
component of
Segue's Silk web
testing toolset.
Notes: no
download.
WEBArt />webart

$ Windows 98,
Windows NT
4.0,
Windows 2000,
SunOS/Solaris,
AIX, Linux
Tool for load testing
of up to 100-200
simulated users;
also includes
functional and
regression testing
capabilities, and
capture/playback

and scripting
language.
Evaluation copy
avail.
Notes:
downloadable
Webload
/>products/platinum/
appdev/fe_iltps.htm
$
AIX,
Windows NT,
Windows 95,
Sun Solaris
Final Exam

WebLoad
integration and pre-
deployment testing
ensures the
reliability,
performance, and
scalability of Web
applications. It
generates and
monitors load stress
tests - which can be
recorded during a
Web session with
any browser - and
assesses Web
application
performance under
user-defined
variable system
loads. Load
scenarios can
include unlimited
numbers of virtual
users on one or
more load servers,
as well as single
users on multiple
client workstations.
Notes:
downloadable,

15day eval. period
Microsoft
WCAT load
test tool
rosoft.
com/workshop/server/
toolbox/wcat.asp
Free
Windows NT,
Windows 2000
Web load test tool
from Microsoft for
load testing of MS
IIS on NT
Webspray hillnet
$199 ($99
with
Windows 98,
Windows NT
Load testing tool;
includes link testing

works.com
discount)
4.0,
Windows 2000
capabilities; can
simulate up to 1,000
clients from a single
IP address; also

supports multiple IP
addresses with or
without aliases.
Notes: not
downloadable
WebSizr,
WebCorder
hnova
tions.com/home.htm
$
Win95(98),
Windows NT,
Windows 2000
Load testing and
capture/playback
tools from
Technovations.
WebSizr load
testing tool supports
authentication,
cookies, redirects
Notes:
downloadable, 30
eval. period.
1.3 Architecture Benchmarking
• Hardware Benchmarking - Hardware benchmarking is performed to size the
application with the planned Hardware platform. It is significantly different from capacity
planning exercise in that it is done after development and before deployment
• Software Benchmarking - Defining the right placement and composition of software
instances can help in vertical scalability of the system without addition of hardware

resources. This is achieved through software benchmark test.
1.4 General Tests
What follows is a list of tests adaptable to assess the performance of most systems. The
methodologies below are generic, allowing one to use a wide range of tools to conduct the
assessments.
Methodology Definitions
• Result: provide information about what the test will accomplish.

• Purpose: explains the value and focus of the test, along with some simple background
information that might be helpful during testing.
• Constraints: details any constraints and values that should not be exceeded during
testing.
• Time estimate: a rough estimate of the amount of time that the test may take to
complete.
• Type of workload: in order to properly achieve the goals of the test, each test requires
a certain type of workload. This methodology specification provides information on the
appropriate script of pages or transactions for the user.
• Methodology: a list of suggested steps to take in order to assess the system under
test.
• What to look for: contains information on behaviors, issues and errors to pay attention
to during and after the test.

2 Performance Metrics
The Common Metrics selected /used during the performance testing is as below
• Response time
• Turnaround time = the time between the submission of a batch job and the completion
of its output.
• Stretch Factor: The ratio of the response time with single user to that of concurrent
users.
• Throughput: Rate (requests per unit of time) Examples:

• Jobs per second
• Requests per second
• Millions of Instructions Per Second (MIPS)
• Millions of Floating Point Operations Per Second (MFLOPS)
• Packets Per Second (PPS)
• Bits per second (bps)
• Transactions Per Second (TPS)
• Capacity:
Nominal Capacity: Maximum achievable throughput under ideal workload conditions.
E.g., bandwidth in bits per second. The response time at maximum throughput is too
high.
• Usable capacity: Maximum throughput achievable without exceeding a pre-specified
response-time limit
• Efficiency: Ratio usable capacity to nominal capacity. Or, the ratio of the performance
of an n-processor system to that of a one-processor system is its efficiency.
• Utilization: The fraction of time the resource is busy servicing requests.
• Average Fraction used for memory.
As tests are executed, metrics such as response times for transactions, HTTP requests per
second, throughput etc., should be collected. It is also important to monitor and collect the
statistics such as CPU utilization, memory, disk space and network usage on individual web,
application and database servers and make sure those numbers recede as load decreases.
Cognizant has built custom monitoring tools to collect the statistics. Third party monitoring tools
are also used based on the requirement.
2.1 Client Side Statistics
• Running Vusers
• Hits per Second
• Throughput
• HTTP Status Code
• HTTP responses per Second

• Pages downloaded per Second
• Transaction response time
• Page Component breakdown time
• Page Download time
• Component size Analysis
• Error Statistics
• Errors per Second
• Total Successful/Failed Transactions
2.2 Server Side Statistics
• System Resources - Processor Utilization, Memory and Disk Space
• Web Server Resources–Threads, Cache Hit Ratio
• Application Server Resources–Heap size, JDBC Connection Pool
• Database Server Resources–Wait Events, SQL Queries
• Transaction Profiling
• Code Block Analysis
2.3 Network Statistics
• Bandwidth Utilization
• Network delay time
• Network Segment delay time
2.4 Conclusion
Performance testing is an independent discipline and involves all the phases as the
mainstream testing lifecycle i.e strategy, plan, design, execution, analysis and reporting.
Without the rigor described in this paper, executing performance testing does not yield
anything more than finding more defects in the system. However, if executed systematically
with appropriate planning, performance testing can unearth issues that otherwise cannot be
done through mainstream testing. It is very typical of the project manager to be overtaken by
time and resource pressures leading not enough budget being allocated for performance
testing, the consequences of which could be disastrous to the final system. There is another flip
side of the coin.
However there is an important point to be noted here. Before testing the system for

performance requirements, the system should have been architected and designed for
meeting the required performance goals. If not, it may be too late in the software development
cycle to correct serious performance issues.
Web-enabled applications and infrastructures must be able to execute evolving business
processes with speed and precision while sustaining high volumes of changing and