Test strategy

1 Test Strategy
1 Introduction
This Document entails you towards the better insight of the Test Strategy and its methodology.
It is the role of test management to ensure that new or modified service products meet the business
requirements for which they have been developed or enhanced.
The Testing strategy should define the objectives of all test stages and the techniques that apply. The testing
strategy also forms the basis for the creation of a standardized documentation set, and facilitates
communication of the test process and its implications outside of the test discipline. Any test support tools
introduced should be aligned with, and in support of, the test strategy. Test Approach/Test Architecture are
the acronyms for Test Strategy.
Test management is also concerned with both test resource and test environment management.
2 Key elements of Test Management:
Test organization –the set-up and management of a suitable test organizational structure and explicit role
definition. The project framework under which the testing activities will be carried out is reviewed, high level
test phase plans prepared and resource schedules considered. Test organization also involves the
determination of configuration standards and the definition of the test environment.
Test planning – the requirements definition and design specifications facilitate in the identification of major
test items and these may necessitate the test strategy to be updated. A detailed test plan and schedule is
prepared with key test responsibilities being indicated.
Test specifications – required for all levels of testing and covering all categories of test. The required
outcome of each test must be known before the test is attempted.
Unit, integration and system testing – configuration items are verified against the appropriate specifications
and in accordance with the test plan. The test environment should also be under configuration control and
test data and results stored for future evaluation.
Test monitoring and assessment – ongoing monitoring and assessment of the integrity of the
development and construction. The status of the configuration items should be reviewed against the phase
plans and test progress reports prepared providing some assurance of the verification and validation
activities.
Product assurance – the decision to negotiate the acceptance testing program and the release and
commissioning of the service product is subject to the ‘product assurance’ role being satisfied with the
outcome of the verification activities. Product assurance may oversee some of the test activity and may

participate in process reviews.
A common criticism of construction programmers is that insufficient time is frequently allocated to the testing
and commissioning of the building systems together with the involvement and subsequent training of the
Facilities Management team. Testing and commissioning is often considered by teams as a secondary
activity and given a lower priority particularly as pressure builds on the program towards completion.
Sufficient time must be dedicated to testing and commissioning as ensuring the systems function correctly is
fairly fundamental to the project’s success or failure. Traditionally the responsibility for testing and
commissioning is buried deep within the supply chain as a sub-contract of a sub-contract. It is possible to
gain greater control of this process and the associated risk through the use of specialists such as Systems
Integration who can be appointed as part of the professional team.
The time necessary for testing and commissioning will vary from project to project depending upon the
complexity of the systems and services that have been installed. The Project Sponsor should ensure that
the professional team and the contractor consider realistically how much time is needed.
Fitness for purpose checklist:
• Is there a documented testing strategy that defines the objectives of all test stages and the
techniques that may apply, e.g. non-functional testing and the associated techniques such as
performance, stress and security etc?
• Does the test plan prescribe the approach to be taken for intended test activities, identifying:
• the items to be tested,
• the testing to be performed,
• test schedules,
• resource and facility requirements,
• reporting requirements,
• evaluation criteria,
• risks requiring contingency measures?
• Are test processes and practices reviewed regularly to assure that the testing processes continue to
meet specific business needs?
For example, e-commerce testing may involve new user interfaces and a business focus on usability may
mean that the organization must review its testing strategies.
3 Test Strategy Flow :

Test Cases and Test Procedures should manifest Test Strategy.
Test Strategy – Selection
Selection of the Test Strategy is based on the following factors
 Product
Test Strategy based on the Application to help people and teams of people in making
decisions.
 Based on the Key Potential Risks
 Suggestion of Wrong Ideas.
 People will use the Product Incorrectly
 Incorrect comparison of scenarios.
 Scenarios may be corrupted.
 Unable to handle Complex Decisions.
 Determination of Actual Risk.
 Understand the underlying Algorithm.
 Simulate the Algorithm in parallel.
 Capability test each major function.
 Generate large number of decision scenarios.
 Create complex scenarios and compare them.
 Review Documentation and Help.
 Test for sensitivity to user Error.
Test Strategy Execution:
Understand the decision Algorithm and generate the parallel decision analyzer using the Perl or Excel that
will function as a reference for high volume testing of the app.
 Create a means to generate and apply large numbers of decision scenarios to the product. This will
be done using the GUI test Automation system
or through the direct generation of Decide Right scenario files that
would be loaded into the product during test.
 Review the Documentation, and the design of the user interface and functionality for its sensitivity
to user error.
 Test with decision scenarios that are near the limit of complexity allowed by the product

 Compare complex scenarios.
 Test the product for the risk of silent failures or corruptions in decision analysis.
 Issues in Execution of the Test Strategy
 The difficulty of understanding and simulating the decision algorithm
 The risk of coincidal failure of both the simulation and the product.
 The difficulty of automating decision tests
4 General Testing Strategies
• Top-down
• Bottom-up
• Thread testing
• Stress testing
• Back-to-back testing
5 Need for Test Strategy
The objective of testing is to reduce the risks inherent in computer systems. The strategy
must address the risks and present a process that can reduce those risks. The system
concerns on risks then establish the objectives for the test process. The two
components of the testing strategy are the Test Factors and the Test Phase.
 Test Factor – The risk or issue that needs to be addressed as part of the test strategy. The strategy
will select those factors that need to be addressed in the testing of a specific application system.
 Test Phase – The Phase of the systems development life cycle in which testing will occur.
Not all the test factors will be applicable to all software systems. The development team will need to
select and rank the test factors for the specific software systems being developed.
The test phase will vary based on the testing methodology used. For example the test phases in as
traditional waterfall life cycle methodology will be much different from the phases in a Rapid
Application Development methodology.
6 Developing a Test Strategy
The test Strategy will need to be customized for any specific software system.
Analysis Coding Errors 36%
and
design

Errors 64%
The applicable test factors would be listed as the phases in which the testing must occur.
Four test steps must be followed to develop a customized test strategy.
 Select and rank Test Factors
 Identify the System Developmental Phases
 Identify the Business risks associated with the System under Development.
 Place risks in the Matrix
TestFactors\T
est Phase
Requirements
Design
Build
Dynamic Test
Integrate
Maintain
7 Conclusion:
Test Strategy should be developed in accordance with the business risks associated with the software when
the test team develop the test tactics. Thus the Test team needs to acquire and study the test strategy that
should question the following:
 What is the relationship of importance among the test factors?
 Which of the high level risks are the most significant?
 What damage can be done to the business if the software fails to perform correctly?
 What damage can be done to the business if the business if the software is not completed on time?
 Who are the individuals most knowledgeable in understanding the impact of the identified business
risks?
Hence the Test Strategy must address the risks and present a process that can reduce those risks. The
system accordingly focuses on risks thereby establishes the objectives for the test process.
Risks
Factors
2 TEST PLAN

1 What is a Test Plan?
A Test Plan can be defined as a document that describes the scope, approach, resources and
schedule of intended test activities. It identifies test items, the features to be tested, the testing tasks,
who will do each task, and any risks requiring contingency planning.
The main purpose of preparing a Test Plan is that everyone concerned with the project are in sync
with regards to the scope, responsibilities, deadlines and deliverables for the project. It is in this
respect that reviews and a sign-off are very important since it means that everyone is in agreement of
the contents of the test plan and this also helps in case of any dispute during the course of the project
(especially between the developers and the testers).
Purpose of preparing a Test Plan
A Test Plan is a useful way to think through the efforts needed to validate the acceptability of a
software product.
The completed document will help people outside the test group understand the 'why' and 'how' of
product validation.
It should be thorough enough to be useful but not so thorough that no one outside the test group will
read it.
Contents of a Test Plan
1. Purpose
2. Scope
3. Test Approach
4. Entry Criteria
5. Resources
6. Tasks / Responsibilities
7. Exit Criteria
8. Schedules / Milestones
9. Hardware / Software Requirements
10. Risks & Mitigation Plans
11. Tools to be used
12. Deliverables
13. References

a. Procedures
b. Templates
c. Standards/Guidelines
14. Annexure
15. Sign-Off
2 Contents (in detail)
Purpose
This section should contain the purpose of preparing the test plan
Scope
This section should talk about the areas of the application which are to be tested by the QA team and
specify those areas which are definitely out of scope (screens, database, mainframe processes etc).
Test Approach
This would contain details on how the testing is to be performed and whether any specific strategy is to be
followed (including configuration management).
Entry Criteria
This section explains the various steps to be performed before the start of a test (i.e.) pre-requisites. For
example: Timely environment set up, starting the web server / app server, successful implementation of the
latest build etc.
Resources
This section should list out the people who would be involved in the project and their designation etc.
Tasks / Responsibilities
This section talks about the tasks to be performed and the responsibilities assigned to the various members
in the project.
Exit criteria
Contains tasks like bringing down the system / server, restoring system to pre-test
environment, database refresh etc.
Schedules / Milestones
This sections deals with the final delivery date and the various milestone dates to be met in the course of the
project.
Hardware / Software Requirements

This section would contain the details of PC’s / servers required (with the configuration) to install the
application or perform the testing; specific software that needs to be installed on the systems to get the
application running or to connect to the database; connectivity related issues etc.
Risks & Mitigation Plans
This section should list out all the possible risks that can arise during the testing and the mitigation plans
that the QA team plans to implement incase the risk actually turns into a reality.
Tools to be used
This would list out the testing tools or utilities (if any) that are to be used in the project (e.g.) WinRunner, Test
Director, PCOM, WinSQL.
Deliverables
This section contains the various deliverables that are due to the client at various points of time (i.e.) daily,
weekly, start of the project, end of the project etc. These could include Test Plans, Test Procedure, Test
Matrices, Status Reports, Test Scripts etc. Templates for all these could also be attached.
References
Procedures
Templates (Client Specific or otherwise)
Standards / Guidelines (e.g.) QView
Project related documents (RSD, ADD, FSD etc)
Annexure
This could contain embedded documents or links to documents which have been / will be used in the course
of testing (e.g.) templates used for reports, test cases etc. Referenced documents can also be
attached here.
Sign-Off
This should contain the mutual agreement between the client and the QA team with both leads / managers
signing off their agreement on the Test Plan.
3 Test Data Preparation - Introduction
A System is programmed by its data. Functional testing can suffer if data is poor, and good data can help
improve functional testing. Good test data can be structured to improve understanding and testability. Its
contents, correctly chosen, can reduce maintenance effort and allow flexibility. Preparation of the data can
help to focus the business where requirements are vague.

The first stage of any recogniser development project is data preparation.
Test data should however, be prepared which is representative of normal business transactions. Actual
customer names or contact details should also not be used for such tests. It is recommended that a full test
environment be set up for use in the applicable circumstances.
Each separate test should be given a unique reference number which will identify the Business Process
being recorded, the simulated conditions used, the persons involved in the testing process and the date the
test was carried out. This will enable the monitoring and testing reports to be co-coordinated with any
feedback received.
Tests must be planned and thought out a head of time; you have to decide such things as what exactly you
are testing and testing for, the way the test is going to be run and applied, what steps are required, etc.
Testing is the process of creating, implementing and evaluating tests.
Effective quality control testing requires some basic goals and understanding:
You must understand what you are testing; if you're testing a specific functionality, you must know how it's
supposed to work, how the protocols behave, etc.
You should have a definition of what success and failure are. In other words, is close enough good enough?
You should have a good idea of a methodology for the test, the more formal a plan the better; you should
design test cases.
You must understand the limits inherent in the tests themselves.
You must have a consistent schedule for testing; performing a specific set of tests at appropriate points in
the process is more important than running the tests at a specific time.
Roles of Data in Functional Testing
Testing consumes and produces large amounts of data. Data describes the initial conditions for a test, forms
the input, is the medium through which the tester influences the software. Data is manipulated, extrapolated,
summarized and referenced by the functionality under test, which finally spews forth yet more data to be
checked against expectations. Data is a crucial part of most functional testing.
This paper sets out to illustrate some of the ways that data can influence the test process, and will show that
testing can be improved by a careful choice of input data. In doing this, the paper will concentrate most on
data-heavy applications; those which use databases or are heavily influenced by the data they hold. The
paper will focus on input data, rather than output data or the transitional states the data passes through
during processing, as input data has the greatest influence on functional testing and is the simplest to

manipulate. The paper will not consider areas where data is important to non-functional testing, such as
operational profiles, massive datasets and environmental tuning.
A SYSTEM IS PROGRAMMED BY ITS DATA
Many modern systems allow tremendous flexibility in the way their basic functionality can be used.
Configuration data can dictate control flow, data manipulation, presentation and user interface. A system can
be configured to fit several business models, work (almost) seamlessly with a variety of cooperative systems
and provide tailored experiences to a host of different users. A business may look to an application's
configurability to allow them to keep up with the market without being slowed by the development process,
an individual may look for a personalized experience from commonly-available
software.
FUNCTIONAL TESTING SUFFERS IF DATA IS POOR
Tests with poor data may not describe the business model effectively, they may be hard to maintain, or
require lengthy and difficult setup. They may obscure problems or avoid them altogether. Poor data tends to
result in poor tests, that take longer to execute.
GOOD DATA IS VITAL TO RELIABLE TEST RESULTS
An important goal of functional testing is to allow the test to be repeated with the same result, and varied to
allow diagnosis. Without this, it is hard to communicate problems to coders, and it can become difficult to
have confidence in the QA team's results, whether they are good or bad. Good data allows diagnosis,
effective reporting, and allows tests to be repeated with confidence,.
GOOD DATA CAN HELP TESTING STAY ON SCHEDULE
An easily comprehensible and well-understood dataset is a tool to help communication. Good data can
greatly assist in speedy diagnosis and rapid re-testing. Regression testing and automated test maintenance
can be made speedier and easier by using good data, while an elegantly-chosen dataset can often allow
new tests without the overhead of new data.
A formal test plan is a document that provides and records important information about a test project, for
example:
project and quality assumptions
project background information
resources
schedule & timeline

entry and exit criteria
test milestones
tests to be performed
use cases and/or test cases
1 Criteria for Test Data Collection
This section of the Document specifies the description of the test data needed to test
recovery of each business process.
 Identify Who is to Conduct the Tests
In order to ensure consistency of the testing process throughout the organization, one or more members of
the Business Continuity Planning (BCP) Team should be nominated to co-ordinate the testing process within
each business unit, a nominated testing and across the organization. Each business process should be
thoroughly tested and the coordinator should ensure that each business unit observes the necessary rules
associated with ensuring that the testing process is carried out within a realistic environment.
This section of the BCP should contain the names of the BCP Team members nominated to co-ordinate the
testing process. It should also list the duties of the appointed co-ordinators.
 Identify Who is to Control and Monitor the Tests
In order to ensure consistency when measuring the results, the tests should be independently monitored.
This task would normally be carried out by a nominated member of the Business Recovery Team or a
member of the Business Continuity Planning Team.
This section of the BCP will contain the names of the persons nominated to monitor the testing process
throughout the organization. It will also contain a list of the duties to be undertaken by the monitoring staff.
 Prepare Feedback Questionnaires
It is vital to receive feedback from the persons managing and participating in each of the tests. This
feedback will hopefully enable weaknesses within the Business Recovery Process to be identified and
eliminated. Completion of feedback forms should be mandatory for all persons participating in the testing
process. The forms should be completed either during the tests (to record a specific issue) or as soon after
finishing as practical. This will enable observations and comments to be recorded whilst the event is still
fresh in the persons mind.
This section of the BCP should contain a template for a Feedback Questionnaire.
Prepare Budget for Testing Phase

Each phase of the BCP process which incurs a cost requires that a budget be prepared
and approved. The 'Preparing for a Possible Emergency' Phase of the BCP process will
involve the identification and implementation of strategies for back up and recovery of
data files or a part of a business process. It is inevitable that these back up and recovery
processes will involve additional costs. Critical parts of the business process such as the
IT systems, may require particularly expensive back up strategies to be implemented.
Where the costs are significant they should be approved separately with a specific
detailed budget for the establishment costs and the ongoing maintenance costs.
This section of the BCP will contain a list of the testing phase activities and a cost for
each. It should be noted whenever part of the costs is already incorporated with the
organization’s overall budgeting process.
 Training Core Testing Team for each Business Unit