Test Logs - Introduction

1 Test Logs - Introduction
Test Problem is a condition that exists within the software system that needs to be addressed.
Carefully and completely documenting a test problem is the first step in correcting the problem.
The following four attributes should be developed for all the test problems:
Statement of condition. –Tells what it is.
Criteria – Tells what should be.
These two attributes are the basis for a finding. If a comparison between the two gives little or no
practical consequence, no finding exists.
Effect: Tells why the difference between what is and what should be is significant
Cause: Tells the reasons for the deviation. Identification of the cause is the necessary as a basis for
corrective action.
A well developed problem statement will include each of these atttributes.When one or more these
attributes is missing, questions almost arise, such as
Criteria: Why is the current state inadequate?
Effect: How significant is it?
Cause: What could have cause of the problem?
1 Factors defining the Test Log Generation
Document Deviation:
Problem statements begin to emerge by process of comparision.Essentially the user compares” what is”
with “what should be”. When a deviation is identified between what is found to actually exist and what
the user thinks is correct or proper , the first essential step toward development of a problem statement
has occurred. It is difficult to visualize any type of problem that is not in some way characterized by this
deviation. The ‘What is”: can be called the statement of condition. The “What should be” shall be called
the “Criteria”. These concepts are the first two and the most basic , attributes of a problem statement.
The documenting of the deviation is describing the conditions, as they currently exist, and the criteria,
which represents what the user desires.
The actual deviation will be the difference or gap between “what –is” and “ what is desired”.
The statement of condition is uncovering and documenting the facts, as they exist.
What is a fact? The statement of condition will of course depend on the nature and extent of the
evidence or support that is examined and noted. For those facts, making up the
statement of condition, the I/S professional will need to ensure that the information is accurate, well

supported, and worded as clearly and precisely as possible.
The statement of condition should document as many of the following attributes as appropriate of the
problem.
Activities Involved:- The specific business or administered activities that are being performed during Test
Log generation are as follows:
Procedures used to perform work. – The specific step-by –step activities that are utilized in producing
the output from the identical activities.
Outputs /Deliverables – The products that are produced from the activity.
Inputs - The triggers,events,or documents that cause this activity to be executed.
Users/Customers served –The organization ,individuvals,or class users/customers serviced by this
activity.
Deficiencies noted – The status of the results of executing this activity and any appropriate
interpretation of those facts.
The Criterion is the user’s statement of what is desired. It can be stated in the either negative or
positive terms. For example , it could indicate the need to reduce the complaints or delays as well as
desired processing turn around time.
Work Paper to describe the problem, and document the statement of condition and the statement of
criteria. For example the following Work paper provides the information for Test Log Documentation:

Field Requirements: Field Instructions for Entering Data
Name of Software Tested : Put the name of the S/W or subsystem tested.
Problem Description: Write a brief narrative description of the variance uncovered from expectations
Statement of Conditions: Put the results of actual processing that occurred here.
Statement of Criteria: Put what testers believe was the expected result from processing
Effect of Deviation: If this can be estimated , testers should indicate what they believe the impact or
effect of the problem will be on computer processing
Cause of Problem: The testers should indicate what they believe is the cause of the problem, if known.
If the testers re unable to do this , the work paper will be given to the development team and they
should indicate the cause of the problem.
Location of the Problem: The Tests should document where problem occurred as closely as possible.

Recommended Action: The testers should indicate any recommended action they believe would be
helpful to the project team. If not approved, the alternate action should be listed or the reason for not
following the recommended action should be documented.
Name of the S/W tested:
Problem Description
Statement of Condition
Statement of Criteria
Effect of Deviation
Cause of a Problem
Location of the Problem
Recommended Action
2 Collecting Status Data
Four categories of data will be collected during testing. These are explained in the
following paragraphs.
Test Results Data
This data will include,
Test factors -The factors incorporated in the plan, the validation of which becomes the Test
Objective.
Business objective –The validation that specific business objectives have been met.
Interface Objectives-Validation that data/Objects can be correctly passed among Software
components.
Functions/Sub functions-Identifiable Software components normally associated with the
requirements of the software.
Units- The smallest identifiable software components
Platform- The hardware and Software environment in which the software system will operate.
Test Transactions, Test Suites, and Test Events
These are the test products produced by the test team to perform testing.
Test transactions/events: The type of tests that will be conducted during the execution of tests, which
will be based on software requirements.
Inspections – A verification of process deliverables against deliverable specifications.

Reviews: Verification that the process deliverables / phases are meeting the user’s true needs.
Defect
This category includes a Description of the individual defects uncovered during the testing process. This
description includes but not limited to :
Data the defect uncovered
Name of the Defect
Location of the Defect
Severity of the Defect
Type of Defect
How the defect was uncovered(Test Data/Test Script)
The Test Logs should add to this information in the form of where the defect originated , when it was
corrected, and when it was entered for retest.
Storing Data Collected during Testing
It is recommended that a database be established in which to store the results collected during testing.
It is also suggested that the database be put in online through client/server systems so that with a
vested interest in the status of the project can be readily accessed for the status update.
As described the most common test Report is a simple Spread sheet , which indicates the project
component for which the status is requested, the test that will be performed to determine the status of
that component, and the results of testing at any point of time.
Developing Test Status Reports
Report Software Status
Establish a Measurement Team
Inventory Existing Project Measures
Develop a Consistent Set of Project metrics
Define Process Requirements
Develop and Implement the Process
Monitor the Process
The Test process should produce a continuous series of reports that describe the status of testing. The
test reports are for use of testers, test managers, and the software development team. The frequency of
the test reports should be based on the discretion of the team and extensiveness of the test process.

Use of Function/Test matrix:
This shows which tests must be performed in order to validate the functions and also used to determine
the status of testing. Many organizations use spreadsheet package to maintain test results. The
intersection can be coded with a number or symbol to indicate the following:
1=Test is needed, but not performed
2=Test currently being performed
3=MINOR DEFECT NOTED
4=Major defect noted
5=Test complete and function is defect free for the criteria included in this test
TEST
FUNCTION 1 2 3 4 5 6 7 8 9
A
B
C
D
E
Function Test Matrix
1 Methods of Test Reporting
Reporting Tools - Use of word processing, database, defect tracking, and graphic tools to prepare test
reports.
Some Database test tools like Data Vision is a database reporting tool similar to Crystal Reports.
Reports can be viewed and printed from the application or output as HTML, LaTeX2e, XML, DocBook,
or tab- or comma-separated text files. From the LaTeX2e and DocBook output files you can in turn
produce PDF, text, HTML, PostScript, and more. Some query tools available for Linux-based databases
include:
QMySQL
dbMetrix
PgAccess
Cognos Powerhouse
This is not yet available for Linux; Cognos is looking into what interest people have in the product to

assess what their strategy should be with respect to the Linux ``market.''
GRG - GNU Report Generator
The GRG program reads record and field information from a dBase3+ file, delimited ASCII text file or a
SQL query to a RDBMS and produces a report listing. The program was loosely designed to produce
TeX/LaTeX formatted output, but plain ASCII text, troff, PostScript, HTML or any other kind of ASCII
based output format can be produced just as easily.
Word –Processing:
One way of increasing the utility of computers and word processors for the teaching of writing may be to
use software that will guide the processes of generating, organizing, composing and revising text. This
allows each person to use the normal functions of the computer keyboard that are common to all word
processors, email editors, order entry systems, and data base management products. From the Report
Manager, however, you can quickly scan through any number of these reports and see how each
person's history compares. A one-page summary report may be printed with either the Report Manager
program or from the individual keyboard or keypad software at any time. Individual Reports include all of
the following information.
Status Report
Word Processing Tests or Keypad Tests
Basic Skills Tests or Data Entry Tests
Progress Graph
Game Scores
Test Report for each test
Test Director:
 Facilitates consistent and repetitive testing process
 Central repository for all testing assets facilitates the adoption of a more consistent
testing process, which can be repeated throughout the application life cycle
 Provides Analysis and Decision Support
 Graphs and reports help analyze application readiness at any point in the testing
process
 Requirements coverage, run schedules, test execution progress, defect statistics can
be used for production planning

 Provides Anytime, Anywhere access to Test Assets
 Using Test Director’s web interface, tester, developers, business analysts and Client
can participate and contribute to the testing process
 Traceability throughout the testing process
 Test Cases can be mapped to requirements providing adequate visibility over the test
coverage of requirements
 Test Director links requirements to test cases and test cases to defects
 Manages Both Manual and Automated Testing
 Test Director can manage both manual and automated tests (Win Runner)
 Scheduling of automated tests can be effectively done using Test Director
Test Report Standards - Defining the components that should be included in a test report.
Statistical Analysis - Ability to draw statistically valid conclusions from quantitative test results.
Testing Data used for metrics
Testers are typically responsible for reporting their test status at regular intervals. The
following measurements generated during testing are applicable:
Total number of tests
Number of Tests executed to date
Number of tests executed successfully to date
Data concerning software defects include
Total number of defects corrected in each activity
Total number of defects entered in each activity.
Average duration between defect detection and defect correction
Average effort to correct a defect
Total number of defects remaining at delivery
Software performance data us usually generated during system testing, once the software has
been integrated and functional testing is complete.
Average CPU utilization
Average memory Utilization
Measured I/O transaction rate
Test Reporting

A final test report should be prepared at the conclusion of each test activity. This includes the following
 Individual Project Test Report
 Integration Test Report
 System Test Report
 Acceptance test Report
These test reports are designed to document the results of testing as defined in the testplan.The test
report can be a combination of electronic data and hard copy. For example, if the function matrix is
maintained electronically, there is no reason to print that, as paper report will summarize the data, draws
appropriate conclusions and present recommendations.9 - Purpose of a Test Report:
The test report has one immediate and three long term purposes. The immediate purpose is to provide
information to customers of the software system so that they can determine whether the system is ready
for production , and if so, to assess the potential consequences and initiate appropriate actions to
minimize those consequences.
The first of the three long term uses is for the project to trace problems in the event the application
malfunctions in production. Knowing which functions have been correctly tested and which ones still
contain defects can assist in taking corrective actions.
The second long term purpose is to use the data to analyze the rework process for making changes to
prevent the defects from occurring in the future. These defect prone components identify tasks/steps
that if improved, could eliminate or minimize the occurrence of high frequency defects. The Third long
term purpose is to show what was accomplished in case of an Y2K lawsuit.
 Individual Project Test Report
These reports focus on the Individual projects(software system),when different testers should test
individual projects, they should prepare a report on their results.
 Integration Test Report
Integration testing tests the interfaces between individual projects. A good test plan will identify the
interfaces and institute test conditions that will validate interfaces. Given is the Individual Project test
report except that conditions tested are interfaces.
1.Scope of Test – This section indicates which functions were and were not tested
2.Test Results – This section indicates the results of testing, including any variance between what
is and what should be

3.What works/What does not work - This section defines the functions that work and do not work
and the interfaces that work and do not work
4. Recommendations – This section recommends actions that should be taken to
Fix functions /Interfaces that do not work.
Make additional improvements
 System Test Reports
A System Test plan standard that identified the objective of testing , what was to be tested, how was it to
be tested, and when tests should occur. The system test Report should present the results of executing
the test plan. If these details are maintained Electronically , then it need only be referenced , not
included in the report.
 Acceptance Test Report
There are two primary objectives of Acceptance testing Report .The first is to ensure that the system as
implemented meets the real operating needs of the user/customer. If the defined requirements are those
true needs, testing should have accomplished this objective.
The second objective is to ensure that software system can operate in the real world user environment,
which includes people skills and attitudes, time pressures, changing business conditions, and so forth.
The Acceptance Test Report should encompass these criteria’s for the User acceptance respectively.
2 Conclusion
The Test Logs obtained from the execution of the test results and finally the test reports should be
designed to accomplish the following objectives:
 Provide Information to the customer whether the system should be placed into production, if so
the potential consequences and appropriate actions to minimize these consequences.
 One Long term objective is for the Project and the other is for the information technology
function.