Relying on ongoing monitoring to test controls performance to reduce the scope of separate testing

Research Series

A Top-Down Approach to
Risk Management and Internal Control

Issue 4

Relying on Ongoing
Monitoring to Test Controls
Performance, to Reduce the
Scope of Separate Testing

Published by
Financial Executives
Research Foundation


FERF Research Series

April 2007

A Top-Down Approach to Risk Management and Internal Control –
Issue #4: Relying on Ongoing Monitoring to Test Controls Performance, to
Reduce the Scope of Separate Testing

By R. Malcolm Schwartz
Purpose
This four-part report presents a business-centric and cost-effective approach to internal
control and risk management using systems thinking and systems. This approach provides
business benefits and helps enable compliance with the Sarbanes-Oxley Act of 2002, and
other laws and regulations. This document is the fourth of the series, and it explores the

use of monitoring to test the performance of controls. This FERF research series is being
sponsored by BWise B.V.

Executive Summary
It is unrealistic to assume that the costs for risk management and internal control will be
reduced simply by repeating the same process year after year. Experience alone will not
generate all of the possible benefits. An approach that specifically addresses business
benefits while enabling compliance is necessary. The purpose of this four-part series is to
suggest how to do that by considering both the technical and managerial tools.
Selecting technical tools -- software -- is not the first step. First, have your managerial
design in place. Otherwise, you will risk using software that does nothing more than make
a marginal approach more efficient and lose the opportunity to become more effective.
This is what is happening to many companies after their early Sarbanes-Oxley compliance
cycles. To improve effectiveness as well as efficiency:
1. Have a business process focus tied to business planning: Integrate management and
governance with operations and transactions processes to reduce costs of overlap and
maintenance;
2. Use an aggregated risk assessment, to reduce documentation costs;
3. Use a process, and not a financial accounts, point of view to reduce further the costs
of documentation as well as testing costs; and
4. Rely on ongoing monitoring to test the performance of controls and to reduce the
scope of separate testing.
These are the issues examined in this four-part report. This part examines issue #4.
You can reduce costs and become more effective if you start with a focus on the business
processes and:
•
•
•
•

Prioritize -- to reduce the effort to what is necessary and valuable,
Organize -- to use accountability as a key to control and performance,
Integrate -- to avoid overlaps and redundancies, and
Manage performance -- by using monitoring to control and improve performance.

1


These four management issues must be addressed first, and then the right projects and
systems support can follow. Furthermore, if a template of a generic solution to the
management design is the basis of your effort, then your work can focus on tailoring that
generic design solution, and not on the larger effort of creating one from scratch.
In sum, begin with a management design that addresses risk management and internal
control from a business-centric focus. Next, select systems and tools that will support this
approach. Then, follow with audit activities as part of your business plans and operations.
Financial executives are well aware that most business processes and most software
applications treat compliance as a standalone function. This leads to added effort to
develop separate programs and then integrate them. The problem is compounded by the
extra work to maintain the integration and connectivity as one or more programs change.
But a new approach to compliance and internal controls reporting will solve the problem:
assess the relevant activities of the business and then develop a top-down approach to
financial controls reporting.

Issue #4: Relying on Ongoing Monitoring to Test Controls Performance, to
Reduce the Scope of Separate Testing
Too often, companies have created a separate program for testing the design and
performance of internal controls, with little or no reliance on ongoing monitoring performed
by persons who are accountable for processes and their activities and controls. That
reliance on separate evaluations fits with an audit-centric perspective, because separate
evaluations are what auditors do. Managers tend to rely on ongoing monitoring, because

that is what managers do. Using ongoing monitoring as the basis for assessing the
performance of controls is consistent with a management-centric approach. The issue is
not whether or not to rely on ongoing monitoring, because you should be able to do so; it
is how to make ongoing monitoring sufficiently rigorous that it can become the basis of
assessing internal control performance.
Relying on rigorous ongoing monitoring of the performance of control activities does not
eliminate separate evaluations. They still are needed to assess:
•
•

The design of internal control activities -- ongoing monitoring can only be used to
assess the performance of internal control activities and not their design -- and
The conduct and effectiveness of ongoing monitoring.

Nonetheless, using separate evaluations to assess the performance of ongoing
monitoring, distinctive from using separate evaluations to assess the performance of
controls, substantially reduces the scope and cost of separate evaluations; and reinforces
the accountability of your people for results and the quick correction of deviations.

2


Relying on ongoing monitoring also is consistent with The COSO Framework,* each of
whose five components need monitoring.
•
•
•
•
•

Control Environment – for the control culture and framework of your organization
Risk Assessment – for the process of linking business objectives through risk
management to internal controls
Control Activities – for the specific reviews, approvals and other forms of control
activities, and for the activities that they control
Information and Communication – for information content and technology, and for how
information is passed to and from various stakeholders
Monitoring – for applying the sub-components of ongoing monitoring, separate
evaluations and reporting deficiencies to the internal control framework.

Relying on ongoing monitoring depends on and enables:
• Integrating managing and monitoring transaction, management and governance
processes;
• Building accountability for monitoring employees’ sense of responsibilities;
• Measuring what is monitored;
• Addressing problems as they occur, and
• Reducing the scope and cost of separate evaluations as a means of testing control
activities, and integrating separate evaluations with ongoing monitoring.
The design and conduct of an ongoing monitoring program can be made even more
efficient if it is supported:
• with software that integrates controls and risk management with business planning
and
• from a process perspective.

________
*Internal Control - Integrated Framework, Committee of the Sponsoring Organizations of the
Treadway Commission, September 1992, American Institute of Certified Public Accountants
Publications Division. Sarbanes-Oxley requires that a complying company use a framework. The
Securities and Exchange Commission in turn cited The COSO Framework, and the Public
Company Accounting Oversight Board uses it extensively. As a consequence, most complying

companies claim that they are using The COSO Framework, but many have confused it with the
illustrative evaluation tools attached to it. This has led to it being both misunderstood and misused.
Too often, the focus for compliance has been on transaction processes and their controls; and on
separate checklists, or spreadsheets, for management and governance – so-called company-level
and entity-level – controls.

3


Integrating Managing and Monitoring
Addressing control as a management-centric, and not an audit-centric, issue makes
sense. Earlier in this series it was stated that elaborate management design is much less
costly than elaborate execution. An analogy was made to quality control, for which it has
been stated that $1 spent on quality design will save $10 in quality inspection, or $100 in
quality correction. This does not eliminate the need for inspection and correction or
monitoring and correcting deficiencies. Effective design cannot eliminate monitoring,
because you must deal with whether or not the design is effectively performed. But, with
effective design, you can expect that monitoring finds fewer problems and corrects them
quickly. Such a design, as discussed earlier, depends on:
•
•

•
•

A top-down, business-focused risk assessment, which in turn depends on a granular,
bottoms-up business design;
Comprehensive business process design, which can enable reduced documentation;
Segmenting the process steps to their component activities; and then, for example,
relating these activities to specific programs, such as the various financial statement

accounts for Sarbanes-Oxley compliance, or the various selling and supply chain
activities for launching a new product;
Having detailed insights to the information contained in business documents, in order
to understand how to integrate them; and
Having accountable ongoing monitoring in place at the activity level, so that testing can
rely largely on ongoing monitoring.

To illustrate this integration of managing and monitoring across The COSO Framework,
consider staff competency, part of the Control Environment component. In order to certify
the accounts receivable process,* the competencies of the staff involved -- accounts
receivable clerks, and the controller -- need to be assessed, by considering:
•
•
•

The position descriptions -- which should include control and monitoring accountability;
The current appraisals -- to assess performance as compared to what is in the position
descriptions; and
The development plans -- to determine that any gaps in competencies are not only
identified but also corrected.

______
*One process example -- “Maintain accounts receivable reserves” -- is being used throughout this
four-part research series, so that a great deal of specifics about the selected process can be shown
and discussed. “Maintain accounts receivable reserves” was selected because it involves:
(1) Both operations and financial reporting objectives, so it helps to explain the value of
integrating business and compliance planning and management;
(2) Judgments and estimates, so it relates to the area of major risk regarding accurate financial
statements;
(3) Transaction, management and governance processes, so it illustrates how these different

types of processes can be integrated; and
(4) A number of different forms of documentation, so it illustrates how they can be integrated.

4


These are outputs of human resources processes. The design of forms and the
procedures for generating and approving position descriptions, appraisals and
development plans also are outputs of human resources processes. So, the subcomponent, in The COSO Framework, of Control Environment that deals with the matter
of competency includes, integrates and leads to monitoring of:
•
•
•

A transaction process dealing with the valuation of accounts receivable;
A management process that identifies the accountability for the design and operation
of these human resources processes; and
A governance process that oversees and monitors the above processes.

Every one of these processes and their activities should be monitored, particularly if their
effect on objectives -- and for Sarbanes-Oxley, these are financial reporting objectives -can be substantial. So, integrating management and monitoring includes integrating the
monitoring of the transaction, management and governance processes. For the example
being used, this leads from gaining comfort in human resources processes to gaining
comfort in an accounts receivable reserve process that is well-controlled, and then leads
to enabling its certification.
The outputs of the human resources processes in this case provide outputs that become
inputs for a number of transaction processes -- all of which depend on competent staff
performing them -- and in turn depend on the evaluation of staff and the ensuing
development plans. These outputs of managerial processes can be monitored, which
increases their visibility and control. This level of control through monitoring is more

difficult with checklists and spreadsheets, because the lack of integration leads to more
effort and cost, and reliance on them can cause control risk.

Building Accountability for Monitoring Employees’ Sense of Responsibility
By integrating control activities with management activities, you should expect personnel
to monitor the activities for which they are accountable. Testing should begin as ongoing
monitoring performed by the owner of the process or activity. Then, independent testing -separate evaluations -- can be done of this ongoing monitoring, and not of the
performance of the controls as such. This leads to better monitoring, faster responses to
problems, and lower costs for separate, built-on testing (only ongoing monitoring is built-in
testing, and separate evaluations by their very nature are built-on testing).
Ongoing monitoring also enables integrated certification, by the process owner – a
“horizontal” certification based on monitoring the process for which the certification is to be
issued. When these horizontal certifications are aggregated, then the “vertical” certification
can be done at both the business unit and corporate levels by the CEO and CFO.

5


Accountability of this sort also is important to cost-effectiveness; recent research*
indicates that managers and how they monitor and how they are motivated -- principles of
good control -- are more important to company performance than other structural factors.
In other words, mediocre management and control correlate with mediocre monitoring and
corporate results. This research notes that, in studying a set of 18 management practices:
•

•

•

One company used monitoring only when output dipped, to spur action, and then

discontinued the monitoring when output rose; so there was no way to track
performance with business objectives; this is sporadic and not ongoing monitoring
A second company monitored performance indicators continually, but did not share
this information with operating personnel, thus depriving them and the company of
improvement efforts; this is non-communicative ongoing monitoring
A third company used displays to show personnel where their performance ranked
with daily targets and other goals. Managers met with operating personnel every
morning to discuss the previous day’s performance and today’s agenda; provided a
monthly overview and summary; and used lunch breaks to provide feedback on
performance, achievements and improvement opportunities. This is effective ongoing
monitoring

There are several lessons from this research, which had a statistically supportable
correlation in performance among these companies:
•
•
•

•

Good people enable good performance.
Good management techniques provide a setting for good people to perform better.
Control as envisioned in the principles of The COSO Framework -- beginning with
a control environment of, among other components, competent people, welldesigned policies and procedures, effective communications, and reinforcing
human resources policies -- is built into those good management techniques.
Good management techniques rely on monitoring of actual performance compared
to targets to provide a focus for goals, for performance in the context of current
practices and for improving current practices. The result is accountable people
working smarter, not harder.

Working smarter and those good management techniques include a good, integrated
approach to monitoring, with a heavy reliance on ongoing monitoring. As noted earlier and
for various reasons, companies often treat monitoring of controls performance as a
separate program that is neither well-linked to their business objectives nor to
accountability. This audit-centric approach can lead to wasted effort, lack of reinforcement
of accountability for performance and control, and wasted time and cost.

_______
* Conducted in 2005 by McKinsey and the Center for Economic Performance at the London School
of Economics

6


Exhibit 1, shown before in this series and repeated below, illustrates how monitoring is
linked with business planning and improvement. This monitoring – for minor, operational
and control risks -- involves the organization broadly. Monitoring this way enables you to
focus on what is done to produce the results that you plan to have, and to make course
corrections. You get your desired results by controlling what people do and their
commitment to doing it. And, by linking monitoring and accountability, you are able to
continually address results in terms of risks and controls.

Exhibit 1. Management and Monitoring for Internal Control and Risk Management

Assess Risks
• In monetary terms
• Prioritize
• Aggregate
Business
Planning

• Strategy,
structure
and process
• Design, execute
and monitor

Focused
documentation

Focus on Processes
• Organize
• Connect
• Integrate

• Train
• Manage information
• Manage change

Operational
risk
Diagnose
• Cost–benfit analysis
• Business case
Minor risks

•Balanced controls
•Remediation programs

Improvement
opportunities

Improve
• Assess opportunity
• Select approach
• Apply
Operational
risks

Business improvement
program
Control
risks

Manage Performance
• Monitor
• Evaluate, test and oversee

7


Link monitoring and accountability for both internal processes and their outputs – such as
a sale posted to the sales ledger – and for processes that rely on external outputs and
stimuli. In the generic business model in Exhibit 2, many important business risks are
consequences of external parties and their actions. For the accounts receivable reserve,
threats to revenue, and market threats and opportunities -- from changes in revenue
patterns, to economic downturns, to natural disasters -- can influence customers’ ability to
pay.
So, ongoing monitoring can address not only how well activities are performed, but also
what might happen in the future. In this regard, the first activity in the accounts receivable
reserve process is “Review economic trends;” this is an operations activity that involves

monitoring external influences on future performance. From the standpoint of internal
control related to financial reporting, the last step of the process -- “Certify accounts
receivable reserve maintenance” -- can include monitoring that the review of economic
trends was performed timely and well.

Exhibit 2. The Generic Business Model in Context

Generic Business Mo del – Context Level

Oth er Sou rces
of
Co n su mp tio n

Pub lic Bo d ies
&
Oth er Parties

Co llab o rato rs
Co mp lian ce
& Persu asion
Sh ared
Ven tures

Rev en u eOppo rtun ities & Th reats

Can d id ates

Vend ors

Staffing Needs

Sk ills & Ex p erien ce

Fun d s

Human
Res ources
Technology
Development

Pu rch ase
Req u ests
Sh ipp ed Pro du ct

Procuremen t
Inbound Operations

Co mp etito rs

Mark et Th reats
& Oppo rtun ities

Rep o rts

Admin

Run the
Enterprise
Av ailab le Tech no logy
Cap ab ilities
Sp ecificatio n s

Pu rch ase Ord ers
Pu rch ased Good s &
Serv ices

Sh areh o d
l ers
Inv esto rs
&
Fin an cial
In stitu tio ns

Outbound

M arketing
& Sales

Services

Serv ice

Bu y ers
&
Distrib u o
t rs

This generic model of the business in the context of its surroundings puts monitoring
accountability in all business processes. And, because an activity might be part of several
processes, accountability for monitoring should be in each activity.
The question then is: how should you do ongoing monitoring on an activity?

8


Measuring What is Monitored
The quick answer is that an activity should be monitored -- ongoing -- by measuring its
output. In the example of maintaining the accounts receivable reserve, the result, or
output, of the connected set of activities being performed is the update of the accounts
receivable reserve value in the general ledger. But, it is more than that. The output – of the
process overall, and of each of the activities -- has certain measurable values associated
with it. These values, where appropriate, can include: accuracy, completeness,
compliance (with both external laws and regulations, and with internal policies) and
timeliness – so the output of this process is better stated as “accurate, compliant and
timely posting of the reserve value to the general ledger (which, by the way, clearly states
that the risks associated with this process involve inaccuracy due to misfeasance or
malfeasance, non-compliance, and/or lack of timeliness).”
By dealing with these dimensions of risk and control, issues of fraud and mismanagement
can be incorporated and addressed as part of the basic process, and not as separate
processes; this also leads to reduced costs and risks, and to better control. For the
illustrative process and in the generic template, Exhibit 3 shows which measures of risk
and control -- which key control indicators, or KCIs -- apply to which of the activities.
Exhibit 3. Key Control Indicators by Activity in “Maintain Accounts Receivable
Reserves”
Activity
Review economic trends
Maintain and communicate credit policy
Calculate accounts receivable reserves
Approve accounts receivable reserve
calculations
Post accounts receivable reserves to general
ledger

Approve accounts receivable reserves posted
to the general ledger
Certify accounts receivable reserves
calculation process

Accuracy

Completeness
X

Compliance

X

X

X

X

Timeliness

X

X

X

X
X

X

The control of the review of economic trends primarily depends on its being complete. The
control of the calculation of the reserve, and approval of it, depends on the accuracy of the
calculation and on its compliance with policy and procedure; the control on timeliness can
be determined following the next activity in the process, which does not need to be
monitored for compliance if the preceding calculation activity is compliant. And, the
certification monitoring depends on the completeness and compliance, of the activities
themselves and of the associated monitoring.

9


As an aside, and as noted previously, these key control indicators correlate well to the
statements of assertion, as shown in Exhibit 4; so using KCIs for monitoring also enables
addressing the statements of assertion, if appropriate for Sarbanes-Oxley compliance. By
serving two purposes, the use of KCIs provides even more a cost-effective solution.

Exhibit 4. Correlation of Key Control Indicators with Financial Statement Assertions
Correlation of Key Control Indicators with Financial Statement Assertions
Key Control Indicators
Financial Statement Assertions
Accuracy
Completeness
Compliance
Account Assertions
Existence
x
x

Completeness
x
Rights and Obligations
x
x
x
Valuation or allocation
x
x
x
Presentation and Disclosure
x
x
Transaction Assertions
Occurrence
x
Completeness
x
Accuracy
x
Cutoff
Classification
x
x

Timeliness
x
x
x
x

x

10


Once the KCIs in the generic template are tailored to your situation, so that you have
identified what should be measured for each activity, data can be collected on the KCIs as
each activity is completed. This can be done either manually or automatically, depending
on the kind and level of automated support.
Using the accounts receivable reserve process, for example, a manual format such as
shown in Exhibit 5 has been used. For this process being illustrated, the nature of this
monitoring is for events, inasmuch as there are no streams of transaction volumes. This
monitoring nonetheless provides a summary that can be the evidence for both certification
as well as support for any separate evaluation.

Exhibit 5. Recording Key Control Indicators for “Maintain Accounts Receivable
Reserves”
Monitoring Accounts Receivable
Reserves
Control Activity

Date & Time: 7 /7/05
Preparer (name & initial): Susan S

Reviewer (name &
initial): Mal S,
7/9/05

Key Control Indicators

Accuracy
N/A

Completeness

Review trends
change in inflation
steady at 5.5%
changes in credit lines
3% upward growth
customer base
3% new customers
payment trends
from 63 to 58 days
changes in CPI
steady at 5.8%
N/A
N/A
Credit policy
reviewed
N/A
Calculate reserve
reviewed factors
applied policy
applied trend information increased by 8%
N/A
Approve calculations
reviewed policy
reviewed policy
reviewed trend information reviewed trend informati

reviewed calculation
reviewed calculation
N/A
Post to G/L
prepared JE form
prepared 7/8
reviewed data
reviewed and posted
N/A
Approve G/L posting
reviewed JE form
reviewed 7/8
approved G/L result
approved 7/8
N/A
Certify process
review activity monitoring
monitoring complete
review activities
activities complete
outputs complete
review activity outputs

Compliance
N/A

N/A

Timeliness
N/A

met clsoing schedule

Comments
OK for certification

N/A
OK for certification
N/A

applied procedure
applied procedure

OK for certification
N/A

policy applied
trend information used
calculation OK
N/A

OK for certification

met closing schedule

OK for certification

met closing schedule

OK for certification

N/A

N/A
OK with procedure
OK with procedure
OK with procedure

OK for certification

11


A similar approach, but one that includes streams of transaction volumes, is illustrated in
Exhibit 6, for the accounts payable process. Specifically, for batches of items to be
vouchered, the person doing the ongoing monitoring has recorded the size of the batch
(42 items), and then has recorded the KCI performance where applicable -- for example,
the receiving report (RR), the purchase order (PO), and the authorization to pay (ATP)
agreed on 41 of the items as received, or 98% of the batch; and the clerk was able to
resolve the other item, so that all 42, or 100% of the batch, were in agreement as the work
was completed (this example is shown for a manual monitoring activity). This also was
true of agreement with the authorization to pay (ATP). The summary is sent to the process
owner, to be part of the documentation used for certifying the process, as well as to post to
summary dashboards presenting overall and process-level controls.
Note that the clerk is addressing problems as they occur. This is an additional benefit of
ongoing monitoring, as distinctive from separate evaluations.
Exhibit 6. Key Control Indicators for Accounts Payable
Monitoring Accounts Payable

Accuracy

Control Activity

Reviewer (name &
initial): Mal S

Date & Time: 5/7/05, 4:00
Batch Size: 42
Preparer (name & initial): Cindy B
Key Control Indicators

#

%

Compliance
#
%

Completeness
#
%

Timeliness
#
%

41
42

98

100

N/A
N/A

N/A
N/A

19
42

45
100

38
38

90
90

partials, wrong vendor;
resolved. Late
submittals
from Bethesda

41
42

98
100

41
42

98
100

41
42

98
100

37
37

88
88

missing ATP; resolved.
Late submittals from
Bethesda

40
42

95
100

N/A

N/A

N/A
N/A

19
42

45
100

42
42

100
100

partials, balances
noted on POs; wrong
codes, resolved

N/A
N/A

N/A
N/A

40
42

95
100

41
42

98
100

N/A
N/A

N/A
N/A

42
41

100
100

42
42

100
100

42
42

100
100

42
42

100
100

42
42

100
100

N/A
N/A

N/A
N/A

42
42

100
100

42
42

100
100

RR & PO agree
as received
as completed

ATP & PO agree
as received
as completed

Invoice & PO agree
as received
as completed

Authorization OK
as received
as completed

A/P voucher OK
as prepared
as reviewed

Comments

missing, and
wrong, authorizations;
resolved
vendor name
wrong; resolved

A/P & G/L agree
as entered
as reviewed

Addressing Problems as They Occur
This type of ongoing monitoring can be done for current performance, and then compared
to a baseline or a target. The baseline is the experienced level of performance using the
relevant KCIs. Current performance is the most recently recorded baseline value. The
target is the level of performance that reduces the uncertainty – the risk – to the
acceptable level, which shows the activity to be “in control.”

12


Current performance can be at or above target; or below target, in which case the
accountable owner of the activity or process is expected, as stated in the integrated
position description, to take immediate steps to bring performance to an acceptable level.
Taking steps to bring performance to an acceptable level is illustrated in Exhibit 7, below.
This consumer products company experienced substantial deductions upon payment of
invoices by their grocery retailer customers. For effective financial reporting, the company
had to maintain a reserve on receivables to reflect the expected level of deductions; so it
maintained a statistical quality control chart showing the level of deductions and the
change in that level from time to time. Importantly, it began to use this information, and
control, to reduce the level of deductions. By providing continual segment information
underlying the deductions -- product and packaging, shipping location, promotional
activity, and so on -- the process owner was able to identify immediately the root causes of
the deductions and to take steps to correct those causes. There was no need to perform
special studies. Some of the major causes could be traced to salesperson training, for
example, and corrective steps were taken. Unfortunately, as can be seen, a major

reorganization of the sales force had led to higher deductions and greater variability. Early
actions reduced this variability; and later actions addressed the increased level of
deductions resulting from the organization change.

Exhibit 7. Example of Statistical Control Chart for Internal Control

15
UCL (+3σ)

12
Mean

9

LCL (-3σ)

6

3

0

Deductions,
% of
revenue

Reorganization

By month

13


So, the use of ongoing monitoring has two values. First, ongoing monitoring enables a
lower cost of testing controls, by building monitoring in to the activities and to the duties of
those accountable for them. Secondly, ongoing monitoring provides more value, by
enabling rapid response to problems as they occur.
Ongoing monitoring of a well-developed bottoms-up business process also improves your
ability to analyze work flows and methods – and related resources – for operational
improvements and for controls remediation. In other words, ongoing monitoring supports
both getting performance to the targeted level, and continually improving the targets.
Ongoing monitoring also can help you deal with end-to-end process controls and
effectiveness. As noted earlier, many companies have difficulty in managing compliance
efforts because they organize them functionally; and then complain about the inordinate
costs and uncertainties “at the hand-offs.” By establishing owners of activities, and then a
process owner who integrates accountability across the process and monitors the overall
process, the concern about hand-offs, and related cost, can be greatly reduced.
But addressing problems as they occur can use some prioritization, some focus. Some
problems are more worth addressing than others. This affects how broadly ongoing
monitoring should be applied; and this affects the scope and cost of separate evaluations,
for confirming the design of controls and the performance of ongoing monitoring.

14


Reducing the Scope and Cost of Separate Evaluations for Testing Controls
Remember that separate evaluations are made to assess both:
•
•

The design of controls
The performance of ongoing monitoring.

The scope of monitoring -- of ongoing monitoring in the first place and of separate
evaluations as both the follow-on of performance monitoring and the assessment of
controls design -- depends on the degree of uncertainty in the output of the control activity.
For example, in the case of the deductions upon payment of invoices, if the level of
deductions is certain to be 7% of revenue, then there is very little need to monitor this
output from the standpoint of effective financial reporting; and, once the design of the
control has been evaluated, the consistent report from ongoing monitoring should lead to a
very limited program of separate evaluation. There may be good reasons for monitoring
this level for operational objectives, to reduce the level of deductions. Whether it is
salespeople presenting a new program correctly or financial personnel calculating
reserves correctly, the level of monitoring depends on the degree of uncertainty and its
relationship to the objectives.
Take the accounts receivable reserves process once again: each activity has a distinctive
risk profile. In the generic template, these activities and their risks were presented earlier
and are as shown below.
Control
Risk
Importance Exposure
• Review economic trends
M
L
• Maintain and communicate credit policies
L
L
• Calculate accounts receivable reserves
L
M

• Approve accounts receivable reserve
calculations
H
L
• Post accounts receivable reserves to the
general ledger
L
M
• Approve accounts receivable reserves posted
to the general ledger
M
L
• Certify accounts receivable reserves maintenance
process
M
L
If the three control activities -- the two approvals, and the certification -- are designed well,
and if they are performed as designed, then the risk in this sub-process should drop to a
low level. However, this can only be determined by monitoring the performance of the
activities with greater degrees of risk -- the calculation of the accounts receivable reserves,
and the postings to the general ledger. It is this ongoing monitoring that is summarized
and provided to the certification activity. The certification activity, by being performed by a
different role -- in the generic template, this certification is performed by the CFO and
reviewed by the audit committee -- is an ongoing monitoring of the process in question,
and it also is a source of information for a further separate evaluation, if either the chief
compliance officer or the external auditor considers it to be warranted. In either case, the
separate evaluation is of the monitoring and not of the control as such, so the extent and
scope of the separate evaluation is reduced.

15

This approach -- ongoing monitoring by the activity owner, and first-level separate
evaluation by the process owner -- dovetails well with risk management, because, as was
noted earlier in this series:
•

•

•

Many key risks do not link to specific accounts. Management override, an employee’s
careless or callous behavior, intentional cheating, and so forth, are not accountspecific, and yet these are among the major reasons that Sarbanes-Oxley was
enacted, and these are among the major concerns that the Securities and Exchange
Commission and the Public Company Accounting Oversight Board want to see
addressed. So, focusing on activity and not account monitoring makes sense.
Beginning with an account tends to focus on coverage of some portion of the financial
statement, as opposed to the risk in having an accurate portrayal of the financial
statements. So it makes sense to start with the magnitude of the risk, and not with the
magnitude of the account.
Beginning with an account leads to documenting everything that affects that account,
once that account is deemed to be large, to require “coverage,” and/or to be subject to
some rule that deals with risk exposure as a percentage of the financial statement. So
it makes sense to isolate the risky activities, and then to focus on controlling them.

The better way, which can lead to better control at less cost, is based on measuring the
dimensions of control, and monitoring them, to show whether or not there is variability in
the output -- the result -- of the activity being measured; and whether the level of variability
is acceptable. Using this approach, the amount of separate evaluations of the activities of
the accounts receivable reserves process differs by the type and purpose of the activity.

•

•

•
•

“Review economic trends,” and “Maintain and communicate credit policy” are important
from on operations perspective, but have little impact on effective financial statements,
so separate evaluations likely are not needed for Section 404 compliance
“Calculate accounts receivable reserves” and “Post the accounts receivable reserves
to the general ledger” are the activities with uncertainty. Monitoring them enables
better performance, through training, supervision and assessment. Periodic separate
evaluations confirm that the right metrics are monitored and in the right way.
Approving each of these activities is where the bulk of the ongoing monitoring should
occur. This ongoing monitoring becomes the basis the certification step, and in turn of
the separate evaluation – the testing – program, to the extent that it is needed.
The certification activity functions somewhat as a control activity, but more importantly
it is the basis for Section 302 compliance – the certification in regard to effective
control over financial reporting and disclosure – for this particular process

In sum, by using ongoing monitoring of the activities as designed, monitoring can focus on
what is important from the standpoint of control. And, the top-down risk assessment can
lead to a process that is designed and understood, in terms of its activities, their outputs,
and the uncertainties about those outputs. By doing this, the amount of testing in the form
of separate evaluations for SOX compliance can be drastically reduced. And, good
management design is much less costly than the ensuing documentation effort.

16

Software Features for Ongoing Monitoring
Today there are some advertised best-practices for systems to support Sarbanes-Oxley
compliance. For example, see “The 2007 Sarbanes-Oxley RFP Template” at
www.SOXRFP.com .
However, these systems tend to focus on technical and operational features, and either do
not include managerial features or treat them at a very high-level.
Some companies also advertise Web-based, best-practices tools and related material to
help you “escape from spreadsheet hell” in the planning process, to reduce planning cycle
times and to improve planning content. For example, you can download related material at
www.adaptive-planning.com .

17


The recommendations for information technology support that apply to this overall series
of four papers are summarized in Exhibit 8.

Exhibit 8. Software Features
Software Features

Recording processes, activities and controls -- end to
end, hierarchical, connected -- for role and associated
position descriptions and accountabilities, and for inputs,
tools and mechanisms, outputs, and constraints and
controls
Identifying outcomes as an array and not just as a singlepoint estimate; as well as estimating the sizes and the
probabilities, or likelihoods, of the outcomes
Aggregating and cross-connecting processes and
process aspects, and their outcomes and attributes, by

process and eventually by business, into a summary of
expected performance, for purposes of control,
documentation, cost and timing analysis, and process
Aggregating "what-if" outcomes for different
assumptions and conditions
Providing a generic template solution to be tailored
Ranking risk
Aggregating risk
Relating process and activity risks, and the processes
and activities, to financial statements
Identifying risk dimensions
Associating inherent and residual risk by activities
Identifying accountability for control by role
Providing a means to document control procedures
Maintaining and connecting source information -policies, procedures, position descriptions, appraisals,
development plans, training material, forms and formats,
improvement opportunities, and "what-if" depictions
Identifying key control indicators, and capturing related
information
Providing control charts and dashboards, for control
targets, baseline and current performance
Recording and reporting monitoring actions

Focus on
Beginning
Business
With Risk
Planning, Assessment
from a
Process

Perspective

X

X

Using a
Process
Point of
View

Ongoing
Monitoring

X

X

X

X

X

X

X

X

X
X
X

X

X

X

X

X

X
X
X
X

X
X

X

X
X
X

18

Summary
In summary, it would be better to consider, and resolve, the management design issues
first, and more directly, when considering software to support your compliance program.
To summarize from a technical perspective what has been discussed in this research
series, when evaluating software as a tool to support Sarbanes-Oxley compliance,
consider the management design as well as the technical and operational design features.
And, in regard to the issue of ongoing monitoring, consider the capability of the software to
support ongoing monitoring, and from a business process/activity perspective, for:
•
•
•
•
•

Documenting accountability and relating it to position descriptions,
Identifying KCIs, and capturing the information for recording and calculating them,
Reporting baseline, target and current performance, in control charts and dashboards,
Recording and reporting monitoring actions and
Notifying auditors of the state of ongoing monitoring.

If the software you are considering or using does not have some of these capabilities, then
at least look for ease of uploading to, and downloading from, that software to a software
that does have the missing features.
Whatever the software selected, make sure that it includes a developed, generic,
connected and integrated model of activities (both operating and control activities) and
their risks, so that your focus can be on tailoring that model and not having to create it.

19

About the Author
Malcolm Schwartz is one of the principal contributors to The COSO Report (“Internal
Control - Integrated Framework”), and has been on the recent COSO task force providing
simplified guidelines for Sarbanes-Oxley compliance. He currently is COO of CRS
Associates LLC. He recently retired from PwC, where he was a senior management
consulting partner. Prior to that, he had been a senior vice-president and CFO of Booz,
Allen & Hamilton; and had held general, financial and operations management and staff
positions at Insilco, Westinghouse Broadcasting, and Procter & Gamble. Malcolm can be
reached at or 908-273-6967.

About the Sponsor, BWise B.V.

BWise is an enterprise risk management (ERM), corporate compliance, and internal
control software provider. BWise delivers solutions to help organizations become “in
control” by increasing corporate accountability; strengthening financial, strategic and
operational efficiencies; and maximizing performance and ROI. More than 1,000
companies with more than 125,000 users rely on BWise solutions, including VNU, TNT,
Connexxion and Crucell. For more information, please, go to:
www.bwise.com

About FERF
Financial Executives Research Foundation (FERF) is the non-profit 501(c)(3) research
affiliate of Financial Executives International (FEI). FERF researchers identify key financial
issues and develop impartial, timely research reports to FEI members and non-members
alike, in a variety of publication formats. FERF relies primarily on voluntary tax-deductible
contributions from corporations and individuals.
For more information, visit or .
The views set forth in this publication do not necessarily reflect those of the Financial
Executives Research Foundation Board as a whole, individual trustees, employees or the

members of the Research Advisory Council. Financial executives Research Foundation
shall be held harmless against any claims, demands, injuries, costs or expenses of any
kind or nature whatsoever except such liabilities as may result from misconduct or
improper performance by the Foundation or any of its representatives.
This and more than 80 other Research Foundation publications can be ordered by logging
onto .

Financial Executives Research Foundation, Inc., would like to thank and
acknowledge BWise B.V. for their generosity and support in underwriting this
report.

20


Copyright © 2007 by Financial Executives Research Foundation, Inc.
All rights reserved. No part of this publication may be reproduced in any form or by any
means without written permission from the publisher and the author.
International Standard Book Number 1-933130-48-2
Printed in the United States of America
First Printing.
Authorization to photocopy items for internal or personal use, or the internal or personal
use of specific clients, is granted by Financial Executives Research Foundation, Inc.,
provided that an appropriate fee is paid to Copyright Clearance Center, 222 Rosewood
Drive, Danvers MA 01923. Fee inquiries can be directed to Copyright Clearance Center at
978-750-8400. For further information please check Copyright Clearance Center online at:
.

21