The oxford handbook of computational and mathematical psychology (oxford library of psychology) 1st edition {PRG} 2015

OXFORD LIBRARY OF PSYCHOLOGY
EDITED BY

JEROME R.

BUSEMEYER
ZHENG

WANG
JAMES T.

TOWNSEND &
AMI

EIDELS

The Oxford Handbook of
COMPUTATIONAL and
MATHEMATICAL
PSYCHOLOGY


The Oxford Handbook of
Computational and
Mathematical Psychology


OX F O R D L I B R A RY O F P S Y C H O L O G Y
E D I T O R-I N-C H I E F

Peter E. Nathan

AREA EDITORS

Clinical Psychology
David H. Barlow

Cognitive Neuroscience
Kevin N. Ochsner and Stephen M. Kosslyn

Cognitive Psychology
Daniel Reisberg

Counseling Psychology
Elizabeth M. Altmaier and Jo-Ida C. Hansen

Developmental Psychology
Philip David Zelazo

Health Psychology
Howard S. Friedman

History of Psychology
David B. Baker

Methods and Measurement
Todd D. Little

Neuropsychology
Kenneth M. Adams

Organizational Psychology

Steve W. J. Kozlowski

Personality and Social Psychology
Kay Deaux and Mark Snyder


OXFORD LIBRARY OF PSYCHOLOGY

Editor-in-Chief

peter e. nathan

The Oxford Handbook
of Computational
and Mathematical
Psychology
Edited by

Jerome R. Busemeyer
Zheng Wang
James T. Townsend
Ami Eidels

1


3
Oxford University Press is a department of the University of
Oxford. It furthers the University’s objective of excellence in research,
scholarship, and education by publishing worldwide.

Oxford New York
Auckland Cape Town Dar es Salaam Hong Kong Karachi
Kuala Lumpur Madrid Melbourne Mexico City Nairobi
New Delhi Shanghai Taipei Toronto
With offices in
Argentina Austria Brazil Chile Czech Republic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
South Korea Switzerland Thailand Turkey Ukraine Vietnam
Oxford is a registered trademark of Oxford University Press
in the UK and certain other countries.
Published in the United States of America by
Oxford University Press
198 Madison Avenue, New York, NY 10016
c Oxford University Press 2015
All rights reserved. No part of this publication may be reproduced, stored in
a retrieval system, or transmitted, in any form or by any means, without the prior
permission in writing of Oxford University Press, or as expressly permitted by law,
by license, or under terms agreed with the appropriate reproduction rights organization.
Inquiries concerning reproduction outside the scope of the above should be sent to the
Rights Department, Oxford University Press, at the address above.
You must not circulate this work in any other form
and you must impose this same condition on any acquirer.
Library of Congress Cataloging-in-Publication Data
Oxford handbook of computational and mathematical psychology / edited by Jerome R. Busemeyer,
Zheng Wang, James T. Townsend, and Ami Eidels.
pages cm. – (Oxford library of psychology)
Includes bibliographical references and index.
ISBN 978-0-19-995799-6
1. Cognition. 2. Cognitive science. 3. Psychology–Mathematical models.
4. Psychometrics. I. Busemeyer, Jerome R.

BF311.O945 2015
150.1 51–dc23
2015002254

9 8 7 6 5 4 3 2 1
Printed in the United States of America
on acid-free paper


Dedicated to the memory of
Dr. William K. Estes (1919–2011) and Dr. R. Duncan Luce (1925–2012)
Two of the founders of modern mathematical psychology

v



SHORT CONTENTS

Oxford Library of Psychology ix
About the Editors xi
Contributors

xiii

Table of Contents
Chapters
Index

xvii

1–390

391

vii



OX F O R D L I B R A R Y O F P SYC H O LO GY

The Oxford Library of Psychology, a landmark series of handbooks, is published
by Oxford University Press, one of the world’s oldest and most highly respected
publishers, with a tradition of publishing significant books in psychology. The
ambitious goal of the Oxford Library of Psychology is nothing less than to span
a vibrant, wide-ranging field and, in so doing, to fill a clear market need.
Encompassing a comprehensive set of handbooks, organized hierarchically,
the Library incorporates volumes at different levels, each designed to meet a
distinct need. At one level are a set of handbooks designed broadly to survey the
major subfields of psychology; at another are numerous handbooks that cover
important current focal research and scholarly areas of psychology in depth and
detail. Planned as a reflection of the dynamism of psychology, the Library will
grow and expand as psychology itself develops, thereby highlighting significant
new research that will impact on the field. Adding to its accessibility and ease
of use, the Library will be published in print and, later on, electronically.
The Library surveys psychology’s principal subfields with a set of handbooks
that capture the current status and future prospects of those major subdisciplines. The initial set includes handbooks of social and personality psychology,
clinical psychology, counseling psychology, school psychology, educational
psychology, industrial and organizational psychology, cognitive psychology,
cognitive neuroscience, methods and measurements, history, neuropsychology,

personality assessment, developmental psychology, and more. Each handbook
undertakes to review one of psychology’s major subdisciplines with breadth,
comprehensiveness, and exemplary scholarship. In addition to these broadlyconceived volumes, the Library also includes a large number of handbooks
designed to explore in depth more specialized areas of scholarship and research,
such as stress, health and coping, anxiety and related disorders, cognitive
development, or child and adolescent assessment. In contrast to the broad
coverage of the subfield handbooks, each of these latter volumes focuses on
an especially productive, more highly focused line of scholarship and research.
Whether at the broadest or most specific level, however, all of the Library
handbooks offer synthetic coverage that reviews and evaluates the relevant past
and present research and anticipates research in the future. Each handbook in
the Library includes introductory and concluding chapters written by its editor
to provide a roadmap to the handbook’s table of contents and to offer informed
anticipations of significant future developments in that field.

ix


An undertaking of this scope calls for handbook editors and chapter authors
who are established scholars in the areas about which they write. Many of
the nation’s and world’s most productive and best-respected psychologists have
agreed to edit Library handbooks or write authoritative chapters in their areas
of expertise.
For whom has the Oxford Library of Psychology been written? Because of
its breadth, depth, and accessibility, the Library serves a diverse audience,
including graduate students in psychology and their faculty mentors, scholars,
researchers, and practitioners in psychology and related fields. Each will find
in the Library the information they seek on the subfield or focal area of
psychology in which they work or are interested.
Befitting its commitment to accessibility, each handbook includes a

comprehensive index, as well as extensive references to help guide research.
And because the Library was designed from its inception as an online as well
as print resource, its structure and contents will be readily and rationally
searchable online. Further, once the Library is released online, the handbooks
will be regularly and thoroughly updated.
In summary, the Oxford Library of Psychology will grow organically to
provide a thoroughly informed perspective on the field of psychology, one
that reflects both psychology’s dynamism and its increasing interdisciplinarity.
Once published electronically, the Library is also destined to become a
uniquely valuable interactive tool, with extended search and browsing
capabilities, As you begin to consult this handbook, we sincerely hope you
will share our enthusiasm for the more than 500-year tradition of Oxford
University Press for excellence, innovation, and quality, as exemplified by the
Oxford Library of Psychology.
Peter E. Nathan
Editor-in-Chief
Oxford Library of Psychology

x

oxford library of psychology


ABOUT THE EDITORS

Jerome R. Busemeyer is Provost Professor of Psychology at Indiana University. He

was the president of Society for Mathematical Psychology and editor of the Journal
of Mathematical Psychology. His theoretical contributions include decision field
theory and, more recently, pioneering the new field of quantum cognition.

Zheng Wang is Associate Professor at the Ohio State University and directs
the Communication and Psychophysiology Lab. Much of her research tries
to understand how our cognition, decision making, and communication are
contextualized.
James T. Townsend is Distinguished Rudy Professor of Psychology at Indiana

University. He was the president of Society for Mathematical Psychology and editor
of the Journal of Mathematical Psychology. His theoretical contributions include
systems factorial technology and general recognition theory.
Ami Eidels is Senior Lecturer at the School of Psychology, University of
Newcastle, Australia, and a principle investigator in the Newcastle Cognition
Lab. His research focuses on human cognition, especially visual perception and
attention, combined with computational and mathematical modeling.

xi



CONTRIBUTORS

Daniel Algom
School of Psychological Sciences
Tel-Aviv University
Israel
F. Gregory Ashby
Department of Psychological and Brain Sciences
University of California, Santa Barbara
Santa Barbara, CA
Joseph L. Austerweil
Department of Cognitive

Linguistic, and Psychological Sciences
Brown University
Providence, RI
Scott D. Brown
School of Psychology
University of Newcastle
Callaghan, NSW
Australia
Jerome R. Busemeyer
Department of Psychological
and Brain Sciences
Cognitive Science Program
Indiana University
Bloomington, IN
Amy H. Criss
Department of Psychology
Syracuse University
Syracuse, NY
Simon Dennis
Department of Psychology
The Ohio State University
Columbus, OH
Adele Diederich
Psychology
Jacobs University Bremen gGmbH
Bremen 28759
Germany
Chris Donkin
School of Psychology
University of New South Wales

Kensington, NSW
Australia

Ami Eidels
School of Psychology
University of Newcastle
Callaghan, NSW
Australia
Samuel J. Gershman
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
Cambridge, MA
Thomas L. Griffiths
Department of Psychology
University of California, Berkeley
Berkeley, CA
Todd M. Gureckis
Department of Psychology
New York University
New York, NY
Robert X. D. Hawkins
Department of Psychological
and Brain Sciences
Indiana University
Bloomington, IN
Andrew Heathcote
School of Psychology
University of Newcastle
Callaghan, NSW
Australia

Marc W. Howard
Department of Psychological
and Brain Sciences
Center for Memory and Brain
Boston University
Boston, MA
Brett Jefferson
Department of Psychological
and Brain Sciences
Indiana University
Bloomington, IN
Michael N. Jones
Department of Psychological and Brain Sciences
Indiana University
Bloomington, IN
xiii


John K. Kruschke
Department of Psychological
and Brain Sciences
Indiana University
Bloomington, IN
Yunfeng Li
Department of Psychological
Sciences
Purdue University
West Lafayette, IN
Gordon D. Logan
Department of Psychology

Vanderbilt Vision Research Center
Center for Integrative and
Cognitive Neuroscience
Vanderbilt University
Nashville, TN
Bradley C. Love
University College London
Experimental Psychology
London, UK
Dora Matzke
Department of Psychology
University of Amsterdam
Amsterdam, the Netherlands
Robert M. Nosofsky
Department of Psychological
and Brain Sciences
Indiana University
Bloomington, IN
Richard W. J. Neufeld
Departments of Psychology and Psychiatry,
Neuroscience Program
University of Western Ontario
London, Ontario
Canada
Thomas J. Palmeri
Department of Psychology
Vanderbilt Vision Research Center
Center for Integrative and
Cognitive Neuroscience
Vanderbilt University

Nashville, TN
Zygmunt Pizlo
Department of Psychological
Sciences
Purdue University
West Lafayette, IN
Timothy J. Pleskac
Center for Adaptive Rationality (ARC)
Max Planck Institute for Human
Development
Berlin, Germany
xiv

contributors

Emmanuel Pothos
Department of Psychology
City University London
London, UK
Babette Rae
School of Psychology
University of Newcastle
Callaghan, NSW
Australia
Roger Ratcliff
Department of Psychology
The Ohio State University
Columbus, OH
Tadamasa Sawada
Department of Psychology

Higher School of Economics
Moscow, Russia
Jeffrey D. Schall
Department of Psychology
Vanderbilt Vision Research Center
Center for Integrative and
Cognitive Neuroscience
Vanderbilt University
Nashville, TN
Philip Smith
School of Psychological Sciences
The University of Melbourne
Parkville, VIC
Australia
Fabian A. Soto
Department of Psychological
and Brain Sciences
University of California, Santa Barbara
Santa Barbara, CA
Joshua B. Tenenbaum
Department of Brain and Cognitive Sciences
Massachusetts Institute of Technology
Cambridge, MA
James T. Townsend
Department of Psychological
and Brain Sciences
Cognitive Science Program
Indiana University
Bloomington, IN
Joachim Vandekerckhove

Department of Cognitive Sciences
University of California, Irvine
Irvine, CA
Wolf Vanpaemel
Faculty of Psychology
and Educational Sciences
University of Leuven
Leuven, Belgium


Eric-Jan Wagenmakers
Department of Psychology
University of Amsterdam
Amsterdam, the Netherlands
Thomas S. Wallsten
Department of Psychology
University of Maryland
College Park, MD

Zheng Wang
School of Communication
Center for Cognitive and Brain Sciences
The Ohio State University
Columbus, OH
Jon Willits
Department of Psychological and Brain Sciences
Indiana University
Bloomington, IN

contributors

xv



CONTENTS

Preface

xix

1. Review of Basic Mathematical Concepts Used in Computational and
Mathematical Psychology 1
Jerome R. Busemeyer, Zheng Wang, Ami Eidels, and James T. Townsend

Part I

•

Elementary Cognitive Mechanisms

2. Multidimensional Signal Detection Theory 13
F. Gregory Ashby and Fabian A. Soto
3. Modeling Simple Decisions and Applications Using a Diffusion
Model 35
Roger Ratcliff and Philip Smith
4. Features of Response Times: Identification of Cognitive Mechanisms
through Mathematical Modeling 63
Daniel Algom, Ami Eidels, Robert X. D. Hawkins, Brett Jefferson, and
James T. Townsend

5. Computational Reinforcement Learning 99
Todd M. Gureckis and Bradley C. Love

Part II

•

Basic Cognitive Skills

6. Why Is Accurately Labeling Simple Magnitudes So Hard? A Past,
Present, and Future Look at Simple Perceptual Judgment 121
Chris Donkin, Babette Rae, Andrew Heathcote, and Scott D. Brown
7. An Exemplar-Based Random-Walk Model of Categorization and
Recognition 142
Robert M. Nosofsky and Thomas J. Palmeri
8. Models of Episodic Memory 165
Amy H. Criss and Marc W. Howard

Part III

•

Higher Level Cognition

9. Structure and Flexibility in Bayesian Models of Cognition 187
Joseph L. Austerweil, Samuel J. Gershman, Joshua B. Tenenbaum, and
Thomas L. Griffiths
xvii

10. Models of Decision Making under Risk and Uncertainty 209
Timothy J. Pleskac, Adele Diederich, and Thomas S. Wallsten
11. Models of Semantic Memory 232
Michael N. Jones, Jon Willits, and Simon Dennis
12. Shape Perception 255
Tadamasa Sawada, Yunfeng Li, and Zygmunt Pizlo

Part IV

•

New Directions

13. Bayesian Estimation in Hierarchical Models 279
John K. Kruschke and Wolf Vanpaemel
14. Model Comparison and the Principle of Parsimony 300
Joachim Vandekerckhove, Dora Matzke, and Eric-Jan Wagenmakers
15. Neurocognitive Modeling of Perceptual Decision Making 320
Thomas J. Palmeri, Jeffrey D. Schall, and Gordon D. Logan
16. Mathematical and Computational Modeling in
Clinical Psychology 341
Richard W. J. Neufeld
17. Quantum Models of Cognition and Decision 369
Jerome R. Busemeyer, Zheng Wang, and Emmanuel Pothos
Index

xviii

contents

391


P R E FA C E

Computational and mathematical psychology has enjoyed rapid growth over
the past decade. Our vision for the Oxford Handbook of Computational and
Mathematical Psychology is to invite and organize a set of chapters that review
these most important developments, especially those that have impacted—
and will continue to impact—other fields such as cognitive psychology,
developmental psychology, clinical psychology, and neuroscience. Together
with a group of dedicated authors, who are leading scientists in their areas,
we believe we have realized our vision. Specifically, the chapters cover the
key developments in elementary cognitive mechanisms (e.g., signal detection,
information processing, reinforcement learning), basic cognitive skills (e.g.,
perceptual judgment, categorization, episodic memory), higher-level cognition
(e.g., Bayesian cognition, decision making, semantic memory, shape perception),
modeling tools (e.g., Bayesian estimation and other new model comparison
methods), and emerging new directions (e.g., neurocognitive modeling,
applications to clinical psychology, quantum cognition) in computation and
mathematical psychology.
An important feature of this handbook is that it aims to engage readers
with various levels of modeling experience. Each chapter is self-contained and
written by authoritative figures in the topic area. Each chapter is designed to be
a relatively applied introduction with a great emphasis on empirical examples
(see New Handbook of Mathematical Psychology (2014) by Batchelder, Colonius,
Dzhafarov, and Myung for a more mathematically foundational and less applied
presentation). Each chapter endeavors to immediately involve readers, inspire
them to apply the introduced models to their own research interests, and refer
them to more rigorous mathematical treatments when needed. First, each chapter

provides an elementary overview of the basic concepts, techniques, and models
in the topic area. Some chapters also offer a historical perspective of their area
or approach. Second, each chapter emphasizes empirical applications of the
models. Each chapter shows how the models are being used to understand human
cognition and illustrates the use of the models in a tutorial manner. Third, each
chapter strives to create engaging, precise, and lucid writing that inspires the use
of the models.
The chapters were written for a typical graduate student in virtually any area of
psychology, cognitive science, and related social and behavioral sciences, such as
consumer behavior and communication. We also expect it to be useful for readers
ranging from advanced undergraduate students to experienced faculty members
and researchers. Beyond being a handy reference book, it should be beneficial as

xix


a textbook for self-teaching, and for graduate level (or advanced undergraduate
level) courses in computational and mathematical psychology.
We would like to thank all the authors for their excellent contributions. Also
we thank the following scholars who helped review the book chapters in addition
to the editors (listed alphabetically): Woo-Young Ahn, Greg Ashby, Scott Brown,
Cody Cooper, Amy Criss, Adele Diederich, Chris Donkin, Yehiam Eldad, Pegah
Fakhari, Birte Forstmann, Tom Griffiths, Andrew Heathcote, Alex Hedstrom,
Joseph Houpt, Marc Howard, Matt Irwin, Mike Jones, John Kruschke, Peter Kvam,
Bradley Love, Dora Matzke, Jay Myung, Robert Nosofsky, Tim Pleskac, Emmanuel
Pothos, Noah Silbert, Tyler Solloway, Fabian Soto, Jennifer Trueblood, Joachim
Vandekerckhove, Wolf Vanpaemel, Eric-Jan Wagenmakers, and Paul Williams.
The authors and reviewers’ effort ensure our confidence in the high quality of this
handbook.
Finally, we would like to express how much we appreciate the outstanding

assistance and guidance provided by our editorial team and production team at
Oxford University Press. The hard work provided by Joan Bossert, Louis Gulino,
Anne Dellinger, A. Joseph Lurdu Antoine and the production team of Newgen
Knowledge Works Pvt. Ltd., and others at the Oxford University Press are essential
for the development of this handbook. It has been a true pleasure working with this
team!
Jerome R. Busemeyer
Zheng Wang
James T. Townsend
Ami Eidels
December 16, 2014

xx

preface


CHAPTER

1

Review of Basic Mathematical Concepts
Used in Computational and
Mathematical Psychology

Jerome R. Busemeyer, Zheng Wang, Ami Eidels, and James T. Townsend

Abstract

Computational and mathematical models of psychology all use some common mathematical

functions and principles. This chapter provides a brief overview.
Key Words: mathematical functions, derivatives and integrals, probability theory,

expectations, maximum likelihood estimation

We have three ways to build theories to explain
and predict how variables interact and relate to each
other in psychological phenomena: using natural
verbal languages, using formal mathematics, and
using computational methods. Human intuitive
and verbal reasoning has a lot of limitations. For
example, Hintzman (1991) summarized at least
10 critical limitations, including our incapability
to imagine how a dynamic system works. Formal
models, including both mathematical and computational models, can address these limitations of
human reasoning.
Mathematics is a “radically empirical” science
(Suppes, 1984, p.78), with consistent and rigorous
evidence (the proof ) that is “presented with a
completeness not characteristic of any other area
of science” (p.78). Mathematical models can help
avoid logic and reasoning errors that are typically
encountered in human verbal reasoning. The
complexity of theorizing and data often requires
the aid of computers and computational languages.
Computational models and mathematical models
can be thought of as a continuum of a theorizing
process. Every computational model is based on
a certain mathematical model, and almost every
mathematical model can be implemented as a

computational model.

Psychological theories may start as a verbal
description, which then can be formalized using
mathematical language and subsequently coded
into computational language. By testing the models
using empirical data, the model fitting outcomes
can provide feedback to improve the models, as well
as our initial understanding and verbal descriptions.
For readers who are newcomers to this exciting
field, this chapter provides a review of basic
concepts of mathematics, probability, and statistics
used in computational and mathematical modeling
of psychological representation, mechanisms, and
processes. See Busemeyer and Diederich (2010) and
Lewandowsky and Ferrel (2010) for a more detailed
presentations.

Mathematical Functions
Mathematical functions are used to map a set of
points called the domain of the function into a set
of points called the range of the function such that
only one point in range is assigned to each point
in the domain.1 As a simple example, the linear
function is defined as f (x) = a·x where the constant
a is the slope of a straight line. In general, we use the
notation f (x) to represent a function f that maps
a domain point x into a range point y = f (x). If a

1

function f (x) has the property that each range point
y can only be reached by a single unique domain
point x, then we can define the inverse function
f −1 (y) = x that maps each range point y = f (x)
back to the corresponding domain point x. For
example, the quadratic function is defined as the
map f (x) = x 2 = x · x, and if we pick the number
x = 3.5, then f (3.5) = 3.52 = 12.25. The quadratic
function is defined on a domain of both positive
and negative real numbers, and it does not have an
inverse because, for example, ( − x)2 = x 2 and so
there are two ways to get back from each range point
y to the domain. However, if we restrict the domain
to the non-negative real numbers, then the inverse
defined
of x 2 exists and it is the square root function
√ √
on non-negative real numbers y = x 2 = x. There
are, of course, a large number of functions used
in mathematical psychology, but some of the most
popular ones include the following.
The power function is denoted x a where the
variable x is a positive real number and the constant
a is called the power. A quadratic function can be
obtained by setting a = 2 but we could instead
choose a = 0.50, which is the square root function
√
x .50 = x, or we could choose a = −1, which

produces the reciprocal x −1 = 1x , or we could
choose any real number such as a = 1.37. Using
a calculator, one finds that if x = 15.25 and a =
1.37, then 15.251.37 = 41.8658. One important
property to remember about power functions is that
x a · x b = x a+b and x b · y b = (x · y)b and (x a )b = x ab .
Also note that x 0 = 1. Note that when working
with the power function, the variable x appears
in the base, and the constant a appears as the
power.
The exponential function is denoted e x where
the exponent x is any real valued variable and
the constant base e stands for a special number
that is approximately e ∼
= 2.7183. Sometimes it is
more convenient to use the notation e x = exp (x)
instead. Using a calculator, we can calculate e 2.5 =
2.71832.5 = 12.1825. Note that the exponent can
be negative −x < 0, in which case we can write
e −x = e1x . If x = 0, then e0 = 1. The exponential
function always returns a positive value, ex > 0, and
it approaches zero as x approaches negative infinity.
More complex forms of the exponential are often
used. For example, you will later see the function
−

x−μ

2

σ
, where x is a variable and μ and σ are
e
constants. In this case, it is more convenient to

write this as e

2

−

x−μ
σ

2

= exp −

x−μ 2
σ

. This tells

you to first compute the squared deviation y =
x−μ 2
and then compute the reciprocal exp1 y .
σ
()
The exponential function obeys the property e x ·
ey = ex+y and (e x )a = e a·x . In contrast to the power

function, the base of the exponential is a constant
and the exponent is a variable.
The (natural) log function is denoted ln (x) for
positive values of x. For example, using a calculator,
for x = 10, we obtain ln (10) = 2.3026. (We
normally use the natural base e = 2.7183. If instead
we used base 10, then log10 (10) = 1.) The log
function obeys the rules ln (x ·y) = ln (x)+ln (y) and
ln (x a ) = a · ln (x). The log function is the inverse
of the exponential function: ln ( exp (x)) = x and
the exponential function is the inverse of the log
function exp ( log (x)) = x. The function ax where a
is a constant and x is a variable can be rewritten in
terms of the exponential function: define b = ln (a),
then e bx = (eb )x = exp ( ln (a))x = ax .
Figure 1.1 illustrates the power, exponential, and
log functions using different coefficient values for
the function. As can be seen, the coefficient changes
the curve of the functions.
Last but not least are the trigonometric functions
based on a circle. Figure 1.2 shows a circle with
its center located at coordinates (0, 0) in an (X , Y )
plane. Now imagine a line segment of radius
r = 1 that extends from the center point to the
circumference of the circle. This line segment
intersects with the circumference at coordinates
( cos (t · π), sin (t · π) ) in the plane. The coordinate
cos (t · π) represents the projection of the point on
the circumference down onto the the X axis, and
the point sin (t · π) is the projection of the point

on the circumference to the Y axis. The variable
t (which, for example can be time) moves this
point around the circle, with positive values moving
the point counter clockwise, and negative values
moving it clockwise. The constant π = 3.1416
equals one-half cycle around the circle, and 2π is
the period of time it takes to go all the way around
once. The two functions are related by a translation
(called the phase) in time: cos (t · π + (π/2)) =
sin (t · π). Note that cos is an even function because
cos (t · π ) = cos (−t · π), whereas sin is an odd
function because − sin (t · π ) = sin ( − t · π ) . Also
note that these functions are periodic in the sense
that for example cos (t · π ) = cos (t · π + 2 · k · π )
for any integer k. We can generalize these two
functions by changing the frequency and the phase.
For example, cos (ω · tπ + θ ) is a cosine function
with a frequency ω (changing the time it takes

review of basic mathematical concepts


Power function: y = xa

5
4

Exponential function: y = exp(x)

5

a=2

3

y

y

y

a = 1/2

y = log(x)

3

3

2

y = log(2x)

4

y = exp(x)

4

Log function: y = log(x)

5

y = log(0.5x)
2

2
y = exp(0)

1
0

a = –1
0

1

2

3

1

1
y = exp(–x)
4

5

0

0

1

2

3

4

5

0

0

10

x

x

20

30

40

50

x

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

Sin(Time)

Sin(x)

Fig. 1.1 Examples of three important functions, with various parameter values. From left to right: Power function, exponential
function, and log function. See text for details.

0
–0.2

0
–0.2

–0.4

–0.4

–0.6

–0.6

–0.8

–0.8

–1
–1 –0.8 –0.6 –0.4 –0.2

0

0.2 0.4 0.6 0.8

–1

1

Cos(x)

0

0.2 0.4 0.6 0.8

0

1.2 1.4 1.6 1.8

2

Time

Fig. 1.2 Left panel illustrates a point on a unit circle with a radius equal to one. Vertical line shows sine, horizontal line shows cosine.
Right panel shows sine as a function of time. The point on the Y axis of the right panel corresponds to the point on the Y axis of the
left panel.

to complete a cycle) and a phase θ (advancing or
delaying the initial value at time t = 0).

Derivatives and Integrals
A derivative of a continuous function is the rate
of change or the slope of a function at some point.
Suppose f (x) is some continuous function. For a
small increment , the change in this function is
df (x) = f (x) − f (x − ) and the rate of change
df (x)
=
is the change divided by the increment
f (x)−f (x− )

. If the function is continuous, then
as
→ 0, this ratio converges to what is called
the derivative of the function at x, denoted as

d
dx f

(x). The derivatives of many functions are
derived in calculus (see Stewart (2012) or any
calculus textbook for an introduction to calculus).
d c·x
For example, in calculus it is shown that dx
e =
c·x
c · e , which says that the slope of the exponential
function at any point x is proportional to the
exponential function itself. As another example, it
d a
x = a · x a−1 , which is
is shown in calculus that dx
the derivative of the power function. For example,
the derivative of a quadratic function a·x 2 is a linear
function 2 · a · x, and the derivative of the linear
function a · x is the constant a. The derivative of
the cosine function is dtd cos (t) = − sin (t) and the
derivative of sine is dtd sin (t) = cos (t).

review of basic mathematical concepts

3


Power function: y = xa; a = 2

4

4

3

3

Power function: y = x a; a = 0.5

y

5

y

5

2

2

1

1

0

0

1

2

3

4

5

0

0

1

x

2

3

4

5

x

Fig. 1.3 Illustration of the power function and its derivatives. The lines in both panels mark the power-function line. The slope of the
dotted line (the tangent to the function) is given by the derivative of that function. (in this example, at x = 1).

Figure 1.3 illustrates the derivative of the power
function at the value x = 1 for two different
coefficients. The curved line shows the power
function, and the straight line touches the curve at
x = 1. The slope of this line is the derivative.
The integral of a continuous function is the area
under the curve within some interval (see Fig. 1.4).
Suppose f (x) is a continuous function of x within
the interval [a, b]. A simple way to approximate this
area is to divide the interval into N very small steps,
with a small increment being a step:
[x0 = a, x1 = a + , x2 = a + 2 , ...,
xj = a + j · , ..., xN −1 = b − , xN = b]
Then, compute the area of the rectangle within each
step, · f xj , and finally sum all the areas of the
rectangles to obtain an approximate area under the
curve:
N

A≈

· f (x1 ) +

· f (x2 ) + · · · +

· f xN =

f xj · .
j=1

As the number of intervals becomes arbitrarily large
and the increments get arbitrarily small so that
N → ∞ and → 0, this sum converges to the
integral
A=

b
a

f (x) · dx.

If we allow the upper limit of the integral to be a
variable, say z, then the integral becomes a function
of the upper limit, which can be written as F (z) =
z
a f (x) · dx . What happens if we take the derivative
of an integral? Let’s examine the change in the area
divided by the increment
4

A (xN ) − A (xN −1 ) = f (xN ),
A (xN ) − A (xN −1 )

= f (xN ) .

This simple idea (proven more rigorously in a
calculus textbook) leads to the first fundamental
d
theorem of integrals which states that dz
F (z) =
z
f (z) , with F (z) = a f (x)dx. The fundamental
theorem can then be used to find the integral of
z
a function. For example, the integral of 0 x a dx =
d
(a + 1)−1 z a+1 = z a . The
(a + 1)−1 z a+1 because dz
z α·x
1 α·z
e dx = α e because dtd eα·z = α ·
integral of
z
α·z
cos (t)dt = sin (z) because
e . The integral of
d
dz sin (z) = cos (z).
Computational and mathematical models are often described by difference or differential equations.
These types of equations are used to describe how
the state of a system changes with time. For example, suppose V (t) represents the strength of a neural
connection between an input and an output at time
t, and suppose x (t) is some reward signal that is

guiding the learning process. A simple, discrete time
linear model of learning can be V (t) = (1 − α) ·
V (t − 1) + α · x (t), where 0 ≤ α ≤ 1 is the learning
rate parameter. We can rewrite this as a difference
equation:
dV (t) = V (t) − V (t − 1)
= − α · V (t − 1) + α · x (t)
= − α · (V (t − 1) − x (t)) .
This model states that the change in strength at time
t is proportional to the negative of the error signal,
which is defined as the difference between the

review of basic mathematical concepts