Ebook Programming interactivity Part 1

Programming Interactivity
A Designer’s Guide to Processing, Arduino, and
openFrameworks

Joshua Noble

Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo


Programming Interactivity
by Joshua Noble
Copyright © 2009 Joshua Noble. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions
are also available for most titles (). For more information, contact our
corporate/institutional sales department: (800) 998-9938 or

Editor: Steve Weiss
Production Editor: Sumita Mukherji
Copyeditor: Kim Wimpsett
Proofreader: Sumita Mukherji
Production Services: Newgen

Indexer: Ellen Troutman Zaig
Cover Designer: Karen Montgomery
Interior Designer: David Futato
Illustrator: Robert Romano

Printing History:
July 2009:

First Edition.

Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of
O’Reilly Media, Inc. Programming Interactivity, the image of guinea fowl, and related trade dress are
trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a
trademark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and author assume
no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

TM

This book uses RepKover™, a durable and flexible lay-flat binding.
ISBN: 978-0-596-15414-1
[M]
1247251884


Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Part I. Introductions
1. Introducing Interaction Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
What This Book Is for
Programming for Interactivity

The Nature of Interaction
Messages and Interaction
Interfaces and Interaction
Languages of Interaction
Design and Interaction
Art and Interaction
Data Exchange and Exploration
Working Process

3
4
5
7
8
10
12
13
15
19

2. Programming Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Why You’ll Read This Chapter More Than Once
The Nature of Code
Variables
Simple Types
Arrays
Casting
Operators
Control Statements
if/then

for Loop
while Loop
continue
break
Functions

22
22
23
24
29
33
33
37
37
38
39
40
40
41
iii


Defining a Function
Passing Parameters to a Method
Some Suggestions on Writing Functions
Overloading Functions
Objects and Properties
Scope
Review

41
42
44
44
46
49
50

3. Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Downloading and Installing Processing
Exploring the Processing IDE
The Basics of a Processing Application
The setup() Method
The draw() Method
The Basics of Drawing with Processing
The rect(), ellipse(), and line() Methods
RGB Versus Hexadecimal
The fill() Method
The background() Method
The line() Method
The stroke() and strokeWeight() Methods
The curve() Method
The vertex() and curveVertex() Methods
Capturing Simple User Interaction
The mouseX and mouseY Variables
The mousePressed() Method
The mouseReleased() and mouseDragged() Methods
The keyPressed and key Variables
Importing Libraries

Downloading Libraries
Loading Things into Processing
Loading and Displaying Images
Displaying Videos in the Processing Environment
Using the Movie Class
Reading and Writing Files
Running and Debugging Applications
Exporting Processing Applications
Conclusion
Review

54
54
56
56
57
60
60
62
63
65
65
65
66
66
67
68
69
70
73

77
77
79
79
81
81
83
85
86
88
89

4. Arduino . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Starting with Arduino
Installing the IDE
iv | Table of Contents

92
93


Configuring the IDE
Touring Two Arduino Boards
The Controller
Duemilanove Versus Mini
Touring the Arduino IDE
The Basics of an Arduino Application
The setup Statement
The loop Method
Features of the Arduino Language

Constants
Methods
Arrays
Strings
How to Connect Things to Your Board
Hello World
Debugging Your Application
Importing Libraries
Running Your Code
Running Your Board Without a USB Connection
Review

96
97
97
97
102
105
106
106
108
109
110
111
112
115
117
122
124
126

126
127

5. Programming Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Object-Oriented Programming
Classes
The Basics of a Class
Class Rules
Public and Private Properties
Inheritance
Processing: Classes and Files
C++: Classes and Files
.cpp and .h
A Simple C++ Application
Pointers and References
Reference
Pointer
When to Use Pointers
Large Data Objects
Pointers and Arrays
When Are You Going to Use This?
Review

129
130
131
132
133
135
137

139
140
142
144
146
146
147
148
149
150
151

6. openFrameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Your IDE and Computer

154
Table of Contents | v


Windows
Mac OS X
Linux
Taking Another Quick Tour of C++
Basic Variable Types
Arrays
Methods
Classes and Objects in C++
Getting Started with oF
Touring an oF Application
Methods

Variables
Creating “Hello, World”
Drawing in 2D
Setting Drawing Modes
Drawing Polygons
Displaying Video Files and Images
Images
Video
Importing Libraries
ofxOpenCv
ofxVectorGraphics
ofxVectorMath
ofxNetwork
ofxOsc
Compiling an oF Program
Compiling in Xcode
Compiling in Code::Blocks
Debugging an oF Application
Using the printf Statement
Using the GNU Debugger
Using the Debugger in Xcode
Using the Debugger in Code::Blocks
Review

154
155
155
156
157
157

158
159
160
166
166
168
168
171
172
174
176
176
178
180
181
181
182
182
182
183
183
184
184
184
185
186
188
188

Part II. Themes

7. Sound and Audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Sound As Feedback
Sound and Interaction
How Sound Works on a Computer
Audio in Processing

vi | Table of Contents

194
197
199
202


Instantiating the Minim Library
Generating Sounds with Minim
Filtering Sounds with Minim
Sound in openFrameworks
openFrameworks and the FMOD Ex Library
The Sound Object Library
The Magic of the Fast Fourier Transform
Physical Manipulation of Sound with Arduino
A Quick Note on PWM
Creating Interactions with Sound
Further Resources
Review

202
204
208

214
221
228
233
238
239
242
242
243

8. Physical Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Interacting with Physical Controls
Thinking About Kinetics
Getting Gear for This Chapter
Controlling Controls
The Button As an Electrical Object
The Button As an Interactive Object
The Button As a Value in Code
Turning Knobs
The Dial As an Interactive Object
Potentiometers
Using Lights
Wiring an LED
Detecting Touch and Vibration
Reading a Piezo Sensor
Getting Piezo Sensors
Communicating with Other Applications
Sending Messages from the Arduino
openFrameworks
Detecting Motion

PIR Motion Sensor
Reading Distance
Reading Input from an Infrared Sensor
Understanding Binary Numbers
Binary Numbers
Bits and Bit Operations
Why Do You Need to Know Any of This?
Detecting Forces and Tilt
Introducing I2C
What Is a Physical Interface?

245
246
247
248
248
248
248
249
249
249
251
252
253
254
255
259
262
263
265

265
267
269
270
270
271
273
273
278
283

Table of Contents | vii


What’s Next
Review

284
286

9. Programming Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
The Screen and Graphics
Seeing Is Thinking, Looking Is Reading
Math, Graphics, and Coordinate Systems
Drawing Strategies
Use Loops to Draw
Use Arrays to Draw
Draw Only What You Need
Use Sprites
Processing and Transformation Matrices

Creating Motion
Shaping the Gaze
Setting the Mood
Creating Tweens
Using Vectors
Using Graphical Controls
ControlP5 Library
Event Handling
Importing and Exporting Graphics
Using PostScript in Processing
Using PostScript Files in oF
What’s Next
Review

289
292
293
296
296
298
303
303
303
307
308
308
310
315
325
326

326
328
329
330
333
334

10. Bitmaps and Pixels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
Using Pixels As Data
Using Pixels and Bitmaps As Input
Providing Feedback with Bitmaps
Looping Through Pixels
Manipulating Bitmaps
Manipulating Color Bytes
Using Convolution in Full Color
Analyzing Bitmaps in oF
Analyzing Color
Analyzing Brightness
Detecting Motion
Using Edge Detection
Using Pixel Data
Using Textures
Textures in oF
viii | Table of Contents

338
340
341
342
345

347
348
349
350
351
353
355
361
368
369


Textures in Processing
Saving a Bitmap
What’s Next
Review

373
375
376
377

11. Physical Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
Using Motors
DC Motors
Stepper Motors
Other Options
Using Servos
Connecting a Servo
Communicating with the Servo

Wiring a Servo
Using Household Currents
Working with Appliances
Introducing the LilyPad Board
Using Vibration
Using an LED Matrix
Using the Matrix Library
Using the LedControl Library
Using the SPI Protocol
Using LCDs
Serial LCD
Using Solenoids for Movement
What’s Next
Review

380
381
384
386
386
387
387
388
392
393
395
397
404
404
407

410
412
416
417
420
421

12. Protocols and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
Communicating Over Networks
Using XML
Understanding Networks and the Internet
Network Organization
Network Identification
Network Data Flow
Handling Network Communication in Processing
Client Class
Server Class
Sharing Data Across Applications
Understanding Protocols in Networking
Using ofxNetwork
Creating Networks with the Arduino
Initializing the Ethernet Library

425
426
429
429
430
431
432

432
433
436
441
442
450
451
Table of Contents | ix


Creating a Client Connection
Creating a Server Connection
Using Carnivore to Communicate
Installing the Carnivore Library
Creating a Carnivore Client
Communicating with Bluetooth
Using Bluetooth in Processing
Using the bluetoothDesktop Library
Using the Arduino Bluetooth
Communicating Using MIDI
Review

452
453
456
457
458
460
461
461

464
467
471

Part III. Explorations
13. Graphics and OpenGL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
What Does 3D Have to Do with Interaction?
Understanding 3D
Working with 3D in Processing
Lighting in Processing
Controlling the Viewer’s Perspective
Making Custom Shapes in Processing
Using Coordinates and Transforms in Processing
Working with 3D in OpenGL
So, What Is OpenGL?
Transformations
OpenGL in Processing
Open GL in openFrameworks
Using Matrices and Transformations in OpenGL
Using Vertices in OpenGL
Drawing with Textures in oF
Lighting in OpenGL
Blending Modes in OpenGL
Using Textures and Shading in Processing
Applying Material Properties
Using Another Way of Shading
What Does GLSL Look Like?
Vertex Shaders
Fragment Shader
Variables Inside Shaders

Using an ofShader Addon
What to Do Next
Review

x | Table of Contents

475
476
477
478
480
484
487
489
489
490
490
492
493
496
496
500
501
506
507
508
508
508
509
510

510
513
514


14. Detection and Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
Computer Vision
Interfaces Without Controls
Example CV Projects
OpenCV
Using Blobs and Tracking
Starting with ofxOpenCV
Tracking Blobs with ofxOpenCV
Using OpenCV in Processing
Exploring Further in OpenCV
Detecting Gestures
Using ezGestures in Processing
Using Gestures in oF
Implementing Face Recognition
Exploring Touch Devices with oF
TouchKit
Tuio
Touchlib
reacTIVision
What’s Next
Review

518
519
520

521
521
522
527
537
542
543
544
548
550
554
554
555
555
555
556
557

15. Movement and Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559
Using Movement As and in Interaction
Using Software-Based Serial Ports
Understanding and Using GPS
Storing Data
Logging GPS Data to an Arduino
Using the Breadcrumbs Library
Implementing Hardware-Based Logging
Sending GPS Data
Determining Location by IP Address
What to Do Next
Review

559
561
563
575
577
578
579
580
583
589
589

16. Interfaces and Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
Examining Tools, Affordances, and Aesthetics
Reexamining Tilt
Exploring InputShield
Understanding Touch
Exploring Open Source Touch Hardware
Nort_/D
Liquidware TouchShield

592
593
597
599
600
600
603
Table of Contents | xi

Drawing to the TouchShield Screen
Controlling Servos Through the TouchShield
Setting Up Communication Between Arduino and TouchShield
Communicating Using OSC
Using the Wiimote
Using the Wii Nunchuck in Arduino
Tracking Wii Remote Positioning in Processing
What’s Next
Review

607
609
611
614
616
616
622
625
626

17. Spaces and Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627
Using Architecture and Space
Sensing Environmental Data
Using an XBee with Arduino
Creating a Simple Test
Configuring the XBee Module
Addressing in the XBee
XBee Library for Processing

Placing Objects in 2D
Using the X10 Protocol
Setting Up an RFID Sensor
Reading Heat and Humidity
What’s Next
Review

627
628
629
632
634
635
637
641
651
654
659
664
664

18. Further Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667
What’s Next?
Software Tools
Construction Processes
Artificial Intelligence
Physics
Hardware Platforms
Bibliography
Interaction Design

Programming
Hardware
Art
Conclusion

xii | Table of Contents

667
667
670
671
677
678
681
681
682
683
683
684


Appendix: Circuit Diagram Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685
Programming Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693

Table of Contents | xiii



Preface

This is a book about creating physical interaction with computer systems. It focuses
on designing hardware and programming for systems that use either physical input or
physical feedback. This book has been a dream of mine since I was an art student
beginning to create interactive installations and finding that there was no simple introduction to the topics that I wanted to explore. At the time, I didn’t know what
platforms, tools, and programming languages were available for creating interactive
art, and I didn’t know where to find more information about these topics that a relative
novice programmer could understand. As I began teaching, I was asked the same question again and again by students: “where do I begin?” Much has changed in the seven
years since then, though, and now many excellent projects are helping beginners program, artists create, and programmers rapidly prototype applications. We’ll cover three
of these projects in this book: Processing, Arduino, and openFrameworks. This book
intends to answer the question “Where do I begin?” in as comprehensive a manner as
possible. It is the intention of this book to be useful for almost any type of project. This
book will provide technical advice, critical commentary for you to consider, code that
you can use, hardware diagrams that you can use, and further resources for you to
explore.
Ten years ago, the idea of artists or designers writing code or designing hardware was
almost unheard of. Today, not only has it become commonplace, but it has become an
important arena of expression and exploration. The dialogue between technology and
design is a vital and vibrant one that shapes art and technology alike. I hope that this
book can be, in some small way, another path into this conversation for more artists
and designers.

Who This Book Is For
This book is aimed at designers, artists, amateur programmers, or anyone interested in
working with physical interaction in computing. No assumption is made about your
technical background or previous experience. The only assumption is that you are
interested in learning to program and build hardware. This book is an introduction to
a great number of topics, and throughout the book we list links to further resources so
you can expand your knowledge or explore a particular topic that interests you.
xv

We encourage you to make as much use as possible of these resources and to use this
book as a map for exploring a great number of technologies and techniques.

How This Book Is Organized
This book is broken into three parts. The first introduces the three projects that will be
used throughout this book, the second introduces some of the most common themes
in creating interaction in designs and applications, and the third introduces some of
the more advanced topics that you may want to explore further. Also included with
some of the chapters are interviews with programmers, artists, designers, and authors
who work with the tools covered in this book. Covering such a massive range of topics
means that this book doesn’t go into great depth about most of them, but it is filled
with references to other books, websites, designers, and artists that you may find helpful
or inspiring.

What Is—and Isn’t—in This Book
My excitement about the ideas and rapid growth of the field of interaction design is
hard to contain. However, as exciting and far-reaching as interaction design is, the
limitations of time and physical book size dictate that I be selective about what is and
isn’t covered in this book.

What’s in
This book covers Processing, Arduino, and openFrameworks. To help novice programmers, it covers some of the core elements of programming in C and C++ for Arduino and openFrameworks and also covers the Processing language. We introduce
dozens of libraries for openFrameworks and Processing—too many to list here. Some
of these are official libraries or add-ons for the two frameworks, and some are simply
extensions that have been created for this book or provided by altruistic coders.
We also introduce some of the basics of electronics and how computer hardware
functions, as well as many tools and components that you can use with an Arduino.
The Arduino and Processing IDEs are covered, as are two different IDEs for

openFrameworks, namely, CodeBlocks, and Xcode. The Arduino Duemilanove and
Mini are covered in depth, and we discuss other boards only briefly. We cover many
electronic components that have designed expressly for the Arduino, called shields, in
depth as well.

What’s Not in
While this book shows how to create some circuits, it doesn’t cover a great deal of the
fundamentals of electronics or hardware, how to create circuits, or electronics theory.

xvi | Preface


Chapter 18 lists some excellent tutorials and references. While the book does cover the
Processing subset of the Java programming language, to conserve space and maintain
focus, it doesn’t cover Java. The book doesn’t cover many aspects of C++, such as
templates, inline functions, operator overloading, and abstract classes. Again, though,
listed in Chapter 18 are several excellent resources that you can use to learn about these
deeper topics in C++.
There are so many Arduino-compatible boards now that it’s almost impossible to cover
them all in depth; the book mentions the Mega, the Nano, and several other boards
only in passing and leaves out many of the Arduino-compatible boards that are not
created by the Arduino team. Quite a few components and other tools that we would
have liked to discuss in depth could not be included to maintain scope and to save
space. A good camera for computer vision was not included either, though a glance at
the openFrameworks or Processing forums will likely provide a more up-to-date discussion than could have been given here.
Many topics that we would have liked to include have been left out because of space
considerations: artificial intelligence, data visualization, and algorithmic music, among
others. Though these are all potentially interesting areas for artists and designers, the
focus of the book is on teaching some of the theory and techniques for interaction
design as well as the basics of hardware and programming. The resources listed at the

end of the book can provide the names of some materials that might help you explore
these topics.

Companion Website
All the code included in this book are available for download from the book’s companion website, />
Typographical Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, file extensions, path
names, and directories.
Constant width

Indicates direct references to code, text output from executing scripts, XML tags,
HTML tags, and the contents of files.
Constant width bold

Shows commands or other text that should be typed literally by the user.
Constant width italic

Shows text that should be replaced with user-supplied values.

Preface | xvii


This icon signifies a tip, suggestion, or general note.

This icon indicates a warning or caution.

Hexadecimal numbers in this book are denoted with the prefix 0x.

Using Code Examples
This book is here to help you get your job done. In general, you may use the code in
this book in your programs and documentation. You do not need to contact us for
permission unless you’re reproducing a significant portion of the code. For example,
writing a program that uses several chunks of code from this book does not require
permission. Answering a question by citing this book and quoting example code does
not require permission. On the other hand, selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Incorporating a significant
amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title,
author, publisher, and ISBN. For example: Programming Interactivity by Joshua Noble.
Copyright 2009 Joshua Noble, 978-0-596-15414-1.
If you think your use of code examples falls outside fair use or the permission given
here, feel free to contact us at

Safari® Books Online
When you see a Safari® Books Online icon on the cover of your favorite
technology book, that means the book is available online through the
O’Reilly Network Safari Bookshelf.
Safari offers a solution that’s better than e-books. It’s a virtual library that lets you easily
search thousands of top tech books, cut and paste code samples, download chapters,
and find quick answers when you need the most accurate, current information. Try it
for free at .

xviii | Preface


We’d Like to Hear from You
Please address comments and questions concerning this book to the publisher:
O’Reilly Media, Inc.
1005 Gravenstein Highway North

Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at:
/>To comment or ask technical questions about this book, send email to:

For more information about our books, conferences, Resource Centers, and the
O’Reilly Network, see our website at:


Acknowledgments
I need, first and foremost, to thank the wonderful engineers, artists, programmers, and
dreamers who created the platforms that I’ve covered in this book. It is to all of them
that I would like to dedicate this book. A woefully short list has to include Massimo
Banzi, Tom Igoe, David Cuartielles, Gianluca Martino, David A. Mellis, Ben Fry, Casey
Reas, Zach Lieberman, Theo Watson, Arturo Castro, and Chris O’Shea, the creators
of the frameworks covered in this book. There are dozens, if not hundreds, of other
names that should be on this list, but space is too limited to list them all. All I can say
is thank to you to all the creators of these frameworks and to everyone who uses them
to inspire, invent, amaze, and enrich the dialogue about design, technology, and art.
This book is a humble attempt to thank you all for everything that you’ve given to me
and to every other programmer, artist, or designer interested for working with computing in novel and interesting ways and bringing more people into the conversation.
I would also like to extend my deepest thanks to all my interviewees for taking the time
to respond to my questions and enrich this book and for so enriching the world of
interaction design and art. To everyone who provided code for this book as well, created
open source code, or answered questions on any of the forums for beginners, thank
you for your efforts to create a community.

Preface | xix


This book is as much my effort as it is the sum of the efforts of the editorial team that
worked on it. My technical editors, Michael Margolis, Adam Parrish, and Jeremy Rotzstain, have been absolutely fantastic. Their expertise, suggestions, and fresh look at
what I was working on shaped not only this book but enlightened me, showed me new
ways of solving problems, introduced me to new tools and techniques, and sharpened
my thinking and broadened my horizons for the better. This book is a collaboration
between all four of us in every sense of the word. I cannot pay them enough thanks for
their excellent work. I would also like to thank Justin Hunyh and Mike Gionfriddo
from LiquidWare as well as Nathan Seidle from Sparkfun for all of their help. My
editors, Robyn Thomas and Kim Wimpsett, have been incredible, helping me with my
sometime torturous grammar and patiently working with my propensity for sending in
extremely rough drafts to bounce ideas off of them. They have made this book better
than it ever could have been without their watchful eyes and guidance. Finally, I need
to thank Steve Weiss for listening to my idea when I first proposed it and helping guide
it through to completion.
I need to thank all of my friends in New York, Amsterdam, Geneva, London, Zurich,
Boston, Paris, and Toulouse for their support, their ideas, their Internet, and their encouragement. I would like to thank my family as well, and particularly my mother, for
their support and humor.

xx | Preface


PART I

Introductions

Part I of this book is an introduction not only to writing code and working with hardware, but also to the three tools that you’ll be learning about in this book: Processing,
Arduino, and openFrameworks. You’ll learn about writing software in general and

about writing code for each of the three platforms in particular. Each of the three platforms approaches the applications that you’ll use to actually write code in a slightly
different way, but the general concepts of working with the code are the same across
all three. One thing you’ll come to realize as you learn more about programming is that
the core concepts of programming are quite similar across a lot of different programming languages. This means that the knowledge you have about Processing, for instance, can help you if you decide to create a project using Arduino. Understanding
how these different frameworks are similar will help you leverage your knowledge in
one to work with the other. Understanding how they’re different will help you choose
the right one for the kind of project that you want to create.
Chapter 1, Introducing Interaction Design, will give you an introduction into what interactive design means and the tools available to build interactivity. In Chapter 2,
Programming Basics, you’ll be introduced to the fundamental concepts of programming. There you’ll learn how programming an application works, learn what code
really is, see some of the key terminology and concepts, and get ready to dive into
actually writing code in each of the three programming tools that you’ll be exploring.
Chapter 3, Processing, introduces you to the Processing language and environment and
shows you some code to get you started working with video, sound, images, and
simple graphics. Chapter 4, Arduino, introduces the Arduino language, hardware platform, and environment. Since the Arduino platform consists of a programming language, a hardware device, and an IDE that you use to write code, you’ll be learning
about how to use all three. In Chapter 5, Programming Revisited, you’ll learn about
some more advanced topics in programming that will prepare you to work with openFrameworks, like classes and object-oriented programming. Finally, in Chapter 6,
openFrameworks, you’ll be introduced to the C++ programming language and the
openFrameworks way of using it.


This first part might involve taking a lot of first steps for you, but once you have stepped
your way through it, you’ll understand a great deal about three tools that can help you
build almost any kind of interactive project by creating your own hardware and
software.


CHAPTER 1

Introducing Interaction Design

The scientist and philosopher Alfred Korzybski once remarked, “The map is not the
territory,” and it’s in that spirit that this book was written. The map may not be the
territory, but it is helpful for getting around the territory and for finding where you are
and where you want to go. This book covers a vast range of topics from programming
to electronics to interaction design to art, but it doesn’t cover any one of them in great
depth. It covers all of these topics because they are part of an emerging territory that is
often called interaction design, and that territory encompasses art, design, psychology,
engineering, and programming. It’s also a territory that is becoming more and more
accessible thanks to excellent projects like the ones that we’ll be exploring in the book—
tools that have been created to make code and coding easier to do.
You should use this book like a map to see what technologies exist and the areas in
interaction design that you might want to explore. This isn’t a cookbook or an in-depth
technical manual, but it will point you in the direction of other books, researchers,
designers, projects, and artists as you go along. This book will also give you the technical
understanding to know how to find information on almost any kind of project that you
want to explore and what to do with that information once you find it.

What This Book Is for
This book was created under the premise that technology and code are not tools solely
for computer scientists or engineers to create applications and that no one be intimidated by or shy away from working with and exploring electronics, hardware, and code.
Artists and designers can be interested in enabling interaction between users and between applications in ways that can be accentuated by the addition of custom computer
applications or that can be realized only through the use of custom computer applications. You can focus on creating applications that emphasize their technological nature
or on creating applications that feel very high-tech or use familiar metaphors like a
keyboard and mouse or touchscreen. You can also choose to accentuate other aspects
of the interaction or hide the technology behind a more organic interface. This book is
specifically about the interactions that users or viewers can have with computers,
3