the definitive guide to sqlite

www.it-ebooks.info
The Definitive Guide
to SQLite
■■■
Michael Owens
Owens_6730 FRONT.fm Page i Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
The Definitive Guide to SQLite
Copyright © 2006 by Michael Owens
All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means,
electronic or mechanical, including photocopying, recording, or by any information storage or retrieval
system, without the prior written permission of the copyright owner and the publisher.
ISBN-13: 978-1-59059-673-9
ISBN-10: 1-59059-673-0
Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1
Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence
of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark
owner, with no intention of infringement of the trademark.
Lead Editors: Jason Gilmore, Keir Thomas
Technical Reviewer: Preston Hagar
Editorial Board: Steve Anglin, Ewan Buckingham, Gary Cornell, Jason Gilmore, Jonathan Gennick,
Jonathan Hassell, James Huddleston, Chris Mills, Matthew Moodie, Dominic Shakeshaft, Jim Sumser,
Keir Thomas, Matt Wade
Project Manager: Beth Christmas
Copy Edit Manager: Nicole LeClerc
Copy Editor: Liz Welch
Assistant Production Director: Kari Brooks-Copony
Production Editor: Katie Stence
Compositor: Susan Glinert
Proofreader: April Eddy
Indexer: Toma Mulligan

Artist: Kinetic Publishing Services, LLC
Cover Designer: Kurt Krames
Manufacturing Director: Tom Debolski
Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor,
New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail , or
visit .
For information on translations, please contact Apress directly at 2560 Ninth Street, Suite 219, Berkeley, CA
94710. Phone 510-549-5930, fax 510-549-5939, e-mail , or visit .
The information in this book is distributed on an “as is” basis, without warranty. Although every precaution
has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to
any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly
by the information contained in this work.
The source code for this book is available to readers at in the Source Code section.
Owens_6730 FRONT.fm Page ii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
To my family: Gintana, Natalie, and Riley
To my parents: Larry and Nancy
And to my grandfather: C. R. Clough
Owens_6730 FRONT.fm Page iii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
Owens_6730 FRONT.fm Page iv Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
v
Contents at a Glance
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
■CHAPTER 1 Introducing SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
■CHAPTER 2 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

■CHAPTER 3 The Relational Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
■CHAPTER 4 SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
■CHAPTER 5 Design and Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
■CHAPTER 6 The Core C API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
■CHAPTER 7 The Extension C API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
■CHAPTER 8 Language Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
■CHAPTER 9 SQLite Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
■APPENDIX A SQL Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
■APPENDIX B C API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
■APPENDIX C Codd’s 12 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
Owens_6730 FRONT.fm Page v Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
Owens_6730 FRONT.fm Page vi Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
vii
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
■CHAPTER 1 Introducing SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
An Embedded Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
A Developer’s Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
An Administrator’s Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
SQLite History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Who Uses SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
The Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
The Compiler. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

The Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
The Back-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Utilities and Test Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
SQLite’s Features and Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Zero Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Portability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Compactness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Simplicity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Liberal Licensing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Convenience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Performance and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Who Should Read This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
How This Book Is Organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Owens_6730 FRONT.fm Page vii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
viii
■CONTENTS
■CHAPTER 2 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Where to Get SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
SQLite on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Getting the Command-Line Program . . . . . . . . . . . . . . . . . . . . . . . . . 18
Getting the SQLite DLL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Compiling the SQLite Source Code on Windows. . . . . . . . . . . . . . . . 21
Building the SQLite DLL with Microsoft Visual C++ . . . . . . . . . . . . . 25
Building a Dynamically Linked SQLite Client with Visual C++ . . . . 28
Building SQLite with MinGW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

SQLite on POSIX Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Binaries and Packages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Compiling SQLite from Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Working with SQLite Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
The CLP in Shell Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
The CLP in Command-Line Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Database Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Creating, Backing Up, and Dropping Databases . . . . . . . . . . . . . . . . 42
Getting Database File Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Other SQLite Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
■CHAPTER 3 The Relational Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
The Three Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
SQL and the Relational Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
The Structural Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
The Information Principle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
The Sanctity of the Logical Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
The Anatomy of the Logical Level. . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Tables: Relation Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Views: Virtual Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
The System Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Owens_6730 FRONT.fm Page viii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
■CONTENTS
ix
The Integrity Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Primary Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Foreign Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Null Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Normal Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
First Normal Form. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Functional Dependencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Second Normal Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Third Normal Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
The Manipulative Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Relational Algebra and Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Relational Query Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
The Advent of SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
The Meaning of Relational . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
■CHAPTER 4 SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
The Relational Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Query Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
The Growth of SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
The Example Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Running the Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Literals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Keywords and Identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Creating a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Creating Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Altering Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Owens_6730 FRONT.fm Page ix Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
x
■CONTENTS
Querying the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Relational Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
The Operational Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Limiting and Ordering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Functions and Aggregates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
Grouping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Removing Duplicates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Joining Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Names and Aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Subqueries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Compound Queries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Conditional Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
The Thing Called Null . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Set Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Modifying Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Inserting Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Updating Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Deleting Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Data Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Entity Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Domain Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Storage Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Manifest Typing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Type Affinity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Transaction Scopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Conflict Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
Database Locks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Deadlocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Transaction Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Database Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Indexes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Attaching Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Cleaning Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Database Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
The System Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Viewing Query Plans. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Owens_6730 FRONT.fm Page x Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
■CONTENTS
xi
■CHAPTER 5 Design and Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
The API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
What’s New in SQLite Version 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
The Principal Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
The Core API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Operational Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
The Extension API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Transaction Lifecycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Lock States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Read Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Write Transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Tuning the Page Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Waiting for Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Using Multiple Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Table Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Fun with Temporary Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
The Importance of Finalizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Shared Cache Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
■CHAPTER 6 The Core C API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Wrapped Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Connecting and Disconnecting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
The exec Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
String Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
The Get Table Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Prepared Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Finalization and Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Fetching Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
Parameterized Queries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Errors and the Unexpected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Handling Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Handling Busy Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Handling Schema Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Owens_6730 FRONT.fm Page xi Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xii

■CONTENTS
Operational Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Commit Hooks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Rollback Hooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Update Hooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Authorizer Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Shared Cache Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
Threads and Memory Management . . . . . . . . . . . . . . . . . . . . . . . . . 252
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
■CHAPTER 7 The Extension C API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
The API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Registering Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
The Step Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Return Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Return Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
A Complete Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
A Practical Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Aggregates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
A Practical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
Collating Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Collation Defined. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
A Simple Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
Collation on Demand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
A Practical Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
■CHAPTER 8 Language Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
Selecting an Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
Perl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
Connecting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
Query Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
Parameter Binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
User-Defined Functions and Aggregates . . . . . . . . . . . . . . . . . . . . . 307
Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
PySQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
APSW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Owens_6730 FRONT.fm Page xii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
■CONTENTS
xiii
Ruby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
Connecting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
Query Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
User-Defined Functions and Aggregates . . . . . . . . . . . . . . . . . . . . . 322
Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Connecting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Query Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
User-Defined Functions and Aggregates . . . . . . . . . . . . . . . . . . . . . 328
JDBC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
Tcl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Connecting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Query Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
PHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

Connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
Queries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
User-Defined Functions and Aggregates . . . . . . . . . . . . . . . . . . . . . 339
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
■CHAPTER 9 SQLite Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
The Virtual Database Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
The Stack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Program Body . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Program Startup and Shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
Instruction Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
The B-Tree and Pager Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Database File Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
The B-Tree API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
The Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
The Tokenizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
The Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
The Code Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
The Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
Owens_6730 FRONT.fm Page xiii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xiv
■CONTENTS
■APPENDIX A SQL Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
■APPENDIX B C API Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
■APPENDIX C Codd’s 12 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
Owens_6730 FRONT.fm Page xiv Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xv

Foreword
When I first began coding SQLite in the spring of 2000, I never imagined that it would be so
enthusiastically received by the programming community. Today, there are millions and millions
of copies of SQLite running unnoticed inside computers and gadgets made by hundreds of
companies from around the world. You have probably used SQLite before without realizing it.
SQLite might be inside your new cell phone or MP3 player or in the set-top box from your cable
company. At least one copy of SQLite is probably found on your home computer; it comes built
in on Apple’s Mac OS X and on most versions of Linux, and it gets added to Windows when you
install any of dozens of third-party software titles. SQLite backs many websites thanks in part to
its inclusion in the PHP5 programming language. And SQLite is also known to be used in aircraft
avionics, modeling and simulation programs, industrial controllers, smart cards, decision-support
packages, and medical information systems. Since there are no reporting requirements on the
use of SQLite, there are without doubt countless other deployments that are unknown to me.
Much credit for the popularity of SQLite belongs to Michael Owens. Mike’s articles on
SQLite in The Linux Journal (June 2003) and in The C/C++ Users Journal (March 2004) intro-
duced SQLite to countless programmers. The traffic at the SQLite website jumped noticeably
after each of these articles appeared. It is good to see Mike apply his expository talents in a
larger work: the book you now peruse. I am sure you will not be disappointed. This volume
contains everything you are likely to ever need to know about SQLite. You will do well to keep
it within arm’s reach.
SQLite is free software. Free as in freedom. Though I am its architect and principal coder,
SQLite is not my program. SQLite does not belong to anyone. It is not covered by copyright.
Everyone who has ever contributed code to the SQLite project has signed an affidavit releasing
their contributions to the public domain and I keep the originals to those affidavits in the fire-
safe at my office. I have also taken great care to ensure that no patented algorithms are used in
SQLite. These precautions mean that you are free to use SQLite in any way you wish without
having to pay royalties or license fees or abide by any other restrictions.
SQLite continues to improve and advance. But the other SQLite developers and I are
committed to maintaining its core values. We will keep the code small—never exceeding 250KB
for the core library. We will maintain backward compatibility both in the published API and the

database file format. And we will continue to work to make sure SQLite is thoroughly tested and
as bug-free as possible. We want you to always be able to drop newer versions of SQLite into
your older programs, in order to take advantage of the latest features and optimizations, with
little or no code change on your part and without having to do any additional debugging. We
did break backward compatibility on the transition from version 2 to version 3 in 2004, but
since then we have achieved all of these goals and plan to continue doing so into the future.
There are no plans for a SQLite version 4.
Owens_6730 FRONT.fm Page xv Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xvi
■FOREWORD
I hope that you find SQLite to be useful. On behalf of all the contributors to SQLite, I charge
you to use it well: make good and beautiful things that are fast, reliable, and simple to use. Seek
forgiveness for yourself and forgive others. And since you have received SQLite for free, please
give something for free to someone else in return. Volunteer in your community, contribute to
some other software project, or find some other way to pay the debt forward.
Richard Hipp
Charlotte, NC
April 11, 2006
Owens_6730 FRONT.fm Page xvi Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xvii
About the Author
■MICHAEL OWENS is the IT director for a major real estate firm in Fort Worth,
Texas, where he’s charged with the development and management of the
company’s core systems. His prior experience includes time spent at
Oak Ridge National Laboratory as a process design engineer, and at
Nova Information Systems as a C++ programmer. He is the original
creator of PySQLite, the Python extension for SQLite. Michael earned
his bachelor’s degree in chemical engineering from the University of

Tennessee in Knoxville.
Michael enjoys jogging, playing guitar, snow skiing, and hunting
with his buddies in the Texas panhandle. He lives with his wife, two
daughters, and two rat terriers in Fort Worth, Texas.
Owens_6730 FRONT.fm Page xvii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
Owens_6730 FRONT.fm Page xviii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xix
About the Technical Reviewer
■PRESTON HAGAR has a broad range of computer skills and experience.
He has served as a system administrator, consultant, DBA, programmer,
and web developer. He currently works for one of the largest single office
real estate companies in the country, where he focuses on programming
and database administration. He is lead developer and maintainer of
iBroker3, a QT/C++ real estate software suite that manages all facets of
a real estate business. Preston is also author of PNF and a partner in
Linterra, a consulting company whose primary focus is to provide Linux
server solutions for small- to medium-sized businesses.
Preston enjoys skiing and playing tennis. He lives with his wife in
North Richland Hills, Texas.
Owens_6730 FRONT.fm Page xix Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
Owens_6730 FRONT.fm Page xx Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
xxi
Acknowledgments
First and foremost, thanks to my family for putting up with all the nights, weekends, vacations,
and holidays that I have spent working on this book. I recall seeing so many instances in other
books where authors beg the forgiveness of their loved ones, and now I understand why.

Thanks to my employer and hunting buddy, Mike Bowman, for his support throughout
this project, and for the years of satisfaction that have come from using open source software to
run the company. He’s given me the most enjoyable job I’ve ever had.
I am grateful to Jamis Buck, Roger Binns, Wez Furlong (Dr. Evil), and Christian Werner for
their comments on the various language extensions. I am also greatful to Vladimir Vukicevic for
telling me how the Mozilla project uses SQLite, Eric Kustarz for his input on NFS, as well as
David Gleason and Ernest Prabhakar at Apple for information on Mac OS X.
I am deeply indebted to Richard Hipp, the creator of SQLite, for his feedback from
reviewing countless drafts, answering endless emails at all hours of the day, and for being very
supportive throughout the project. His suggestions, advice, and encouragement made all the
difference.
Thanks to Stéphane Faroult for his input on the book, especially the relational model and
SQL chapters. Thanks also to Jonathan Gennick who from the start has patiently but firmly
forced me to confront my addiction to passive construction. An ongoing battle it is.
Thanks to all the great people who write open source software. All of the code for this book
was developed using open source software: Gentoo and Ubuntu Linux, KDE, GCC, Emacs,
Firefox, OpenOffice, Ruby. . . the list goes on. I want to specifically thank the creators of the Dia
drawing program. It has been invaluable for creating the conceptual illustrations for this book.
I am also greatful to my colleague, John Starke, for introducing me to it.
To all the people at Apress, who have consistently provided me with more support than I
could ever need and then some. Thank you! Jason, Keir, Beth, Liz, Katie, and Julie, you have all
been a pleasure to work with.
Finally, to my wife, Gintana, who’s been my partner in crime for 14 years now. You are the
reason I ever stuck with anything, the reason I even tried.
Owens_6730 FRONT.fm Page xxi Friday, April 21, 2006 1:38 PM
fa938d55a4ad028892b226aef3fbf3dd
www.it-ebooks.info
Owens_6730 FRONT.fm Page xxii Friday, April 21, 2006 1:38 PM
www.it-ebooks.info
1

■ ■ ■
CHAPTER 1
Introducing SQLite
SQLite is an open source embedded relational database. Originally released in 2000, it was
designed to provide a convenient way for applications to manage data without the overhead
that often comes with dedicated relational database management systems. SQLite has a reputation
for being highly portable, easy to use, compact, efficient, and reliable.
An Embedded Database
SQLite is an embedded database. Rather than running independently as a standalone process,
it symbiotically coexists inside the application it serves—within its process space. Its code is
intertwined, or embedded, as a part of the program that hosts it. To an outside observer, it
would never be apparent that such a program had a relational database management system
(RDBMS) on board. The program would just do its job and manage its data somehow, making
no fanfare about how it went about doing so. But inside, there is a complete, self-contained
database engine at work.
One advantage of having a database server inside your program is that no network config-
uration or administration is required. Both client and server run together in the same process.
This reduces overhead related to network calls, simplifies database administration, and makes
it easier to deploy your application. Everything you need is compiled right into your program.
Consider the processes found in Figure 1-1. One is a Perl script, another is a standard C/C++
program, and the other is an Apache process with PHP, all using SQLite. The Perl script imports
the DBI::SQLite module, which in turn is linked to the SQLite C API, pulling in the SQLite library.
The PHP library works similarly, as does the C++ program. Ultimately, all three processes inter-
face with the SQLite C API. All three therefore have SQLite embedded in their process spaces,
and all three are independent database servers in and of themselves. Furthermore, even though
each process represents an independent server, they can still operate on the same database
file(s), as SQLite uses the operating system to manage synchronization and locking.
Today there is a wide variety of relational database products on the market specifically
designed for embedded use—products such as Sybase SQL Anywhere, InterSystems Caché,
Pervasive PSQL, and Microsoft’s Jet Engine. Some vendors have retrofitted their large-scale

databases to create embedded variants. Examples of these include IBM’s DB2 Everyplace,
Oracle’s 10g, and Microsoft’s SQL Server Desktop Engine. The open source databases MySQL
and Firebird both offer embedded versions as well. Of all these products, only two are both
open source and unencumbered by licensing fees: Firebird and SQLite. Of these remaining
two, only one is designed exclusively for use as an embedded database: SQLite.
Owens_6730 C01.fm Page 1 Monday, April 17, 2006 7:13 AM
www.it-ebooks.info
2
CHAPTER 1
■ INTRODUCING SQLITE
Figure 1-1. SQLite embedded in host processes
A Developer’s Database
SQLite is quite versatile. It is a database, a programming library, and a command-line tool, as
well an excellent learning tool that provides a good introduction to relational databases. There
are indeed many ways to use it—in embedded environments, websites, operating system
services, scripts, and applications. For programmers, SQLite is like digital duct tape, providing
an easy way to bind applications and their data. Like duct tape, there is no end to its potential
uses. In a web environment, SQLite can help with managing complex session information. Rather
than serializing session data into one big blob, individual pieces can be selectively written to
and read from individual session databases. SQLite also serves as a good stand-in relational
database for development and testing: there are no external RDBMSs or networking to configure,
or usernames and passwords to bother with. SQLite might also serve as a cache, hold configuration
data, or because of its binary compatibility across platforms, even work as an application
file format.
Besides being just a storage receptacle, SQLite can serve as a purely functional tool as well
for general data processing. Depending on size and complexity, it may be easier to represent
some application data structures as a table or tables in an in-memory database. This way, you
can operate on the data relationally, using SQLite to do the heavy lifting rather than having to
write your own algorithms to manipulate and sort data structures. If you are a programmer,
Owens_6730 C01.fm Page 2 Monday, April 17, 2006 7:13 AM

www.it-ebooks.info