IT training assembly language step by step programming with linux (3rd ed ) duntemann 2009 10 05

Assembly Language
Step-by-Step



Assembly Language
Step-by-Step
Programming with Linux®
Third Edition

Jeff Duntemann

Wiley Publishing, Inc.


Assembly Language Step-by-Step
Published by
Wiley Publishing, Inc.
10475 Crosspoint Boulevard
Indianapolis, IN 46256

www.wiley.com
Copyright © 2009 by Jeff Duntemann
Published by Wiley Publishing, Inc., Indianapolis, Indiana
Published simultaneously in Canada
ISBN: 978-0-470-49702-9
Manufactured in the United States of America
10 9 8 7 6 5 4 3 2 1
No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means,
electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108

of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization
through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers,
MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the
Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008,
or online at />Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with
respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including
without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or
promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work
is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional
services. If professional assistance is required, the services of a competent professional person should be sought. Neither
the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is
referred to in this work as a citation and/or a potential source of further information does not mean that the author or the
publisher endorses the information the organization or Web site may provide or recommendations it may make. Further,
readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this
work was written and when it is read.
For general information on our other products and services please contact our Customer Care Department within the
United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available
in electronic books.
Library of Congress Control Number: 2009933745
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its
affiliates, in the United States and other countries, and may not be used without written permission. Linux is a registered
trademark of Linus Torvalds. All other trademarks are the property of their respective owners. Wiley Publishing, Inc. is
not associated with any product or vendor mentioned in this book.


To the eternal memory of
Kathleen M. Duntemann, Godmother
1920– 1999
who gave me books when all I could do was put teeth marks on them.

There are no words for how much I owe you!



About the Author

Jeff Duntemann is a writer, editor, lecturer, and publishing industry analyst. In
his thirty years in the technology industry he has been a computer programmer
and systems analyst for Xerox Corporation, a technical journal editor for
Ziff-Davis Publications, and Editorial Director for Coriolis Group Books and
later Paraglyph Press. He is currently a technical publishing consultant and also
owns Copperwood Press, a POD imprint hosted on lulu.com. Jeff lives with
his wife Carol in Colorado Springs, Colorado.

vii



Credits

Executive Editor
Carol Long
Project Editor
Brian Herrmann
Production Editor
Rebecca Anderson
Copy Editor
Luann Rouff
Editorial Director
Robyn B. Siesky

Editorial Manager
Mary Beth Wakefield
Production Manager
Tim Tate

Vice President and Executive
Publisher
Barry Pruett
Associate Publisher
Jim Minatel
Project Coordinator, Cover
Lynsey Stanford
Proofreader
Dr. Nate Pritts, Word One
Indexer
J&J Indexing
Cover Image
© Jupiter Images/Corbis/
Lawrence Manning

Vice President and Executive
Group Publisher
Richard Swadley

ix



Acknowledgments

First of all, thanks are due to Carol Long and Brian Herrmann at Wiley, for
allowing this book another shot, and then making sure it happened, on a much
more aggressive schedule than last time.
As for all three previous editions, I owe Michael Abrash a debt of gratitude
for constant sane advice on many things, especially the arcane differences
between modern Intel microarchitectures.
Although they might not realize it, Randy Hyde, Frank Kotler, Beth, and
all the rest of the gang on alt.lang.asm were very helpful in several ways, not
least of which was hearing and answering requests from assembly language
newcomers, thus helping me decide what must be covered in a book like this
and what need not.
Finally, and as always, a toast to Carol for the support and sacramental
friendship that has enlivened me now for 40 years, and enabled me to take on
projects like this and see them through to the end.

xi



Contents at a Glance

Introduction: ‘‘Why Would You Want to Do That?’’

xxvii

Chapter 1

Another Pleasant Valley Saturday

1

Chapter 2

Alien Bases

15

Chapter 3

Lifting the Hood

45

Chapter 4

Location, Location, Location

77

Chapter 5

The Right to Assemble

109

Chapter 6

A Place to Stand, with Access to Tools

155

Chapter 7

Following Your Instructions

201

Chapter 8

Our Object All Sublime

237

Chapter 9

Bits, Flags, Branches, and Tables

279

Chapter 10 Dividing and Conquering

327

Chapter 11 Strings and Things

393

Chapter 12 Heading Out to C

439

Conclusion: Not the End, But Only the Beginning

503

Appendix A Partial x86 Instruction Set Reference

507

Appendix B Character Set Charts

583

Index

587
xiii



Contents

Introduction: ‘‘Why Would You Want to Do That?’’

xxvii

Chapter 1

Another Pleasant Valley Saturday
It’s All in the Plan

Steps and Tests
More Than Two Ways?
Computers Think Like Us
Had This Been the Real Thing . . .
Do Not Pass Go
The Game of Big Bux
Playing Big Bux
Assembly Language Programming As a Board Game
Code and Data
Addresses
Metaphor Check!

1
1
2
3
4
4
5
6
8
9
10
11
12

Chapter 2

Alien Bases
The Return of the New Math Monster

Counting in Martian
Dissecting a Martian Number
The Essence of a Number Base
Octal: How the Grinch Stole Eight and Nine
Who Stole Eight and Nine?
Hexadecimal: Solving the Digit Shortage
From Hex to Decimal and from Decimal to Hex
From Hex to Decimal
From Decimal to Hex
Practice. Practice! PRACTICE!

15
15
16
18
20
20
21
24
28
28
29
31
xv


xvi

Contents

Chapter 3

Arithmetic in Hex
Columns and Carries
Subtraction and Borrows
Borrows across Multiple Columns
What’s the Point?
Binary
Values in Binary
Why Binary?
Hexadecimal As Shorthand for Binary
Prepare to Compute

32
35
35
37
38
38
40
42
43
44

Lifting the Hood
RAXie, We Hardly Knew Ye . . .
Gus to the Rescue
Switches, Transistors, and Memory
One If by Land . . .
Transistor Switches

The Incredible Shrinking Bit
Random Access
Memory Access Time
Bytes, Words, Double Words, and Quad Words
Pretty Chips All in a Row
The Shop Foreman and the Assembly Line
Talking to Memory
Riding the Data Bus
The Foreman’s Pockets
The Assembly Line
The Box That Follows a Plan
Fetch and Execute
The Foreman’s Innards
Changing Course
What vs. How: Architecture and Microarchitecture
Evolving Architectures
The Secret Machinery in the Basement
Enter the Plant Manager
Operating Systems: The Corner Office
BIOS: Software, Just Not as Soft
Multitasking Magic
Promotion to Kernel
The Core Explosion
The Plan

45
45
46
47
48

48
50
52
53
54
55
57
58
59
60
61
61
63
64
65
66
67
68
70
70
71
71
73
73
74


Contents
Chapter 4

Chapter 5

Location, Location, Location
The Joy of Memory Models
16 Bits’ll Buy You 64K
The Nature of a Megabyte
Backward Compatibility and Virtual 86 Mode
16-Bit Blinders
The Nature of Segments
A Horizon, Not a Place
Making 20-Bit Addresses out of 16-Bit Registers
16-Bit and 32-Bit Registers
General-Purpose Registers
Register Halves
The Instruction Pointer
The Flags Register
The Three Major Assembly Programming Models
Real Mode Flat Model
Real Mode Segmented Model
Protected Mode Flat Model
What Protected Mode Won’t Let Us Do Anymore
Memory-Mapped Video
Direct Access to Port Hardware
Direct Calls into the BIOS
Looking Ahead: 64-Bit ‘‘Long Mode’’
64-Bit Memory: What May Be Possible Someday vs.
What We Can Do Now

77
77

79
82
83
83
85
88
88
90
91
93
95
96
96
97
99
101
104
104
105
106
106

The Right to Assemble
Files and What’s Inside Them
Binary Files vs. Text Files
Looking at File Internals with the Bless Editor
Interpreting Raw Data
‘‘Endianness’’
Text In, Code Out
Assembly Language

Comments
Beware ‘‘Write-Only’’ Source Code!
Object Code and Linkers
Relocatability
The Assembly Language Development Process
The Discipline of Working Directories
Editing the Source Code File

109
110
111
112
116
117
121
121
124
124
125
128
128
129
131

107

xvii


xviii Contents

Assembling the Source Code File
Assembler Errors
Back to the Editor
Assembler Warnings
Linking the Object Code File
Linker Errors
Testing the .EXE File
Errors versus Bugs
Are We There Yet?
Debuggers and Debugging

Chapter 6

131
132
133
134
135
136
136
137
138
138

Taking a Trip Down Assembly Lane
Installing the Software
Step 1: Edit the Program in an Editor
Step 2: Assemble the Program with NASM
Step 3: Link the Program with LD
Step 4: Test the Executable File

Step 5: Watch It Run in the Debugger
Ready to Get Serious?

139
139
142
143
146
147
147
153

A Place to Stand, with Access to Tools
The Kate Editor
Installing Kate
Launching Kate
Configuration
Kate Sessions
Creating a New Session
Opening an Existing Session
Deleting or Renaming Sessions
Kate’s File Management
Filesystem Browser Navigation
Adding a File to the Current Session
Dropping a File from the Current Session
Switching Between Session Files in the Editor
Creating a Brand-New File
Creating a Brand-New Folder on Disk
Deleting a File from Disk (Move File to Trash)
Reloading a File from Disk

Saving All Unsaved Changes in Session Files
Printing the File in the Editor Window
Exporting a File As HTML
Adding Items to the Toolbar
Kate’s Editing Controls
Cursor Movement
Bookmarks
Selecting Text

155
157
157
158
160
162
162
163
163
164
165
165
166
166
166
166
166
167
167
167
167

167
168
169
169
170


Contents

Chapter 7

Searching the Text
Using Search and Replace
Using Kate While Programming
Creating and Using Project Directories
Focus!

171
172
172
173
175

Linux and Terminals
The Linux Console
Character Encoding in Konsole
The Three Standard Unix Files
I/O Redirection
Simple Text Filters
Terminal Control with Escape Sequences

So Why Not GUI Apps?
Using Linux Make
Dependencies
When a File Is Up to Date
Chains of Dependencies
Invoking Make from Inside Kate
Using Touch to Force a Build
The Insight Debugger
Running Insight
Insight’s Many Windows
A Quick Insight Run-Through
Pick Up Your Tools . . .

176
176
177
178
180
182
183
185
186
187
189
189
191
193
194
195
195

197
200

Following Your Instructions
Build Yourself a Sandbox
A Minimal NASM Program
Instructions and Their Operands
Source and Destination Operands
Immediate Data
Register Data
Memory Data
Confusing Data and Its Address
The Size of Memory Data
The Bad Old Days
Rally Round the Flags, Boys!
Flag Etiquette
Adding and Subtracting One with INC and DEC
Watching Flags from Insight
How Flags Change Program Execution
Signed and Unsigned Values
Two’s Complement and NEG
Sign Extension and MOVSX

201
201
202
204
204
205
207

209
210
211
211
212
215
215
216
218
221
221
224

xix


xx

Contents

Chapter 8

Implicit Operands and MUL
MUL and the Carry Flag
Unsigned Division with DIV
The x86 Slowpokes
Reading and Using an Assembly Language Reference
Memory Joggers for Complex Memories
An Assembly Language Reference for Beginners
Flags

NEG: Negate (Two’s Complement; i.e., Multiply by -1)
Flags affected
Legal forms
Examples
Notes
Legal Forms
Operand Symbols
Examples
Notes
What’s Not Here . . .

225
227
228
229
230
230
231
232
233
233
233
233
233
234
234
235
235
235

Our Object All Sublime
The Bones of an Assembly Language Program
The Initial Comment Block
The .data Section
The .bss Section
The .text Section
Labels
Variables for Initialized Data
String Variables
Deriving String Length with EQU and $
Last In, First Out via the Stack
Five Hundred Plates per Hour
Stacking Things Upside Down
Push-y Instructions
POP Goes the Opcode
Storage for the Short Term
Using Linux Kernel Services Through INT80
An Interrupt That Doesn’t Interrupt Anything
Getting Home Again
Exiting a Program via INT 80h
Software Interrupts versus Hardware Interrupts
INT 80h and the Portability Fetish
Designing a Non-Trivial Program
Defining the Problem
Starting with Pseudo-code

237
237
239
240

240
241
241
242
242
244
246
246
248
249
251
253
254
254
259
260
261
262
264
264
265


Contents
Successive Refinement
Those Inevitable ‘‘Whoops!’’ Moments
Scanning a Buffer
‘‘Off By One’’ Errors
Going Further

Chapter 9

Bits, Flags, Branches, and Tables
Bits Is Bits (and Bytes Is Bits)
Bit Numbering
‘‘It’s the Logical Thing to Do, Jim. . .’’
The AND Instruction
Masking Out Bits
The OR Instruction
The XOR Instruction
The NOT Instruction
Segment Registers Don’t Respond to Logic!
Shifting Bits
Shift By What?
How Bit Shifting Works
Bumping Bits into the Carry Flag
The Rotate Instructions
Setting a Known Value into the Carry Flag
Bit-Bashing in Action
Splitting a Byte into Two Nybbles
Shifting the High Nybble into the Low Nybble
Using a Lookup Table
Multiplying by Shifting and Adding
Flags, Tests, and Branches
Unconditional Jumps
Conditional Jumps
Jumping on the Absence of a Condition
Flags
Comparisons with CMP
A Jungle of Jump Instructions

‘‘Greater Than‘‘ Versus ’’Above’’
Looking for 1-Bits with TEST
Looking for 0 Bits with BT
Protected Mode Memory Addressing in Detail
Effective Address Calculations
Displacements
Base + Displacement Addressing
Base + Index Addressing
Index × Scale + Displacement Addressing
Other Addressing Schemes

266
270
271
273
277

279
279
280
280
281
282
283
284
285
285
286
286
287

287
288
289
289
292
293
293
295
298
298
299
300
301
301
302
303
304
306
307
308
309
310
310
312
313

xxi


xxii

Contents
LEA: The Top-Secret Math Machine
The Burden of 16-Bit Registers

Character Table Translation
Translation Tables
Translating with MOV or XLAT
Tables Instead of Calculations
Chapter 10 Dividing and Conquering
Boxes within Boxes
Procedures As Boxes for Code
Calling and Returning
Calls within Calls
The Dangers of Accidental Recursion
A Flag Etiquette Bug to Beware Of
Procedures and the Data They Need
Saving the Caller’s Registers
Local Data
More Table Tricks
Placing Constant Data in Procedure Definitions
Local Labels and the Lengths of Jumps
’’Forcing’’ Local Label Access
Short, Near, and Far Jumps
Building External Procedure Libraries
Global and External Declarations
The Mechanics of Globals and Externals
Linking Libraries into Your Programs
The Dangers of Too Many Procedures and Too Many
Libraries

The Art of Crafting Procedures
Maintainability and Reuse
Deciding What Should Be a Procedure
Use Comment Headers!
Simple Cursor Control in the Linux Console
Console Control Cautions
Creating and Using Macros
The Mechanics of Macro Definition
Defining Macros with Parameters
The Mechanics of Invoking Macros
Local Labels Within Macros
Macro Libraries As Include Files
Macros versus Procedures: Pros and Cons

315
317

318
318
320
325
327
328
329
336
338
340
341
342
343

346
347
349
350
353
354
355
356
357
365
366
367
367
368
370
371
377
378
379
385
386
387
388
389


Contents xxiii
Chapter 11 Strings and Things
The Notion of an Assembly Language String
Turning Your ‘‘String Sense’’ Inside-Out

Source Strings and Destination Strings
A Text Display Virtual Screen
REP STOSB, the Software Machine Gun
Machine-Gunning the Virtual Display
Executing the STOSB Instruction
STOSB and the Direction Flag (DF)
Defining Lines in the Display Buffer
Sending the Buffer to the Linux Console
The Semiautomatic Weapon: STOSB without REP
Who Decrements ECX?
The LOOP Instructions
Displaying a Ruler on the Screen
MUL Is Not IMUL
Adding ASCII Digits
Adjusting AAA’s Adjustments
Ruler’s Lessons
16-bit and 32-bit Versions of STOS
MOVSB: Fast Block Copies
DF and Overlapping Block Moves
Single-Stepping REP String Instructions with Insight
Storing Data to Discontinuous Strings
Displaying an ASCII Table
Nested Instruction Loops
Jumping When ECX Goes to 0
Closing the Inner Loop
Closing the Outer Loop
Showchar Recap
Command-Line Arguments and Examining the Stack
Virtual Memory in Two Chunks
Anatomy of the Linux Stack

Why Stack Addresses Aren’t Predictable
Setting Command-Line Arguments with Insight
Examining the Stack with Insight’s Memory View
String Searches with SCASB
REPNE vs. REPE
Pop the Stack or Address It?
For Extra Credit . . .

393
393
394
395
395
402
403
404
405
406
406
407
407
408
409
410
411
413
414
414
414
416

418
419
419
420
421
421
422
423
424
424
427
429
429
430
432
435
436
438


xxiv

Contents
Chapter 12 Heading Out to C
What’s GNU?
The Swiss Army Compiler
Building Code the GNU Way
How to Use gcc in Assembly Work
Why Not gas?
Linking to the Standard C Library

C Calling Conventions
A Framework to Build On
Saving and Restoring Registers
Setting Up a Stack Frame
Destroying a Stack Frame
Characters Out via puts()
Formatted Text Output with printf()
Passing Parameters to printf()
Data In with fgets() and scanf()
Using scanf() for Entry of Numeric Values
Be a Time Lord
The C Library’s Time Machine
Fetching time_t Values from the System Clock
Converting a time_t Value to a Formatted String
Generating Separate Local Time Values
Making a Copy of glibc’s tm Struct with MOVSD
Understanding AT&T Instruction Mnemonics
AT&T Mnemonic Conventions
Examining gas Source Files Created by gcc
AT&T Memory Reference Syntax
Generating Random Numbers
Seeding the Generator with srand()
Generating Pseudorandom Numbers
Some Bits Are More Random Than Others
Calls to Addresses in Registers
How C Sees Command-Line Arguments
Simple File I/O
Converting Strings into Numbers with sscanf()
Creating and Opening Files
Reading Text from Files with fgets()

Writing Text to Files with fprintf()
Notes on Gathering Your Procedures into Libraries

439
440
441
441
443
444
445
446
447
447
448
450
451
452
454
456
458
462
462
464
464
465
466
470
470
471
474

475
476
477
482
483
484
487
487
489
490
493
494

Conclusion: Not the End, But Only the Beginning
Where to Now?
Stepping off Square One

503
504
506