Metal by Tutorials beginning game engine development with metal by raywenderlich tutorial team, caroline begbie, marius horga

Metal by Tutorials

Metal by Tutorials
By Caroline Begbie & Marius Horga
Copyright ©2019 Razeware LLC.

Notice of Rights
All rights reserved. No part of this book or corresponding materials (such as text,
images, or source code) may be reproduced or distributed by any means without prior
written permission of the copyright owner.

Notice of Liability
This book and all corresponding materials (such as source code) are provided on an
“as is” basis, without warranty of any kind, express or implied, including but not
limited to the warranties of merchantability, fitness for a particular purpose, and
noninfringement. In no event shall the authors or copyright holders be liable for any
claim, damages or other liability, whether in action of contract, tort or otherwise,
arising from, out of or in connection with the software or the use of other dealing in
the software.

Trademarks
All trademarks and registered trademarks appearing in this book are the property of
their own respective owners.

raywenderlich.com

2


Metal by Tutorials

Table of Contents: Overview
About the Cover ...................................................................................... 13
What You Need ........................................................................................ 17
Book License ............................................................................................. 18
Book Source Code & Forums ............................................................. 19
Book Updates ........................................................................................... 21
Introduction .............................................................................................. 22

Section I: The Player .......................................................... 26
Chapter 1: Hello, Metal! .......................................................... 28
Chapter 2: 3D Models ............................................................... 42
Chapter 3: The Rendering Pipeline ..................................... 64
Chapter 4: Coordinate Spaces............................................... 89
Chapter 5: Lighting Fundamentals .................................... 121
Chapter 6: Textures ................................................................. 154
Chapter 7: Maps & Materials .............................................. 185
Chapter 8: Character Animation ....................................... 216

Section II: The Scene ....................................................... 254
Chapter 9: The Scene Graph................................................ 256
Chapter 10: Fragment Post-Processing.......................... 289
Chapter 11: Tessellation & Terrains ................................. 313
Chapter 12: Environment ..................................................... 343
Chapter 13: Instancing & Procedural Generation ...... 373
raywenderlich.com

3

Metal by Tutorials

Chapter 14: Multipass & Deferred Rendering ............. 406
Chapter 15: GPU-Driven Rendering ................................ 434

Section III: The Effects ................................................... 470
Chapter 16: Particle Systems .............................................. 472
Chapter 17: Particle Behavior ............................................ 499
Chapter 18: Rendering with Rays ..................................... 517
Chapter 19: Advanced Shadows ........................................ 543
Chapter 20: Advanced Lighting.......................................... 572
Chapter 21: Metal Performance Shaders ...................... 605
Chapter 22: Integrating SpriteKit & SceneKit .............. 640
Chapter 23: Debugging & Profiling ................................... 671
Chapter 24: Performance Optimization ......................... 698
Conclusion .............................................................................................. 722

raywenderlich.com

4


Metal by Tutorials

Table of Contents: Extended
About the Cover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
About the Editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
About the Artist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

What You Need . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Book License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Book Source Code & Forums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Book Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
About this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
How did Metal come to life? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Why would you use Metal? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
When should you use Metal? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Who this book is for. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
How to read this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

22
23
23
24
25
25

Section I: The Player . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Chapter 1: Hello, Metal! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
What is rendering? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
What is a frame? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Your first Metal app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29
30

31
32
37
41

Chapter 2: 3D Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
What are 3D models? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Creating models with Blender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
raywenderlich.com

5


Metal by Tutorials

3D file formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Exporting to Blender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The .obj file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The .mtl file format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Material groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Vertex descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Metal coordinate system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Submeshes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47
48
50
51
54

56
59
60
63

Chapter 3: The Rendering Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . 64
The GPU and the CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Metal project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The rendering pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Send data to the GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65
66
74
86
88

Chapter 4: Coordinate Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Vectors and matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Matrices on the GPU. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Coordinate spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Upgrade the engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Uniforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Chapter 5: Lighting Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . 121
The starter project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Representing color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
raywenderlich.com

6


Metal by Tutorials

Normals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hemispheric lighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Light types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Directional light . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Phong reflection model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The dot product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Point lights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Spotlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

125
128
131
132
133
135
136

146
150
152
153

Chapter 6: Textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Textures and UV maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Texture the model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
sRGB color space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
GPU frame capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Samplers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Mipmaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The asset catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Texture compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

155
158
165
167
169
175
178
182
184

Chapter 7: Maps & Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Normal maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Function specialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Physically based rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Channel packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

186
201
204
208
212
214
214

Chapter 8: Character Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
raywenderlich.com

7


Metal by Tutorials

The starter project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Procedural animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Animation using physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Keyframes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Quaternions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
USD and USDZ files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Animating meshes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Blender for animating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Skeletal Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Loading the animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Joint matrix palette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The inverse bind matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Vertex function constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

217
218
218
221
227
230
231
235
241
244
245
247
250
253

Section II: The Scene . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
Chapter 9: The Scene Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Scenes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The scene graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Grouping nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
First-person camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Orthographic projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Third-person camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Animating the player . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Simple collisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

257
259
266
269
276
279
281
282
288

Chapter 10: Fragment Post-Processing . . . . . . . . . . . . . . . . . . . . 289
Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Alpha testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Depth testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Stencil testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
raywenderlich.com

290
291
296
296
8


Metal by Tutorials

Scissor testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Alpha blending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Antialiasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

297
298
306
308
311
312

Chapter 11: Tessellation & Terrains . . . . . . . . . . . . . . . . . . . . . . . . 313
Tessellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The starter project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Kernel (compute) functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Multiple patches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Tessellation by distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Displacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Shading by slope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

314
315
319
326
328
332

338
340
341

Chapter 12: Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The skybox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Procedural skies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Image-based lighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

344
345
351
357
360
372
372

Chapter 13: Instancing & Procedural Generation . . . . . . . . . . 373
The starter project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Instancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Morphing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Texture arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Procedural systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
raywenderlich.com

374

375
382
393
396
9


Metal by Tutorials

Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

Chapter 14: Multipass & Deferred Rendering . . . . . . . . . . . . . 406
Shadow maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Multipass rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Deferred rendering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

407
411
417
433

Chapter 15: GPU-Driven Rendering . . . . . . . . . . . . . . . . . . . . . . . 434
Argument buffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Resource heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Indirect Command Buffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
GPU driven rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

435
441
449
459
469

Section III: The Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
Chapter 16: Particle Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472
Particle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Emitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Compute. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Threads and threadgroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fireworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Particle systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

473
474
476
478
483
488
495
497

Chapter 17: Particle Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
Behavioral animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Swarming behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Behavioral rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

raywenderlich.com

500
501
502
504
506
10


Metal by Tutorials

Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516

Chapter 18: Rendering with Rays . . . . . . . . . . . . . . . . . . . . . . . . . . 517
Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Ray casting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Ray tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Path tracing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Raymarching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Signed distance functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The raymarching algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Creating random noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Marching clouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

518

519
521
522
523
526
527
533
540
542

Chapter 19: Advanced Shadows . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
Hard shadows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Soft shadows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Ambient occlusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Percentage closer filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

544
551
560
568
571

Chapter 20: Advanced Lighting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
The rendering equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Refraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Raytraced water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Rasterized water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

573
574
575
580
584
586
603
604

Chapter 21: Metal Performance Shaders . . . . . . . . . . . . . . . . . . 605
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606
Image processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607
raywenderlich.com

11


Metal by Tutorials

Matrix/vector mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
Ray tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638

Chapter 22: Integrating SpriteKit & SceneKit . . . . . . . . . . . . . . 640
SceneKit starter project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SceneKit shaders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Toon shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The fwidth function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Cel shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SpriteKit rendering in Metal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
SKRenderer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Core Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

641
643
649
650
651
653
657
661
670
670

Chapter 23: Debugging & Profiling . . . . . . . . . . . . . . . . . . . . . . . . . 671
Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672
Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684
Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697

Chapter 24: Performance Optimization. . . . . . . . . . . . . . . . . . . . 698
CPU-GPU synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Multithreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
GPU families . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Memory management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Best practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Where to go from here? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

699
703
707
709
712
721

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 722

raywenderlich.com

12


A

About the Cover

The comb jellyfish — or Ctenophora — has been swimming in both shallow and deep
waters for nearly 500 million years.
While just as seemingly alien as other jelly species, comb jellies are especially
unusual, boasting a completely transparent body with groups of cilia — or "combs" —
arranged in rows along the axis of the jelly's body. While appearing to be a
bioluminescent species, these jellies only appear to glow as their moving cilia scatter
the light, causing a glowing rainbow-like effect.
Perhaps most incredibly — and much like Metal's own low-level, low-overhead
features — these jellies' bodies use only what they need to live and produce their
amazing shading effects: They have no stomach, intestines or lungs, and they are

known to have one of the most basic nervous systems of any multicellular animal on
the planet.
You can learn more about comb jellies and see them in action in this video clip:
/>
raywenderlich.com

13


Metal by Tutorials

About the Cover

Dedications
"To Warren Moore, who first made it possible for me to learn
Metal, to my wonderful children Robin and Kayla, and to my
best friends who patiently waited for me to indulge my
dream."
— Caroline Begbie
"To my wife, Adina, and my son, Victor Nicholas, without
whose patience, support and understanding I could not have
made it. To Warren Moore who first whet my appetite for
Metal, offered his advice when needed and motivated me to
get involved with Metal too. To Chris Wood who taught me
that most of the times all you need to render is a ray, a camera
and a few distance fields. To Simon Gladman whose amazing
work with compute kernels inspired me to write more about
particles and fluid dynamics. To Jeff Biggus who keeps the
GPU programming community in Chicago alive. Our daily
conversations motivate me to stay hungry for more. To

everyone else who believes in me. A huge Thanks to all of
you!"
— Marius Horga

raywenderlich.com

14


Metal by Tutorials

About the Cover

About the Authors
Caroline Begbie is a co-author of this book. Caroline is an indie
iOS developer. When she's not developing, she's playing around
with 2D and 3D animation software, or learning Arduino and
electronics. She has previously taught the elderly how to use their
computers, done marionette shows for pre-schools, and created
accounting and stock control systems for mining companies.
Marius Horga is a co-author of this book. Marius is an iOS
developer and Metal API blogger. He is also a computer scientist.
He has more than a decade of experience with systems, support,
integration and development. You can often see him on Twitter
talking about Metal, GPGPU, games and 3D graphics. When he's
away from computers, he enjoys music, biking or stargazing.

About the Editors
Adrian Strahan is the technical editor of this book. Adrian is a
freelance iOS developer and Project Manager living in the South

West of England. He's worked on iPhone and iPad apps since 2010
(iOS3) and specializes in mobile- and web-based application
development.
Tammy Coron is the final pass editor of this book. Tammy is an
independent creative professional and the host of Roundabout:
Creative Chaos. She’s also the founder of Just Write Code. Find out
more at tammycoron.com.

About the Artist
Vicki Wenderlich is the designer and artist of the cover of this
book. She is Ray’s wife and business partner. She is a digital artist
who creates illustrations, game art and a lot of other art or design
work for the tutorials and books on raywenderlich.com. When she’s
not making art, she loves hiking, a good glass of wine and

raywenderlich.com

15


Metal by Tutorials

About the Cover
attempting to create the perfect cheese plate.

raywenderlich.com

16

W
What You Need

To follow along with the tutorials in this book, you need the following:
• A Metal-capable Mac running macOS Catalina 10.15 or later. All Macs built since
2012 should run Metal, although not all of them will be able to run the most recent
features in Metal 2. Nvidia GPUs will have issues, in some cases serious, as drivers
have not been updated since macOS High Sierra.
• Xcode 11.0 or later.
• [optional] A Metal-capable iPhone or iPad running iOS 13 or later. Any iOS device
running the A7 chip or later will run Metal. The latest features, such as tile shading
and imageblocks, will only run on the A11 (or later) chipset. The projects will build
and run on macOS, and most of them will run on the iOS Simulator, so using an
iOS device is optional. If you wish to make an iOS game, the game engine you build
while reading this book will have an iOS target as well. The Metal API, with a few
exceptions, works the same on macOS as it does on iOS so it won’t be difficult to
add an iOS target to your project later on.

raywenderlich.com

17


L

Book License

By purchasing Metal by Tutorials, you have the following license:
• You are allowed to use and/or modify the source code in Metal by Tutorials in as
many apps as you want, with no attribution required.

• You are allowed to use and/or modify all art, images and designs that are included
in Metal by Tutorials in as many apps as you want, but must include this
attribution line somewhere inside your app: “Artwork/images/designs: from Metal
by Tutorials, available at www.raywenderlich.com”.
• The source code included in Metal by Tutorials is for your personal use only. You
are NOT allowed to distribute or sell the source code in Metal by Tutorials without
prior authorization.
• This book is for your personal use only. You are NOT allowed to sell this book
without prior authorization, or distribute it to friends, coworkers or students; they
would need to purchase their own copies.
All materials provided with this book are provided on an “as is” basis, without
warranty of any kind, express or implied, including but not limited to the warranties
of merchantability, fitness for a particular purpose and noninfringement. In no event
shall the authors or copyright holders be liable for any claim, damages or other
liability, whether in an action of contract, tort or otherwise, arising from, out of or in
connection with the software or the use or other dealings in the software.
All trademarks and registered trademarks appearing in this guide are the properties
of their respective owners.

raywenderlich.com

18


B

Book Source Code &
Forums

If you bought the digital edition

The digital edition of this book comes with the source code for the starter and
completed projects for each chapter. These resources are included with the digital
edition you downloaded from .

If you bought the print version
You can get the source code for the print edition of the book here:
• />
Forums
We’ve also set up an official forum for the book at forums.raywenderlich.com.This is
a great place to ask questions about the book or to submit any errors you may find.

Digital book editions
We have a digital edition of this book available in both ePUB and PDF, which can be
handy if you want a soft copy to take with you, or you want to quickly search for a
specific term within the book.
Buying the digital edition version of the book also has a few extra benefits: free
updates each time we update the book, access to older versions of the book, and you
can download the digital editions from anywhere, at anytime.

raywenderlich.com

19


Metal by Tutorials

Book Source Code & Forums

Visit our book store page here:
• />And if you purchased the print version of this book, you’re eligible to upgrade to the

digital editions at a significant discount! Simply email with
your receipt for the physical copy and we’ll get you set up with the discounted digital
edition version of the book.

raywenderlich.com

20


B

Book Updates

Since you’ve purchased the digital edition version of this book, you get free access to
any updates we may make to the book!
The best way to get update notifications is to sign up for our monthly newsletter.
This includes a list of the tutorials that came out on raywenderlich.com that month,
any important news like book updates or new books, and a list of our favorite iOS
development links for that month. You can sign up here:
• www.raywenderlich.com/newsletter

raywenderlich.com

21


I

Introduction

Welcome to Metal by Tutorials!
Metal is a unified, low-level, low-overhead application programming interface (API)
for the graphics processing unit, or GPU. It’s unified because it applies to both 3D
graphics and data-parallel computation paradigms. Metal is a low-level API because
it provides programmers near-direct access to the GPU. Finally, Metal is a lowoverhead API because it reduces the runtime cost by multi-threading and precompiling of resources.
But beyond the technical definition, Metal is the most appropriate way to use the
GPU’s parallel processing power to visualize data or solve numerical challenges. It’s
also tailored to be used for machine learning, image/video processing or, as this book
describes, graphics rendering.

About this book
This book introduces you to low-level graphics programming in Metal — Apple’s
framework for programming on the graphics processing unit (GPU). As you progress
through this book, you’ll learn many of the fundamentals that go into making a game
engine and gradually put together your own engine. Once your game engine is
complete, you’ll be able to put together 3D scenes and program your own simple 3D
games. Because you’ll have built your 3D game engine from scratch, you’ll be able to
customize every aspect of what you see on your screen.

raywenderlich.com

22


Metal by Tutorials

Introduction

How did Metal come to life?
Historically, you had two choices to take advantage of the power of the GPU: OpenGL

and the Windows-only DirectX. In 2013, the GPU vendor AMD announced the Mantle
project in an effort to revamp GPU APIs and come up with an alternative to Direct3D
(which is part of DirectX) and OpenGL. AMD were the first to create a true lowoverhead API for low-level access to the GPU. Mantle promised to be able to generate
up to 9 times more draw calls (the number of objects drawn to the screen) than
similar APIs and also introduced asynchronous command queues so that graphics
and compute workloads could be run in parallel. Unfortunately, the project was
terminated before it could become a mainstream API.
Metal was announced at the Worldwide Developers Conference (WWDC) on June 2,
2014 and was initially made available only on A7 or newer GPUs. Apple created a new
language to program the GPU directly via shader functions. This is the Metal
Shading Language (MSL) based on the C++11 specification. A year later at WWDC
2015, Apple announced two Metal sub-frameworks: MetalKit and Metal
Performance Shaders (MPS). In 2018, MPS made a spectacular debut as a Ray
Tracing accelerator.
The API has continued to evolve, and WWDC 2017 introduced an exciting new
version of the API: Metal 2. Metal 2 adds support for Virtual Reality (VR),
Augmented Reality (AR) and accelerated machine learning (ML), among many new
features. Fall 2017 brought new updates to Metal including image blocks, tile
shading and threadgroup sharing, which are available on iOS devices powered by the
A11 Bionic chip, which comes with the first GPU ever designed in-house by Apple.
MSL was also updated to version 2.0 in Fall 2017 and is now based on the C++14
specification.

Why would you use Metal?
Metal is a top-notch graphics API. That means Metal can empower graphics pipelines
and, more specifically, game engines such as the following:
• Unity and Unreal Engine: The two leading cross-platform game engines today
are ideal for game programmers who target a range of console, desktop and mobile
devices. However, these engines haven’t always kept pace with new features in
Metal. For example, Unity announced that tessellation on iOS was to be released in

2018, despite it being demonstrated live at WWDC 2016. If you to use cutting-edge
Metal developments, you can’t always depend on third-party engines.
raywenderlich.com

23


Metal by Tutorials

Introduction

• The Witness: This award-winning puzzle game has a custom engine that runs on
top of Metal. By taking advantage of Metal, the iPad version is every bit as
stunning as the desktop version and is highly recommended for puzzle game fans.
• Many Others: From notable game titles such as Hitman, BioShock, Deus Ex, Mafia,
Starcraft, World of Warcraft, Fortnite, Unreal Tournament, Batman and even the
beloved Minecraft.
But Metal isn’t limited to the world of gaming. There are many apps that benefit
from GPU acceleration for image and video processing:
• Procreate: An app for sketching, painting and illustrating. Since converting to
Metal, it runs four times faster than it did before.
• Astropad: An app for drawing using Apple Pencil. The most notable
improvements since adding Metal show a 30% increase over wifi and two times
faster overall performance on most Macs.
• Pixelmator: A Metal-based app that provides image distortion tools. In fact, they
were able to implement a new painting engine and dynamic paint blending
technology powered by Metal 2.
• Affinity Photo: A recent release, available on the iPad. According to the developer
Serif, “Using Metal allows users to work easily on large, super high-resolution
photographs, or complex compositions with potentially thousands of layers.”

Metal, and in particular, the MPS sub-framework, is incredibly useful in the realm of
machine and deep learning on convolutional neural networks. Apple presented a
practical machine learning application at WWDC 2016 that demonstrated the power
of CNNs in high-precision image recognition.

When should you use Metal?
GPUs belong to a special class of computation that Flynn’s taxonomy terms Single
Instruction Multiple Data (SIMD). Simply, GPUs are processors that are optimized for
throughput (how much data can be processed in one unit of time), while CPUs are
optimized for latency (how much time it takes a single unit of data to be processed).
Most programs execute serially: they receive input, process it, provide output and
then the cycle repeats.
Those cycles sometimes perform computationally-intensive tasks, such as large
matrix multiplication, which would take CPUs a lot of time process serially, even in a
multithreaded manner on a handful of cores.
raywenderlich.com

24


Metal by Tutorials

Introduction

In contrast, GPUs have hundreds or even thousands of cores which are smaller and
have less memory than CPU cores, but perform fast parallel mathematical
calculations.
Choose Metal when:
• You want to render 3D models as efficiently as possible.
• You want your game to have its own unique style, perhaps with custom lighting

and shading.
• You will be performing intensive data processes, such as calculating and changing
the color of each pixel on the screen every frame, as you would when processing
images and video.
• You have large numerical problems, such as scientific simulations, that you can
partition into independent sub-problems to be processed in parallel.
• You need to process multiple large datasets in parallel, such as when you train
models for deep learning.

Who this book is for
This book is for intermediate Swift developers interested in learning 3D graphics or
gaining a deeper understanding of how game engines work. If you don’t know Swift,
you can still follow along, as all the code instructions are included in the book. You’ll
gain general graphics knowledge, but it would be less confusing if you cover Swift
basics first. We recommend the Swift Apprentice book, available from our store:
/>A smattering of C++ knowledge would be useful too. The Metal Shader Language that
you’ll use when writing GPU shader functions is based on C++. But, again, all the
code you’ll need is included in the book.

How to read this book
If you’re a beginner to iOS/macOS development or Metal, you should read this book
from cover to cover.
If you’re an advanced developer, or already have experience with Metal, you can skip
from chapter to chapter or use this book as a reference.
raywenderlich.com

25