3D Graphics with OpenGL ES and M3G- P12 pps

94 LOW-LEVEL RENDERING CHAPTER 3
both depth fail and pass). A very advanced use case for stenciling is volumetric shadow
casting [Hei91].
Depth test
Depth testing is used for hidden surface removal: the depth value of the incoming frag-
ment is compared against the one already stored at the pixel, and if the comparison
fails, the fragment is discarded. If the comparison function is LESS only fragments with
smaller depth value than already in the depth buffer pass; other fragments are discarded.
This can be seen in Figure 3.2, where the translucent object is clipped to the depth values
wr itten by the opaque object. The passed fragments continue along the pipeline and are
eventually committed to the frame buffer.
There are other ways of determining the visibility. Conceptually the simplest approach is
the painter’s algorithm, which sorts the objects into a back-to-front order from the camera,
and renders them so that a closer object always draws over the previous, farther objects.
There are several drawbacks to this. The sorting may require significant extra time and
space, particularly if there are a lot of objects in the scene. Moreover, sorting the prim-
itives simply does not work when the primitives interpenetrate, that is, a triangle pokes
through another. If you instead sort on a per-pixel basis using the depth buffer, visibility
is always resolved correctly, the storage requirements are fixed, and the running time is
proportional to the screen resolution rather than the number of objects.
With depth buffer ing it may make sense to have at least a partial front-to-back rendering
order, the opposite that is needed without a depth buffer. This way most fragments that
are behind other objects will be discarded by the depth test, avoiding a lot of useless frame
buffer updates. At least blending and writing to the frame buffer can be avoided, but
some engines even perform texture mapping and fogging only after they detect that the
fragment survives the depth test.
Depth offset
As already discussed in Section 2.5.1, the depth buffer has only a finite resolution. Deter-
mining the correct depth ordering for objects that are close to each other but not close to
the near frustum plane may not always be easy, and may result in z-fighting, as shown in
Figure 2.11. Let us examine why this happens.

Figure 3.22 shows a situation where two surfaces are close to each other, and how the
distance between them along the viewing direction increases with the slope or slant of the
surfaces. Let us interpret the small squares as pixel extents (in the horizontal direction as
one unit of screen x, in the vertical direction as one unit of depth buffer z), and study the
image more carefully. On the left, no matter where on the pixel we sample the surfaces, the
lower surface always has a higher depth value, but at this z-resolution and at this particular
depth, both will have the same quantized depth value. In the middle image, if the lower
surface is sampled at the left end of the pixel and the higher surface at the right end, they
SECTION 3.5 PER-FRAGMENT OPERATIONS 95
z
x
z
x
z
x
Figure 3.22: The slope needs to be taken into account with polygon offset. The two lines are two
surfaces close to each other, the arrow shows the viewing direction, and the coordinate axes illustrate
x and z axis orientations. On the left, the slope of the surfaces with respect to the viewing direction
is zero. The slope grows to 1 in the middle, and to about 5 on the right. The distance between the
surfaces along the viewing direction also grows as the slope increases.
will have the same depth. On the rightmost image, the depth order might be inverted
depending on where the surfaces are evaluated. In general, due to limited precisions in the
depth buffer and transformation arithmetic, if two surfaces are near each other, but have
different vertex values and different transformations, it is almost random which surface
appears in the front at any given pixel.
The situation in Figure 2.11 is contrived, but z-fighting can easily occur in real applica-
tions, too. For example, in a shooter game, after you spray a wall with bullets, you may
want to paint bullet marks on top of the wall. You would try to align the patches with the
wall, but want to guarantee that the bullet marks will resolve to be on top. By adding a
polygon offset, also known as depth offset, to the bullet marks, you can help the rendering

engine to determine the correct order. The depth offset is computed as
d = m · factor + units,
(3.13)
where m is the maximum depth slope of the polygon, computed by the rendering engine
for each polygon, while factor and units are user-given constants.
3.5.2 BLENDING
Blending takes the incoming fragment color (the source color) and the current value in
the color buffer (the destination color) and mixes them. Typically the value in the alpha
channel determines how the blending is done.
Some systems do not reserve storage for alpha in the color buffer, and do not therefore
support a destination alpha. In such a case, all computations assume the destination alpha
to be 1, allowing all operations to produce meaningful results. If destination alpha is sup-
ported, many advanced compositing effects become possible [PD84].
96 LOW-LEVEL RENDERING CHAPTER 3
Two interpretations of alpha
The transparency, or really opacity (alpha=1typically means opaque, alpha = 0, transpar-
ent) described by alpha has two different interpretations, as illustrated in Figure 3.23. One
interpretation is that the pixel is partially covered by the fragment, and the alpha denotes
that coverage value. Both in the leftmost image and in the middle image two triangles
each cover about one-half of the pixel. On the left the triangle orientations are indepen-
dent from each other, and we get the expected coverage value of 0.5 + 0.5 · 0.5 = 0.75,
as the first fragment covers one-half, and the second is expected to cover also one-half of
what was left uncovered. However, if the triangles are correlated, the total coverage can
be anything between 0.5 (the two polygons overlap each other) and 1.0 (the two triang les
abut, as in the middle image).
The other interpretation of alpha is that a pixel is fully covered by a transparent film that
adds a factor of alpha of its own color and lets the rest (one minus alpha) of the existing
color to show through, as illustrated on the right of Figure 3.23. In this case, the total
opacity is also 1 − 0.5 · 0.5 = 0.75.
These two interpretations can also be combined. For example, when drawing transparent,

edge-antialiased lines, the alpha is less than one due to transparency, and may be further
reduced by partial coverage of a pixel.
Blend equations and factors
The basic blend equation adds the source and destination colors using blending factors,
producing C = C
s
S + C
d
D. The basic blending uses factors (S, D) =(SRC_ALPHA,
ONE_MINUS_SRC_ALPHA). That is, the alpha component of the incoming fragment
determines how much of the new surface color is used, e.g., 0.25, and the remaining
Figure 3.23: Left: Two opaque polygons each cover half of a pixel, and if their orientations are
random, the chances are that 0.75 of the pixel will be covered.
Center: If it is the same polygon
drawn twice, only half of the pixel should be covered, whereas if the polygons abut as in the image,
the whole pixel should be covered.
Right: Two polygons with 50% opacity fully cover the pixel, creating
a compound film with 75% opacity.
SECTION 3.5 PER-FRAGMENT OPERATIONS 97
portion comes from the destination color already in the color buffer, e.g., 1.0 − 0.25 =
0.75. This kind of blending is used in the last image in Figure 3.2.
There are several additional blending factors that may be used. The simplest ones are
ZERO and ONE where all the color components are multiplied with 0 or 1, that is,
either ignored or taken as is. One can use either the destination or source alpha, or
one minus alpha as the blending factor (SRC_ALPHA, ONE_MINUS_SRC_ALPHA,
DST_ALPHA, ONE_MINUS_DST_ALPHA). Using the ONE_MINUS version flips the
meaning of opacity to transparency and vice versa.
With all the factors described so far, the factors for each of the R, G, B, and A channels
are the same, and they can be applied to both source or destination colors. However, it is
also possible to use the complete 4-component color as the blending factor, so that each

channel gets a unique factor. For example, using SRC_COLOR as the blending factor for
destination color produces ( R
s
R
d
, G
s
G
d
, B
s
B
d
, A
s
A
d
). In OpenGL ES, SRC_COLOR and
ONE_MINUS_SRC_COLOR are legal blending factors only for destination color, while
DST_COLOR and ONE_MINUS_DST_COLOR can only be used with the source color.
Finally, SRC_ALPHA_SATURATE can be used with the source color, producing a blend-
ing factor (f, f, f, 1) where f = min(A
s
, 1 − A
d
).
Here are some examples of using the blending factors. The default rendering that does not
use blending is equivalent to using (ONE, ZERO) as the (src, dst) blending factors. To add
a layer with 75% transparency, use 0.25 as the source alpha and select the (SRC_ALPHA,
ONE_MINUS_SRC_ALPHA) blending factors. To equally mix n layers, set the factors to

(SRC_ALPHA, ONE) and render each layer with alpha = 1/n.Todrawacoloredfilteron
top of the frame, use (ZERO, SRC_COLOR).
A later addition to OpenGL, which is also available in some OpenGL ES implementa-
tions through an extension,
2
allows you to subtract C
s
S from C
d
D and vice versa. Another
extension
3
allows you to define separate blending factors for the color (RGB) and alpha
components.
Rendering transparent objects
OpenGL renders primitives in the same order as they are sent to the engine. With depth
buffering, one can use an arbitrary rendering order, as the closest surface will always
remain visible. However, for correct results in the presence of transparent surfaces in
the scene, the objects should be rendered in a back-to-front order. On the other hand,
this is usually the slowest approach, since pixels that will be hidden by opaque objects
are unnecessarily rendered. The best results, in terms of both perfor mance and quality,
are obtained if you sort the objects, render the opaque objects front-to-back with depth
2 OES_blend_subtract
3 OES_blend_func_separate
98 LOW-LEVEL RENDERING CHAPTER 3
testing and depth writing tur ned on, then turn depth write off and enable blending, and
finally draw the transparent objects in a back-to-front order.
To see why transparent surfaces need to be sorted, think of a white object behind blue
glass, both of which are behind red glass, both glass layers being 50% transparent. If you
draw the blue glass first (as you should) and then the red glass, you end up with more red

than blue: (0.75, 0.25, 0.5), whereas if you draw the layers in opposite order you get more
blue: (0.5, 0.25, 0.75).
As described earlier, if it is not feasible to separate transparent objects from opaque objects
otherwise, you can use the alpha test to render them in two passes.
Multi-pass rendering
The uses of blending are not limited to rendering translucent objects and compositing
images on top of the background. Multi-pass rendering refers to techniques where objects
and materials are synthesized by combining multiple rendering passes, typically of the
same geometry, to achieve the final appearance. Blending is a fundamental requirement
for all hardware-accelerated multi-pass rendering approaches, though in some cases the
blending machinery of texture mapping units can be used instead of the later blending
stage.
An historical example of multi-pass rendering is light mapping, discussed in Section 3.4.3:
back in the days of old, when graphics hardware only used to have a single texture unit,
light mapping could be implemented by rendering the color texture and light map tex-
ture as separate passes with (DST_COLOR, ZERO)or(ZERO, SRC_COLOR) blend-
ing in between. However, this is the exact same operation as combining the two using
a MODULATE texture function, so you will normally just use that if you have multi-
texturing capability.
While multi-texturing and multi-pass rendering can substitute for each other in simple
cases, they are more powerful combined. Light mapping involves the single operation AB,
which is equally doable with either multi-texturing or multi-pass rendering. Basically, any
series of operations that can be evaluated in a straightforward left-to-right order, such
as AB + C, can be decomposed into either texturing stages or rendering passes. More
complex operations, requiring one or more intermediate results, can be decomposed into
a combination of multi-texturing and multi-pass rendering: AB + CD can be satisfied
with two multi-textured rendering passes, AB additively blended with CD.
While you can render an arbitrary number of passes, the number of texture units quickly
becomes the limiting factor when proceeding toward more complex shading equations.
This can be solved by storing intermediate results in textures, either by copying the frame

buffer contents after rendering an intermediate result or by using direct render-to-texture
capability.
Multi-pass rendering, at least in theory, makes it possible to construct arbitrarily complex
rendering equations from the set of basic blending and texturing operations. This has
SECTION 3.5 PER-FRAGMENT OPERATIONS 99
been demonstrated by systems that translate a high-level shading language into OpenGL
rendering passes [POAU00, PMTH01]. In practice, the computation is limited by the
numeric accuracy of the individual operations and the intermediate results: with 8 bits
per channel in the frame buffer, rounding errors accumulate fast enough that great care
is needed to maximize the number of useful bits in the result.
3.5.3 DITHERING, LOGICAL OPERATIONS, AND MASKING
Before the calculated color at a pixel is committed to the frame buffer, there are two more
processing steps that can be taken: dithering and logical operations. Finally, writing to
each of the different buffers can also be masked, that is, disabled.
Dithering
The human eye can accommodate to great changes in illumination: the ratio of the light
on a bright day to the light on a moonless overcast night can be a billion to one. With a
fixed lighting situation, the eye can distinguish a much smaller range of contrast, perhaps
10000 :1. However, in scenes that do not have ver y bright lights, 8 bits, or 256 levels, are
sufficient to produce color transitions that appear continuous and seamless. Since 8 bits
also matches pretty well the limits of current displays, and is a convenient unit of storage
and computation on binary computers, using 8 bits per color channel on a display is a
typical choice on a desktop.
Some displays cannot even display all those 256 levels of intensity, and some frame buffers
save in memory costs by storing fewer than 8 bits per channel. Having too few bits avail-
able can lead to banding. Let us say you calculate a color channel at 8 bits where values
range from 0 to 255, but can only store 4 bits with a range from 0 to 15. Now all values
between 64 and 80 (0100000 and 0101000 in binary) map to either 4 or 5 (0100 or 0101).
If you simply quantize the values in an image where the colors vary smoothly, so that
values from 56 to 71 map to 4 and from 72 to 87 map to 5, the flat areas and the sudden

jumps between them become obvious to the viewer. However, if you mix pixels of values
4 and 5 at roughly equal amounts where the original image values are around 71 or 72,
the eye fuses them together and interprets them as a color between 4 and 5. This is called
dithering, and is illustrated in Figure 3.24.
Figure 3.24: A smooth ramp (left) is quantized (middle) causing banding. Dithering (right) produces
smoother transitions even though individual pixels are quantized.
100 LOW-LEVEL RENDERING CHAPTER 3
OpenGL allows turning dithering on and off per drawing command. This way, internal
computations can be calculated at a higher precision, but color ramps are dithered just
after blending and before committing to the frame buffer.
Another approach to dithering is to have the internal frame buffer at a higher resolution
than the display color depth. In this case, dithering takes place only when the frame is
complete and is sent to the display. This allows allows reasonable results even on displays
that only have a single bit per pixel, such as the monochrome displays of some low-end
mobile devices, or newspapers printed with only black ink. In such situations, dithering
is absolutely required so that any impression of continuous intensity v ariations can be
conveyed.
Logical operations
Logical operations,orlogic ops for shor t, are the last processing stage of the OpenGL
graphics pipeline. They are mutually exclusive with blending. With logic ops, the source
and destination pixel data are considered bit patterns, rather than color values, and a logi-
cal operation such as AND, OR, XOR, etc., is applied between the source and the destination
before the values are stored in the color buffer.
In the past, logical operations were used, for example, to draw a cursor w ithout having
to store the background behind the cursor. If one draws the cursor shape with XOR, then
another XOR will erase it, reinstating the original background. OpenGL ES 1.0 and 1.1
support logical operations as they are fast to implement in software renderers and allow
some special effects, but both M3G and OpenGL ES 2.0 omit this functionality.
Masking
Before the fragment values are actually stored in the frame buffer, the different data fields

can be masked. Writing into the color buffer can be turned off for each of red, green, blue,
or alpha channels. The same can be done for the depth channel. For the stencil buffer, even
individual bits may be masked before writing to the buffer.
3.6 LIFE CYCLE OF A FRAME
Now that we have covered the whole low-level 3D graphics pipeline, let us take a look at
the full life cycle of an application and a frame.
In the beginning of an application, resources have to be obtained. The most important
resource is the frame buffer. This includes the color buffer, how many bits there are for
each color channel, existence and bit depth of the alpha channel, depth buffer, s tencil
buffer, and multisample buffers. The geometry data and texture maps also require mem-
or y, but those resources can be allocated later.
SECTION 3.6 LIFE CYCLE OF A FRAME 101
The viewport transformation and projection matrices describe the type of camera that is
being used, and are usually set up only once for the whole application. The modelview
matrix, however, changes whenever something moves, whether they are objects in the
scene or the camera viewing the scene.
After the resources have been obtained and the fixed parameters set up, new frames are
rendered one after another. In the beginning of a new frame, the color, depth, and other
buffers are usually cleared. We then render the objects one by one. Before rendering each
object, we set up its rendering state, including the lights, texture maps, blending modes,
and so on. Once the frame is complete, the system is told to display the image. If the
rendering was quick, it may make sense to wait for a while before starting the next frame,
instead of rendering as many frames as possible and using too much power. This cycle
is repeated until the application is finished. It is also possible to read the contents of the
frame buffer into user memory, for example to grab screen shots.
3.6.1 SINGLE VERSUS DOUBLE BUFFERING
In a simple graphics system there may be only a single color buffer, into which new
graphics is drawn at the same time as the display is refreshed from it. This single buffer-
ing has the benefits of simplicity and lesser use of graphics memory. However, even if the
graphics drawing happens very fast, the rendering and the display refresh are usually not

synchronized with each other, which leads to annoying tearing and flickering.
Double buffering avoids tearing by rendering into a back buffer and notifying the sys-
tem when the frame is completed. The system can then synchronize the copying of the
rendered image to the display with the display refresh cycle. Double buffering is the
recommended way of rendering to the screen, but single-buffering is still useful for off-
screen surfaces.
3.6.2 COMPLETE GRAPHICS SYSTEM
Figure 3.25 presents a conceptual high-level model of a graphics system. Applications
run on a CPU, which is connected to a GPU with a first-in-first-out (FIFO) buffer. The
GPU feeds pixels into various frame buffers of different APIs, from which the display
subsystem composites the final displayed image, or which can be fed back to graphics
processing through the texture-mapping unit. The Graphics Dev ice Interface (GDI) block
implements functionality that is typically present in 2D graphics APIs of the operating
systems. The Compositor block handles the mixing of different types of content surfaces
in the system, such as 3D rendering surfaces and native OS graphics.
Inside the GPU a command processor processes the commands coming from the CPU
to the 2D or 3D graphics subsystems, which may again be buffered. A typical 3D subsys-
tem consists of two executing units: a vertex unit for transformations and lighting, and
a fragment unit for the rear end of the 3D pipeline. Real systems may omit some of the
components; for example, the CPU may do more (even all) of the graphics processing,
102 LOW-LEVEL RENDERING CHAPTER 3
CPU FIFO
FIFO
FIFO
FIFO
FIFO
GPU
OpenVG
Composition
Display

Command
processor
Vertex
Unit
Fragment
Unit
Texture
Memory
Graphics Device Interface (GDI)
BUFFER
BUFFER
BUFFER
Figure 3.25: A conceptual model of a graphics system.
some of the FIFO buffers may be direct unbuffered bus connections, or the compositor
is not needed if the 3D subsystem executes in a full-screen mode. Nevertheless, look-
ing at the 3D pipeline, we can separate roughly four main execution stages: the CPU,
the vertex unit that handles transformations and lighting (also known as the geometry
unit), the rasterization and fragment-processing unit (pixel pipeline), and the display
composition unit.
Figure 3.26 shows an ideal case when all four units can work in parallel. While the CPU
is processing a new frame, the vertex unit performs geometry processing for the previ-
ous frame, the rasterization unit works on the frame before that, and the display subunit
displays a frame that was begun three frames earlier. If the system is completely balanced,
and the FIFOs are large enough to mask temporary imbalances, this pipelined system
can produce images four times faster than a fully sequential system such as the one in
Figure 3.27. Here, one opportunity for parallelism vanishes from the lack of double buffer-
ing, and all the stages in general wait until the others have completed their frame before
proceeding with the next frame.
3.6.3 SYNCHRONIZATION POINTS
We call the situation where one unit of the graphics system has to wait for the input of a

previous unit to complete, or even the whole pipeline to flush, a synchronization point.
Even if the graphics system has been designed to be able to execute fully in parallel, use of
certain API features may create a synchronization point. For example, if the application
asks to read back the current frame buffer contents, the CPU has to stall and wait until
all the previous commands have fully executed and have been committed into the frame
buffer. Only then can the contents be delivered to the application.
Another synchronization point is caused by binding the rendering output to a texture
map. Also, creating a new texture map and using it for the first time may create a bottle-
neck for transferring the data from the CPU to the GPU and organizing it into a format
SECTION 3.6 LIFE CYCLE OF A FRAME 103
CPU
T&L
Rasterizer
Flip
N
N
N
N
N 2 1
N 1 1N 1 2N 1 3
N 1 1N 1 2N 1 3
N 1 1N 1 2
N 1 2N 1 2N 1 3
N 1 3N 2 2
N 2 3N 2 2
N 2 1
N 2 1
Figure 3.26: Parallelism of asynchronous multibuffered rendering.
CPU
T&L

Rasterizer N
NN
1 1
NN 1 1
N
1 1
Figure 3.27: Nonparallel nature of single-buffered or synchronized rendering.
that is native to the texturing unit. A similar synchronization point can result from the
modification of an existing texture map.
In general, the best performance is obtained if each hardware unit in the system executes
in parallel. The first rule of thumb is to keep most of the traffic flowing in the same direc-
tion, and to query as little data as possible back from the graphics subsystem. If you must
read the results back, e.g., if you render into a texture map, delaying the use of that data
until a few frames later may help the system avoid stalling . You should also use server-
side objects wherever possible, as they allow the data to be cached on the GPU. For best
performance, such cached data should not be changed after it has been loaded. Finally,
you can try to increase parallelism, for example, by executing application-dependent CPU
processing immediately after GPU-intensive calls such as clear ing the buffers, drawing a
large textured mesh, or swapping buffers. Another way to improve parallelism is to move
non-graphics–related processing into another thread altogether.