Microsoft SQL Server 2008 R2 Unleashed- P118 doc

ptg
1114
CHAPTER 34 Data Structures, Indexes, and Performance
If SQL Server had to search throughout an entire database file to find free extents, it would-
n’t be efficient. Instead, SQL Server uses two special types of pages to record which extents
have been allocated to tables or indexes and whether it is a mixed or uniform extent:
. Global allocation map pages (GAMs)
. Shared global allocation map pages (SGAMs)
Global and Shared Global Allocation Map Pages
The allocation map pages track whether extents have been allocated to objects and
indexes and whether the allocation is for mixed extents or uniform extents. As mentioned
in the preceding section, there are two types of GAMs:
. Global allocation map (GAM)—The GAM keeps track of all allocated extents in a
database, regardless of what it’s allocated to. The structure of the GAM is straightfor-
ward: each bit in the page outside the page header represents one extent in the file,
where 1 means that the extent is not allocated, and 0 means that the extent is allo-
cated. Nearly 8,000 bytes (64,000 bits) are available in a GAM page after the header
and other overhead bytes are taken into account. Therefore, a single GAM covers
approximately 64,000 extents, or 4GB (64,000 * 64KB) of data.
. Shared global allocation map (SGAM)—The SGAM keeps track of mixed extents
that have free space available. An SGAM has a structure similar to a GAM, with each
bit representing an extent. A value of 1 means that the extent is a mixed extent and
there is free space (at least one unused page) available on the extent. A value of 0
means that the extent is not currently allocated, that the extent is a uniform extent,
or that the extent is a mixed extent with no free pages.
Table 34.6 summarizes the meaning of the bit in GAMs and SGAMs.
When SQL Server needs to allocate a uniform extent, it simply searches the GAM for a bit
with a value of 1 and sets it to 0 to indicate it has been allocated. To find a mixed extent
with free pages, it searches the SGAM for a bit set to 1. When all pages in a mixed extent
are used, its corresponding bit is set to 0. When a mixed extent needs to be allocated, SQL
Server searches the GAM for an extent whose bit set to 1 and sets the bit to 0, and the

corresponding SGAM bit is set to 1. There is some more processing involved as well, such
as spreading the data evenly across database files, but the allocation algorithms are still
relatively simple.
TABLE 34.6 Meaning of the GAM and SGAM Bits
Extent Usage GAM Bit SGAM Bit
Free, not used
1 0
Uniform or mixed with no free pages
0 0
Mixed, with free pages available
0 1
Download from www.wowebook.com
ptg
1115
Space Allocation Structures
34
SQL Server is able to easily locate GAM pages in a database because the first GAM page is
located at the third page in the file (page number 2). There is another GAM every 511,230
pages after the first GAM. The fourth page (page number 3) in each database file is the
SGAM page, and there is another SGAM each 511,230 pages after the first SGAM.
Page Free Space Pages
A page free space (PFS) page records whether each page is allocated and the amount of free
space available on the page. Each PFS covers 8,088 contiguous pages in the file. For each
of the 8,088 pages, the PFS has a 1-byte record that contains a bitmap for each page indi-
cating whether the page is empty, 1 to 50% full, 51 to 80% full, 81 to 95% full, or more
than 95% full. The first PFS page in a file is located at page number 1, the second PFS page
is located at page 8088, and each additional PFS page is located every 8,088 pages after
that. SQL Server uses PFS pages to find free pages on extents and to find pages with space
available on extents when a new row needs to be added to a table or index.
Figure 34.6 shows the layout of GAM, SGAM, and PFS pages in a database file. Note that

every file has a single file header located at page 0.
Index Allocation Map Pages
Index allocation map (IAM) pages keep track of the extents used by a heap or index. Each
heap table and index has at least one IAM page for each file where it has extents. An IAM
cannot reference pages in other database files; if the heap or index spreads to a new data-
base file, a new IAM for the heap or index is created in that file. IAM pages are allocated as
needed and are spread randomly throughout the database files.
An IAM page contains a small header that has the address of the first extent in the range
of pages being mapped by the IAM. It also contains eight page pointers that keep track of
index or heap pages that are in mixed extents. These pointers might or might not contain
any information, depending on whether any data has been deleted from the tables and
the page(s) released. Remember, an index or heap will have no more than eight pages in
mixed extents (after eight pages, it begins using uniform extents), so only the first IAM
page stores this information. The remainder of the IAM page is for the allocation bitmap.
The IAM bitmap works similarly to the GAM, indicating which extents over the range of
extents covered by the IAM are used by the heap or index the IAM belongs to. If a bit is
on, the corresponding extent is allocated to the table.
Each IAM covers a possible range of 63,903 extents (511,224 pages), covering a 4GB
section of a file. Each bit represents an extent within that range, whether or not the
Page 0
File
Heaader
Page 2
GAM
Page
Page 3
SGAM
Page
Page
8089

PFS
Page
…
Page 1
PFS
Page
Page
16177
PFS
Page
…
Page
509545
PFS
Page
Page
511232
GAM
Page
Page
511233
SGAM
Page
… … …
FIGURE 34.6 The layout of GAM, SGAM, and PFS pages in a database file.
Download from www.wowebook.com
ptg
1116
CHAPTER 34 Data Structures, Indexes, and Performance
extent is allocated to the object that the IAM belongs to. If the bit is set to 1, the relative

extent in the range is allocated to the index or heap. If the bit is set to 0, the extent is
either not allocated or might be allocated to another heap or index.
For example, assume that an IAM page resides at page 649 in the file. If the bit pattern in
the first byte of the IAM is 1010 0100, the first, third, and sixth extents within the range
of the IAM are allocated to the heap or index. The second, fourth, fifth, seventh, and
eighth extents are not.
NOTE
For a heap table, the data pages and rows within them are not stored in any specific
order. Unlike versions of SQL Server prior to 7.0, the pages in a heap structure are not
linked together in a page chain. The only logical connection between data pages is the
information recorded in the IAM pages, which are linked together. The structure of heap
tables is examined in more detail later in this chapter.
Differential Changed Map Pages
The seventh page (page number 6), and every 511,232
nd
page thereafter, in the database
file is the differential changed map (DCM) page. This page keeps track of which extents in a
file have been modified since the last full database backup. When an extent has been
modified, its corresponding bit in the DCM is turned on. This information is used when a
differential backup is performed on the database. A differential backup copies only the
extents changed since the last full backup was made. Using the DCM, SQL Server can
quickly tell which extents need to be backed up by examining the bits on the DCM pages
for each data file in the database. When a full backup is performed for the database, all
the bits are set back to
0.
Bulk Changed Map Pages
The eighth page (page number 7), and every 511,232
nd
page thereafter, in the database file
is the bulk changed map (BCM). When you perform a minimally or bulk-logged operation

in SQL Server 2008 in BULK_LOGGED recovery mode, SQL Server logs only the fact that the
operation occurred and doesn’t log the actual data changes. The operation is still fully
recoverable because SQL Server keeps track of what extents were actually modified by the
bulk operation in the BCM page. Similar to the DCM page, each bit on a BCM page repre-
sents an extent within its range, and if the bit is set to 1, that indicates that the corre-
sponding extent has been changed by a minimally logged bulk operation since the last
full database backup. All the bits on the BCM page are reset to 0 whenever a full database
backup or log backup occurs.
When you initiate a log backup for a database using the BULK_LOGGED recovery model, SQL
Server scans the BCM pages and backs up all the modified extents along with the contents
of the transaction log itself. You should be aware that the log file itself might be small, but
the backup of the log can be many times larger if a large bulk operation has been
performed since the last log backup.
Download from www.wowebook.com
ptg
1117
Data Compression
34
Data Compression
SQL Server 2008 introduced a new data compression feature that is available in Enterprise
and Datacenter Editions. Data compression helps to reduce both storage and memory
requirements as the data is compressed both on disk and when brought into the SQL
Server data cache.
When compression is enabled and data is written to disk, it is compressed and stored in
the designated compressed format. When the data is read from disk into the buffer cache,
it remains in its compressed format. This helps reduce both storage requirements and
memory requirements. It also reduces I/O because more data can be stored on a data page
when it’s compressed. When the data is passed to another component of SQL Server,
however, the Database Engine then has to uncompress the data on the fly. In other words,
every time data has to be passed to or from the buffered cache, it has to be compressed or

uncompressed. This requires extra CPU overhead to accomplish. However, in most cases,
the amount of I/O and buffer cache saved by compression more than makes up for the
CPU costs, boosting the overall performance of SQL Server.
Data compression can be applied on the following database objects:
. Tables (clustered or heap)
. Nonclustered indexes
. Indexed views
As the DBA, you need to evaluate which of the preceding objects in your database could
benefit from compression and then decide whether you want to compress it using either row-
level or page-level compression. Compression is enabled or disabled at the object level There is
no single option you can enable that turns compression on or off for all objects in the data-
base. Fortunately, other than turning compression on or off for the preceding objects, you
don’t have to do anything else to use data compression. SQL Server handles data compression
transparently without your having to re-architect your database or your applications.
Row-Level Compression
Row-level compression isn’t true data compression. Instead, space savings are achieved by
using a more efficient storage format for fixed-length data to use the minimum amount of
space required. For example, the int data type uses 4 bytes of storage regardless of the value
stored, even NULL. However, only a single byte is required to store a value of 100. Row-level
compression allows fixed-length values to use only the amount of storage space required.
Row-level compression saves space and reduces I/O by
. Reducing the amount of metadata required to store data rows
. Storing fixed-length numeric data types as if they were variable-length data types,
using only as many bytes as necessary to store the actual value
. Storing CHAR data types as variable-length data types
. Not storing NULL or 0 values
Download from www.wowebook.com
ptg
1118
CHAPTER 34 Data Structures, Indexes, and Performance

Row-level data compression provides less compression than page-level data compression,
but it also incurs less overhead, reducing the amount of CPU resources required to
implement it.
Row-level compression can be enabled when creating a table or index or using the ALTER
TABLE or ALTER INDEX commands by specifying the WITH (DATA_COMPRESSION = ROW)
option. The following example enables row compression on the titles table in the
bigpubs2008 database:
ALTER TABLE titles REBUILD WITH (DATA_COMPRESSION=ROW)
Additionally, if a table or index is partitioned, you can apply compression at the parti-
tion level.
When row-level compression is applied to a table, a new row format is used that is unlike
the standard data row format discussed previouslywhich has a fixed-length data section
separate from a variable-length data section (see Figure 34.3). This new row format is
referred to as column descriptor, or CD, format. The name of this row format refers to the
fact the every column has description information contained in the row itself. Figure 34.7
illustrates a representative view of the CD format (a definitive view is difficult because,
except for the header, the number of bytes in each region is completely dependent on the
values in the data row).
The row header is always 1 byte in length and contains information similar to Status Bits
A in a normal data row:
. Bit 0—This bit indicates the type of record (1 = CD record format).
. Bit 1—This bit indicates whether the row contains versioning information.
. Bits 2–4—This three-bit value indicates what kind of information is stored in the
row (such as primary record, ghost record, forwarding record, index record).
. Bit 5—This bit indicates whether the row contains a Long data region (with values
greater than 8 bytes in length).
. Bits 6 and 7—These bits are not used.
The CD region consists of two parts. The first is either a 1- or 2-byte value indicating the
number of short columns (8 bytes or less). If the most significant bit of the first byte is set
to

0, it’s a 1-byte field representing up to 127 columns; if it’s 1, it’s a 2-byte field represent-
ing up to 32,767 columns. Following the first 1 or 2 bytes is the CD array. The CD array
uses 4 bits for each column in the table to represent information about the length of the
Header
(1 byte)
CD Region Short Data Region
Long Data Region
Special Information
FIGURE 34.7 A representative structure of a CD format row.
Download from www.wowebook.com
ptg
1119
Data Compression
34
column. A bit representation of 0 indicates the column is NULL. A bit representation of the
values 1 to 9 indicates the column is 0 to 8 bytes in length, respectively. A bit representa-
tion of 10 (0xa) indicates that the corresponding column value is a long data value and
uses no space in the short data region. A bit representation of 11 (0xb) represents a bit
column with a value of 1, and a bit representation of 12 (0xc) indicates that the corre-
sponding value is a 1-byte symbol representing a value in the page compression dictionary
(the page compression dictionary is discussed next in the page-level compression section).
The short data region contains each of the short data values. However, because accessing
the last columns can be expensive if there are hundreds of columns in the table, columns
are grouped into clusters of 30 columns. At the beginning of the short data region, there is
an area called the short data cluster array. Each entry in the array is a single byte, which
indicates the sum of the sizes of all the data in the previous cluster in the short data
region; the value is essentially a pointer to the first column of the cluster (no row offset is
needed for the first cluster because it starts immediately after the CD region).
Any data value in the row longer than 8 bytes is stored in the long data region. This can
include LOB and row-overflow pointers. Long data needs an actual offset value to allow

SQL Server to locate each value. This offset array looks similar to the offset array used in
the
standard data row structure. The long data region consists of three parts: an offset
array, a long data cluster array, and the long data. The long data cluster array is similar to
the short data cluster array; it has one entry for each 30-column cluster (except for the last
one) and serves to limit the cost of locating columns near the end of a long list of columns.
The special information section at the end of the row contains three optional pieces of
information. The existence of any or all of this information is indicated by bits in the first
1-byte header at the beginning of the row. The three special pieces of information are
. Forwarding pointer—This pointer is used in a heap when a row is forwarded due
to an update (forward pointers are discussed later in this chapter).
. Back pointer—If the row is a forwarded row, it contains a pointer back to the origi-
nal row location.
. Versioning information—If snapshot isolation is being used, 14 bytes of version-
ing information are appended to the row.
Page-Level Compression
Page-level compression is an implementation of true data compression, using both column
prefix and dictionary-based compression. Data is compressed be storing repeating values or
common prefixes only once and then referencing those values from other columns and
rows. When you implement page compression for a table, row compression is applied as
well. Page-level compression offers increased data compression over row-level compression
alone but at the expense of greater CPU utilization. It works using these techniques:
. First, row-level data compression is applied to fit as many rows as it can on a
single page.
Download from www.wowebook.com
ptg
1120
CHAPTER 34 Data Structures, Indexes, and Performance
. Next, column prefix compression is run. Essentially, repeating patterns of data at the
beginning of the values of a given column are removed and substituted with an

abbreviated reference, which is stored in the compression information (CI) structure
stored after the page header.
. Finally, dictionary compression is applied on the page. Dictionary compression
searches for repeated values anywhere on a page and stores them in the CI.
Page compression is applied only after a page is full and if SQL Server determines that
compressing a page will save a meaningful amount of space.
The amount of compression provided by page-level data compression is highly dependent
on the data stored in a table or index. If a lot of the data repeats itself, compression is
more efficient. If the data is more randomly discrete values, fewer benefits are gained from
using page-level compression.
Column prefix compression looks at the column values on a single page and chooses a
common prefix that can be used to reduce the storage space required for values in that
column. The longest value in the column that contains the prefix is chosen as the anchor
value. A row that represents the prefix values for each column is created and stored in the
CI structure that immediately follows the page header. Each column is then stored as a
delta from the anchor value, where repeated prefix values in the column are replaced by a
reference to the corresponding prefix. If the value in a row does not exactly match the
selected prefix value, a partial match can still be indicated.
For example, consider a page that contains the following data rows before prefix compres-
sion as shown in Figure 34.8.
After you apply column prefix compression on the page, the CI structure is stored after
the page header holding the prefix values for each column. The columns then are stored
as the difference between the prefix and column value, as shown in Figure 34.9.
In the first column in the first data row, the value
4b represents that the first four charac-
ters of the prefix (aaab) are present at the beginning of the column for that row and also
the character b. If you append the character b to the first four values of the prefix, it
rebuilds the original value of aaabb. For any columns values that are [empty], the column
matches the prefix value exactly. Any column value that starts with 0 means that none of
the first characters of the column match the prefix. For the fourth column, there is no

common prefix value in the columns, so no prefix value is stored in the CI structure.
Page Header
aaabb aaaab abcd abc
aaabccc bbbbb abcd mno
aaaccc aaaacc bbbb xyz
Data Rows
FIGURE 34.8 Sample page of a table before prefix compression.
Download from www.wowebook.com
ptg
1121
Data Compression
34
aaabccc
Page Header
Data Rows
aaabccc aaabcccaaabccc
4b4b abcd[empty]
0bbbb[empty] mno[empty]
[empty]3ccc xyz0bbbb
FIGURE 34.9 Sample page of a table after prefix compression.
After column prefix compression is applied to every column individually on the page, SQL
Server then looks to apply dictionary compression. Dictionary compression looks for
repeated values anywhere on the page and also stores them in the CI structure after the
column prefix values. Dictionary compression values replace repeated values anywhere on
a page. The following illustrates the same page shown previously after dictionary compres-
sion has been applied:
The dictionary is stored as a set of these duplicate values and a symbol to represent these
values in the columns on the page. As you can see in this example, 4b is repeated in
multiple columns in multiple rows, and the value is replaced by the symbol 0 throughout
the page. The value 0bbbb is replaced by the symbol 1. SQL Server recognizes that the

value stored in the column is a symbol and not a data value by examining the coding in
the CD array, as discussed earlier.
Not all pages contain both the prefix record and a dictionary. Having them both depends
on whether the data has enough repeating values or patterns to warrant either a prefix
record or a dictionary.
Data Rows
Page Header
0 0 [empty] abcd
[empty] 1 [empty] mno
3ccc
[empty] 1 xyz
aaabccc aaaacc abcd [NULL]
4b 0bbbb
FIGURE 34.10 Sample page of a table after dictionary compression.
Download from www.wowebook.com
ptg
1122
CHAPTER 34 Data Structures, Indexes, and Performance
The CI Record
The CI record is the only main structural change to a page when it is page compressed
versus a page that uses row compression only. As shown in the previous examples, the CI
record is located immediately after the page header. There is no entry for the CI record in
the row offset table because its location is always the same. A bit is set in the page header
to indicate whether the page is page compressed. When this bit is present, SQL Server
knows to look for the CI record. The CI record contains the data elements shown in
Table 34.7.
Implementing Page Compression
Page compression can be implemented for a table at the time it is created or by using the
ALTER TABLE command, as in the following example:
ALTER TABLE sales_big REBUILD WITH (DATA_COMPRESSION=PAGE)

Unlike row compression, which is applied immediately on the rows, page compression
isn’t applied until the page is full. The rows cannot be compressed until SQL Server can
determine what encodings for prefix and dictionary substitution are going to be used to
replace the actual data. When you enable page compression for a table or a partition, SQL
Server examines every full page to determine the possible space savings. Any pages that
are not full are not considered for compression. During the compression analysis, the
prefix and dictionary values are created, and the column values are modified to reflect the
prefix and dictionary values. Then row compression is applied. If the new compressed
page can hold at least five additional rows, or 25% more rows than the page currently
TABLE 34.7 Data Elements Within the CI Record
Name Description
Header This structure contains 1 byte to keep track of information about the CI. Bit 0 is
the version (currently always 0), Bit 1 indicates the presence of a column prefix
anchor record, and Bit 2 indicates the presence of a compression dictionary.
PageModCount
This value keeps track of the number of changes to the page to determine
whether the compression on the page should be reevaluated and the CI record
rebuilt.
Offsets This element contains values to help SQL Server find the dictionary. It contains
the offset of the end of the Column prefix anchor record and offset of the end of
the CI record itself.
Anchor Record This record looks exactly like a regular CD record (see Figure 34.7). Values
stored are the common prefix values for each column, some of which might be
NULL.
Dictionary The first 2 bytes represent the number of entries in the dictionary, followed by
an offset array of 2-byte entries, which indicate the end offset of each dictionary
entry, and then the actual dictionary values.
Download from www.wowebook.com
ptg
1123

Data Compression
34
holds, the page is compressed. If neither one of these criteria is met, the compressed
version of the page is discarded.
New rows inserted into a compressed page are compressed as they are inserted. However,
new entries are not added to the prefix list or dictionary based on a single new row. The
prefix values and dictionary symbols are rebuilt only on an all-or-nothing basis. After the
page is changed a sufficient number of times, SQL Server evaluates whether to rebuild the
CI record. The PageModCount field in the CI record is used to keep track of the number of
changes to the page since the CI record was last built or rebuilt. This value is updated
every time a row is updated, deleted, or inserted. If SQL Server encounters a full page
during a data modification and the PageModCount is greater than 25 or the PageModCount
divided by the number of rows on the page is greater than 25%, SQL Server reapplies the
compression analysis on the page. Again, only if recompressing the page creates room for
five additional rows, or 25% more rows than the page currently holds, the new
compressed page replaces the existing page.
In B-tree structures (nonclustered indexes or a clustered table), only the leaf-level and data
pages are considered for compression. When you insert a new row into a leaf or data page,
if the compressed row fits, it is inserted and nothing more is done. If it doesn’t fit, SQL
Server attempts to recompress the page and then recompress the row based on the new CI
record. If the row fits after recompression, it is inserted and nothing more is done. If the
row still doesn’t fit, the page needs to be split. When a compressed page is split, the CI
record is copied to the new page exactly as it was, along with the rows moved to the new
page. However, the
PageModCount value is set to 25, so that when the new page gets full, it
will be immediately analyzed for recompression. Leaf and data pages are also checked for
recompression whenever you run an index rebuild or shrink operation.
If you enable compression on a heap table, pages are evaluated for compression only
during rebuild and shrink operations. Also, if you drop a clustered index on a table,
turning it into a heap, SQL Server runs compression analysis on any full pages.

Compression is avoided during normal data modification operations on a heap to avoid
changes to the Row IDs, which are used as the row locators for any indexes on the heap.
(See the “Understanding Index Structures” section later in this chapter for a discussion of
row locators.) Although the
RowModCounter is still maintained, SQL Server essentially
ignores it and never tries to recompress a page based on the RowModCounter value.
Evaluating Page Compression
Before choosing to implement page compression, you should determine if the overhead of
page compression will provide sufficient benefit in space savings. To determine how
changing the compression state will affect a table or an index, you can use the SQL Server
2008 sp_estimate_data_compression_savings stored procedure, which is available only
in the editions of SQL Server that support data compression. This stored procedure evalu-
ates the effects of compression by sampling up to 5,000 pages in the table and creating a
copy of these 5,000 pages of the table in tempdb, performing the compression, and then
using the sample to estimate the overall size for the table after compression. The syntax
for sp_estimate_data_compression_savings is as follows:
Download from www.wowebook.com