Hands-On Microsoft SQL Server 2008 Integration Services part 58 docx

548 Hands-On Microsoft SQL Server 2008 Integration Services
In this model, the dimensions are denormalized and a business view of a dimension
is represented as a single table in the model. Also, the dimensions have simple
primary keys and the fact table has a set of foreign keys pointing to dimensions; the
combination of these keys forms a compound primary key for the fact table. This
structure provides some key benefits to this model by making it simple for users to
understand. Writing select queries for this model is easier due to simple joins between
dimensions and the fact tables. Also, the query performance is generally better due
to the reduced number of joins compared to other models. Finally, a star schema can
easily be extended by simply adding new dimensions as long as the fact entity holds the
related data.
Snowflake Model
Sometimes it is difficult to denormalize a dimension, or in other words, it makes
more sense to keep a dimension in a normalized form, especially when multiple levels
of relationships exist in dimension data or the child member has multiple parents
in a dimension. In this model, the dimension suitable for snowflaking is split into
its hierarchies and results into multiple tables linked to each other via relationships,
generally, one-to-many. The many-to-many relationship is also handled using a
bridge table between the dimensions, sometimes called a FactLessFact table. For
example, the AdventureWorksDW2008 products dimension DimProduct is a
snowflake dimension that is linked to DimProduct SubCategory, which is further
linked to the DimProductCategory table (refer to Figure 12-2). This structure makes
much sense to database developers or data modelers and helps users to write useful
queries, especially when an OLAP tool such as SSAS supports such a structure
and optimizes running of snowflaked queries. However, business users might find
it a bit difficult to work with and would prefer a star schema, so you have to find
a balance in choosing when to go for a snowflake schema. Though there is some
space saving by breaking a dimension into a snowflake schema, it is not high on the
preference list because first, the space is not very costly, and second, the dimension
tables are not huge and space savings are not expected to be considerable in many
cases. The snowflaking is done for the functional reasons rather than savings in disk

space. Finally, the queries written against a snowflake schema tend to use more joins
(because more dimension tables are involved) compared to a star schema, and this
will affect the query performance. You need to test for user acceptance for the speed
at which results are being returned.
Chapter 12: Data Warehousing and SQL Server 2008 Enhancements 549
Building a Star Schema
The very first requirement in designing a data warehouse is your focus on the subject
area for which a business has engaged you to build a star schema. It’s easy to sway in
different directions while building a data warehouse, but whatever you do later in the
process, you must always keep a focus on the business value you are delivering with the
model. As a first step, capture all the business requirements and the purposes for which
they want to do this activity. You might end up meeting several business managers
and process owners to understand the requirements and match them against the data
available in source systems. At this stage, you will be creating a high-level data source
mappings document to meet the requirements. Once you have identified at a high
Figure 12-2 AdventureWorksDW2008 simplified snowflake schema
550 Hands-On Microsoft SQL Server 2008 Integration Services
level that the requirements can be met with the available data, the next step is to define
the dimensions and the measures in the process. While defining measures or facts,
it is important that you discuss with the business and understand clearly the level of
detail they will be interested in. This will decide the grain of your fact data. Typically,
you would want to keep lowest grain of data so that you have maximum flexibility for
future changes. In fact, defining the grain is one of the first steps while building a data
warehouse, as this is the cornerstone to collating the required information, for instance,
defining the roll-up measures.
At this stage, you are ready to create a low-level star schema data model and will be
defining attributes and fields for the dimension and fact tables. As part of the process,
you will also define primary keys for dimension tables that will also exist in fact tables
as a foreign key. These primary keys will need to be the new set of keys known as the
surrogate keys. The use of surrogate keys instead of source system keys or business

keys provides many benefits in the design; for instance, they provide protection against
changes in source systems keys, maintain history (by using SCD transformation),
integrate data from multiple source, and handle late arriving members, including facts
for which dimension members are missing. Generally, a surrogate key will be an auto-
incrementing non-null integer value like an identity and will form a clustered index on
the dimension. However, the Date dimension is a special dimension, commonly used
in the data warehouses, with a primary key based on the date instead of a continuous
number; for instance, 20091231 and 20100101 are date-based consecutive values, but
not in a serial number order.
While working on dimensions, you will need to identify some special dimensions.
First, look for role-playing dimensions. Refer to Figure 12-2 and note that the DateKey
of the DimDate dimension is connected multiple times with the fact table, once
each for the OrderDateKey, DueDateKey, and ShipDateKey columns. In this case,
DimDate is acting as a role-playing dimension. Another case is to figure out degenerate
dimensions in your data and place them alongside facts in the fact table. Next, you need
to identify any indicators or flags used in the facts that are of low cardinality and can be
grouped together in one junk dimension table. These miscellaneous flags held together
at one place make the user’s life much easier when they need to classify the analysis by
some indicators or flags. Finally, you will complete the exercise by listing the attributes’
change types, that is, whether they are Type 1 or Type 2 candidates. These attributes,
especially Type 2, help in maintaining history in the data warehouse. Keeping history
using SCD transformations has been discussed earlier in the chapter as well as in
Chapter 10. Though your journey to implement a data warehouse still has a long way
to go, after implementing the preceding measures, you will have implemented a basic
star schema structure. From here it will be easy to work with and extend this model
further to meets specific business process requirements, such as real-time data delivery
or snowflaking.
Chapter 12: Data Warehousing and SQL Server 2008 Enhancements 551
SQL Server 2008 R2 Features and Enhancements
Several features have been provided in SQL Server 2008 R2 that can help you in

your data warehousing project by either providing new functionality or improving the
performance of commonly used features. It is not possible to cover them all here unless
I want to stretch myself beyond the scope of this book, especially when those features
do not reside in the realm of Integration Services. But I will still try to cover some
features that most data warehousing projects will use, while other features such as data
compression, sparse columns, new data types, Large UDTs, and minimal logging are
not covered.
SQL Server 2008 R2 Data Warehouse Editions
Microsoft has just launched SQL Server 2008 R2, which is built upon the strong
foundations and successes of SQL Server 2008 and is targeted at very large-scale data
warehouses, and at higher mission-critical-scale and self-service business intelligence.
SQL Server 2008 R2 has introduced two new premium editions to meet the demands
of large-scale data warehouses.
SQL Server 2008 R2 Datacenter
This edition is built on the Enterprise Edition code base, but provides the highest
levels of scalability and manageability. SQL Server 2008 R2 Datacenter is designed for
the highest levels of scalability that SQL Server platform can provide, virtualization,
and consolidation, and it delivers a high-performing data platform. Typical
implementations include a large-scale data warehouse server that can scale up to
support tens of terabytes of data, provide Master Data services, and implement very
large-scale BI applications such as self-service or power pivot for SharePoint. Following
are the key features:
As the Enterprise Edition is restricted to up to 25 editions and 4 virtual machines
c
(VMs), the Datacenter Edition is the next level if you need more than 25 instances
or more VMs. is also provides application and Multi-Server Management for
enrolling and gaining insights.
e Datacenter Edition has no limits on server maximum memory; rather, it is c
restricted by the limits of the operating system only. For example, it can support
up to 2TB of RAM if running on the Windows Server 2008 R2 Datacenter

Edition.
552 Hands-On Microsoft SQL Server 2008 Integration Services
It supports more than 8 processors and up to 256 logical processors for the highest c
levels of scale.
It has the highest virtualization support for maximum ROI on consolidation and
c
virtualization.
It provides high-scale complex event processing with SQL Server StreamInsight.
c
Advanced features such as the Resource Governor, data compression, and backup c
compression are included.
SQL Server 2008 R2 Parallel Data Warehouse
Since acquiring DATAllegro, a provider of large-volume, high-performance data
warehouse appliances, Microsoft has been working on consolidating hardware and
software solutions for high-end data warehousing under a project named Madison.
The SQL Server 2008 R2 Parallel Data Warehouse is the result of Project Madison.
The Parallel Data Warehouse is an appliance-based, highly scalable, highly reliable,
and high-performance data warehouse solution. Using SQL Server 2008 on Windows
Server 2008 in a massively parallel processing (MPP) configuration, Parallel Data
Warehouse can scale from tens to hundreds of terabytes, providing better and more
predictable performance, increased reliability, and lower cost per terabyte. It comes
with preconfigured hardware and software that is carefully balanced in one appliance,
thus making deployment quick and easy. Massively parallel processing enables it to
perform ultra-fast loading and high-speed backups, thus addressing two of the major
challenges facing modern data warehouses: data load and the backup times. You can
integrate existing SQL Server 2008–based data marts or mini–data warehouses with
parallel data warehouses via a hub-and-spoke architecture. This product was targeted
to be shipped alongside the SQL Server 2008 R2 release; however, the release of this
product has been slightly delayed awaiting customer feedback from the customer
Technology Adoption Program (TAP). The parallel data warehouse principles and

architecture are detailed later in this chapter.
SQL Server 2008 R2 Data Warehouse Solutions
Microsoft has recognized the need to develop data warehouse solutions to build
upon the successes of SQL Server 2008. This has resulted in Microsoft partnering
with several industry-leading hardware vendors to create best-of-breed balanced
configurations combining hardware and software to achieve the highest levels of
performance. Two such solutions are now available under the names of Fast Track
Data Warehouse and Parallel Data Warehouse.
Chapter 12: Data Warehousing and SQL Server 2008 Enhancements 553
Fast Track Data Warehouse
The Fast Track Data Warehouse solution implements a CPU core-balanced approach
on the symmetric multiprocessor (SMP)–based SQL Server data warehouse using a
core set of configurations and database best practice guidelines. The fast track reference
architecture is a combination of hardware that is balanced for performance across all the
components and software configurations, such as Windows OS configurations, SQL
Server database layout, and indexing, along with a whole raft of other settings, best
practices, and documents to implement all of these objectives. The Fast Track Data
Warehouse servers can have two-, four-, or eight-processor configurations and can
scale from 4 terabytes to 30-plus terabytes and even more if compression capabilities
are used. The earlier available reference architectures are found to suffer from various
performance issues due to the simple fact that they have not been specifically designed
to suit the needs of one particular problem and hence, suffer from an unbalanced
architecture. For example, you may have seen that a server is busier and processing
too much I/O, but still the CPU utilization is not high enough to indicate the work
load. This is a simple example of mismatch or unbalance existing between various
components in currently available servers. The Fast Track Data Warehouse servers are
built on the use cases of a particular scenario—i.e., they are built to capacity to match
the required workload on the server rather than with a one-size-fits-all approach.
This approach of designing a balanced server provides predictable performance and
minimizes the risk of going over spec on the components such as by providing CPU

or storage that will never be utilized. The predictable performance and scalability is
achieved by adopting core principles, best practices, and methodologies, some of which
have been listed next.
It is built for data warehouse workloads.
c First of all, understand that the data
warehouse workload is quite different from that on the OLTP servers. While
OLTP transactions are made up of small read and write operations, data warehouse
queries usually perform large read and write operations. e data warehouse
queries are generally fewer in number, but they are more complex, requiring high
aggregations, and generally have date range restrictions. Furthermore, OLTP
transactions generate more random I/O, which causes slow response. To overcome
this, a large number of disks have been traditionally used, along with some other
optimization techniques such as building heavy indexes. ese techniques have
their own maintenance overheads and cause the data warehouse loading time
to increase. e Fast Track Data Warehouse uses a new way of optimizing data
warehouse workloads by laying data in a sequential architecture. Considering that
a data warehouse workload requires ranges of data, reading sequential data off the
disk drives is much efficient compared to random I/Os. All efforts are targeted to
554 Hands-On Microsoft SQL Server 2008 Integration Services
preserving sequential storage. e data is preferred to be served from disk rather
than from memory, as the performance achieved is much higher with sequential
storage. is results in fewer indexes, yielding savings in maintenance, decreased
loading time, lesser fragmentation of data, and reduced storage requirements.
It offers a holistic approach to component architecture. c e server is built with a balance
across all components, starting with disks, disk controllers, fiber channels HBAs,
and the Windows operating system, and ranging up to the SQL Server and then
to the CPU core. For example, on the basis of how much data can be consumed
per CPU core (200 MBps), the number of CPU cores is calculated for a given
workload and then the backward calculations are applied to all the components to
support the same bandwidth or capacity. is balance or the synchronization in

response by individual components provides the required throughput to match the
capabilities of the data warehouse application.
It is optimized for workload type.
c e Fast Track Data Warehouse servers are
designed and built considering the very nature of the database application. To
capture these details, templates and tools are provided to design and build the fast
track server. Several industry-leading vendors are participating in this program to
provide you out-of-the-box performance reference architecture servers. Businesses
can benefit from reduced hardware testing and tuning and rapid deployment.
Parallel Data Warehouse
When your data growth needs can no longer be satisfied with the scale-up approach
of the Fast Track Data Warehouse, you can choose the scale-out approach of Parallel
Data Warehouse, which has been built for very large data warehouse applications
using Microsoft SQL Server 2008. The symmetric multiprocessing (SMP) architecture
used in the Fast Track Data Warehouse is limited by the capacity of the components
such as CPU, memory, and hard disk drives that form part of the single computer.
This limitation is addressed by scaling out to a configuration consisting of multiple
computing nodes that have dedicated resources—i.e., CPUs, memory, and hard disk
space, along with an instance of SQL Server—connected in an MPP configuration.
Architecture and Hardware
The appliance hardware is built on industry-standard technology and is not proprietary
to one manufacturer, so you can choose from well-known hardware vendors such as
HP, Dell, IBM, EMC
2
, and Bull. This way, you can keep your hardware maintenance
costs low, as it will integrate nicely with already-existing infrastructure.
Chapter 12: Data Warehousing and SQL Server 2008 Enhancements 555
As mentioned earlier, the Parallel Data Warehouse is built on Microsoft Windows
2008 Enterprise server nodes with their own dedicated hardware connected via a
high-speed fiber channel link, with each node running an instance of the SQL Server

2008 database server. The MPP appliance basically has one control node and several
compute nodes, depending on the data requirements. This configuration is extendable
from single-rack to multirack configurations; in the latter case, one rack could act as a
control node. The nodes are connected in a configuration called Ultra Shared Nothing
(refer to Figure 12-3), in which the large database tables are partitioned across multiple
nodes to improve the query performance. This architecture has no single point of
failure, and redundancy has been built in at all component levels.
Applications or users send requests to a control node that balances the requests
intelligently across all the compute nodes. Each compute node processes the request
it gets from the control node using its local resources and passes back the results to
the control node, which then collates the results before returning to the requesting
application or user. As the data is evenly distributed across multiple nodes and the
nodes process the requests in parallel, queries run many times faster on an MPP
appliance than on an SMP database server.
Like a Fast Track Data Warehouse server, an MPP appliance is also built under
tight specifications and carefully balanced configurations to eliminate performance
bottlenecks. Reference configurations have been designed for different use case
scenarios taking into account different types of workloads such as data loading,
reporting, and ad hoc queries. A control node automatically distributing workload
evenly, compute nodes being able to work on queries autonomously, system resources
balanced against each other, and design of reference configurations on use case
Figure 12-3 Parallel Data Warehouse architecture
Compute nodes
Node -1
Node -2
Node -N
Control node
High speed
fiber channel network
Node -0

556 Hands-On Microsoft SQL Server 2008 Integration Services
scenarios enable an MPP appliance to achieve predictable performance. Scalability
follows from here with the simple addition of capacity as the data volumes grow.
Hub-and-Spoke Architecture
Another important advantage with an MPP appliance is that it can be deployed
in a hub-and-spoke architecture. In this way, you can use an MPP appliance as
a hub while the spokes could be either MPP appliances or standard SQL Server
2008–based symmetric multiprocessing (SMP) servers (see Figure 12-4). Typically,
department users will connect to spokes to access data in their required formats. In
this configuration, the MPP appliance at the hub will host the enterprise data in the
lowest granularity and the spokes will contain data for their relevant department in
the schema and aggregations they require. This is possible because the spokes could
host any database application such as the SQL Server 2008 data mart or SQL Server
Analysis Services data mart, as best fits the user requirements. So, this architecture with
an MPP appliance at the hub and SMP database servers or MPP appliances as spokes
is a specialized configuration in which a grid of computers forms a very large-scale
data warehouse in a federated model. This grid of computers can be connected via a
high-speed network. Also, the nodes of an MPP appliance are connected via a high-
speed link and the hub processes data differently in different nodes, enabling parallel
Figure 12-4 A parallel data warehouse in hub-and-spoke architecture
SQL Server 2008
Fast Track Data Warehouse
SQL Server 2008
Analysis Services
SQL Server 2008
Reporting Services
SQL Server 2008 R2
Parallel Data Warehouse
SQL Server 2008
Fast Track Data Warehouse

SQL Server 2008
Chapter 12: Data Warehousing and SQL Server 2008 Enhancements 557
high-speed data transfer from node to node between hub and spoke units. Data transfer
speeds approaching 500GB per minute can be achieved, thus minimizing the overhead
associated with export and load operations.
The SQL Server 2008 R2 Parallel Data Warehouse MPP appliance integrates very
well with BI applications such as Integration Services, Reporting Services, and Analysis
Services. So, if you have an existing SQL Server 2008 data mart, it can be easily added
as a node in the grid. As spokes can be any SQL Server 2008 database application,
this architecture provides a best-fit approach to the problem. The enterprise data is
managed at the center in the MPP appliance under the enforcement of IT policies
and standards, while a business unit can still have a spoke that they can manage
autonomously. This flexible model is a huge business benefit and provides quick
deployments of data marts, bypassing the sensitive political issues. This way, you can
easily expand an enterprise data warehouse by adding an additional node that can be
configured according to the business unit requirements.
The Parallel Data Warehouse hub-and-spoke architecture utilizes the available
processing power in the best possible way by distributing work across multiple locations
in the grid. While the basic data management such as cleansing, standardization, and
metadata management is done in the hub according to the enterprise policies, the
application of business rules relevant to the business units and the analytical processing
is handled in the relevant spokes. The hub-and-spoke model offers benefits such as
parallel high-speed data movement among different nodes in the grid, the distribution
of workload, and the massively parallel architecture of the hub where all nodes work in
parallel autonomously, thus providing outstanding performance.
With all these listed benefits and many more that can be realized in individual
deployment scenarios, the hub-and-spoke reference architecture provides the best of
both worlds—i.e., the ease of data management of centralized data warehouses and the
flexibility to build data marts on use-case scenarios as with federated data marts.
SQL Server 2008 R2 Data Warehouse

Enhancements
In this section some of the SQL Server 2008 enhancements are covered, such as backup
compression, MERGE T-SQL statements, change data capture, and partition-aligned
Indexed views.
Backup Compression
Backup compression is a new feature provided in SQL Server 2008 Enterprise Edition
and above and due to its popularity, has since been included in the Standard Edition
and in the SQL Server 2008 R2 release as well. Backup compression helps to speed up