Hands-On Microsoft SQL Server 2008 Integration Services part 20 ppt

168 Hands-On Microsoft SQL Server 2008 Integration Services
If you have chosen ADO or ADO.NET Connection Manager in the ConnectionType
field earlier, the IsQueryStoredProcedure field becomes available for use and you will be
able to specify that the SQL statement to be run is a stored procedure.
When you select OLE DB Connection Manager in the ConnectionType field,
you can specify in the BypassPrepare field whether the task should skip preparing the
query step before running it. When SQL queries are run, the SQL statement is parsed
and compiled in the first step and then executed based on the execution plan prepared
earlier in the compilation step. By default the BypassPrepare field is set to True, which
means when the query is run the first time, it parses and compiles the SQL statement
in the first step and keeps the execution plan in the cache of the database engine and
in the next step uses this execution plan to execute the query. So when the query is run
the next time, it doesn’t parse and compile the statement but rather uses the existing
execution plan, saving time required to parse and compile. If you’re running a query
multiple times, it is faster and efficient if you use a prepared execution—however, if
you’re running the query only once, it is not a recommended option.
Parameter Mapping Page
If you are running a parameterized SQL statement, you can map variables to the parameters
in this page. The interface on this page is relatively simple and provides self-explanatory
fields and options. You can click Add to add a parameter mapping and then click
in each field to select the available values from the drop-down lists; however, in the
Parameter Name field, you have to type in a value. The configuration of this page
has been covered in the “Using System Variables to Create Custom Logs” Hands-On
exercise in Chapter 3.
Result Set Page
If you are using a query that will return a result set and you have selected a value other
than None in the ResultSet field on the General page, you can use this page to map the
row set to a variable, which can be used later in the package.
Expressions Page
Using this page, you can build a property expression using SSIS expression language
to update the property dynamically at run time. In the Expression Builder, you can use

System or User variables, functions, type casts, and operators exposed by expression
language to build an expression.
Chapter 5: Integration Services Control Flow Tasks 169
Bulk Insert Task
When you get large amounts of data in the flat files, say from the mainframe systems
or from third parties, and you want to import this data from the flat files into an SQL
Server, you can use the Bulk Insert task. This task is the quickest way to copy large
amounts of data into an SQL Server table or view.
Note the following when deciding whether to use this task:
e Bulk Insert task does not perform any validation or transformation on the
c
data while importing data from the source text file to the SQL Server database
table or view. So, it is more suitable in cases where the data is coming from a
reliable source or you intend to apply business processes in later stages. A similar
component—SQL Server Destination—is provided in the data flow task that
can provide the performance of a bulk insert. You will be using SQL Server
Destination if you want to perform transformations on data before loading
and your target server is the local server. SQL Server Destination is covered in
Chapter 9 in detail.
When you embed this task in an SSIS package Control Flow, only the members
c
of the sysadmin fixed server role can run the package.
You can use an XML- or non-XML-formatted file in the Bulk Insert task.
c
A formatted file is used to store format information for each field in a data file
in relationship to a specific table and provides all the format information that
is required to bulk-import data. is format file must be located on the server
executing the SSIS package containing the Bulk Insert task.
During the import process, some of the rows may fail to import and in turn can fail
c

the task. You can still run this task successfully by increasing the allowed number of
errors by specifying a value in the MaxErrors option. However, the rows that fail to
import cannot be logged or extracted out. If you want to capture the failing rows,
you need to consider alternative ways of importing data using the Data Flow task
and capture the failing rows in the exception files by way of error outputs.
Message Queue Task
Microsoft Message Queuing (MSMQ) is a Windows service that allows applications or
SSIS packages to send and receive messages using a queue. Message queuing is available
in Windows 2000 and later released as a standard operating system component that
has the advantages of Active Directory Integration. This service can be extended using
the Message Queuing Connector to heterogeneous data sources such as CICS (IBM’s
Customer Information Control System) or UNIX.
170 Hands-On Microsoft SQL Server 2008 Integration Services
In the SSIS world, message queuing means one package will be sending a message
to a queue while another package will be receiving that message from the queue. These
packages can be on different servers and may not be running simultaneously. Using
MSMQ, you can reliably exchange data between packages that may not be running on
the same server and might be separated in time throughout your enterprise. Following
are some of the scenarios in which you may be using the Message Queue task:
Send a message from an executing package to the package that is waiting for it to
c
complete so that it can start running.
If you have a small window of time to finish a large workload and you decide to
c
distribute your workload across many servers to utilize the processing power, you
can coordinate the operations on different servers using message queuing between
packages.
SSIS packages can communicate with the applications that utilize message c
queuing.
Send output from a processing package to a waiting package on the other computer,

c
where the data enclosed in the message will be processed further.
You can configure this task into either send mode or receive mode. Select Send
Message in the Message field to configure it into send mode or select Receive Message
in the Message field to configure it into receive mode. The package that is running the
message queue task in the receive mode can be configured to either run and wait for the
message in the queue using a looping construct or run on a schedule that is later than
when the message is dropped in the queue just to pick up the message. SSIS is more
suitable for the applications that can be satisfied with the latter implementation. If you
need to wait for messages, for example, your application is sending data in message
format as the transactions happen, Biztalk is more suitable for such implementations.
Biztalk server is designed to support such requirements and puts negligible load on
the server if it has to wait for messages, whereas SSIS will consume resources, as it has
to keep looping and stay in memory. The message queue task is provided in SSIS to
transfer the information that is relevant for the remaining ETL to complete rather than
read or load data from a transactional system.
You can choose from one of the different types of messages to send or receive using
the Message Queue task: Data file message, String message, or Variable message.
When the task is configured in the receiving mode, you also get an additional String
message to variable MessageType choice. As in other Integration Services tasks, the
Chapter 5: Integration Services Control Flow Tasks 171
available fields in this task are dynamic and change depending upon your choice to send
or receive a message or on message type.
Data file message
c Used to send or receive a file that contains the message. To
send a data file message, you specify Data File Message in the MessageType field
and specify the path of the file in the DataFileMessage field on the Send page
of the Message Queue Task Editor. When receiving this type of messages, you
select Data File Message in the MessageType field and specify the name of the
file in the SaveFileAs field into which you want to save the message. Also, you can

choose to overwrite the existing file, and optionally apply a filter to receive the
message only from the specified package. You specify the full path and package
name in the Identifier field on the Receive page.
Variable message c Used to send or receive one or more variables. To send variables,
you specify Variable Message in the MessageType field and one or more variables
in the VariableMessage field on the Send page of the Message Queue Task Editor.
While receiving the variables, you specify Variable Message in the MessageType
field, specify the name of the variable in the Variable field to receive the message
into, and optionally choose to apply a filter to receive the message only from the
package specified in the Identifier field on the Receive page.
String c message Used to send a text string. To send a text string, you specify String
message in the MessageType field and type a text string in the StringMessage field
on the Send page of the Message Queue Task Editor. While receiving the text
string, you specify String message in the MessageType field and optionally specify
to compare the incoming string in the Compare field with a user-defined string
specified in the CompareString field on the Receive page. e string comparison
options in the Compare field can be Exact Match for an exact comparison, Ignore
Case for case-insensitive comparison, or Containing for a substring match.
String message to variable
c Used to pass the source message that has been sent
as a string to a destination variable and is available only when receiving messages.
To configure a text string message to be passed to a variable, you specify String
Message To Variable in the MessageType field, specify the name of the variable
in the Variable field to receive the text string into, and optionally specify to
compare the incoming string in the Compare field with a user-defined string
specified in the CompareString field on the Receive page. e string comparison
options in the Compare field can be Exact Match for exact comparison, Ignore
Case for case-insensitive comparison, or Containing for a substring match.
172 Hands-On Microsoft SQL Server 2008 Integration Services
To use the Message Queue task to send and receive messages, you first need to

install the Message Queuing service and then create messaging queues. You can create
either a public or a private queue, depending upon whether you have installed the
Active Directory Integration component of the Message Queuing service. A public
queue is created in an Active Directory environment to publish its properties and
description to the Active Directory. A private queue does not publish itself to the
Active Directory but works on the local computer that holds the queue.
The Message Queuing service has the following components available in Windows
2008 Server for installation:
Message Queuing Server
c is component enables you to perform Message
Queuing functions such as guaranteed message delivery, efficient routing, improved
security, support for sending messages within transactions, and priority-based
messaging. It can be used to implement both synchronous and asynchronous
messaging solutions.
Directory Service Integration
c Provides integration with Active Directory
whenever the computer belongs to a Windows domain (Windows 2000 and
later). With this component, you can publish queue properties to the directory,
authenticate and encrypt messages using digital certificates registered in the
directory, and route messages across directory sites. Public queues are configured
and used when using this component.
Message Queuing Triggers
c Associates the arrival of messages at a queue
with triggering functionality of a COM component or a standalone application,
depending upon the filters that you define for the incoming messages in a given
queue.
HTTP Support c Enables sending or receiving of messages over HTTP
transport with proxy settings configured using the proxycfg.exe tool.
Multicasting support
c Enables queuing and sending of multicast messages to

a multicast IP address.
Routing Service
c Available in Windows 2008 Server to provide routing of
messages using the store and forward mechanism in Active Directory transport
between different sites or within the same site.
Windows 2000 Client Support
c Available in Windows 2008 Server to provide
support for Message Queuing clients on Windows 2000 computers in the
domain.
Message Queuing DCOM Proxy
c Enables a computer to act as a DCOM client
of a remote Message Queuing Server.
Chapter 5: Integration Services Control Flow Tasks 173
For proper functioning of the Message Queue task, make sure you have installed the
Message Queuing Server and the SQL Server Integration Services service. When you
install SQL Server 2008 without specifically selecting the Integration Services service,
you may still be able to use BIDS to design and run Integration Services packages.
However, as this is a partial installation of SSIS, not all tasks will run properly, and
the Message Queue task is one of those that won’t. For the Message Queue task to be
functioning properly, you must install Integration Services fully during installation of
SQL Server 2008.
Next, you will import the expanded files into the Campaign database in SQL Server
using the Message Queue task and the Bulk Insert task in the following Hands-On
exercises.
Hands-On: Importing Expanded Files
Dealers of Adventure Works Bikes submit the sales reports to an FTP server, and they need
to be downloaded from this server and imported into the SQL Server database. You have
already downloaded, expanded, and archived these report files in the previous exercises,
and now you have DealerSales01.txt and DealerSales02.txt files in the Downloads
folder ready to be imported to the Campaign database. However, you want to run this

new package independent of earlier packages, and this may occur at a different time
during the day.
Method
We can meet this objective in several ways: We can use the Bulk Insert task to import
the data to our existing table—but we have multiple files, so we can use the Foreach
Loop Container to read the filenames one by one and pass them to the Bulk Insert
task to import multiple files one at a time. However, as we want to keep the packages
independent from one another and want to run the second package only when the
first has completed successfully, the use of messages from the first package to pass the
information to the other will be a better solution. Though in this case you will not be
following the shortest and easiest solution option to achieve your goal, the solution
you are going to follow is quite interesting, and I’m sure it will be relevant in real-life
scenarios.
You will add the Message Queue task at the end of the Archiving Downloaded
Files package to send filenames in the messages, which will be read by the Importing
Expanded Files package using the Message Queue task in the receiving mode. As
the package won’t have to read filenames from the file system, you will not be using
174 Hands-On Microsoft SQL Server 2008 Integration Services
the Foreach Loop Container; instead, you will use the For Loop Container to read
multiple messages one by one. Here’s the step-by-step method:
1. Install the Message Queuing service and create message queues.
2. To keep it simple and applicable to most of the users, for the sake of this exercise
I’ve used a Windows Server 2008 machine with a private queue only. If you’ve a
different environment, refer to Microsoft SQL Server 2008 Books Online for more
details to create a message queuing environment suitable to your requirements.
3. Configure the Archiving Downloaded Files package to send filenames. In
this step, we will add Message Queue task in the already created Archiving
Downloaded Files package.
4. Build the Importing Expanded Files package. And the final step is to execute this
package and check that the two messages have been picked up from the queue

and the text files have been imported into the SQL Server Campaign database.
Exercise (Install Message Queuing Service)
Let us start this exercise by installing the Message Queuing service. You will also create
message queues while learning about private and public queues.
1. From the Server Manager, click the Add Features link to start the Add Features
Wizard.
2. In the Select Features dialog box, expand the Message Queuing group and
select the Message Queuing Server from the list. Click Next to go to Confirm
Installation Selections page.
3. Verify the selection and click Install to install the Message Queuing Server
component.
4. When the installation is completed successfully, click the Close button in the
Results page to close the wizard.
5. Again in the Server Manager, expand the Features node in the left pane of the
window to see the installed Message Queuing feature.
6. Right-click the Private Queues folder and choose New | Private Queue from the
context menu. This will open the New Private Queue window. In the Queue
name field, type SSISprivQ and then click OK.
Note that another difference between a private queue and a public one is that
the private$ is attached to the full name of a private queue. The New Private
Queue window shows that the queue will be created in your computer name with
private$ added in front of the name you type in. So, the full path of the queue
will be ComputerName\private$\SSISprivQ. If this were the public queue in
which we specify the queue name as SSISpubQ, the full path would have been
ComputerName\SSISpubQ.
Chapter 5: Integration Services Control Flow Tasks 175
7. Expand the Private Queues folder under the Message Queuing from the left pane
of the Computer Management window. Further expand the SSISPrivQ folder
and you will see a Queue messages subfolder. This is where the Message Queue
task will be delivering the messages and the messages will wait for a receiving

Message Queue task to pick them up.
Exercise (Configure Archiving Downloaded
Files Package to Send Filenames)
Remember that in the Archiving Downloaded Files package, the Foreach Loop
Container reads the filenames of zipped files lying in the C:\SSIS\downloads folder
and then passes those names to the File System task, which moves the zipped files to
the Archive subfolder. To accomplish this, the package uses a variable named fname
to pass the zipped files names. Because you want to import text files, not the zipped
files, you will derive a new variable called txtfname to contain the text files names from
fname variable. This new variable will then be sent through the Message Queue task as
a variable message.
8. Run BIDS and open the Control Flow Tasks project. In the Solution Explorer
window, double-click the Archiving downloaded files.dtsx package to open it.
9. Open the Variables window. Click the Auto Hide pushpin to dock the Variables
window on the left side of the screen. On the Control Flow surface, click the
Enumerating Files Foreach Loop Container and switch to the Variables window.
Add a new variable named txtfname in the Enumerating Files Scope and set Data
Type to string. Press  to open the Properties window. The Properties window
will open up by default on the right side of the screen, showing properties of the
txtfname variable. (Sometimes you may not see properties for the object in which
you are interested—this is because of your selection of the items on the designer.
If this happens, make sure you’ve selected the Enumerating Files Container and
then clicked the txtfname variable to see the properties for this variable.)
Scroll through the properties and locate the EvaluateAsExpression property. This
property allows you to enable the variable to use the results of an expression as its
value. That is, you can write an expression, which will be evaluated at run time
and the result of this evaluation will be used as the value of the variable.
Select True for the EvaluateAsExpression property. Type the following text in
the Expression property:
SUBSTRING(@[User::fname] , 1, LEN(@[User::fname]) -3) + "txt"

This expression will evaluate txtfname as C:\SSIS\downloads\DealerSales01.txt
from the fname variable when it equals C:\SSIS\downloads\DealerSales01.zip. If
you look further down in the properties window in the Value field, you will notice
176 Hands-On Microsoft SQL Server 2008 Integration Services
(see Figure 5-9) that the full filename in fact has been resolved as txt. You can
see that nothing appears in the name portion of the file path name because the
fname variable has not yet been populated (by the Foreach Loop Container) in the
design mode. However, at run time, the Foreach Loop Container provides a value
for the fname variable and the name portion of the filename gets populated in the
file path name.
10. Drag the Message Queue task from the Toolbox and drop it in the Enumerating
Files (Foreach Loop) Container below the Archive Downloaded Files (File
System) task. Drag and drop the green arrow, the precedence constraint, from
below the Archive Downloaded Files task on to the Message Queue task. The
package will look as shown in Figure 5-10 after you’ve configured the Message
Queue task.
11. Right-click the Message Queue task and choose Edit from the context menu.
Type the following on the General Page of the task editor:
Name Send variable message
Description This task sends the txtfname variable to the Importing expanded files.
Figure 5-9 Deriving the txtfname variable using property expressions
Chapter 5: Integration Services Control Flow Tasks 177
If you want to send this message in the format that is acceptable to SQL Server
2000 Message Queue task, you can use the Use2000Format field. For now, leave
the default False value set in this field.
12. In the MSMQConnection field, select to create a new connection. In the
MSMQ Connection Manager Editor, type the following:
Name SSISprivQ
Description Connection Manager for private Queue SSISprivQ
Path YourComputerName\private$\SSISprivQ

You can use a dot (.) instead of YourComputerName to indicate your local computer.
Figure 5-10 Archiving the downloaded files package after adding Message Queue task