Tracing and Profiling SQL Server potx

1
1
ISQL Users Group May 2009
ISQL Users Group May 2009


Tracing and Profiling
Tracing and Profiling
SQL Server
SQL Server




/> />2
2
Over the next 1 hour:
Over the next 1 hour:

SQL Trace Architecture and
SQL Trace Architecture and
Terminology
Terminology

Security and Permissions
Security and Permissions

Profiler
Profiler

Server side Tracing

Server side Tracing

Saving and Replaying Traces
Saving and Replaying Traces

Querying Server-Side Trace Metadata
Querying Server-Side Trace Metadata
3
3
Introduction
Introduction



Query tuning, optimization and
Query tuning, optimization and
troubleshooting are all possible by
troubleshooting are all possible by
the ability to view what's going on
the ability to view what's going on
inside SQL Server
inside SQL Server



SQL Trace provides a real-time
SQL Trace provides a real-time
view into what’s going on inside the
view into what’s going on inside the
database engine and at a very

database engine and at a very
granular level
granular level
4
4
SQL Trace Architecture and Terminology
SQL Trace Architecture and Terminology
5
5



SQL Trace is an SQL Server database engine
SQL Trace is an SQL Server database engine
technology
technology



SQL Server Profiler is a .NET application that
SQL Server Profiler is a .NET application that
uses system stored procedures exposing the
uses system stored procedures exposing the
functionality of SQL Trace
functionality of SQL Trace
Microsoft.SqlServer.Management.Trace
Microsoft.SqlServer.Management.Trace
/> />us/library/microsoft.sqlserver.management.t
us/library/microsoft.sqlserver.management.t
race.aspx

race.aspx
6
6



When tracing we monitor for specific
When tracing we monitor for specific
events which are generated when
events which are generated when
various actions occur in the database
various actions occur in the database
engine
engine
–


For example, a user login or the execution of a query
For example, a user login or the execution of a query
are each actions that cause events to fire
are each actions that cause events to fire



Each event has an associated
Each event has an associated
collection of columns that contain the
collection of columns that contain the
collected data when the event fires
collected data when the event fires




SQL Server uses a model that
SQL Server uses a model that
selectively enables data collection as
selectively enables data collection as
required
required
–


Data is never collected until someone asks for it
Data is never collected until someone asks for it



There are around 171 events and 65
There are around 171 events and 65
columns to select from
columns to select from


8
8



Trace I/O providers send the data
Trace I/O providers send the data

to the final destination
to the final destination



The available output formats for
The available output formats for
trace data are either a file on the
trace data are either a file on the
database server (or a network
database server (or a network
share) or a rowset to a client
share) or a rowset to a client



Both providers use internal buffers
Both providers use internal buffers
to ensure that if the data is not
to ensure that if the data is not
consumed quickly enough (written
consumed quickly enough (written
to disk or read from the rowset) it
to disk or read from the rowset) it
will be queued
will be queued
9
9


The file provider is designed with a
The file provider is designed with a
guarantee that no event data will be
guarantee that no event data will be
lost
lost



If an I/O slowdown occurs so disk
If an I/O slowdown occurs so disk
writes are not occurring quickly
writes are not occurring quickly
enough the internal buffers begin to
enough the internal buffers begin to
fill
fill



To monitor for these waits watch
To monitor for these waits watch
the SQLTRACE_LOCK and
the SQLTRACE_LOCK and
IO_COMPLETION wait types
IO_COMPLETION wait types
10
10


The rowset provider is not designed for
The rowset provider is not designed for
data loss guarantee
data loss guarantee



If data is not being consumed quickly
If data is not being consumed quickly
enough and its internal buffers fill it waits
enough and its internal buffers fill it waits
up to 20 seconds before it begins dropping
up to 20 seconds before it begins dropping
events in order to free buffers for the sake
events in order to free buffers for the sake
of getting things moving
of getting things moving



SQL Server Profiler will then send an
SQL Server Profiler will then send an
information message informing that some
information message informing that some
events had been lost and had not been
events had been lost and had not been
captured
captured




You can find out if you're headed in that
You can find out if you're headed in that
direction by monitoring SQL Server's
direction by monitoring SQL Server's
TRACEWRITE wait type which is
TRACEWRITE wait type which is
incremented as threads are waiting for
incremented as threads are waiting for
buffers to free up
buffers to free up
11
11



A background trace management
A background trace management
thread is started whenever at least
thread is started whenever at least
one trace is active on the server
one trace is active on the server
–


SELECT * FROM sysprocesses
SELECT * FROM sysprocesses

WHERE cmd = N'TRACE QUEUE TASK' ;
WHERE cmd = N'TRACE QUEUE TASK' ;



The background thread is
The background thread is
responsible for:
responsible for:


–


Flushing file provider buffers (done every 4 seconds)
Flushing file provider buffers (done every 4 seconds)
–


Can be seen as a wait_type of
Can be seen as a wait_type of
SQLTRACE_BUFFER_FLUSH
SQLTRACE_BUFFER_FLUSH
–


Closing rowset-based traces that are considered to be
Closing rowset-based traces that are considered to be
expired (dropping events for more than 10 minutes)

expired (dropping events for more than 10 minutes)



There is no provider that writes
There is no provider that writes
trace data directly to a table
trace data directly to a table
12
12
Security and Permissions
Security and Permissions


13
13



SQL Server 2005 introduces a new
SQL Server 2005 introduces a new
permission called ALTER TRACE. This is
permission called ALTER TRACE. This is
a server-level permission granted to a
a server-level permission granted to a
login principal and allows access to
login principal and allows access to
start, stop, or modify a trace (in
start, stop, or modify a trace (in
addition to being able to generate user-

addition to being able to generate user-
defined events)
defined events)



GRANT ALTER TRACE TO
GRANT ALTER TRACE TO
<LoginName>
<LoginName>
;
;



SQL Trace has a couple of built-in
SQL Trace has a couple of built-in
security features to keep passwords
security features to keep passwords
secured
secured
–
SQL Trace will automatically omit data if an event
SQL Trace will automatically omit data if an event
contains a call to a password-related stored procedure or
contains a call to a password-related stored procedure or
statement. For example, a call to CREATE LOGIN including
statement. For example, a call to CREATE LOGIN including
the WITH PASSWORD option will be blanked out by SQL
the WITH PASSWORD option will be blanked out by SQL

Trace.
Trace.
–
SQL Trace will not return statement text or query plans
SQL Trace will not return statement text or query plans
generated within an encrypted stored procedure, user-
generated within an encrypted stored procedure, user-
defined function, or view
defined function, or view
14
14
Profiler
Profiler


15
15



The General tab allows you to control how the trace
The General tab allows you to control how the trace
will be processed by the consumer
will be processed by the consumer



The default setting is to use the rowset provider
The default setting is to use the rowset provider
displaying the events in real time in the SQL Server

displaying the events in real time in the SQL Server
Profiler tool window
Profiler tool window



Other available options are
Other available options are


–
Save the events to a table
Save the events to a table
–
Save the events to a file (on either the server or the client)
Save the events to a file (on either the server or the client)



When saving to a table Profiler uses the rowset
When saving to a table Profiler uses the rowset
provider and routes the data back into a table
provider and routes the data back into a table



When saving to a server side file Profiler actually
When saving to a server side file Profiler actually
starts two equivalent traces—one using the rowset
starts two equivalent traces—one using the rowset

provider and the other using the file provider
provider and the other using the file provider



Saving to a client-side file does not use the file
Saving to a client-side file does not use the file
provider at all. Rather, the data is routed to the Profiler
provider at all. Rather, the data is routed to the Profiler
tool via the rowset provider and then saved from there
tool via the rowset provider and then saved from there
to a file.
to a file.


–
This is more efficient than using Profiler to write to a server-side file
This is more efficient than using Profiler to write to a server-side file
–
Remember that doing so you do not take benefit of the lossless data guarantee
Remember that doing so you do not take benefit of the lossless data guarantee
offered by the file provider
offered by the file provider
16
16



The Events Selection tab allows you to select events
The Events Selection tab allows you to select events

that you'd like to trace along with their associated data
that you'd like to trace along with their associated data
columns
columns



In order to narrow your scope and help ensure that
In order to narrow your scope and help ensure that
tracing does not cause performance issues SQL Trace
tracing does not cause performance issues SQL Trace
offers the ability to filter the events based on various
offers the ability to filter the events based on various
criteria
criteria



In SQL Server Profiler filtration is exposed via the
In SQL Server Profiler filtration is exposed via the
“Column Filters” tab
“Column Filters” tab



Remember to check the “Show All Columns”
Remember to check the “Show All Columns”
checkbox in order to see the complete list of columns
checkbox in order to see the complete list of columns


Once you click the Run button data will begin
Once you click the Run button data will begin
streaming immediately and be displayed at the Profiler
streaming immediately and be displayed at the Profiler
window (this is because Profiler uses the rowset
window (this is because Profiler uses the rowset
provider)
provider)
–
If you find that data is coming in too quickly for you to be able to read
If you find that data is coming in too quickly for you to be able to read
it you may disable scrolling using the Auto Scroll icon on the SQL Server
it you may disable scrolling using the Auto Scroll icon on the SQL Server
Profiler toolbar
Profiler toolbar
–
If you are on a slow remote connection and your trace is not well
If you are on a slow remote connection and your trace is not well
filtered you may be in a situation that the icons in the Profiler tool bar
filtered you may be in a situation that the icons in the Profiler tool bar
are seen as if they were inactive thus disallowing you to stop the trace.
are seen as if they were inactive thus disallowing you to stop the trace.
The only way out here is to stop the trace using
The only way out here is to stop the trace using
sp_trace_setstatus
sp_trace_setstatus
18
18

Saving and Replaying
Saving and Replaying
Traces
Traces
19
19



Profiler ships with eight predefined templates
Profiler ships with eight predefined templates
–
A template is a collection of event and column selections, filters, and other
A template is a collection of event and column selections, filters, and other
settings that you can save to create a reusable trace definitions
settings that you can save to create a reusable trace definitions



TSQL_Replay template selects a variety of columns for 15
TSQL_Replay template selects a variety of columns for 15
events that are all required for Profiler to be able to play back
events that are all required for Profiler to be able to play back
(Replay) a collected trace at a later time.
(Replay) a collected trace at a later time.
–
Collecting data using this template allows you to reproduce a problem
Collecting data using this template allows you to reproduce a problem
experienced on a production system by replaying events collected on the
experienced on a production system by replaying events collected on the

production system at a test environment
production system at a test environment
–
Very useful for support departments
Very useful for support departments



Collected trace data has to be saved and then reopened
Collected trace data has to be saved and then reopened
before a replay can begin
before a replay can begin



SQL Server Profiler offers the following options for saving
SQL Server Profiler offers the following options for saving
trace data available from the File menu:
trace data available from the File menu:
–
The Trace File option is used to save the data to a file formatted using a
The Trace File option is used to save the data to a file formatted using a
proprietary binary format. This is generally the fastest way to save the data, and
proprietary binary format. This is generally the fastest way to save the data, and
also the smallest in terms of bytes on disk
also the smallest in terms of bytes on disk
–
The Trace Table option is used to save the data to a new or previously created
The Trace Table option is used to save the data to a new or previously created
table in a database. This option is useful if you need to manipulate or report on the

table in a database. This option is useful if you need to manipulate or report on the
data using T-SQL
data using T-SQL
–
The Trace XML File option saves the data to a text file formatted as XML
The Trace XML File option saves the data to a text file formatted as XML
–
The Trace XML File For Replay option also saves the data to an XML text file, but
The Trace XML File For Replay option also saves the data to an XML text file, but
only those events and columns needed for replay functionality are saved
only those events and columns needed for replay functionality are saved
20
20



Once the data has been saved to a file or table the
Once the data has been saved to a file or table the
original trace window can be closed and the file or
original trace window can be closed and the file or
table reopened via SQL Server Profiler's File menu
table reopened via SQL Server Profiler's File menu



A trace reopened in this way will have a Replay menu
A trace reopened in this way will have a Replay menu
on the Profiler tool bar allowing you to start replaying
on the Profiler tool bar allowing you to start replaying
the trace, stop the replay, or set a breakpoint

the trace, stop the replay, or set a breakpoint



During the course of the replay, the same events
During the course of the replay, the same events
used to produce the trace being replayed will be traced
used to produce the trace being replayed will be traced
from the server on which you replay
from the server on which you replay
–
The ”Save To File” and “Save To Table” options are used for a client-
The ”Save To File” and “Save To Table” options are used for a client-
side save. No server-side option exists for saving playback results
side save. No server-side option exists for saving playback results



The trace will be replayed on multiple threads (2005
The trace will be replayed on multiple threads (2005
only), corresponding to at most the “Number Of Replay
only), corresponding to at most the “Number Of Replay
Threads” specified
Threads” specified
–
The “Replay Events In The Order They Were Traced” option ensures
The “Replay Events In The Order They Were Traced” option ensures
that all events will be played back in exactly the order in which they
that all events will be played back in exactly the order in which they
occurred as based upon the EventSequence column. Multiple threads will

occurred as based upon the EventSequence column. Multiple threads will
still be used to simulate multiple spids.
still be used to simulate multiple spids.
–
The “Replay Events Using Multiple Threads” option allows SQL Server
The “Replay Events Using Multiple Threads” option allows SQL Server
Profiler to reorder the order in which each spid starts to execute events,
Profiler to reorder the order in which each spid starts to execute events,
in order to enhance playback performance. However, the order of events
in order to enhance playback performance. However, the order of events
will remain consistent with the EventSequence within a given spid
will remain consistent with the EventSequence within a given spid
21
21
Server-Side Tracing
Server-Side Tracing
22
22



sp_trace_create
sp_trace_create
is used to define a trace
is used to define a trace
and specify an output file location. This
and specify an output file location. This
stored procedure returns a handle to the
stored procedure returns a handle to the
created trace in the form of an integer

created trace in the form of an integer
TraceId
TraceId



sp_trace_setevent
sp_trace_setevent
is used to
is used to
add/remove event/column combinations
add/remove event/column combinations
to traces based on the TraceId
to traces based on the TraceId



sp_trace_setfilter
sp_trace_setfilter
is used to define
is used to define
event filters based on trace columns
event filters based on trace columns



sp_trace_setstatus
sp_trace_setstatus
is called to start,
is called to start,

stop and and delete a trace.
stop and and delete a trace.
–
Traces can be started and stopped multiple times over
Traces can be started and stopped multiple times over
their lifespan
their lifespan
23
23
sp_trace_setfilter
sp_trace_setfilter




@TraceID
@TraceID
,@columnid
,@columnid
,@logical_operator
,@logical_operator
,@comparison_operator
,@comparison_operator
,@value
,@value
/*
/*
@logical_operator: AND (0) or OR (1)
@logical_operator: AND (0) or OR (1)
@comparison_operator:

@comparison_operator:
0 = Equal, 1 = Not equal, 2 = Greater than, 3 =
0 = Equal, 1 = Not equal, 2 = Greater than, 3 =
Less than, 4 = Greater than or equal,
Less than, 4 = Greater than or equal,
5 = Less than or equal, , 6 = Like, 7 = Not like
5 = Less than or equal, , 6 = Like, 7 = Not like
*/
*/
25
25
Querying Server-Side Trace
Querying Server-Side Trace
Metadata
Metadata


26
26



Get Traces
Get Traces
SELECT
SELECT
*
*
FROM sys.traces WHERE [status] = 1
FROM sys.traces WHERE [status] = 1

;
;


This query returns the trace status, which will
This query returns the trace status, which will
be 1 (started) or 0 (stopped); the server-side
be 1 (started) or 0 (stopped); the server-side
path to the trace file (or NULL if the trace is
path to the trace file (or NULL if the trace is
using the rowset provider); the maximum file
using the rowset provider); the maximum file
size (or again, NULL in the case of the rowset
size (or again, NULL in the case of the rowset
provider); information about how many
provider); information about how many
buffers of what size are in use for processing
buffers of what size are in use for processing
the I/O; the number of events captured; and
the I/O; the number of events captured; and
the number of dropped events (NULL if your
the number of dropped events (NULL if your
trace is using the file provider(loseless
trace is using the file provider(loseless
guarantee) ).
guarantee) ).
27
27


Get Events
Get Events


SELECT
SELECT
e.name AS
e.name AS
[e
[e
vent
vent
]
]
,
,
c.name AS
c.name AS
[c
[c
olumn
olumn
]
]
FROM
FROM
fn_trace_geteventinfo(
fn_trace_geteventinfo(
@TraceId

@TraceId
) ei
) ei
INNER
INNER
JOIN sys.trace_events e
JOIN sys.trace_events e
ON ei.eventid = e.trace_event_id
ON ei.eventid = e.trace_event_id
INNER
INNER
JOIN sys.trace_columns c
JOIN sys.trace_columns c
ON ei.columnid =
ON ei.columnid =
c.trace_column_id;
c.trace_column_id;
28
28



Get Filters
Get Filters
SELECT
SELECT


columnid
columnid

,c.name AS
,c.name AS
[c
[c
olumn
olumn
]
]
,logical_operator
,logical_operator
,comparison_operator
,comparison_operator
,value
,value
FROM fn_trace_getfilterinfo(
FROM fn_trace_getfilterinfo(
TraceId
TraceId
)
)
ei
ei
INNER
INNER
JOIN sys.trace_columns c
JOIN sys.trace_columns c
ON ei.columnid =
ON ei.columnid =
c.trace_column_id;
c.trace_column_id;