1430250771 test 8432

CYAN
MAGENTA

YELLOW
BLACK
PANTONE 123 CV

BOOKS FOR PROFESSIONALS BY PROFESSIONALS ®

THE EXPERT’S VOICE ® IN .NET
Companion
eBook Available

Dear Reader,
This book shows you how to write lightweight but powerful test automation in
a .NET programming environment. By lightweight, I mean short (generally less
than two pages of code) and quick (generally less than two hours). If you’ve ever
had to perform manual software testing, you probably found the process slow,
inefficient, and often just plain boring. Using the automation techniques in this
book you can test your software systems quickly and efficiently. During my
years as a software tester at Microsoft and other companies, I discovered that it
was fairly easy to find good information about software testing theory, but
when it came to finding actual concrete test automation examples, there just
wasn’t much information available. I set out to put together in one place all the
test automation techniques I had discovered, and this book is the result.
In Part I of this book, I present techniques for API (Application Programming
Interface) testing. Also called unit testing or module testing, this is the most
fundamental type of software testing. I also show you how to write automated
UI (user interface) tests for Windows form-based applications and how to design
test harness structures. In Part II of this book, I present techniques to write test
automation for Web-based applications. These techniques include automated

HTTP request-response testing, automated UI testing, and automated Web services testing. In Part III of the book, I present test automation techniques that
are related to data. I show you how to automatically generate combinations
and permutations of test case input data. I also present techniques for testing
SQL stored procedures and ADO.NET (data-based) applications. And I give you
code to perform a wide range of XML data tasks.
In short, if you are a software developer, tester, or manager in a .NET environment, you’ll find this book a useful addition to your resources.
RELATED TITLES

Dr. James D. McCaffrey

A Tester’s Guide to
.NET Programming
1-59059-600-5

Join online discussions:

Companion eBook

forums.apress.com

SOURCE CODE ONLINE

FOR PROFESSIONALS BY PROFESSIONALS ™

www.apress.com

Shelve in
Software Development

90000

6

89253 59663

0

McCaffrey

ISBN 1-59059-663-3

See last page for details
on $10 eBook version

.NET Test Automation Recipes

.NET Test Automation Recipes:
A Problem-Solution Approach

.NET Test
Automation
Recipes
A Problem-Solution Approach
Discover how to write lightweight yet powerful test tools in .NET

James D. McCaffrey

9 781590 596630

User level:

Intermediate–Advanced

this print for content only—size & color not accurate

7" x 9-1/4" / CASEBOUND / MALLOY


6633FM.qxd

4/3/06

1:54 PM

Page i

.NET Test Automation
Recipes
A Problem-Solution Approach

James D. McCaffrey


.NET Test Automation Recipes
Copyright © 2012 by James D. McCaffrey
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with
reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed

on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts
thereof is permitted only under the provisions of the Copyright Law of the Publisher's location, in its current
version, and permission for use must always be obtained from Springer. Permissions for use may be obtained
through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the
respective Copyright Law.
ISBN-13 (pbk): 978-1-4302-5077-7
ISBN-13 (electronic): 978-1-4302-5078-4
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the
trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the
material contained herein.
President and Publisher: Paul Manning
Lead Editor: Jonathan Hassell
Technical Reviewer: Josh Kelling
Editorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Louise Corrigan, Morgan
Ertel, Jonathan Gennick, Jonathan Hassell, Robert Hutchinson, Michelle Lowman, James Markham,
Matthew Moodie, Jeff Olson, Jeffrey Pepper, Douglas Pundick, Ben Renow-Clarke, Dominic
Shakeshaft, Gwenan Spearing, Matt Wade, Tom Welsh
Coordinating Editor: Katie Stence
Copy Editor: Julie McNamee
Compositor: Lynn L’Heureux
Indexer: Becky Hornak
Artist: Kurt Krames

Cover Designer: Anna Ishchenko
Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street,
6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail , or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner)
is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware
corporation.
For information on translations, please e-mail , or visit www.apress.com.
Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use.
eBook versions and licenses are also available for most titles. For more information, reference our Special
Bulk Sales–eBook Licensing web page at www.apress.com/bulk-sales.
Any source code or other supplementary materials referenced by the author in this text is available to readers
at www.apress.com. For detailed information about how to locate your book’s source code, go to
www.apress.com/source-code/.


6633FM.qxd

4/3/06

1:54 PM

Page v

Contents at a Glance
About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

■CHAPTER 1

API Testing

■CHAPTER 2

Reflection-Based UI Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

■CHAPTER 3

Windows-Based UI Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

■CHAPTER 4

Test Harness Design Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

■CHAPTER 5

Request-Response Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

■CHAPTER 6

Script-Based Web UI Testing

■CHAPTER 7

Low-Level Web UI Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

■CHAPTER 8

Web Services Testing

■CHAPTER 9

SQL Stored Procedure Testing

....................................................3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

■CHAPTER 10 Combinations and Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
■CHAPTER 11 ADO.NET Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
■CHAPTER 12 XML Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

v


6633FM.qxd

4/3/06

1:54 PM

Page vii

Contents
About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

PART 1

■■■

■CHAPTER 1

Windows Application Testing

API Testing

...................................................3

1.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Storing Test Case Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Reading Test Case Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Parsing a Test Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Converting Data to an Appropriate Data Type . . . . . . . . . . . . . . . . . . . . . 9
1.5 Determining a Test Case Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6 Logging Test Case Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.7 Time-Stamping Test Case Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.8 Calculating Summary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.9 Determining a Test Run Total Elapsed Time . . . . . . . . . . . . . . . . . . . . . 19
1.10 Dealing with null Input/null Expected Results . . . . . . . . . . . . . . . . . . 20
1.11 Dealing with Methods that Throw Exceptions . . . . . . . . . . . . . . . . . . 22
1.12 Dealing with Empty String Input Arguments . . . . . . . . . . . . . . . . . . . . 24
1.13 Programmatically Sending E-mail Alerts on Test Case Failures . . . 26
1.14 Launching a Test Harness Automatically . . . . . . . . . . . . . . . . . . . . . . . 28

1.15 Example Program: ApiTest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

vii


6633Index.qxd

4/3/06

2:00 PM

Page 379

■INDEX

value
determining pass/fail result when stored
procedure under test returns
no, 256–258
of HTML element
manipulating, 194–195
verifying, 195–198
testing stored procedure
that does not return, 314–317
that returns, 306–309
Verification, 208, 227, 247
verify() function, 174–175
verifying value of HTML element on Web
AUT, 174–175, 195–198
ViewState value, programmatically

determining, 152–155
Visual Studio.NET
calling Web methods and, 208
Object Browser, 195
proxy mechanism, 212–214
SOAP message and, 216–217, 224

W
Web application state, determining, 172–173
Web services testing
example program, 232–234
HTTP, using, 220–221
in-memory test case data store
saving to, 229–231
using, 226–228
key code from application that calls Web
service, 212
overview of, 207–208
proxy mechanism, using, 212–214
sockets, using, 214–219, 223
SQL database, creating and
populating, 209–211
TCP, using, 222–225
test run with validation, 208
Web UI testing. See low-level Web UI testing;
script-based Web UI testing
WebApp.aspx application, 168–170
WebClient class, 137–138
WebRequest - WebResponse class, 137
WebRequest object, 139

Weekly Test Runs (WTRs), 126
while loop
Combination.Successor() method
and, 273–274, 287
FindWindow() function and, 71
iterating through test case file with, 7–8
NetworkStream object, 222–225
Permutation.Successor() method
and, 286–287
Socket.Receive() method and, 219
StreamReader.ReadLine() method
inside, 141–142

white box testing, 127
Win32 functions
C# method alias for, 69
FindWindow()
message box, dealing with, 82–86
obtaining handle to main window of
AUT, 68–73
FindWindowByIndex(), 75–77
FindWindowEx(), 73–75
GetMenu(), 86–89
GetMenuItem(), 86–89
GetSubMenu(), 86–89
PostMessage(), 80–82
SendMessage()
application state, checking, 89–91
menu, dealing with, 86–89
PostMessage() compared to, 80

sending characters to control, 78–80
Windows-based UI testing and, 65
window handle, obtaining
to main window of AUT, 68–73
to named control, 73–75
to non-named control, 75–77
Windows Authentication, 241
windows of Internet Explorer, 193
Windows Task Scheduler, launching test
harness automatically using, 28–29
Windows-based UI testing
application state, checking, 89–91
clicking on control, 80–82
example program, 91–96
launching application under test, 66–67
menu, dealing with, 86–89
message box, dealing with, 82–86
obtaining handle
to main window of AUT, 68–73
to named control, 73–75
to non-named control, 75–77
overview of, 65–66
sending characters to control, 78–80
WM CHAR notification message, 78–80
WM_COMMAND notification
message, 86–89
WM_GETTEXT notification message, 89–91
WM_LBUTTONDOWN and
WM_LBUTTONUP notification
messages, 80–82

wrapping test harness code in try-catchfinally block, 102
WriteAttributeString() method, 355–356
WriteEndAttribute() method, 110
WriteEndElement() method, 110
WriteLine() method, 177–178, 249–250,
327–329
WriteStartAttribute() method, 110
WriteStartElement() method, 110, 355–356
WriteString() method, 110, 355–356

379


6633Index.qxd

380

4/3/06

2:00 PM

Page 380

■INDEX

writing
automated tests, 6
procedural versus object-oriented style, 66
reflection-based UI testing, 34
test case results to external storage, 13–15

test results directly to text file from T-SQL
test harness, 249–250
to XML file using XmlTextWriter
class, 355–356
WTRs (Weekly Test Runs), 126

X
XML auto-documentation, 127
XML file data
buffered model test harness, creating
for, 113–117
streaming model test harness, creating
for, 108–112
XML testing
comparing files
for canonical equivalence, 359–361
for exact equality, 356–357
for exact equality, except for
encoding, 358
example program, 361–362
modifying using XSLT, 353–355
overview of, 335–336
parsing
DataSet object, using, 347–350
XmlDocument class, using, 339–341
XmlSerializer class, using, 343–347
XmlTextReader class, using, 337–339
XPathDocument class, using, 341–343
validating
DTD file, using, 353

XSD schema, using, 350–352
writing to XML file using XmlTextWriter
class, 355–356
XML Web services, 208
XmlDataDocument class, 341
XmlDocument class, parsing XML data
using, 339–341
XmlDsigC14NTransform class, 359–361
XmlSerializer class, parsing XML data
using, 343–347
XmlTextReader class, parsing XML data
using, 337–339
XmlTextReader object, 108–112, 117
XmlTextWriter class, writing to XML file
using, 355–356
XmlTextWriter object, 108–112
XmlValidatingReader object, 350–352
XPathDocument class, parsing XML data
using, 341–343
XPathNavigator object, 343
XPathNodeIterator object, 341–343

XSD schema, validating XML data
using, 350–352
XSLT (Extensible Stylesheet Language
Transformations), modifying XML
data using, 353–355
XslTransform object, 353–355