Ebook Designing clinical research (3/E): Part 1

Designing
Clinical Research
THIRD

EDITION

STEPHEN B. HULLEY, M.D., M.P.H.
Professor and Chair, Department of Epidemiology & Biostatistics
Director, Clinical and Translational Sciences Training Program
University of California, San Francisco

STEVEN R. CUMMINGS, M.D.

Founding Director, San Francisco Coordinating Center
Senior Scientist, California Pacific Medical Center Research Institute
Professor Emeritus, University of California, San Francisco

WARREN S. BROWNER, M.D., M.P.H.
Scientific Director, Research Institute
Vice President, Academic Affairs
California Pacific Medical Center
Adjunct Professor, University of California, San Francisco

DEBORAH G. GRADY, M.D., M.P.H.
Professor of Epidemiology & Biostatistics, and of Medicine
Director, Women’s Health Clinical Research Center
Associate Dean for Clinical & Translational Research
University of California, San Francisco

THOMAS B. NEWMAN, M.D., M.P.H.
Professor of Epidemiology & Biostatistics, and of Pediatrics
Chief, Division of Clinical Epidemiology
Attending Physician, Department of Pediatrics
University of California, San Francisco


Acquisitions Editor: Sonya Seigafuse
Managing Editor: Nancy Winter
Project Manager: Jennifer Harper
Manufacturing Coordinator: Kathleen Brown
Marketing Manager: Kimberly Schonberger
Designer: Stephen Druding
Production Services: Laserwords Private Limited, Chennai, India
Printer: Data Reproductions Corporation
© 2007 by LIPPINCOTT WILLIAMS & WILKINS, a Wolters Kluwer business
530 Walnut Street
Philadelphia, PA 19106 USA
LWW.com
1998 Williams & Wilkins
2001 Lippincott Williams & Wilkins
All rights reserved. This book is protected by copyright. No part of this book may be reproduced in any
form or by any means, including photocopying, or utilized by any information storage and retrieval system
without written permission from the copyright owner, except for brief quotations embodied in critical articles
and reviews. Materials appearing in this book prepared by individuals as part of their official duties as U.S.
government employees are not covered by the above-mentioned copyright.
Printed in the USA

Library of Congress Cataloging-in-Publication Data
Designing clinical research / by Stephen B. Hulley . . . [et al.].— 3rd ed.

p. ; cm.
Includes bibliographical references and index.
ISBN-13: 978-0-7817-8210-4
ISBN-10: 0-7817-8210-4
1. Clinical trials. 2. Medicine—Research—Methodology.
3. Epidemiology—Research—Methodology. I. Hulley, Stephen B.
[DNLM: 1. Epidemiologic Methods. 2. Research Design. WA 950 D457 2007]
R853.C55D47 2007
610.72—dc22
2006028271
Care has been taken to confirm the accuracy of the information presented and to describe generally
accepted practices. However, the authors, editors, and publisher are not responsible for errors or omissions or
for any consequences from application of the information in this book and make no warranty, expressed or
implied, with respect to the currency, completeness, or accuracy of the contents of the publication. Application
of this information in a particular situation remains the professional responsibility of the practitioner.
The authors, editors, and publisher have exerted every effort to ensure that drug selection and dosage set
forth in this text are in accordance with current recommendations and practice at the time of publication.
However, in view of ongoing research, changes in government regulations, and the constant flow of
information relating to drug therapy and drug reactions, the reader is urged to check the package insert for
each drug for any change in indications and dosage and for added warnings and precautions. This is
particularly important when the recommended agent is a new or infrequently employed drug.
Some drugs and medical devices presented in this publication have Food and Drug Administration
(FDA) clearance for limited use in restricted research settings. It is the responsibility of the health care
provider to ascertain the FDA status of each drug or device planned for use in their clinical practice.
To purchase additional copies of this book, call our customer service department at (800) 638-3030 or
fax orders to (301) 223-2320. International customers should call (301) 223-2300.
Visit Lippincott Williams & Wilkins on the Internet: at LWW.com. Lippincott Williams & Wilkins
customer service representatives are available from 8:30 am to 6 pm, EST.
10 9 8 7 6 5 4 3 2 1

To our families and our students



CONTENTS

Contributing Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Section I: Basic Ingredients
1. Getting Started: The Anatomy and
Physiology of Clinical Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Anatomy of Research: What It’s Made Of . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Physiology of Research: How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Designing the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Appendix 1.1: Outline of a Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2. Conceiving the Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Origins of a Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Characteristics of a Good Research Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Developing the Research Question and Study Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Translational Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

19
22
23
25
25

3. Choosing the Study Subjects: Specification, Sampling,
and Recruitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Basic Terms and Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Selection Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Recruitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 3.1: Selecting a Random Sample from a Table of Random Numbers . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

28
29
32
33
35
36
36

v


vi

Contents

4. Planning the Measurements: Precision and Accuracy . . . . . . . . . 37
Measurement Scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Other Features of Measurement Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Measurements on Stored Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
In Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Appendix 4.1: Operations Manual: Operational Definition of a Measurement of Grip
Strength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5. Getting Ready to Estimate Sample Size: Hypotheses
and Underlying Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Underlying Statistical Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Additional Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51
54
59
62
63

6. Estimating Sample Size and Power: Applications
and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Sample Size Techniques for Analytic Studies and Experiments . . . . . . . . . . . . . . . . . . .

Other Considerations and Special Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sample Size Techniques for Descriptive Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
What to do When Sample Size is Fixed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Strategies for Minimizing Sample Size and Maximizing Power . . . . . . . . . . . . . . . . . . . .
How to Estimate Sample Size When there is Insufficient Information . . . . . . . . . . . . . .
Common Errors to Avoid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 6A: Sample Size Required per Group When Using the t Test
to Compare Means of Continuous Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 6B: Sample Size Required per Group When Using the Chi-Squared
Statistic or Z Test to Compare Proportions of Dichotomous Variables . . . . . . . . . . . .
Appendix 6C: Total Sample Size Required When Using the Correlation
Coefficient (r ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 6D: Sample Size for a Descriptive Study of a Continuous
Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 6E: Sample Size for a Descriptive Study of a Dichotomous
Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix 6F: Use and Misuse of t Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65
71
73
75
76
81
82
83
84
86

89
90
91
92
93


Contents

vii

Section II: Study Designs
7. Designing a Cohort Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Prospective Cohort Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Retrospective Cohort Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Nested Case–Control and Case–Cohort Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Multiple-Cohort Studies and External Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Other Cohort Study Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

8. Designing Cross-Sectional and Case–Control Studies . . . . . . . . 109
Cross-Sectional Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Case–Control Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Choosing Among Observational Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Appendix 8A: Calculating Measures of Association . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Appendix 8B: Why the Odds Ratio Can Be Used as an Estimate for Relative Risk
in a Case–Control Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

9. Enhancing Causal Inference in Observational Studies . . . . . . . . 127
Spurious Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Real Associations Other than Cause–Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Coping With Confounders in the Design Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Coping with Confounders in the Analysis Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Underestimation of Causal Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Choosing a Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Appendix 9A: Hypothetical Example of Confounding and Interaction . . . . . . . . . . . . 144
Appendix 9B: A Simplified Example of Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

10.

Designing a Randomized Blinded Trial . . . . . . . . . . . . . . . . . . . . . . . . 147
Selecting the Intervention and Control Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Choosing Outcome Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
Selecting the Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
Measuring Baseline Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Randomizing and Blinding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160


viii

11.

Contents

Alternative Trial Designs and Implementation Issues . . . . . . . . . 163
Alternative Clinical Trial Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Conducting a Clinical Trial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Appendix 11.1: Interim Monitoring of Trial Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

12.

Designing Studies of Medical Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Determining Whether a Test is Useful . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Studies of Test Reproducibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Studies of the Accuracy of Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Studies of the Effect of Test Results on Clinical Decisions . . . . . . . . . . . . . . . . . . . . . . . 192
Studies of Feasibility, Costs, and Risks of Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Studies of the Effect of Testing on Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Pitfalls in the Design or Analysis of Diagnostic Test Studies . . . . . . . . . . . . . . . . . . . . . 196
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Appendix 12A: Calculation of Kappa to Measure Interobserver Agreement . . . . . 200
Appendix 12B: Numerical Example of Verification Bias: 1 . . . . . . . . . . . . . . . . . . . . . . . 202
Appendix 12C: Numerical Example of Verification Bias: 2 . . . . . . . . . . . . . . . . . . . . . . . 203
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

13.

Utilizing Existing Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Advantages and Disadvantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Secondary Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Ancillary Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Systematic Reviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Appendix 13.1: Statistical Methods for Meta-Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Section III: Implementation
14.

Addressing Ethical Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Ethical Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Federal Regulations for Research on Human Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
Research Participants who Require Additional Protections . . . . . . . . . . . . . . . . . . . . . . 231
Responsibilities of Investigators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Ethical Issues Specific to Certain Types of Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Other Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236


Contents

ix

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

15.

Designing Questionnaires and Interviews . . . . . . . . . . . . . . . . . . . . . 241
Designing Good Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Steps in Assembling the Instruments for the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
Administering the Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Appendix 15.1: An Example of a Questionnaire about Smoking . . . . . . . . . . . . . . . . . 254
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

16.

Data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Data Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Data Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
Extracting Data (Queries) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
Identifying and Correcting Errors in the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Analysis of the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Confidentiality and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

17.

Implementing the Study and Quality Control . . . . . . . . . . . . . . . . . . 271
Assembling Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
Finalizing the Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Quality Control During the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
Appendix 17.1: Example of an Operations Manual Table of Contents . . . . . . . . . . . . 286
Appendix 17.2: Quality Control Tables and Checklists . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

18.

Community and International Studies . . . . . . . . . . . . . . . . . . . . . . . . . 291

Why Community and International Research? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
Community Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
International Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

19.

Writing and Funding a Research Proposal . . . . . . . . . . . . . . . . . . . . . 301
Writing Proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
Elements of a Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
Characteristics of Good Proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309


x

Contents

Finding Support for Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353


CONTRIBUTING AUTHORS

Norman Hearst M.D., M.P.H.
Professor of Family & Community Medicine, and of Epidemiology & Biostatistics

Attending Physician, Department of Family and Community Medicine
University of California, San Francisco

Michael A. Kohn M.D., M.P.P.
Associate Clinical Professor of Epidemiology & Biostatistics
Director, Data Management Consulting Services,
Clinical & Translational Sciences Institute
University of California, San Francisco

Bernard Lo, M.D.
Professor and Director, Program in Medical Ethics
Attending Physician, Department of Medicine, UCSF Medical Center
University of California, San Francisco

Jeffrey N. Martin, M.D., M.P.H.
Associate Professor of Epidemiology & Biostatistics, and of Medicine
Director, Training In Clinical Research Program
Attending Physician, Department of Medicine, San Francisco General Hospital
University of California, San Francisco

Thomas E. Novotny, M.D., M.P.H.
Professor in Residence, Epidemiology & Biostatistics
Director, International Programs, School of Medicine
University of California, San Francisco

xi



INTRODUCTION

This book is about the science of doing clinical research in all its forms: translational
research, clinical trials, patient-oriented research, epidemiologic studies, behavioral
science and health services research. Codifying the nature of this broad-based science
and how to do it is not straightforward, and there is no single approach that everyone
agrees is best. Our first two editions drew on the terms and principles of epidemiology
in a practical and reader-friendly way, emphasizing systematic and common sense
approaches to the many judgments involved in designing a study.
The Third Edition of Designing Clinical Research (DCR) follows the same
path, adding new developments along the way. New material on observational
studies includes case-crossover designs, and the use of propensity scores, instrumental
variables and Mendelian randomization to control confounding. Reorganized chapters
on clinical trials introduce adaptive designs, and those on studying medical tests and on
utilizing existing datasets present expanded options that will be attractive to beginning
investigators. The chapter on research ethics is extensively updated, the one on data
management entirely new (reflecting current approaches to information technology),
and a rewritten chapter on study implementation and quality control introduces
practicalities of study startup and regulatory issues (‘‘Good Clinical Practices’’). An
updated chapter on getting funded brings help for the challenges facing young
investigators.
The Third Edition is also fresh throughout, with updated examples and references
in every chapter. And, perhaps most important, it reflects a continued maturation
of our thinking, aided by feedback from nearly 1000 health professionals that we
helped design their own studies in our Designing Clinical Research workshop in the
past 6 years since DCR 2. The syllabus for that workshop, which can be used by
others who wish to teach this material or desire a self-instruction guide, has been
combined with useful tools like a sample size calculator on our new DCR Website at
www.epibiostat.ucsf.edu/dcr/.
Many things have not changed in the Third Edition. It is still a simple book
that leaves out unnecessary technicalities and invites the investigator to focus on the

most important things: finding a good research question and planning an efficient,
effective, ethical design. The two chapters on sample size estimation, which have
received a larger number of favorable comments from readers than any other part
of the book, continue to demystify the process and enable readers to make these
calculations themselves without the need for formulas. We still use the feminine
pronoun in the first half of the book, masculine in the second, reasoning that we
are writing for clinical investigators of both genders. And we still do not address the
important area of statistical analysis, nor how to go about disseminating the findings
of clinical research—topics that many readers of this book will wish to pursue (1–5).
New investigators often find the choice of a research question to be the most
difficult step in designing a project. Fortunately, studies tend to generate more
questions than they answer, and an investigator’s awareness of researchable questions

xiii


xiv

Introduction

will grow as she gains experience. In the meantime, she should seek out her own
version of the most important companion to this book, a long-term relationship with
an excellent mentor or two.
Other benefits come from experience. Clinical research becomes easier and more
rewarding as investigators gain familiarity with the particulars of recruitment, measurement and design that pertain to their area of specialization. A higher percentage of
their applications for funding are successful. They acquire staff and junior colleagues
and develop lasting friendships with scientists working on the same topic in distant
places. And because most increments in knowledge are small and uncertain—major
scientific breakthroughs are rare—they begin to see substantial changes in the state
of medicine as an aggregate result of their efforts.

It is gratifying to know many people who have used this book who have found
they like doing research, and have settled into a great career. For those with inquiring
minds, the pursuit of truth can become a lifelong fascination. For perfectionists and
craftsmen, there are endless challenges in creating an elegant study that produces
conclusive answers to important questions at an affordable cost in time and money.
And for those with the ambition to make a lasting contribution to society, there is the
prospect that skill, tenacity and luck may lead to important advances in knowledge.

REFERENCES
1. Vittinghoff E, Glidden DV, Shiboski SC, et al. Regression methods in biostatistics: linear,
logistic, survival, and repeated measures models. New York: Springer-Verlag, 2005.
2. Katz MH. Multivariable analysis: a practical guide for clinicians, 2nd ed. New York:
Cambridge University Press, 2006.
3. Glantz SA. Primer of biostatistics, 5th ed. McGraw-Hill, 2005.
4. Browner WS. Publishing and presenting clinical research, 2nd ed. Philadelphia, PA: Lippincott Williams & Wilkins, 2006.
5. Fletcher RH, Fletcher SW. Clinical epidemiology: the essentials, 4th ed. Philadelphia, PA:
Lippincott Williams & Wilkins, 2005.


ACKNOWLEDGMENTS

We are grateful to the Andrew P. Mellon Foundation for bringing us together more
than two decades ago and stimulating the first edition; to our publisher for steadily
inviting second and now third editions until resistance became futile; to our families
for their patient support as we labored over this opus during family time; to many
colleagues at UCSF and beyond, whose ideas and skills have influenced ours; to
our students over the years, whose accomplishments have been fun to watch and
stimulating to our thinking; and to our readers who have put this book to use.

xv

SECTION I

Basic Ingredients



1 Getting Started: The
Anatomy and Physiology
of Clinical Research
Stephen B. Hulley, Thomas B. Newman,
and Steven R. Cummings

This chapter introduces clinical research from two viewpoints, setting up themes that
run together through the book. One is the anatomy of research—what it’s made
of. This includes the tangible elements of the study plan: research question, design,
subjects, measurements, sample size calculation, and so forth. An investigator’s goal
is to create these elements in a form that will make the project feasible, efficient, and
cost-effective.
The other theme is the physiology of research—how it works. Studies are useful
to the extent that they yield valid inferences, first about what happened in the study
sample and then about how these study findings generalize to people outside the
study. The goal is to minimize the errors, random and systematic, that threaten
conclusions based on these inferences.
Separating the two themes is artificial in the same way that the anatomy of the
human body doesn’t make much sense without some understanding of its physiology.
But the separation has the same advantage: it clarifies our thinking about a complex
topic.

ANATOMY OF RESEARCH: WHAT IT’S MADE OF
The structure of a research project is set out in its protocol, the written plan of the
study. Protocols are well known as devices for seeking grant funds, but they also have
a vital scientific function: helping the investigator organize her research in a logical,
focused, and efficient way. Table 1.1 outlines the components of a protocol. We
introduce the whole set here, expand on each component in the ensuing chapters of
the book, and return to put the completed pieces together in Chapter 19.

Research Question
The research question is the objective of the study, the uncertainty the investigator
wants to resolve. Research questions often begin with a general concern that must be

3


4

Basic Ingredients

TABLE 1.1

Outline of the Study Protocol

Element

Purpose

Research questions

What questions will the study address?

Background and significance

Why are these questions important?

Design

How is the study structured?

Time frame
Epidemiologic approach
Subjects

Who are the subjects and how will they be selected?

Selection criteria
Sampling design
Variables

What measurements will be made?

Predictor variables
Confounding variables
Outcome variables
Statistical issues

How large is the study and how will it be analyzed?

Hypotheses

Sample size
Analytic approach

narrowed down to a concrete, researchable issue. Consider, for example, the general
question:
Should people eat more fish?
This is a good place to start, but the question must be focused before planning
efforts can begin. Often this involves breaking the question into more specific
components, and singling out one or two of these to build the protocol around. Here
are some examples:
How often do Americans eat fish?
Does eating fish lower the risk of cardiovascular disease?
Is there a risk of mercury toxicity from increasing fish intake in older adults?
Do fish oil supplements have the same effects on cardiovascular disease as dietary fish?
Which fish oil supplements don’t make people smell like fish?
A good research question should pass the ‘‘So what?’’ test. Getting the answer
should contribute usefully to our state of knowledge. The acronym FINER denotes


Chapter 1 ■ Getting Started: The Anatomy and Physiology of Clinical Research

5

five essential characteristics of a good research question: it should be feasible,
interesting, novel, ethical, andrelevant (Chapter 2).

Background and Significance
The background and significance section of a protocol sets the proposed study
in context and gives its rationale: What is known about the topic at hand? Why is
the research question important? What kind of answers will the study provide? This

section cites previous research that is relevant (including the investigator’s own work)
and indicates the problems with the prior research and what uncertainties remain. It
specifies how the findings of the proposed study will help resolve these uncertainties
and lead to new scientific knowledge and influence practice guidelines or public
health policy. Often, work on the significance section will lead to modifications in the
research question.

Design
The design of a study is a complex issue. A fundamental decision is whether to
take a passive role in observing the events taking place in the study subjects in an
observational study or to apply an intervention and examine its effects on these events
in a clinical trial (Table 1.2). Among observational studies, two common designs are
cohort studies, in which observations are made in a group of subjects that is followed
over time, and cross-sectional studies, in which observations are made on a single
occasion. Cohort studies can be further divided into prospective studies that begin in
the present and follow subjects into the future, and retrospective studies that examine
information and specimens that have been collected in the past. A third common
option is the case-control design, in which the investigator compares a group of people
who have a disease or condition with another group who do not. Among clinical trial
options, the randomized blinded trial is usually the best design but nonrandomized
or unblinded designs may be more suitable for some research questions.
No one approach is always better than the others, and each research question
requires a judgment about which design is the most efficient way to get a satisfactory
answer. The randomized blinded trial is often held up as the best design for establishing
causality and the effectiveness of interventions, but there are many situations for which
an observational study is a better choice or the only feasible option. The relatively
low cost of case-control studies and their suitability for rare outcomes makes them
attractive for some questions. Special considerations apply to choosing designs for
studying diagnostic tests. These issues are discussed in Chapters 7 through 12, each
dealing with a particular set of designs.

A typical sequence for studying a topic begins with observational studies of a
type that is often called descriptive. These studies explore the lay of the land—for
example, describing distributions of diseases and health-related characteristics in the
population:
What is the average number of servings of fish per week in the diet of Americans with
a history of coronary heart disease (CHD)?
Descriptive studies are usually followed or accompanied by analytic studies that
evaluate associations to permit inferences about cause-and-effect relationships:
Is there an association between fish intake and risk of recurrent myocardial infarction
in people with a history of CHD?


6

Basic Ingredients

TABLE 1.2

Examples of Common Clinical Research Designs Used to Find Out
Whether Fish Intake Reduces Coronary Heart Disease Risk

Study Design

Key Feature

Example

Observational Designs
Cohort study

A group followed over time

The investigator measures fish intake
at baseline and periodically examines
subjects at follow-up visits to see if
those who eat more fish have fewer
coronary heart disease (CHD) events

Cross-sectional study

A group examined at one point
in time

She interviews subjects about current
and past history of fish intake and
correlates results with history of CHD
and current coronary calcium score

Case-control study

Two groups selected based on
the presence or absence of an
outcome

She examines a group of patients with
CHD (the ‘‘cases’’) and compares
them with a group who did not have
CHD (the controls), asking about past
fish intake

Clinical Trial Design
Randomized blinded trial

Two groups created by a
random process, and a blinded
intervention

She randomly assigns subjects to
receive fish oil supplements or
placebo, then follows both treatment
groups for several years to observe
the incidence of CHD

The final step is often a clinical trial to establish the effects of an intervention:
Does treatment with fish oil capsules reduce total mortality in people with CHD?
Clinical trials usually occur relatively late in a series of research studies about a
given question, because they tend to be more difficult and expensive, and to answer
more definitively the narrowly focused questions that arise from the findings of
observational studies.
It is useful to characterize a study in a single sentence that summarizes the design
and research question. If the study has two major phases, the design for each should
be mentioned.
This is a cross-sectional study of dietary habits in 50- to 69-year-old people with
a history of CHD, followed by a prospective cohort study of whether fish intake is
associated with low risk of subsequent coronary events.
This sentence is the research analog to the opening sentence of a medical
resident’s report on a new hospital admission: ‘‘This 62-year-old white policewoman
was well until 2 hours before admission, when she developed crushing chest pain radiating
to the left shoulder.’’ Some designs do not easily fit into the categories listed above,
and classifying them with a single sentence can be surprisingly difficult. It is worth the

Chapter 1 ■ Getting Started: The Anatomy and Physiology of Clinical Research

7

effort—a precise description of design and research question clarifies the investigator’s
thoughts and is useful for orienting colleagues and consultants.

Study Subjects
Two major decisions must be made in choosing the study subjects (Chapter 3). The
first is to specify inclusion andexclusion criteria that define the target population: the
kinds of patients best suited to the research question. The second decision concerns
how to recruit enough people from an accessible subset of this population to be the
subjects of the study. For example, the study of fish intake in people with coronary
heart disease (CHD) might identify subjects seen in the clinic with diagnosis codes
for myocardial infarction, angioplasty, or coronary artery bypass grafting in their
electronic medical record. Decisions about which patients to study represent tradeoffs; studying a random sample of people with CHD from the entire country (or at
least several different states and medical care settings) would enhance generalizability
but be much more difficult and costly.

Variables
Another major set of decisions in designing any study concerns the choice of which
variables to measure (Chapter 4). A study of fish intake in the diet, for example,
might ask about different types of fish that contain different levels of -3 fatty acids,
and include questions about portion size, whether the fish was fried or baked, and
whether the subject takes fish oil supplements.
In an analytic study the investigator studies the associations among variables
to predict outcomes and to draw inferences about cause and effect. In considering
the association between two variables, the one that occurs first or is more likely on

biologic grounds to be causal is called the predictor variable; the other is called the
outcome variable.1 Most observational studies have many predictor variables (age,
race, sex, smoking history, fish and fish oil supplement intake), and several outcome
variables (heart attacks, strokes, quality of life, unpleasant odor).
Clinical trials examine the effects of an intervention (a special kind of predictor
variable that the investigator manipulates), such as treatment with fish oil capsules. This
design allows her to observe the effects on the outcome variable using randomization
to control for the influence of confounding variables—other predictors of the
outcome such as intake of red meat or income level that could be related to dietary
fish and confuse the interpretation of the findings.

Statistical Issues
The investigator must develop plans for estimating sample size and for managing and
analyzing the study data. This generally involves specifying a hypothesis (Chapter 5).
Hypothesis: 50- to 69-year-old women with CHD who take fish oil supplements will
have a lower risk of myocardial infarction than those who do not.
This is a version of the research question that provides the basis for testing the
statistical significance of the findings. The hypothesis also allows the investigator to
calculate the sample size—the number of subjects needed to observe the expected
difference in outcome between study groups with reasonable probability or power
1

Predictors are sometimes termed independent variables and outcomes dependent variables, but we find this usage
confusing, particularly since independent means something quite different in the context of multivariate analyses.