Election Auditing Bibliography potx

Election Auditing Bibliography
Joseph Lorenzo Hall
∗
February 12, 2010 (v3.8)
†
Abstract
This bibliography lists all work in the field of election auditing of which the author is aware. It
takes an expansive view of the term election audit to encompass pre-election audit activities as well as
post-election audit activities such as manual tallies, ballot reconciliation and forensic machine audits.
Entries are roughly sorted by publication date. No attempt has been made to include articles from
the popular press; this list represents scholarly, advocate, election official and vendor publications
relevant to election auditing.
1. Roy G. Saltman, Effective Use of Computing Technology in Vote-Tallying.
1
National Bureau of Stan-
dards, Mar. 1975.
2. Walter R. Mebane, Jr., Jasjeet S. Sekhon, and Jonathan Wand, Detecting and Correcting Election
Irregularities.
2
Stanford University, 2003.
3. C. Andrew Neff, Election Confidence—A Comparison of Methodologies and Their Relative Effective-
ness at Achieving It (Revision 6).
3
Dec. 2003.
4. Stephen Ansolabehere and Andrew Reeves, Using Recounts to Measure the Accuracy of Vote Tab-
ulations: Evidence from New Hampshire Elections 1946-2002.
4
CalTech/MIT Voting Technology
Project, 2004.
5. Jonathan Wand, Auditing an Election Using Sampling: The impact of bin size on the probability of
detecting manipulation.

5
Stanford University, 2004.
6. Douglas W. Jones, “Auditing Elections”.
6
Communications of the ACM 47:10, 46–50 (Oct. 2004).
7. Kenneth C. Johnson, Election Certification by Statistical Audit of Voter-Verified Paper Ballots.
7
Oct.
2004.
8. Kathy Dopp and Ron Baiman, How Can Independent Paper Audits Detect and Correct Vote Mis-
counts?
8
, 2005.
∗
Joe is the ACCURATE Postdoctoral Research Associate at the University of California at Berkeley, School of Information
and Princeton University, Center for Information Technology Policy. Please feel free to contact the author by email (joehall@
berkeley.edu) or via other methods (see: Others that have contributed substantially to this
bibliography include Joseph Calandrino.
†
The most current version of this document is always available here: The
associated BIBT
E
X file is here: />1
/>2
/>3
/>4
/>5
/>6
/>7
/>8

/>1
9. Ellen Theisen, Auditing Election Equipment—The Real Scoop!
9
, Aug. 2005.
10. Michael J. Alvarez, Jonathan N. Katz, and Sarah A. Hill, Machines versus Humans: The Counting
and Recounting of Pre-Scored Punchcard Ballots.
10
CalTech/MIT Voting Technology Project, 2005.
11. Arel Cordero, David Wagner, and David Dill, “The Role of Dice in Election Audits—Extended Ab-
stract”.
11
IAVoSS Workshop on Trustworthy Elections 2006 (WOTE 2006) (June 2006).
12. William D. Kalsbeek and Lei Zhang, An Assessment of the May 2006 Election Recount and a Proposed
Permanent Recount Sample Design.
12
University of North Carolina at Chapel Hill, School of Public
Health’s Survey Research Unit, July 2006.
13. Steven Hertzberg, DRE Analysis for May 2006 Primary Cuyahoga County, Ohio.
13
Election Science
Institute, 2006.
14. Lawrence Norden and Eric Lazarus, The Machinery of Democracy: Protecting Elections in an Elec-
tronic World: Brennan Center Task Force on Voting System Security.
14
Brennan Center for Justice
at NYU School of Law, Aug. 2006.
15. Howard Stanislevic, Random Auditing of E-Voting Systems: How Much Is Enough?
15
, 2006.
16. Kathy Dopp and Frank Stenger, The Election Integrity Audit.

16
National Election Data Archive, Sept.
2006.
17. Jonathan D. Simon and Bruce O’Dell, An End to “Faith-Based” Voting: Universal Precinct-Based
Handcount Sampling to Check Computerized Vote Counts in Federal and Statewide Elections.
17
Election Defense Alliance, Sept. 2006.
18. Jerry Lobdill, Election Audit Sampling Plan—It’s Not Just About Sampling Without Replacement.
18
Oct. 2006.
19. Anonymous, The Titanium Standard for Election Verification and Security.
19
Oct. 2006.
20. Jerry Lobdill, Considering Vote Count Distribution in Designing Election Audits.
20
Nov. 2006.
21. Judy Bertelsen, 1% Manual Tally Observer Report, Congressional District 11.
21
Counted as Cast,
2007.
22. Kathy Dopp, How Big Should an Election Audit Be? Fixed Rate Audits Do Not Work For Elections.
22
Jan. 2007.
23. Kathy Dopp, Tiered Election Audits Based on Margins between Candidates.
23
Jan. 2007.
24. Kathy Dopp, Fool Me Once: Checking Vote Count Integrity.
24
Mar. 2007.
9

/>10
/>11
/>12
/>13
/>14
/>15
/>16
/>17
/>18
/>pdf
19
/>20
/>pdf
21
/>22
/>23
/>24
/>2
25. Joseph Lorenzo Hall, A Quick Primer on the Mathematics of Post-Election Audit Confidence.
25
Mar.
2007.
26. Mark Halvorson and Laura Wolff, Report and Analysis of the 2006 Post-Election Audit of Minnesota’s
Voting Systems.
26
Apr. 2007.
27. Collaborative Public Audit of the November 2006 General Election.
27
The Cuyahoga County Col-
laborative Audit Committee & Cleveland State University Center for Election Integrity, Apr. 2007.

28. Voter Verified Paper Audit Trail Pilot Project Report.
28
Georgia Office of the Secretary of State
Elections Division, Apr. 2007.
29. William D. Kalsbeek and Lei Zhang, An Assessment of the November 2006 Election Recount. Univer-
sity of North Carolina at Chapel Hill, School of Public Health’s Survey Research Unit, May 2007.
30. David Marker, John Gardenier, and Arlene Ash, “Statistics Can Help Ensure Accurate Elections”.
29
Amstat News (June 2007).
31. Anthony Stevens, Hand Counting Paper Ballots.
30
Democracy Fest Annual National Convention,
June 2007.
32. David Jefferson, Elaine Ginnold, Kathleen Midstokke, Kim Alexander, Philip B. Stark, and Amy
Lehmkuhl, Evaluation of Audit Sampling Models and Options for Strengthening California’s Manual
Count.
31
California Secretary of State, July 2007.
33. Joseph A. Calandrino, J. Alex Halderman, and Edward W. Felten, “Machine-Assisted Election Audit-
ing”.
32
USENIX/ACCURATE Electronic Voting Technology Workshop 2007 (Aug. 2007).
34. Stephen N. Goggin and Michael D. Byrne, “An Examination of the Auditability of Voter Verified
Paper Audit Trail (VVPAT) Ballots”.
33
USENIX/ACCURATE Electronic Voting Technology Workshop
2007 (Aug. 2007).
35. Lawrence Norden, Aaron Burstein, Joseph Lorenzo Hall, and Margaret Chen, Post-Election Audits:
Restoring Trust in Elections.
34

Brennan Center for Justice at The New York University School of
Law and The Samuelson Law, Technology and Public Policy Clinic at the University of California,
Berkeley School of Law (Boalt Hall), Aug. 2007.
36. Javed A. Aslam, Raluca A. Popa, and Ronald L. Rivest, “On Estimating the Size and Confidence of a
Statistical Audit”.
35
USENIX/ACCURATE Electronic Voting Technology Workshop 2007 (Aug. 2007).
37. Andrew W. Appel, Effective Audit Policy for Voter-Verified Paper Ballots.
36
Center for Information
Technology Policy / Department of Computer Science, Princeton University, Sept. 2007.
38. Ronald L. Rivest, A Simple Rule of Thumb for Election Audit Size Determination.
37
Massachusetts
Institute of Technology, 2007.
39. Howard Stanislevic, NY Election Audits: Is Three Perecent Enough?
38
, Nov. 2007.
40. White Paper: FSASE Consideration of Audit Legislation in 2008.
39
Florida State Association of
25
/>26
/>27
/>28
/>29
/>30
/>31
/>32
/>33

/>34
/>35
/>36
/>37
/>38
/>39
/>3
Supervisors of Elections, Nov. 2007.
41. Alexander Shvartsman, Aggelos Kiayias, Laurent Michel, Alex Russell, Seda Davtyan, Sotiris Ken-
tros, Nicolas Nicolaou, Andrew See, and Narasimha Shashidhar, Pre-Election Audit of Memory Cards
for the November 2007 Connecticut Elections.
40
UConn Voting Technology Research Center, 2008.
42. John McCarthy, Howard Stanislevic, Mark Lindeman, Arlene Ash, Vittorio Addona, and Mary Batcher,
“Percentage-Based versus Statistical-Power-Based Vote Tabulation Audits”.
41
American Statistician
62, 11–16 (Feb. 2008).
43. Philip B. Stark, Election Audits by Sampling with Probability Proportional to an Error Bound: dealing
with discrepancies.
42
University of California at Berkeley Department of Statistics, 2008.
44. Consideration of Ohio Audits in 2008.
43
Ohio Joint Audit Working Group, Feb. 2008.
45. Joseph Lorenzo Hall, Research Memorandum: On Improving the Uniformity of Randomness with
Alameda County’s Random Selection Process.
44
University of California at Berkeley, School of In-
formation, Mar. 2008.

46. Joseph Lorenzo Hall, Dice Binning Calculator for Post-Election Audits.
45
University of California at
Berkeley School of Information, Mar. 2008.
47. Arlene Ash and John Lamperti, “Florida’s District 13 Election in 2006: Can Statistics Tell Us Who
Won?”.
46
Chance 21:2 (Apr. 2008).
48. Joseph Lorenzo Hall, “Statistical Solutions to Election Mysteries”.
47
Chance 21:2 (Apr. 2008).
49. Joseph Lorenzo Hall, Procedures for California’s 1% Manual Tally.
48
University of California at
Berkeley, School of Information, Apr. 2008.
50. Arel Cordero, DiceHash: Publicly verifiable random number generator.
49
University of California
at Berkeley, Department of Computer Science, Apr. 2008.
51. Ronald L. Rivest, A “Sum of Square Roots” (SSR) Pseudorandom Sampling Method for Election Au-
dits.
50
Apr. 2008.
52. John Gardenier, “Roles for Statisticians in Elections”.
51
Mathematics Awareness Month 2008 (June
2008).
53. Recommended Audit Procedures.
52
Ohio Joint Audit Working Group, June 2008.

54. Javed A. Aslam, Raluca A. Popa, and Ronald L. Rivest, “On Auditing Elections When Precincts Have
Different Sizes”.
53
USENIX/ACCURATE Electronic Voting Technology Workshop 2008 (July 2008).
55. Joseph A. Calandrino, J. Alex Halderman, and Edward W. Felten, “In Defense of Pseudorandom
Sample Selection”.
54
USENIX/ACCURATE Electronic Voting Technology Workshop 2008 (July 2008).
40
/>41
/>42
/>43
/>44
/>45
/>46
/>47
/>48
/>49
/>50
/>pdf
51
/>52
/>53
/>54
/>4
56. Stephen N. Goggin, Michael D. Byrne, Juan E. Gilbert, Gregory Rogers, and Jerome McClendon,
“Comparing the Auditability of Optical Scan, Voter Verified Paper Audit Trail (VVPAT) and Video
(VVVAT) Ballot Systems”.
55
USENIX/ACCURATE Electronic Voting Technology Workshop 2008 (July

2008).
57. Joseph Lorenzo Hall, “Improving the Security, Transparency and Efficiency of California’s 1% Man-
ual Tally Procedures”.
56
USENIX/ACCURATE Electronic Voting Technology Workshop 2008 (July
2008).
58. Philip B. Stark, “Conservative Statistical Post-Election Audits”.
57
The Annals of Applied Statistics 2,
550–581 (July 2008).
59. Matt Bishop, Mark Graff, Candice Hoke, David Jefferson, and Sean Peisert, Resolving the Unexpected
in Elections: Election Officials’ Options.
58
Oct. 2008.
60. Philip B. Stark, “A Sharper Discrepancy Measure for Post-Election Audits”.
59
The Annals of Applied
Statistics 2, 982–985 (Nov. 2008).
61. Principles and Best Practices for Post-Election Audits.
60
ElectionAudits.org, Sept. 2008.
62. Linda H. Lamone, Ross Goldstein, Nikki Baines Trella, B. Gary Dando, Espey T. Browning, John
T. Willis, Dennis McGrath, Mark Brown, and Kon Woo Kim, Development of a Pilot Election Audit
Program: A Project for The Pew Charitable Trusts—Make Voting Work. Maryland State Board of
Elections, Dec. 2008.
63. Lonna Rae Atkeson, R. Michael Alvarez, and Thad E. Hall, The New Mexico 2006 Post Election Audit
Report.
61
Sept. 2008.
64. Report and Feedback November 2008 Connecticut Post-Election Audit Observation.

62
Connecticut
Citizen Election Audit Coalition, Jan. 2009.
65. Judy Duffy, Norman Turrill, Ed Gracely, Mark Halvorson, Barbara Hankins, Kurt Miller, Lu Pierson,
and Barbara Simons, Report on Election Auditing by the Election Audits Task Force.
63
Jan. 2009.
66. Ron Olson, Ohio 2008 Post Election Audit Review.
64
CASE Ohio, Feb. 2009.
67. Philip B. Stark, “CAST: Canvass Audits by Sampling and Testing”.
65
IEEE Transactions on Informa-
tion Forensics and Security 4, 708–717 (Feb. 2009).
68. Philip B. Stark, “Risk-limiting post-election audits: P-values from common probability inequali-
ties”.
66
IEEE Transactions on Information Forensics and Security 4, 1005–1014 (Feb. 2009).
69. Operational Review of Ballot Handling Processes.
67
Riverside County Executive Office, Feb. 2009.
70. Luke W. Miratrix and Philip B. Stark, Election Audits using a Trinomial Bound.
68
University of Cali-
fornia at Berkeley Department of Statistics, 2009.
55
/>56
/>57
/>58
/>59

/>60
/>61
/>62
/>63
/>64
/>65
/>66
/>67
/>68
/>5
71. Absentee Voting [Audit]: New York State Board of Elections.
69
Office of the New York State Comptroller,
Division of State Government Accountability, Mar. 2009.
72. Philip B. Stark, Auditing a Collection of Races Simultaneously.
70
University of California at Berkeley
Department of Statistics, 2009.
73. Mark Halvorson and Sarah Martyn Crowell, Eyes on the Vote Count: Non-partisan Observer Reports
of Minnesota’s 2008 Post-Election Audit and Recount.
71
Citizens for Election Integrity Minnesota,
May 2009.
74. Joseph A. Calandrino, William Clarkson, and Edward W. Felten, “Some Consequences of Paper
Fingerprinting for Elections”.
72
2009 Electronic Voting Technology Workshop / Workshop on Trust-
worthy Elections (Aug. 2009).
75. Gabrielle A. Gianelli, Jennifer D. King, Edward W. Felten, and William P. Zeller, “Software Support
for Software-Independent Auditing”.

73
2009 Electronic Voting Technology Workshop / Workshop
on Trustworthy Elections (Aug. 2009).
76. Joseph Lorenzo Hall, Philip B. Stark, Luke W. Miratrix, Melvin Briones, Elaine Ginnold, Freddie Oak-
ley, Martin Peaden, Gail Pellerin, Tom Stanionis, and Tricia Webber, “Implementing Risk-Limiting
Post-Election Audits in California”.
74
2009 Electronic Voting Technology Workshop / Workshop on
Trustworthy Elections (Aug. 2009).
77. Eric Rescorla, “Understanding the Security Properties of Ballot-Based Verification Techniques”.
75
2009 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections (Aug. 2009).
78. Eric Rescorla, “On the Security of Election Audits with Low Entropy Randomness”.
76
2009 Elec-
tronic Voting Technology Workshop / Workshop on Trustworthy Elections (Aug. 2009).
79. Cynthia Sturton, Eric Rescorla, and David Wagner, “Weight, Weight, Don’t Tell Me: Using Scales to
Select Ballots for Auditing”.
77
2009 Electronic Voting Technology Workshop / Workshop on Trust-
worthy Elections (Aug. 2009).
80. Philip B. Stark, “Efficient post-election audits of multiple contests: 2009 California tests”.
78
In:
CELS 2009 4th Annual Conference on Empirical Legal Studies, University of Southern California,
Los Angeles, California 2009.
81. John L. McCarthy, “Election Plumbing Standards: Data Format Requirements for Inter-operability,
Data Publication and Election Auditing”.
79
In: NIST Workshop on a Common Data Format for Elec-

tronic Voting Systems, 2009.
82. Philip B. Stark, The status and near future of post-election auditing.
80
(working draft), University of
California, Berkeley Department of Statistics, 2009.
83. Neal McBurnett, “Obtaining Batch Reports for Audits from Election Management Systems: Elec-
tionAudits and the Boulder 2008 Election”.
81
In: NIST Workshop on a Common Data Format for
Electronic Voting Systems, 2009.
69
/>70
/>71
/>72
/>73
/>74
/>75
/>76
/>77
/>78
/>79
/>80
/>81
/>6