Chapter 3: Usability, Accessibility, and Privacy Requirements

3.1 Overview

The importance of usability and accessibility in the design of voting systems has become increasingly apparent. It is not sufficient that the internal operation of these systems be correct; in addition, voters and election officials must be able to use them effectively and efficiently.

There are some properties of voting systems that make good design especially difficult:

• The voting task itself can be fairly complex; the voter may have to navigate an electronic ballot, choose multiple candidates in a single contest, understand the effect of party-line voting, or decide on ballot questions written in legal language;
• Voting is performed infrequently (compared with tasks such as using an ATM), so there is relatively limited opportunity for voters and poll workers to gain familiarity with the process;
• Changes in the election process, including new voting equipment, may require voters and poll workers to use new and unfamiliar procedures; and
• The set of "users" for voting equipment is exceptionally diverse. The voting public encompasses a broad range of factors, including physical and cognitive abilities, language skills, and technology experience.

3.1.1 Purpose

The challenge, then, is to provide a voting system that voters can use comfortably, efficiently, and with justified confidence that they have cast their votes correctly. The requirements within this section are intended to serve that goal. Three broad principles motivate this section:

1. All eligible voters are to have access to the voting process without discrimination. The voting process must be accessible to individuals with disabilities. The voting process includes access to the polling place, instructions on how to vote, initiating the voting session, selecting among contest choices, review of the ballot, final submission of the ballot, and getting help when needed.
2. Each cast ballot must accurately capture the selections made by the voter. The ballot must be presented to the voter in a manner that is clear and usable. Voters should encounter no difficulty or confusion regarding the process for recording their votes.
3. The voting process must preserve the secrecy of the ballot. The voting process should preclude anyone else from determining the content of a voter's ballot, without the voter's cooperation. If such a determination is made against the wishes of the voter, then his or her privacy has been violated.

Note that these principles refer to the entire voting process. The VVSG applies only to voting systems; other aspects of the process (such as administrative rules and procedures) are outside the scope of the VVSG, but are nonetheless crucial for the full achievement of the principles.

3.1.2 Special terminology

Several uncommon terms are used in this section. For the convenience of the reader, they are defined below. Many other technical terms frequently used throughout the VVSG are defined in Appendix A. Note in particular the distinctions among these terms: voting process, voting system, voting device, voting session, and voting station.

• Accessible Voting Station (Acc-VS) - the voting station specially equipped for individuals with disabilities referred to in HAVA 301 (a)(3)(B).
• Audio-Tactile Interface (ATI) - a voter interface designed not to require visual reading of a ballot. Audio is used to convey information to the voter and sensitive tactile controls allow the voter to convey information to the voting system.
• Common Industry Format (CIF) - the format to be used for summative usability test reporting, described in ISO/IEC 25062:2006 "Common Industry Format (CIF) for Usability Test Reports" [ISO06e].
• Summative Usability Testing - evaluation of a product with representative users and tasks designed to measure the usability (defined as effectiveness, efficiency and satisfaction) of the complete product. The purpose of a summative test is to evaluate a product through defined measures, rather than diagnosis and correction of specific design problems, as in formative testing.
• Voter-Editable Ballot Device (VEBD) - voting systems such as DREs and EBMs that present voters with an editable ballot (as opposed to manually-marked paper ballots), allowing them easily to change their votes prior to final casting of the ballot. "VEBD-V" denotes the visual interface of such systems and "VEBD-A" denotes the audio interface.
• Voting Performance Protocol (VPP) - a carefully defined method for measuring how well subjects perform various voting tasks within a controlled experiment.

3.1.3 Interaction of usability and accessibility requirements

All the requirements in Section 3 have the purpose of improving the quality of interaction between voters and voting systems. Please note how Sections 3.2 and 3.3 work together:

• The requirements for general usability in Section 3.2 apply to ALL voting systems as indicated by their “Applies to” clause, including the Acc-VS. They cover the features that are applicable both to the general population and to voters with disabilities. In particular, note that the Acc-VS is classified as a Voter-Editable Ballot Device and therefore all VEBD requirements apply to the Acc-VS. Requirements for any alternative languages required by state or federal law are also included under Section 3.2.
• The requirements for accessibility in Section 3.3 cover only those features that are mandatory for the accessible voting station (Acc-VS) in addition to the general usability requirements. For instance, an audio interface would be of interest mainly to those with vision or other reading disabilities, but not to those who can use a visual interface. Therefore, to determine what usability features are required of the Acc-VS, one must examine both Sections 3.2 and 3.3. The features of the Acc-VS may also assist those not usually described as having a disability, e.g., voters with poor reading vision or somewhat limited dexterity.

3.2 General Usability Requirements

The voting system should support a process that provides a high level of usability for all voters. The goal is for voters to be able to negotiate the process effectively, efficiently, and comfortably.

Many of the mandatory voting system standards in HAVA Section 301 [HAVA02] relate to the interaction between the voter and the voting system:

---

a. Requirements.--Each voting system used in an election for federal office shall meet the following requirements:

1. In general.--

A. Except as provided in subparagraph (B), the voting system (including any lever voting system, optical scanning voting system, or direct recording electronic system) shall--

i. Permit the voter to verify (in a private and independent manner) the votes selected by the voter on the ballot before the ballot is cast and counted;

ii. Provide the voter with the opportunity (in a private and independent manner) to change the ballot or correct any error before the ballot is cast and counted (including the opportunity to correct the error through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or correct any error); and

iii. If the voter selects votes for more than one candidate for a single office -

I. Notify the voter that the voter has selected more than one candidate for a single office on the ballot;

II. Notify the voter before the ballot is cast and counted of the effect of casting multiple votes for the office; and

III. Provide the voter with the opportunity to correct the ballot before the ballot is cast and counted.

B. A state or jurisdiction that uses a paper ballot voting system, a punch card voting system, or a central count voting system (including mail-in absentee ballots and mail-in ballots), may meet the requirements of subparagraph (A)(iii) by -

i. Establishing a voter education program specific to that voting system that notifies each voter of the effect of casting multiple votes for an office; and

ii. Providing the voter with instructions on how to correct the ballot before it is cast and counted (including instructions on how to correct the error through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or correct any error).

C. The voting system shall ensure that any notification required under this paragraph preserves the privacy of the voter and the confidentiality of the ballot.

---

The requirements of this section are intended to support these basic usability standards of HAVA.

3.2.1 Performance Requirements

Usability is defined generally as a measure of the effectiveness, efficiency, and satisfaction achieved by a specified set of users with a given product in the performance of specified tasks. In the context of voting, the primary user is the voter (although the equipment is used by poll workers as well), the product is the voting system, and the primary task is the correct recording of the votes (although other tasks are associated with poll workers as users, e.g. system setup).

Additional requirements for task performance are independence and privacy: the voter should normally be able to complete the voting task without assistance from others, and the votes should be private. Lack of independence or privacy may adversely affect effectiveness (e.g., by possibly inhibiting the voter's free choice) and efficiency (e.g., by slowing down the process).

General usability is covered by both high-level performance-based requirements (in this section) and design requirements (in following sections). Whereas the latter require the presence of specific features generally thought to promote usability, the former directly address metrics for effectiveness (e.g., correct capture of voter selections), efficiency (e.g., time taken to vote), and satisfaction. The voting system is tested by having groups of people (representing voters) attempt to perform various typical voting tasks. The requirement is met only if those tasks are accomplished with a specified degree of success.

3.2.1.1 Overall performance metrics

The requirements of this section set benchmarks for the usability of the voting system as a whole. There are three performance requirements that deal with effectiveness and two reporting requirements, one for efficiency and one for satisfaction.The metrics are defined as follows:

• Total Completion Score – the proportion of users who successfully cast a ballot (whether or not the ballot contains erroneous votes). Failure to cast a ballot might involve problems such as a voter simply “giving up” during the voting session because of an inability to operate the system, or a mistaken belief that one has successfully operated the casting mechanism.
• Perfect Ballot Index – the ratio of the number of cast ballots containing no erroneous votes to the number of cast ballots containing one or more errors (either a vote for an unintended choice, or a missing vote).
• Voter Inclusion Index – a measure of both voting accuracy and consistency. It is based on mean accuracy and the associated standard deviation. Accuracy per voter depends on how many “voting opportunities” within each ballot are performed correctly. A low value for the standard deviation of these individual accuracy scores indicates higher consistency of performance across voters..
• Average Voting Session Time – mean time taken per voter to complete the process of activating, filling out, and casting the ballot.
• Average Voter Confidence – mean confidence level expressed by the voters that the system successfully recorded their votes.

Because of the statistical nature of the testing, numerical results must be interpreted very carefully. The numbers have meaning only within the context of the Voting Performance Protocol (VPP). Note especially that the tests associated with these requirements are designed as repeatable controlled experiments and not as “realistic” measures of voting behavior, as might be found in a wide variety of voting contexts. Please see [HFP07] for full details.

Preliminary research at the direction of the TGDC that included experimentation with a variety of voting systems has allowed the Human Factors Subcommittee of the TGDC to judge that the following benchmark values would allow better systems to pass the test, while preventing certification of poorer systems:

• Total Completion Score : 98%
• Perfect Ballot Index: 2.33
• Voter Inclusion Index: 0.35

These tentative values may be adjusted based on planned research to be conducted with additional systems. The TGDC may also consider whether the benchmarks should be strengthened in anticipation of improvements in the design of future voting systems.

3.2.1.2 Manufacturer testing

3.2.2 Functional capabilities

The usability of the voting process is enhanced by the presence of certain functional capabilities. These capabilities differ somewhat depending on whether or not the system presents an editable interface within which voters can easily change their votes (typically an electronic screen) or an interface in which voters must obtain a new ballot to make changes (typically a manually-marked paper ballot).

3.2.2.1 Editable interfaces

Voting systems such as DREs and EBMs present voters with an editable interface, allowing them to easily change their votes prior to final casting of the ballot.

3.2.2.2 Non-Editable interfaces

Non-Editable interfaces, such as manually-marked paper ballots (MMPB) do not have the same flexibility as do editable interfaces. Nonetheless, certain features are required, especially in the case of precinct-based optical scanners. Note that the technical definition of "marginal mark" may be found in Appendix A. Basically, a marginal mark is one that, according the manufacturer specifications, is neither clearly countable as a vote nor clearly countable as a non-vote.

3.2.3 Privacy

The voting process must preclude anyone else from determining the content of a voter's ballot without the voter's cooperation. Privacy ensures that the voter can cast votes based solely on his or her own preferences without intimidation or inhibition.

3.2.3.1 Privacy at the polls

3.2.3.2 No recording of alternative format usage

When voters use non-typical ballot interfaces, such as large print or alternative languages, their anonymity may be vulnerable. To the extent possible, only the logical contents of their ballots should be recorded, not the special formats in which they were rendered. In the case of paper ballots, where the interface is the record, some format information is unavoidably preserved.

3.2.4 Cognitive issues

The features specified in this section are intended to minimize cognitive difficulties for voters. They should always be able to operate the voting system and understand the effect of their actions.

3.2.5 Perceptual issues

The requirements of this section are designed to minimize perceptual difficulties for the voter. Some of these requirements are designed to assist voters with poor reading vision. These are voters who might have some difficulty in reading normal text, but are not typically classified as having a visual disability and thus might not be inclined to use the Accessible Voting Station.

3.2.6 Interaction issues

The requirements of this section are designed to minimize interaction difficulties for the voter.