United States Election Assistance Comittee

Register to Vote!

Use the National Mail Voter Registration Form to register to vote, update your registration information with a new name or address, or register with a political party.

Note: If you wish to vote absentee and are a uniformed service member or family member or a citizen living outside the U.S., contact the Federal Voting Assistance Program to register to vote.

EAC Newsletters
and Updates

Sign up to receive information about EAC activities including public meetings, webcasts, reports and grants.

Give Us Your Feedback

Share your feedback on EAC policy proposalsElection Resource Library materials, and OpenEAC activities. Give feedback on general issues, including the Web site, through our Contact Us page.

Military and Overseas Voters

EAC has several projects under way to assist states in serving military and overseas citizens who register and vote absentee under the Uniformed and Overseas Citizens Absentee Voting Act. Learn more

Chapter 3

The importance of usability and accessibility in the design of voting systems has become increasingly apparent. It is not sufficient that the internal operation of these systems be correct; in addition, voters and election officials must be able to use them effectively and efficiently.

There are some properties of voting systems that make good design especially difficult:

  • The voting task itself can be fairly complex; the voter may have to navigate an electronic ballot, choose multiple candidates in a single contest, understand the effect of party-line voting, or decide on ballot questions written in legal language;
  • Voting is performed infrequently (compared with tasks such as using an ATM), so there is relatively limited opportunity for voters and poll workers to gain familiarity with the process;
  • Changes in the election process, including new voting equipment, may require voters and poll workers to use new and unfamiliar procedures; and
  • The set of "users" for voting equipment is exceptionally diverse. The voting public encompasses a broad range of factors, including physical and cognitive abilities, language skills, and technology experience.

11 Comments

Comment by B. Iversen (General Public)

The system must NOT be networked, have an internal UPS for power regulation, and be fully RF shielded and hardened. For examples of computers which have been tampered with using RF interference, do research on slot machines as several vulnerabilities have been discovered with them. There should be no access or diagnostic ports anywhere on the surface of the machine. As well, the complete system should be evaluated using Orange book specs, with an emphasis on mandatory access controls and tamper-evident design. All software and hardware specifications, including source code, should be in public domain and available for inspection at any time. Lastly, auditing of the system should be performed before, during, and after use to verify tampering has not occurred. A paper record, or other tamper-evident physical medium, should be used by each device. Again, I cannot stress enough the importance of not networking the voting machines together. Networking the machines would be purely for convenience, not necessity. Not only that, but threat exposure goes up dramatically when using networked devices. Real-time monitoring and collation of results is not a priority here; accuracy and tamper-evident systems IS with these systems. My #1 question, should I cast my vote on an electronic voting machine, is this: By mere visual inspection, how can I confirm that my vote, and the machine it is recorded on, has not been tampered with? A ticker-tape is the only viable solution I can see with present technology.

Comment by C.M. (Voter)

Comment: VVSG 2007 Recommended Guidelines, Chapter 3: Usability I do not know if elections were ever simple, they certainly are simple no more. While many of us were quite happy with the "out of date" punch card system and its 2% error rate, it is understandable in this day and age to look for a newer, easier, and more accurate system. However, if a new system, with all its costs does not provide the sought benefits, where have we gotten. These recommended guidelines show great care and attention, but seem to stand in contrast to the dysfunctional reality experienced around our country with some of our voting systems. The needs of the public, to vote accurately, efficiently, and confidently require a voting system or voting systems that have shown themselves to accomplish these goals. Without a doubt the need for usable and accessible voting system design is crucial to effective and efficient elections. I commend the recognition that the users of voting equipment are exceptionally diverse. The inclusion of considerations such as physical and cognitive abilities, language skills, and technology experience show a real desire to attain usability for everyone. While 2005 guidelines relied on assessment methods that were rather unspecific, the new guidelines define standard testing protocol, including test ballot, set of tasks to be performed, and demographic characteristics of test participants. This seems to be an improvement, however, I fear that in creating these controlled experiments something might get lost. While I certainly see the benefit of looking at actual human interaction with the voting machine, I am not convinced that such an elaborate system is justified. The defining of usability as a measure of the effectiveness, efficiency, and satisfaction achieved by the system user seems an appropriate place to begin such an evaluation. Including performance-based requirements (effectiveness, efficiency and satisfaction) and design requirements (usability) in the voting system testing scheme are sound focuses of attention. While I had no problem using the punch card system, I do enjoy using the new touch screens. However, my confidence that my vote will be accurately recorded and counted has greatly diminished. I do not have the answer, but here lies an important issue. Voter confidence must be maintained with whatever voter system is put in use.

Comment by Diane Golden (Advocacy Group)

The current organization is extremely complex and difficult to follow when trying to determine which standards make up the minimum access requirements for the one required "accessible" voting machine per polling place. The Chapter 3, Section 3 access standards do not include all the required access features for a HAVA mandated "accessible" voting system. Some access features are required by usability standards and other requirements are part of the overall typology structure. Specifically, the typology structure indicates that the accessible voting system (Acc-VS) must have an electronic interface or be a VEBD in the schema used by the VVSG. However, that fact is only implied via the typology system, never explicitly stated in Chapter 3. If an election official, advocacy group or any other reader just looked at the Chapter 3 access standards, it would be difficult to impossible to understand comprehensively what is required of an "accessible" voting system. The VVSG should be organized either in a way that provides a central location for the "access standards" or a way of readily identifying all the "access" features that must be available on the one legally mandated "accessible" system. If one location is not feasible, the organization should at least provide a way to identify and pull out the required access features so that election officials and others can use the standards as the benchmark for determining what meets the legal requirement for "accessible".

Comment by Janice Redish (Advocacy Group)

The points made in this Overview are very important for all designers to remember. An infrequent and complex process with new equipment or procedures must be particularly easy to use. Our democracy requires that all voters be able to accomplish the task accurately within a reasonable time. Other factors that might be included here would be that voting is often done under time pressure, either imposed legally or imposed by voters knowing that a long line of people are waiting for them to complete the task.

Comment by E Smith/P Terwilliger (Manufacturer)

3.1. "party-line voting" is not defined. 3.1.1. "Without discrimination" is an untestable (and impossible) absolute. Is it discrimination if voting via an audio interface takes 3-5 times longer? 3.2.1.2-A. "Individuals" implies more than 1. Is a test using 2 individuals sufficient?. 3.2.1.2-A. "representative of the general population" is impossible to define or satisfy. The makeup of the general population varies in different regions. 3.2.2-A. The discussion is incorrect. A VEBD that is compliant with HAVA 301.a.1.A is required to notify the voter of overvotes and the effect of overvoting, and only provide the "opportunity" to correct the overvote. Nowhere in HAVA is allowance for prohibiting overvoting granted. In fact, the 2005 VVSG was changed in its final released form to reflect HAVA, in part by DELETING the previous requirement (2002 VSS and 2005 VVSG drafts) that DRE not allow overvoting. See also 3.2.2.1-A; this requirement violates HAVA. 3.2.2.1-C. If the requirement in 3.2.2.1-A is met, there would never be a need to support modifying an overvote. 3.2.2.1-E. Are navigation controls to move between contests mandatory for a full-face ballot presentation? A ballot style that prsents several contests at once? 3.2.2.2. It is inappropriate to point to appendix A for the definition of marginal mark, and then redefine it here. 3.2.2.2-C. What purpose does allowing this feature to be disabled serve? In particular, the notification is a special case of undervoting, and as such is required by HAVA. 3.2.3.1-A. "prevent" is untestable. 3.2.3.1-A.1. This is not possible; there is always a way to peek. Suggest "Any attempt to view the voter's ballot SHALL be easily noticed by either the voter or a poll worker." 3.2.3.1-A.2. "audible only to the voter" is an absolute and untestable. There will always be some leakage from headphones, no matter how expensive. The discussion's "low sound leakage" does not meet the requirement. Suggest framing the requirement in terms of the minimum attenuation of the headphones (on a standard head...anything that breaks the headphone/skin seal, such as eyeglasses, will impact the performance.) 3.2.3.1-A.3. Untestable. 3.2.3.1-A.4. "issue", "receipt" and "proof" are undefined. 3.2.3.2-A. This requirement forces all write-in entries to be with a standard English keyboard. This discriminates against people using an alternate language to vote. 3.2.4-A. "valid" is not defined. Does this requirement apply to the voter? Pollworker? Election Judge? Administrator? Why is this requirement only applied to the voting station? Are instructions not required for the EMS? 3.2.4-B. Should be voting equipment, not voting system. 3.2.4-C. "norms and practices" is vague and untestable. It is also not a vendor issue - the jurisdiction customizes instructions for their specific needs. 3.2.4-C.1. This says "voting system", but discussion and the parent requirement both imply voter-only, so is it "voting equipment"? 3.2.4-C.2 through 3.2.4-C.7 all use "The system". Is it "voting system" or "voting equipment"? 3.2.4-D. This is not testable. 3.2.4-E. "high level" is not testable. 3.2.4-E.2. "clearly" is untestable. 3.2.4-E.4. "near to" is untestable. 3.2.4-F. This requirement has existed since the 1990 standard. At no time has the "common convention" ever been defined. In today's world of 32-bit color graphics, this is simplistic and archaic. 3.2.5. The introductory narrative indicates that the requirements are for the voter, yet many of the subsections expand to include the poll worker. 3.2.5 subsections. There is inconsistent usage of "voting system", "voting station", and even "the system". 3.2.5-B. "non-manual input mode" (automatic input mode, perhaps?) is not defined. Nor is "manual input mode" or "input mode". 3.2.5-B. How is a "standard default value" different from a "default value"? 3.2.5-D. This applies to printed material (ballots, data plates, etc.) too? As with the 2002 and 2005 standards, there is no aspect ratio requirement, so a tall skinny font is acceptable. (Why not use the words of 3.2.5-E?) 3.2.5-F. "Research has shown". Please cite this research; the same statement was in earlier standards with no reference. 3.2.5-I. Implies that the high contrast mode is under voter control, yet "all information" is to be included. Does this include printed instructions? Pollworker prompts/interfaces? 3.2.5-J. "correct perception" is untestable. 3.2.6-A. Should be "voting station". 3.2.6-B. "Unambiguous" is untestable. 3.2.6-C.1. It would be wise to include a specification for "guard bands" around each sensitive area, as our experience is that when the sensitive areas abut, the liklihood of problems increases significantly. A 5% to 10% non-sensitive area at the perimeter of sensitive areas is recommended. 3.2.6-C.2. Does this apply to the entire voting system, including EMS? Suggest that this is for the voting station. 3.2.6.1-A. "system" should be "station". 3.2.6.1-B. "completed system response time", in this context and as defined in appendix A, should instead be indicating the equipment/terminal/station response time. 3.2.6.1-C. Does this also apply to a VVPAT printing a page for review? The time to save and verify a ballot? 3.2.6.1-E. Who is to be warned? What happens after this period elapses? What is the purpose of this requirement? Does it apply to audio ballots, where listening to the review of a lengthy ballot will take well in excess of 5 minutes? Also, this should be "voting station". 3.2.6.1-F. What is the purpose of this? The combination of 3.2.6.1-E and this section serve to make the voting system LESS usable. Also, this should be "voting station". 3.2.7-A. Is the entire voting system subject to this requirement or just the voting station? Also, the "discussion" is borderline silly. 3.2.7-A.1. "The system". 3.2.7-A.3. This presents serious problems for VVPAT devices. To meet the requirement, alternate language VVPAT records must be printed bilingually. But what then happens for an English language voter? If the record is printed only in English, it will indicate the language used, AND print faster. If it is to be multilingual, how is it determined which alternate language to include? 3.2.7-A.4. "fluent" and "not fluent" are not defined. 3.2.8-A. "The system". 3.2.8.1-A. "reasonably easy" is not testable. "System" is not defined. 3.2.8.1-B. "Voting system" or "voting terminals"? "representative" is not defined, and will vary for different regions. 3.2.8.1-C. "Clear complete and detailed" is untestable. 3.2.8.1-C.1. "appropriate" is not testable. 3.2.8.1-C.2. "suitable for practical use" is not testable. Further, it is not under vendor control. 3.2.8.1-C.3. This is subjective and untestable. 3.3.1-A. "integrated" is not testable. 3.3.1-A.1. "fully implement" is not testable. 3.3.1-C. Using "accesible voting station" instead of "Acc-VS". Is this on purpose? 3.3.1-D. "voting system" or "Acc-VS"? 3.3.1-E. Under the definitions of permissable voting technology (ie IVVR, etc.), how can the Acc-VS NOT provide a durable, human-readable record? "Verification record" is a new term, not defined. 3.3.1-E.1. This requirement needs to be clarified as to whether the actual information printed on the paper record must be read back, or whether machine-readable marks (such as the marks on an optical scan ballot) may be read and interpreted for playback. Also the discussion to section 4.2.4 and 4.2.4-A appears to negate the audio readback requirement stated here. 3.3.2-A. "voting system" should be Acc-VS. 3.3.2-B. "high and "low" are not defined. 3.3.2-C. "Distinguishable" is not testable. "Sizeable group" is not defined. How many keys must be in a group? 3.3.2-D. "synchronized" is not defined. One imagines "follow the bouncing ball" Karaoke. 3.3.3-A. "Voting system" should be Acc-VS. 3.3.3-B. Using "accesible voting station" instead of "Acc-VS". Is this on purpose? 3.3.3-C.1 This requirement potentially violates 3.2.3.1-A.2. 3.3.3-C.2. "Voting system" should be "ATI". 3.3.3-C.3. This is a jurisdiction issue and is untestable. 3.3.3-C.4, 3.3.3-C.5. As has been the case since the 2002 standards, these are untestable requirements. To be testable, here is some of the information required: * Peak or average SPL? If average, over what period? * Raw or using one of several standard "weighting" functions? * Single-tone, actual voice? * If a single-tone, what frequency? * To what precision must the SPL be met? Is a maximum of 98 dB or 102 dB SPL allowed? 3.3.3-C.4. "voting system" should be "ATI". 3.3.3-C.5. "audio system" should be "ATI". 3.3.3-C.6. How is this tested? To what degree of flatness is this response required? 3.3.3-C.6. "audio system" should be "ATI". 3.3.3-C.7. "readily comprehensible" is not testable. 3.3.3-C.8. "audio system" should be "ATI". 3.3.4-A. "voting system" should be "Acc-VS". 3.3.4-B. "non-manual input" is a confusing term, as it implies "automatic input". Suggest rewording to state specifically what the discussion is asking for. ALternately, this term should be defined in Appendix A, not in the discussion following the requirement. 3.3.4-D. "tight" is untestable. 3.3.5-B. "adequate" and "clearance" are untestable. "voting station" should be "Acc-VS". 3.3.5-C. "easily" in untestable. 3.3.5.1-B.3.a. "maximum" should be "minimum". 3.3.5.1-B.4.a. "maximum" should be "minimum". 3.3.6-B. "voting system" should be "Acc-VS". 3.3.6-C. This is very confusing. Why would the voting equipment be considered a wireless device when all forms of wireless are prohibited? Should this section be referring to the Acc-VS instead of voting equipment? 3.3.8-A. "voting equipment" should be Acc-VS.

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

Accuracy and Usability A critical factor in election accuracy is accurately capturing a voter’s intent. The voting process always starts with the voter’s intent, which must be converted from a selection on the machine through the user interface. Therefore, designing usable interfaces by building upon a body of best practices and knowledge for interface design is a critical first-step toward accuracy in elections. The VVSG should reflect the above principle. We present some specific comments below to help clarify and increase focus on the importance of usability in its affect on accuracy. Section 3.2 of Part 1 of the draft VVSG cites the basic usability standards of the HAVA. These requirements define important and fundamental functional capabilities of the voting system, but are incomplete in that they do not specify any goal or mechanism to achieve usability in the initial casting of the ballot. By omitting usability requirements for the primary vote-casting activity, an incorrect impression is given that this is not a point of emphasis. While the standards themselves should reflect the goal of designing usable interfaces to capture voter intent, the law should as well. If the EAC puts forward amendments to the law in the future, we suggest that it address this gap. Specifically, we would recommend the following new clause be amended to HAVA: "i. Have a vote-casting mechanism presented, following best practices for user interface design, to enhance the ability of voters to accurately make selections that represent their intent. The design approaches for reaching this goal may include such basic principles as consistency, visibility, feedback, mapping between selections and candidates, and clear visual design" Accessibility and Usability While these terms are used separately within the VVSG, accessibility and usability have the same goal — making the voting experience and the voting equipment as easy as possible for the voter (and in the case of setup, shutdown and auditing, the poll worker) to use. While it is important to make sure that those with disabilities are able to vote with privacy and the other election guarantees provided to all voters, it is a mistake to restrict accessibility and usability concerns to only those with disabilities. Limiting accessibility features to machines specifically designated for users with disabilities may limit the ability of other voters to benefit from technologies and innovations that could improve their voting experience. Similarly, by restricting features to a limited number of machines, costs for those machines will be greater. They will likely be used more often, and reach their mean time to failure faster than other machines. To the extent feasible, we recommend that accessibility features be included with as many voting machines as possible and practical. Assistive devices that must be connected to voting systems may raise security concerns. Specifically, devices that must interface with the voting system software may introduce viruses, or the interaction of two disparate systems may prompt unintentional problems with the voting system. We recommend that jurisdictions should provide common assistive devices that can be connected via industry standard interfaces (such as USB) with voting systems. This would allow for testing of the interface as part of the certification process. Other assistive devices are either external to the voting system or connect through some mechanism that does not require a software interface (such as the audio devices currently available with some voting systems); Voters who need such devices should be allowed to bring such a device with them to vote.

Comment by Cem Kaner (Academic)

Chapter 3 repeatedly identifies three types of users of the electronic voting system: voters, poll workers and election officials. .......... Chapter 3 does NOT identify a fourth type of user, the election auditor. This is a critical role for these systems, especially because of the low level of public confidence in electronic voting systems and the high level of mistrust expressed by the media. If these systems are not easy to audit, and if audits are not obviously thorough and trustworthy, they will never be accepted by the public or the press. Nor should they be. .......... It is commonplace, in requirements analysis and user interface design, to consider the several roles (types of users, personas) involved in the system and to evaluate the system through the eyes of each role. .......... VVSG consider usability or utility of the voting system for the auditor only briefly (Part 1, Section 4.2), explaining that "audits are considered part of the election procedures and cannot be mandated by the VVSG." Thoroughgoing assessment of the usability and utility of the voting system for the auditor does not mandate that audits happen; it only mandates that if audits are attempted, the equipment will doing an acceptable job of supporting them. This omission is serious and has broad implications for the VVSG document. I believe that this Chapter, Part 1: Chapter 7, and much of Part 3 would change if such an analysis was undertaken. .......... (Affiliation Note: IEEE representative to TGDC)

Comment by Electronic Privacy Information Center (Advocacy Group)

Access to ATMs for banking purposes is not universal because affordable banking options are not available to all communities. Penetration of telephone services, although not universal is much more widespread then ready access to ATM machines for a broad spectrum of the voting age public.

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

A voter's ability to vote may also be affected by where they are as pertinent to absentee voting. Voters may also have technical literacy but adversarial position on the technology, they may be honest but curious and systems must remain usable considering a very wide range of voter curiosity, positive or otherwise.

Comment by Cem Kaner (Academic)

A majority of respondents, in a poll of the IEEE SCC38 Voting Standards committee, supported the recommendation that: "The use of electronic voting machines be limited insofar as practical to use by disabled voters who cannot otherwise vote unassisted. And that limitation remain in place until reasonable standards and security for electronic voting machines that provide for public trust and transparency can be developed and implemented." .......... Implicit in this recommendation is a lack of confidence in the currently proposed VVSG. .......... Until these systems' code and tests become publicly reviewable, doubts about the trustworthiness of these systems will persist, fueled by widely-publicized examples of problems, and equally-widely publicized comments about the secrecy of the ensuing investigations. .......... (Affiliation Note: IEEE representative to TGDC)

Comment by Jim Tobias (Voting System Test Laboratory)

As is correctly recognized, voting is infrequent. Training and awareness materials and programs must be highly usable and clear if even a well-designed system is to be implemented and used successfully. Ballot design is another part of the voting process; there should be usability requirements of ballot design utilities. Finally, it is essential that usability be analyzed and imporved all along the value chain. There are interaction effects between and among system design, system option selection and configuration, ballot design, pollworker training, voter information, etc. None of these lives in a vacuum. The usability of the voter's experience is a function of the individual elements and how the elements work together. This obvious conclusion is yet to be factored into how election officials actually manage the procurement and implementation of voting systems.

3.1.1 Purpose

The challenge, then, is to provide a voting system that voters can use comfortably, efficiently, and with justified confidence that they have cast their votes correctly. The requirements within this section are intended to serve that goal. Three broad principles motivate this section:

  1. All eligible voters are to have access to the voting process without discrimination. The voting process must be accessible to individuals with disabilities. The voting process includes access to the polling place, instructions on how to vote, initiating the voting session, selecting among contest choices, review of the ballot, final submission of the ballot, and getting help when needed.
  2. Each cast ballot must accurately capture the selections made by the voter. The ballot must be presented to the voter in a manner that is clear and usable. Voters should encounter no difficulty or confusion regarding the process for recording their votes.
  3. The voting process must preserve the secrecy of the ballot. The voting process should preclude anyone else from determining the content of a voter's ballot, without the voter's cooperation. If such a determination is made against the wishes of the voter, then his or her privacy has been violated.

Note that these principles refer to the entire voting process. The VVSG applies only to voting systems; other aspects of the process (such as administrative rules and procedures) are outside the scope of the VVSG, but are nonetheless crucial for the full achievement of the principles.

5 Comments

Comment by ted selker (Academic)

Part 1, Chapter 3, page 25, section 3.1.1. voters can use comfortably Should change to: Voters can use with confidence in the selections they have made Part 1, Chapter 3, page 26, section 3.1.1 clear and usable Should be changed to: clear and showing their selection Also "clear" and "usable" need definitions. Summative usability testing comments. This definition needs an exemplar to demonstrate what these properties are and how to test compliance.

Comment by David Cary, Californians for Electoral Reform (Advocacy Group)

A more precise statement of principle 3 should distinguish between preventing others from knowing the content of a ballot versus inhibiting their ability to associate the ballot content with a voter. The goal should be the latter; the former is only a method for achieving that goal. The former can conflict with other goals if it is relied upon exclusively. For example, a VVPAT is designed to allow others to view the content of a ballot while reducing the availability of information about who cast the ballot. It becomes more important to clarify this distinction in the specification of requirements supporting the principle. A balanced approach is needed and should be acknowledged.

Comment by Bryan Pfaffenberger (Academic)

I am currently studying the history of secret voting and early voting machines with the assistance of a National Science Foundation Scholar's Award. The secrecy provisions of this section are important and appropriate. One shortcoming of current implementations is that, in the rush to provide a separate independent audit trail (by such means as voter-verifiable paper audit trails), secrecy is too often forgotten; an individual voter's choices can be reconstructed from the serial order in which votes were cast. It is possible to provide an independent audit trail without compromising secrecy. The secrecy requirement should therefore be phrased in such a way that it refers to the entire voting process, including paper audit trails. Bryan Pfaffenberger, Ph.D. Univ. of Virginia

Comment by Harrine Freeman, IEEE member (Voter)

What is the definition of an eligible voter, i.e. if a voter has been a victim of identity theft and the identity thief uses the voter's name to vote and is successful and the actual voter tries to go vote but is unable to when person's use deceased voter information to cast votes - what provisions are in place to handle those types of situations?

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

System user interface design needs to adhere to defacto standards for kiosks and forms which have been in place and have been used by large numbers of voters. That is, web forms and ATM machines are the most likely form behaviors known to voters even though these forms have very limited functionality compared to user interfaces of software applications. The reason web forms serve as a better model is that they are are based on the HTML Standard while computer interfaces are much more diverse. In this way, usability requirements should not be ideal, but be reduced to the most likely known model.

3.1.2 Special terminology

Several uncommon terms are used in this section. For the convenience of the reader, they are defined below. Many other technical terms frequently used throughout the VVSG are defined in Appendix A. Note in particular the distinctions among these terms: voting process, voting system, voting device, voting session, and voting station.

  • Accessible Voting Station (Acc-VS) - the voting station specially equipped for individuals with disabilities referred to in HAVA 301 (a)(3)(B).
  • Audio-Tactile Interface (ATI) - a voter interface designed not to require visual reading of a ballot. Audio is used to convey information to the voter and sensitive tactile controls allow the voter to convey information to the voting system.
  • Common Industry Format (CIF) - the format to be used for summative usability test reporting, described in ISO/IEC 25062:2006 "Common Industry Format (CIF) for Usability Test Reports" [ISO06e].
  • Summative Usability Testing - evaluation of a product with representative users and tasks designed to measure the usability (defined as effectiveness, efficiency and satisfaction) of the complete product. The purpose of a summative test is to evaluate a product through defined measures, rather than diagnosis and correction of specific design problems, as in formative testing.
  • Voter-Editable Ballot Device (VEBD) - voting systems such as DREs and EBMs that present voters with an editable ballot (as opposed to manually-marked paper ballots), allowing them easily to change their votes prior to final casting of the ballot. "VEBD-V" denotes the visual interface of such systems and "VEBD-A" denotes the audio interface.
  • Voting Performance Protocol (VPP) - a carefully defined method for measuring how well subjects perform various voting tasks within a controlled experiment.

3 Comments

Comment by Carolyn Coggins (Voting System Test Laboratory)

Are all DREs Accessible Voting Stations? If not, by what mechanism can a DRE that is not an accessible voting station be certified? Is the idea of DREs that do not include Accessible Voting Station functionality in sync with the direction of the EAC?

Comment by Jim Tobias (Academic)

Wherever possible, the "Accessible voting station" should be the same as the other voting stations. Either per-day or per-voter configuration should be enough to provide the accessibility features needed. If the Acc-VS is entirely unique, and only one is provided each polling place, a single technical failure would disenfranchise all disabled voters at that location.

Comment by Electronic Privacy Information Center (Advocacy Group)

Bullet item 5 "Voter-Editable Ballot Device (VEBD) "…(As opposed to manually- marked paper ballots) allowing them easily to change their votes prior to casting of ballot." This may be a subjective conclusion and not a measurable fact. The options for correcting ballots that are physically market and those that are marked with the assistance of DREs and EBMs should be further studied to assist in making more clear the benefits or shortcomings of each as they related to this point.

3.1.3 Interaction of usability and accessibility requirements

All the requirements in Section 3 have the purpose of improving the quality of interaction between voters and voting systems. Please note how Sections 3.2 and 3.3 work together:

  • The requirements for general usability in Section 3.2 apply to ALL voting systems as indicated by their "Applies to" clause, including the Acc-VS. They cover the features that are applicable both to the general population and to voters with disabilities. In particular, note that the Acc-VS is classified as a Voter-Editable Ballot Device and therefore all VEBD requirements apply to the Acc-VS. Requirements for any alternative languages required by state or federal law are also included under Section 3.2.
  • The requirements for accessibility in Section 3.3 cover only those features that are mandatory for the accessible voting station (Acc-VS) in addition to the general usability requirements. For instance, an audio interface would be of interest mainly to those with vision or other reading disabilities, but not to those who can use a visual interface. Therefore, to determine what usability features are required of the Acc-VS, one must examine both Sections 3.2 and 3.3. The features of the Acc-VS may also assist those not usually described as having a disability, e.g., voters with poor reading vision or somewhat limited dexterity.

4 Comments

Comment by Diane Golden (Advocacy Group)

This section should make it clear that an accessible voting system as mandated by HAVA MUST have a voter editable interface. Simply saying an Acc-VS is classified as a Voter –Editable Ballot Device does not make it clear that manually-marked paper ballot systems will not conform to the requirements for the legally mandated accessible voting system.

Comment by Jim Tobias (Voting System Test Laboratory)

Most voters with impaired vision will need/want to use their residual vision in addition to the audio interface. Many of these voters are not frequent users of audio interfaces, and in any event will not be familiar with the particular interface of the Acc-VS they are using. For this reason, attention should be paid to the interoperation of the audio and visual aspects of the interface.

Comment by Susan Pynchon (Advocacy Group)

This is a comment on Part 1, Chapter 3.1.3, dated 2008/05/05 15:59:16.221 GMT-4 The behavior of disabled accessible DREs varies from firmware version to firmware version, and also from machine to machine, depending on what is right or wrong with any given machine at any moment in time. The following two email exchanges between the Sarasota County Supervisor of Elections and two different poll workers following the September 2006 primary election are instructive because they are a clear demonstration that problems can be experienced in an actual election that were not caught in laboratory testing and also that many problems are never disclosed in any post-election report. _____________________________________________ From: Dent, Kathy Sent: Thursday, August 31, 2006 11:04 AM To: Brennan, Elizabeth Cc: IT Department Subject: RE: Voter feedback Thanks, Liz. This is a firmware issue. I don't know whether this is fixed in the next version, but I will take it to the next users' group meeting in September. -----Original Message----- From: Brennan, Elizabeth Sent: Thursday, August 31, 2006 9:40 AM To: Dent, Kathy Subject: Voter feedback Good morning Kathy. I just wanted to share some feedback from a voter. We had a young lady request the use of the large print ballot. She was able to see everything in bold, but the box where she pushed to mark her selection was to light and she couldn't see it. I was asked to pass on the information and request to bold the outline of the block just as dark as the words on the ballot. She would have been able to vote unassisted in that case. Thanks, Liz _________________________________________________________________________________________________ The second email exchange is a long one, so I will only show the relevant section here: From: Dent, Kathy Sent: Wednesday, September 27, 2006 3:10 PM To: 'a charlie original' Subject: RE: Sept 5 06 election report Charlie: I agree that the large print is a disaster but that takes a firmware change. We will be discussing this with our vendor. -----Original Message----- From: a charlie original [mailto:charliehc@verizon.net] Sent: Friday, September 08, 2006 8:55 AM To: Dent, Kathy Cc: Crete, Karen Subject: Sept 5 06 election report _______________________________________________________________________ September 8, 2005 To: Kathy Dent From: C H Cullen Clerk Precinct 36 Subject: September 5 election report The ‘large print’ option is still a disaster. I instructed my inspectors not to even mention it unless the voter requested it. Using the option, one ballot was 21 pages long. The worst part was that in many instances a single contest covered multiple pages. The font is much too big for a person to read on a computer screen. If you personally haven’t tried using it, may I suggest you take a look. Several voters who tried to use it, ended up having us read it to them.

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

USACM Comment #5. Section 3.1.3. Usability and Accessibility [imprecise] The distinction between Acc-VS and VEBD sets up a false choice between accessible stations and other voting stations. This is problematic in several ways. USACM recommends that the following text be added to section 1.3 - All VEBD systems — even those that are not specifically designated Acc-VS systems — should implement all reasonable accessibility features from section 3.3. Reasonable accessibility features would include any that could benefit all voters, regardless of ability. DISCUSSION: Relegating accessibility features to a completely separate class of machines can increase costs (as a result of relatively lower volume of production) and lead to reliability concerns (the added cost and difficulty of testing these special-purpose machines may make appropriate testing difficulty). Many accessibility features such as magnified text and speech output can be inexpensively supported on the same platforms that would likely be used as VEBD systems. Including these features will let many users with disabilities use VEBDs to vote.

3.2 General Usability Requirements

The voting system should support a process that provides a high level of usability for all voters. The goal is for voters to be able to negotiate the process effectively, efficiently, and comfortably.

Many of the mandatory voting system standards in HAVA Section 301 [HAVA02] relate to the interaction between the voter and the voting system:


a. Requirements.--Each voting system used in an election for federal office shall meet the following requirements:

1. In general.--

A. Except as provided in subparagraph (B), the voting system (including any lever voting system, optical scanning voting system, or direct recording electronic system) shall--

i. Permit the voter to verify (in a private and independent manner) the votes selected by the voter on the ballot before the ballot is cast and counted;

ii. Provide the voter with the opportunity (in a private and independent manner) to change the ballot or correct any error before the ballot is cast and counted (including the opportunity to correct the error through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or correct any error); and

iii. If the voter selects votes for more than one candidate for a single office -

I. Notify the voter that the voter has selected more than one candidate for a single office on the ballot;

II. Notify the voter before the ballot is cast and counted of the effect of casting multiple votes for the office; and

III. Provide the voter with the opportunity to correct the ballot before the ballot is cast and counted.

B. A state or jurisdiction that uses a paper ballot voting system, a punch card voting system, or a central count voting system (including mail-in absentee ballots and mail-in ballots), may meet the requirements of subparagraph (A)(iii) by -

i. Establishing a voter education program specific to that voting system that notifies each voter of the effect of casting multiple votes for an office; and

ii. Providing the voter with instructions on how to correct the ballot before it is cast and counted (including instructions on how to correct the error through the issuance of a replacement ballot if the voter was otherwise unable to change the ballot or correct any error).

C. The voting system shall ensure that any notification required under this paragraph preserves the privacy of the voter and the confidentiality of the ballot.


The requirements of this section are intended to support these basic usability standards of HAVA.

5 Comments

Comment by Electronic Privacy Information Center (Advocacy Group)

The section in general is very good. Addition: 1. In general B. iii. Establish a usability review process for ballot design that screens for identified usability problems Discussion: Experience with usability and ballot design should be to the degree possible in consideration of new voting systems or review of ballot development process

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

USACM Comment #6. Section 3.2 General Usability Requirements [vague] This section does not adequately describe why usability is an important part of voting system accuracy. As we described in the introduction, the voting process always starts with the machine capturing the voter’s intent. Careful interface design is a critical first-step toward accuracy in elections. The specified usability requirements define important and fundamental functional capabilities of the voting system, but are incomplete in that they do not specify any goal or mechanism to achieve usability in the initial casting of the ballot. USACM recommends that the introductory paragraph be changed as follows: "The voting system should support a process that provides a high-level of usability for all voters. The goal is for voters to be able to accurately cast their votes as intended while negotiating the process effectively, efficiently, and comfortably."

Comment by Susan Pynchon (Advocacy Group)

Usability studies in a test environment are far different than usability in an actual election. DREs perform differently depending on what is right or wrong with each individual machine at any given moment in time. What we observed in our year-long study of the 2006 general election in Sarasota County, for example, was that each voter’s ability to vote easily and/or successfully was affected by a number of different problems being experienced by the machines, including screen failures, screens out of calibration, slow touch response, hard touch response (having to press hard or use knuckles to get a response), votes disappearing on the review screen, and myriad other problems resulting from low battery problems, power supply failures, and failures of video cards, power cords, IRDA (infra-red) boards, and more. Each machine that experienced one or more of these problems provided a different voting experience to the voters using it than another machine that was experiencing no problems or a different set of problems. Our conclusion is that there can NEVER be uniformity with DREs, because each voter’s experience is different depending on what is happening with each individual machine at any given moment in time. The same thing holds true for "ballot style." We became aware that a so-called ballot style on a DRE can be quite different on different machines in the same jurisdiction if there is a problem with one or more memory cards or the PEBs (personalized electronic ballots). Each PEB, for example, is a mini-processor that holds the ballot style, and faulty programming of one batch of PEBs in a jurisdiction could mean that some machines display the ballot properly and others don’t. Thus, what a jurisdiction SAYS is the ballot style, or a PDF file presented as evidence of a ballot style, might be true for some machines but different for others. We believe that the inherent lack of uniformity in DRE voting has not been adequately considered, and that this lack of uniformity alone makes DREs unfit to use in any election. It is noteworthy that this lack of uniformity does NOT occur with paper ballots. Paper ballots can be easily checked to ensure they are uniform. DRE machines, and DRE "ballot styles" can never be checked to ensure uniformity because something can change on any given machine from one minute to the next. And, as also evidenced in Sarasota, very little credence is given to the complaints of affected voters when problems do occur. Thus, the lack of transparency inherent in all electronic voting systems is compounded on DREs because any one machine, or more than one machine, can become erratic or unreliable at any moment in time for any number of reasons.

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

Is there a requirement that the voter use the voting machine in a predetermined amount of time? If so, what is the limit, if not, why not?

Comment by Marybeth Kuznik (Local Election Official)

Please note that the current voting systems, forced upon us by HAVA, are supposed to meet these requirements for independent voting, etc. AND THEY DO NOT. My own mother, who votes from a wheelchair, can no longer vote independently because she can not reach the "Vote" button on the DRE chosen by my county under HAVA (supposedly certified to the 2002 VVSG.) Marybeth Kuznik Elected Majority Inspector of Elections Penn Township, Westmoreland County PA

3.2.1 Performance Requirements

Usability is defined generally as a measure of the effectiveness, efficiency, and satisfaction achieved by a specified set of users with a given product in the performance of specified tasks. In the context of voting, the primary user is the voter (although the equipment is used by poll workers as well), the product is the voting system, and the primary task is the correct recording of the votes (although other tasks are associated with poll workers as users, e.g. system setup).

Additional requirements for task performance are independence and privacy: the voter should normally be able to complete the voting task without assistance from others, and the votes should be private. Lack of independence or privacy may adversely affect effectiveness (e.g., by possibly inhibiting the voter's free choice) and efficiency (e.g., by slowing down the process).

General usability is covered by both high-level performance-based requirements (in this section) and design requirements (in following sections). Whereas the latter require the presence of specific features generally thought to promote usability, the former directly address metrics for effectiveness (e.g., correct capture of voter selections), efficiency (e.g., time taken to vote), and satisfaction. The voting system is tested by having groups of people (representing voters) attempt to perform various typical voting tasks. The requirement is met only if those tasks are accomplished with a specified degree of success.

1 Comments

Comment by Janice Redish (Advocacy Group)

Setting performance metrics for voting systems is crucial. We must have voting systems that allow voters to be successful at the task of voting, where success means completion, accuracy, and confidence. Traditionally, we have learned about successes – and failures – only after an election. But that is too late. Voters have been deprived of the opportunity to participate in the election if they cannot complete the ballot for the candidates and positions that they want to vote for. If the lack of clarity and usability in the voting system causes voters to not vote or to cast a vote for a candidate or position that was not what they wanted, the outcome of the election does not reflect the will of the people. Failures due to usability are just as serious as failures due to the system functioning incorrectly for technological reasons. Voting systems must be tested to be sure they function correctly. They must also be tested to be sure they are usable. Waiting until after an election to know that a system fails for usability is not necessary. Human factors specialists have long had a methodology – Usability Testing – that allows developers to know how well their system will work for its users before they deploy the system. The particular version of the methodology known as Summative Usability Testing is appropriate and necessary for acceptance testing, as specified in Section 3.2.1 of the VVSG.

3.2.1.1 Overall performance metrics

The requirements of this section set benchmarks for the usability of the voting system as a whole. There are three performance requirements that deal with effectiveness and two reporting requirements, one for efficiency and one for satisfaction.The metrics are defined as follows:

  • Total Completion Score – the proportion of users who successfully cast a ballot (whether or not the ballot contains erroneous votes). Failure to cast a ballot might involve problems such as a voter simply "giving up" during the voting session because of an inability to operate the system, or a mistaken belief that one has successfully operated the casting mechanism.
  • Perfect Ballot Index – the ratio of the number of cast ballots containing no erroneous votes to the number of cast ballots containing one or more errors (either a vote for an unintended choice, or a missing vote).
  • Voter Inclusion Index – a measure of both voting accuracy and consistency. It is based on mean accuracy and the associated standard deviation. Accuracy per voter depends on how many "voting opportunities" within each ballot are performed correctly. A low value for the standard deviation of these individual accuracy scores indicates higher consistency of performance across voters..
  • Average Voting Session Time – mean time taken per voter to complete the process of activating, filling out, and casting the ballot.
  • Average Voter Confidence – mean confidence level expressed by the voters that the system successfully recorded their votes.

Because of the statistical nature of the testing, numerical results must be interpreted very carefully. The numbers have meaning only within the context of the Voting Performance Protocol (VPP). Note especially that the tests associated with these requirements are designed as repeatable controlled experiments and not as "realistic" measures of voting behavior, as might be found in a wide variety of voting contexts. Please see [HFP07] for full details.

Preliminary research at the direction of the TGDC that included experimentation with a variety of voting systems has allowed the Human Factors Subcommittee of the TGDC to judge that the following benchmark values would allow better systems to pass the test, while preventing certification of poorer systems:

These tentative values may be adjusted based on planned research to be conducted with additional systems. The TGDC may also consider whether the benchmarks should be strengthened in anticipation of improvements in the design of future voting systems.

12 Comments

Comment by ted selker (Academic)

Part 1, Chapter 3, page 29, section 3.2.1.1 perfect ballot index Could be changed to: completed ballot index Part 1, Chapter 3, page 29, section 3.2.1.1 voting opportunities Should be changed to: selection opportunities Part 1, Chapter 3, page 29, section 3.2.1.1 Repeatable controlled experiments and not as "realistic" measures of voting behaviors as might be found in a wide variety of voting contexts. Should be deleted: research has found that controlled experiments can be very representative and reliable [http://vote.caltech.edu/journals/PSJ-revised-6-20-06.pdf].

Comment by Carolyn Coggins (Voting System Test Laboratory)

The test lab SHALL report the Average Voter Confidences, as measure by the VPP" The VPP (Voter Performance Protocol) has numbers with meaning only within the context of itself and does not represent "realisteic" measure of voting behavior (1:3.2.1.1). How SHALL the labs then report this unrealistic results of a repeatable controlled experiment consistently?

Comment by Frank Padilla (Voting System Test Laboratory)

It is not clear whether the lab or the manufacturer is required to do this testing. Numerous sections state the lab and some state nothing.

Comment by Janice Redish (Advocacy Group)

The measures set in 3.2.1.1 are appropriate for the three primary criteria of success in voting: completion, accuracy, and confidence.

Comment by David Beirne, Executive Director, Election Technology Council (Manufacturer)

Average Voter Confidence is a purely subjective criteria and should be stricken from the listing of performance metrics; It should be used only for purposes of validating the usability benchmarks. Since it is a subjective measurement, it should be only used for validation and internal purposes or reporting back to the applicant. It should not be used for public consumption as it is uncontrolled and speaks to the need for the EAC to focus on performance, not subjective impressions or assessments on voting system design that should be left to the marketplace.

Comment by E Smith/L Korb (Manufacturer)

The Overall performance metrics referenced in this section while laudable are not based on any real past or present voting systems and may require compromises and constraints that the administrators of the these systems will find unworkable or unaffordable. Additionally many of the jurisdiction-definable election system parameters – i.e. ballot layout and formatting, voter instructions and prompts, number of IVVR capture devices in a precinct / location, state or local election laws, etc. are not under the control of the voting system manufacturer and has a significant effect on the overall performance of the voting system. A further problem comes from the lack of a standardized test suite that should be used to measure these performance metrics and to enable comparisons between different voting systems, although we note the EAC's and NIST's efforts to have a suite in place for the VSTLs to utilize. A better approach would be to provide a standardized test suite, with well defined test parameters and require each voting systems manufacturer to report the performance metrics of their system in the system's recommended configuration. This would allow jurisdictions looking to purchase a voting system to directly compare competitive systems.

Comment by E Smith/L Korb (Manufacturer)

The Overall performance metrics referenced in this section while laudable are not based on any real past or present voting systems and may require compromises and constraints that the administrators of the these systems will find unworkable or unaffordable. Additionally many of the jurisdiction-definable election system parameters – i.e. ballot layout and formatting, voter instructions and prompts, number of IVVR capture devices in a precinct / location, state or local election laws, etc. are not under the control of the voting system manufacturer and has a significant effect on the overall performance of the voting system. A further problem comes from the lack of a standardized test suite that should be used to measure these performance metrics and to enable comparisons between different voting systems, although we note the EAC's and NIST's efforts to have a suite in place for the VSTLs to utilize. A better approach would be to provide a standardized test suite, with well defined test parameters and require each voting systems manufacturer to report the performance metrics of their system in the system's recommended configuration. This would allow jurisdictions looking to purchase a voting system to directly compare competitive systems.

Comment by ACCURATE (Aaron Burstein) (Academic)

The five usability benchmarks used in performance testing in this section are welcome and encouraging. The first three of these benchmarks will mean that there is an absolute target that voting systems must meet to satisfy the recognized goal of effectiveness. While there are no specific targets for efficiency and satisfaction, requiring vendors to report data about those measures may also lead to improvements in those areas. NIST has since published the Voting Performance Protocol (VPP) which establishes a standard testing methodology for testing each of these benchmarks. While perhaps not all of the details of the VPP represent our preferred choices, the fact that such usability benchmarks exist at all is a critical and welcome step. It is worth noting that the usability measures collected using the VPP are likely to overstate the true usability of voting systems because the VPP instructs voters on who to vote for in the test, which tends to decrease error rates. Thus, these benchmarks should be adopted in the VVSG

Comment by Jim Tobias (Voting System Test Laboratory)

Certain other factors "outside" the typical metrics would also be worth considering. For example, some percentage of voters avoid certain voting systems because of their beliefs about accuracy and vote by absentee ballot instead. Some voters with disabilities vote absentee because they believe that the in-person system may not meet their needs. The behaviors of these voters should also be collected and analyzed.

Comment by Kitty Garber (Advocacy Group)

Summative usability testing has serious limitations. In a real voting environment, any number of variables can affect how difficult it is for voters to cast their ballots--which, of course,increaes voter errors. Problems that could affect residual votes include machine malfunctions or performance problems, long lines, ballot format issues, environmental conditions, and a host of other problems. If a system is not tested in a realist voting environment, then we cannot be sure how it will perform in such an environment. Certainly, "repeatable, controlled experiments" have the advantage of being comparable across systems, but they must also give us information that is relevant to actual performance in the real world. Otherwise, what is the point? Our most recent research on Florida's 2006 general election shows that the all touch screen counties (53% of Florida voters)had an 84 percent higher undervote rate on the five top-of-the-ballot statewide than did the optical scan counties (46% of Florida voters). These are races that appeared on every ballot in the state, in the same position, with the same candidates in the same order. The higher undervote rates were experienced on each of the five races and as a summary rate as well. Further examination of the data reveals that of the two touch screen systems--Sequoia AVC Edge and ES&S iVotronic--the iVotronic was by far the worst. Yet we suspect that the performance problems that the iVotronics had would be unlikely to show up in the sort of testing envisioned under these guidelines. Yet the tangible results of poorer performance by a voting system can lead to the disenfranchisement of hundreds of thousands of voters. "Average Voter Confidence." Voters only interact with the voter interface so any "confidence" that their vote is counted correctly is based on nothing whatsoever. And how would one possibly quantify such confidence? Wouldn't it be better to measure satisfaction with the voter interface, based on certain performance characteristics, such as ease of moving through the ballot, time to vote, clarity of instructions, etc.? "Average Voting Time" It would be helpful here to develop a measure that is more complex--just as is done with the voter inclusion index. Use of standard deviation here could help pinpoint system characteristics that lengthen voting time for certain types of voters or under certain circumstances.

Comment by Kitty Garber (Advocacy Group)

Summative usability testing has serious limitations. In a real voting environment, any number of variables can affect how difficult it is for voters to cast their ballots--which, of course,increaes voter errors. Problems that could affect residual votes include machine malfunctions or performance problems, long lines, ballot format issues, environmental conditions, and a host of other problems. If a system is not tested in a realist voting environment, then we cannot be sure how it will perform in such an environment. Certainly, "repeatable, controlled experiments" have the advantage of being comparable across systems, but they must also give us information that is relevant to actual performance in the real world. Otherwise, what is the point? Our most recent research on Florida's 2006 general election shows that the all touch screen counties (53% of Florida voters)had an 84 percent higher undervote rate on the five top-of-the-ballot statewide than did the optical scan counties (46% of Florida voters). These are races that appeared on every ballot in the state, in the same position, with the same candidates in the same order. The higher undervote rates were experienced on each of the five races and as a summary rate as well. Further examination of the data reveals that of the two touch screen systems--Sequoia AVC Edge and ES&S iVotronic--the iVotronic was by far the worst. Yet we suspect that the performance problems that the iVotronics had would be unlikely to show up in the sort of testing envisioned under these guidelines. Yet the tangible results of poorer performance by a voting system can lead to the disenfranchisement of hundreds of thousands of voters. "Average Voter Confidence." Voters only interact with the voter interface so any "confidence" that their vote is counted correctly is based on nothing whatsoever. And how would one possibly quantify such confidence? Wouldn't it be better to measure satisfaction with the voter interface, based on certain performance characteristics, such as ease of moving through the ballot, time to vote, clarity of instructions, etc.? "Average Voting Time" It would be helpful here to develop a measure that is more complex--just as is done with the voter inclusion index. Use of standard deviation here could help pinpoint system characteristics that lengthen voting time for certain types of voters or under certain circumstances.

Comment by Cem Kaner (Academic)

The emphasis on repeatable, controlled experiments that are not "realistic" measures of voting behaviors is not a good way to assess the usability of the system and not a good foundation for discovering or reporting usability problems. .......... The creation of a Voting Performance Protocol that is so standardized that even the test ballot is fully specified runs the risk of design optimization by the vendor in ways that pass the test easily without improving the usability of the system as a whole. A test ballot is only a tiny sample of the population of possible ballots, and it stops being a representative sample when application designers consider it specifically in their designs and test designers know that they will be required to use this ballot in their tests of each version of each voting device they test. Metrics based on such a test are likely to show apparent improvement over time, as systems are optimized toward better performance on this particular test. This improvement may or may not correlate with the underlying level of usability. .......... Usability tests (content and results) should be public records. Trends in test results should be compared from version to version of a voting system. .......... (Affiliation Note: IEEE representative to TGDC)
3.2.1.1-A Total completion performance

The system SHALL achieve a Total Completion Score of at least 98% as measured by the VPP.

Applies To: Voting System

Test Reference: Performance

1 Comment

Comment by ted selker (Academic)

The 98 percent is only as good as the 2000 election. This is not a good or motivated number. All numbers should refer to references.
3.2.1.1-B Perfect ballot performance

The system SHALL achieve a Perfect Ballot Index of at least 2.33 as measured by the VPP.

Applies To: Voting System

Test Reference: Performance

1 Comment

Comment by Jason Good (General Public)

A perfect ballot index of 2.33 corresponds to 30% of the general population not being able to use the voting system to accurately record their votes. This requirement is much too lax, especially given that the usability testing will be done by the vendors, rather than independent test labs. The requirement should be strengthened so that to require an index of at least that the 9 (failure rates at 10% or less) or even 19 (failure rates at 5% or less).
3.2.1.1-C Voter inclusion performance

The system SHALL achieve a voter inclusion index of at least 0.35 as measured by the VPP.

Applies To: Voting System

Test Reference: Performance

2 Comments

Comment by Alan A. Jorgensen, Ph.D., for the Association for Software Testing Special Interest Group on eVoting (Advocacy Group)

Please include in this section a link to an authoritative reference on the Voter Inclusion test methodology.

Comment by Audrey N. Glickman (Advocacy Group)

In assessing voter inclusion, practical consideration must be given to design characteristics, also. For illustration, I offer: Where one might say "This machine can be used by persons in wheelchairs with other disabilities, if and only if the machine is removed from its holder, several external arrangements are made, and the voter holds an official stick in her mouth," consideration should be given that (a) during a busy day at the polls, pollworkers may be disinclined to gather the necessary equipment and personnel to achieve this method; (b) there may not be privacy accommodations accessible or available under such circumstances; (c) if assistance is required anyway, all concerned may well opt for straightforward voter assistance. All levels of such practical consideration should be addressed - simple removal from a stand may present insurmountable obstacles if (a) pollworkers do not have the key, (b) the lines are long, (c) the voter came with others in a bus and time is of the essence, (d) the voting place may be too small to safeguard the privacy of others simultaneously, etc. If a voter in a wheelchair cannot reach some portion of the voting device without undue exertion or assistance, and without reconfiguring the device somehow, it should be considered inaccessible.
3.2.1.1-D Usability metrics from the Voting Performance Protocol

The test lab SHALL report the metrics for usability of the voting system, as measured by the VPP.

Applies To: Voting system

Source: New requirement

3.2.1.1-D.1 Effectiveness metrics for usability

The test lab SHALL report all the effectiveness metrics for usability as defined and measured by the VPP.

Applies To: Voting system

Source: New requirement

3.2.1.1-D.2 Voting session time

The test lab SHALL report the Average Voting Session Time, as measured by the VPP.

Applies To: Voting system

DISCUSSION

This requirement encourages systems to enable voters to vote with reasonable speed. Note that this requirement does not apply to the audio interface of a system, or to the use of special input devices for voters with dexterity disabilities.

Source: New requirement

2 Comments

Comment by Carolyn Coggins (Voting System Test Laboratory)

What is the standard ballot? Are there standard instructions? How do you calculate the start time, enter the booth, activation of ballot or start of actual candidate selection?

Comment by Electronic Privacy Information Center (Advocacy Group)

We are in support of this requirement because it serves the purpose of transparency on an important area of voting system’s performance. There may be statutory limits on the amount of time each voter may spend casting a ballot. For example: PA Title 25 §3057. Time allowed elector in voting booth or voting machine compartment. No elector shall remain in a voting compartment or voting machine booth an unreasonable length of time, and in no event, for more than three minutes, and if he shall refuse to leave after said period, he shall be removed by the election officers: Provided, however, That they may grant him a longer time if other electors are not waiting to vote. 1937, June 3 P.L. 1333, art. XLL, § 1217.
3.2.1.1-D.3 Average voter confidence

The test lab SHALL report the Average Voter Confidence, as measured by the VPP.

Applies To: Voting system

Source: New requirement

5 Comments

Comment by Gail Audette (Voting System Test Laboratory)

The test lab SHALL report the Average Voter Confidences, as measure by the VPP" The VPP (Voter Performance Protocol) has numbers with meaning only within the context of itself and does not represent a "realistic" measure of voting behavior (1:3.2.1.1). How SHALL the labs then report this unrealistic result of a repeatable controlled experiment consistently?

Comment by Gail Audette (Voting System Test Laboratory)

The test lab SHALL report the Average Voter Confidences, as measure by the VPP" The VPP (Voter Performance Protocol) has numbers with meaning only within the context of itself and does not represent "realistic" measure of voting behavior (1:3.2.1.1). How SHALL the labs then report this unrealistic results of a repeatable controlled experiment consistently?

Comment by ACCURATE (Aaron Burstein) (Academic)

We suggest paying more attention to the last two reporting requirements of Part 1:3.2.1.1, which cover the time taken to vote and voter confidence in the system. For example, the System Usability Scale (SUS) was developed to facilitate comparison of subjective usability assessments across similar, but slightly different, systems and its use is supported by recent psychometric research. The SUS score (or a subset of the SUS questionnaire) would likely make more sense as a reporting requirement for satisfaction than the undefined "Average Voter Confidence" measurement in this section. (The VPP defines "Average Voter Confidence" in more detail but we still feel that a standard metric like the SUS score would be more appropriate.)

Comment by Susan Pynchon (Advocacy Group)

It really isn't relevant if voters FEEL confident about any given type of machine. People felt confident about their Ford Pintos until they discovered the gas tanks blew up. People felt confient sailing on the Titanic at the beginning of its first and final voyage. In general, voters have no way to judge whether their votes are being accurately counted or not, and it is a false premise to assume that they can knowledgeably answer any question about whether they have "confidence" in the machines. This would be like asking someone if they have confidence in the new design of the space shuttle. A respondent could feel confident, but that confidence would be based on something other than actual understanding or knowledge. A more accurate questionnaire would ask voters specific questions about the usability of a voting system or specific problems they encountered, but "confidence" is generally based on a number of factors, and reflects perceived rather than actual reliability.

3.2.1.2 Manufacturer testing

3.2.1.2-A Usability testing by manufacturer for general population

The manufacturer SHALL conduct summative usability tests on the voting system using individuals who are representative of the general population and SHALL report the test results, using the Common Industry Format, as part of the TDP.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

Voting system developers are required to conduct realistic usability tests on the final product before submitting the system to conformance testing. This is to encourage early detection and resolution of usability problems.

6 Comments

Comment by Brian V. Jarvis (Local Election Official)

Since 3.2.1.2-A has the first usage of the term "TDP" in Chapter 3, recommend adding the phrase (Technical Data Package) immediately following it. Also, recommend adding "TDP" or "Technical Data Package" to the glossary.

Comment by Carolyn Coggins (Voting System Test Laboratory)

Are there any requirements for the sample group such as age, gender, race, education, literacy, disabilities, etc?

Comment by Jason Good (General Public)

The usability testing, like other testing, should be done by independent testing labs, not by the vendors. Vendors will not conduct fair testing of their own systems. It is not appropriate for federal certification to rely on vendor testing.

Comment by Frank Padilla (Voting System Test Laboratory)

What are the requirements for this testing? Who reviews this testing?

Comment by David Beirne, Executive Director, Election Technology Council (Manufacturer)

"Manufacturer is required to conduct summative usability tests on the voting system using individuals who are representative of the general population and shall report the results;" I would suggest defining "individuals who are representative of the general population" so as to avoid future confusion. In particular, what is the data source?

Comment by ACCURATE (Aaron Burstein) (Academic)

Along with the six other summative usability tests defined in the VVSG draft, testing the system for the general population is vital for demonstrating that the system is usable by a wide variety of users, from the general population of voters to voters with specific disabilities.

3.2.2 Functional capabilities

The usability of the voting process is enhanced by the presence of certain functional capabilities. These capabilities differ somewhat depending on whether or not the system presents an editable interface within which voters can easily change their votes (typically an electronic screen) or an interface in which voters must obtain a new ballot to make changes (typically a manually-marked paper ballot).

2 Comments

Comment by ted selker (Academic)

These capabilities differ somewhat Should be changed to: These capabilities differ by many dimensions especially relative to feedback. Active systems give feedback as the voter is in the process of voting. Passive systems give feedback after the ballot is given to a system or person for evaluation and require the voter to remember the intended interpretation. People make remarkably fewer errors with active systems.

Comment by Diane Golden (Advocacy Group)

This section should include a statement that an accessible voting system, as mandated by HAVA, MUST have a voter editable interface.
3.2.2-A Notification of effect of overvoting

If the voter selects more than the allowable number of choices within a contest, the voting system SHALL notify the voter of the effect of this action before the ballot is cast and counted.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

In the case of manual systems, this may be achieved through appropriately placed instructions. This requirement has no force for VEBD systems, since they prevent overvoting in the first place.

3.2.2-B Undervoting to be permitted

The voting system SHALL allow the voter, at the voter’s choice, to submit an undervoted ballot without correction.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.2-C Correction of ballot

The voting system SHALL provide the voter the opportunity to correct the ballot for either an undervote or overvote before the ballot is cast and counted.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

In the case of manual systems, this may be achieved through appropriately placed written instructions. Some corrections may require the voter to obtain a new paper ballot from a poll worker. Also, note the requirements on precinct-count optical scanners in Section 3.2.2.2 below.

4 Comments

Comment by Gail Audette (Voting System Test Laboratory)

If a blank ballot consists of only 1 contest, how does the system distinguish between an undervote versus a blank ballot? The requirements of 3.2.2.2-A and C conflict.

Comment by Alan A. Jorgensen, Ph.D., for the Association for Software Testing Special Interest Group on eVoting (Advocacy Group)

This section only mentions over-voting and under-voting and does not mention mis-voting. The voter must be allowed to change a vote before it is finally cast whether by manual or automated machine. We strongly recommend that this requirement be restated so as to be consistent with the intent of Part 1, Ch. 3, 3.2.2.1-C, Independent correction of ballot, which is designed to ensure that voters get to review and correct their ballot before casting it.

Comment by Electronic Privacy Information Center (Advocacy Group)

We are in support of this subsection because it protects voter privacy and ballot secrecy. Usability and accessibility of IVVRs and CVRs for final review by voter must be made clear by instruction provided by ballot marking interface and the physical design of the voting device. • Instructions and physical directions should allow voters to access IVVR and CVR • Poll worker assistance should not be required • Allow a voter to discard the ballot as spoiled and re-engage the voting process • If limits exists on the number of times that a voter may discard spoiled ballots or engage in voting this should be communicated • Removal of ballot access device, token, or card should remove IVVR and CVR from display mode • Initialization of voting device to service another voter should clear the ballot review facility of the ballot and deposit it in the tabulation pool.

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

USACM Comment #8. Section 3.2.2-C [Incomplete] USACM recommends that the discussion of "Correction of ballot" should be amended to include the ability to modify a vote. Specifically, the text should be changed to read: The voting system SHALL provide the voter the opportunity to correct the ballot for either an undervote or overvote, or to change any votes, before the ballot is cast and counted. DISCUSSION: Voter review of selections may lead to the identification of incorrectly cast votes, which would not necessarily be undervotes or overvotes. Voters should have the ability to change all such votes.
3.2.2-D Notification of ballot casting

If and only if the voter successfully casts the ballot, then the system SHALL so notify the voter.

Applies To: DRE, PCOS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The purpose of this requirement is to provide feedback to voters to assure them that the voting session has been completed. Note that either a false notification of success or a missing confirmation of actual success violates this requirement.

3.2.2.1 Editable interfaces

Voting systems such as DREs and EBMs present voters with an editable interface, allowing them to easily change their votes prior to final casting of the ballot.

3.2.2.1-A Prevention of overvotes

The VEBD SHALL prevent voters from selecting more than the allowable number of choices for each contest.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement does not specify exactly how the system must respond when a voter attempts to select an "extra" candidate. For instance, the system may prevent the selection and issue a warning, or, in the case of a single-choice contest, simply change the vote.

2 Comments

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

We propose that there be two recognized modes and that the first is preferable 1. Attempting to choose an additional candidate is blocked with a message rendered 2. Attempting to choose an additional candidate un-chooses the first candidate chosen Once the voter is alerted to the fact they have made too many choices, we propose two correction modes 1. Choosing an already chosen candidate un-chooses that candidate 2. The most recent candidate chosen is un-chosen as a feature of accepting a dialogue prompt

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

USACM Comment #9. 3.2.2.1-A Prevention of Overvotes [Imprecise] USACM recommends that the requirements for changing overvotes should be changed to read as follows: The VEBD SHALL prevent voters from selecting more than the allowable number of choices for each contest. If this process causes the VEBD to make any changes to the selection of votes, the exact nature of any changes must be clearly presented to the user. DISCUSSION: As currently stated, the discussion of this requirement reads as follows: This requirement does not specify exactly how the system must respond when a voter attempts to select an "extra" candidate. For instance, the system may prevent the selection and issue a warning, or, in the case of a single-choice contest, simply change the vote. Unfortunately, simply ‘changing the vote’ may not match the voter’s goals and intentions. Voters must be given clear and appropriate feedback whenever such changes are made. If the vote is changed the voter must have the opportunity to review that change.
3.2.2.1-B Warning of undervotes

The VEBD SHALL provide feedback to the voter, before final casting of the ballot that identifies specific contests for which the voter has selected fewer than the allowable number of choices (i.e., undervotes).

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For VEBD systems, no allowance is made for disabling this feature. Also, see requirement below on "Clarity of Warnings."

3.2.2.1-C Independent correction of ballot

The VEBD SHALL provide the voter the opportunity to correct the ballot before it is cast and counted. This correction process SHALL NOT require external assistance. The corrections to be supported include modifying an undervote or overvote, and changing a vote from one candidate to another.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

2 Comments

Comment by Carolyn Coggins (Voting System Test Laboratory)

Must this warning list the specific undervoted contests or is a general warning of undervoted contests sufficient? Must there be a final summary screen warning?

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

We propose that any confirmation of an under-voted ballot include a statement to the effect that "There are N candidates not chosen" where N is the number of candidates below the minimum required to cast a well formed vote.
3.2.2.1-D Ballot editing per contest

The VEBD SHALL allow the voter to change a vote within a contest before advancing to the next contest.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The point here is that voters using an editable interface should not have to wait for a final ballot review screen in order to change a vote.

1 Comment

Comment by Alan A. Jorgensen, Ph.D., for the Association for Software Testing Special Interest Group on eVoting (Advocacy Group)

The word "independent" correction of the ballot is used in this requirement to ensure that voters will not require assistance to make their changes. We do not believe this is a legitimate issue, and recommend that it be removed from the requirement. Voters should be allowed to change and correct their ballot as many times as needed, whether the system they use is electronic or not. If it not electronic, and they need the assistance of a poll worker to refresh their ballot and dispose of their old one, they should be allowed to do so. Similarly with an electronic device, an otherwise secure and fit device that requires a poll worker to perform an operation to allow a user to fix her balloting error should not be disqualified.
3.2.2.1-E Contest navigation

The VEBD SHALL provide navigation controls that allow the voter to advance to the next contest or go back to the previous contest before completing a vote on the contest(s) currently being presented (whether visually or aurally).

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, voters should not be forced to proceed sequentially through all the contests before going back to check their votes within a previous contest.

2 Comments

Comment by ted selker (Academic)

editable interfaces Might be changed to: Interfaces with active feedback. How do we feel about whiteout or erasers on optical-scan ballots? Election officials have illegally turned off overvote detection in the past. Overvote detection settings and any other election-official-settable state needs to be part of all records of use of any equipment.

Comment by David Beirne, Executive Director, Election Technology Council (Manufacturer)

From a usability standpoint, the forward advance of a voting system can reduce ballot errors and speed ballot efficiency (time spent in the voting booth); This requirement should be stricken or further reviewed before incorporated.
3.2.2.1-F Notification of ballot casting failure (DRE)

If the voter takes the appropriate action to cast a ballot, but the system does not accept and record it successfully, including failure to store the ballot image, then the DRE SHALL so notify the voter and provide clear instruction as to the steps the voter should take to cast the ballot.

Applies To: DRE

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

If a DRE fails at the point of casting a ballot, it must clearly indicate to the voter and to election officials responding to the failure whether or not the ballot was cast. Otherwise, election officials may be unable to provide substantial confirmation that the vote was or was not counted, possibly resulting in disenfranchisement or the casting of two ballots by a single voter.

A device that "freezes" when the voter attempts to cast the ballot, providing no evidence one way or the other whether the ballot was cast, would violate this requirement.

Source: 2002 VSS I.2.4.3.3.k / VVSG'05 I.2.3.3.3.m

1 Comments

Comment by ted selker (Academic)

The Voter Editable Ballot Device SHALL provide navigation controls Should be changed to: The Voter Editable Ballot Device SHALL orient the voter as to their location on the ballot and provide navigation controls.

3.2.2.2 Non-Editable interfaces

Non-Editable interfaces, such as manually-marked paper ballots (MMPB) do not have the same flexibility as do editable interfaces. Nonetheless, certain features are required, especially in the case of precinct-based optical scanners. Note that the technical definition of "marginal mark" may be found in Appendix A. Basically, a marginal mark is one that, according the manufacturer specifications, is neither clearly countable as a vote nor clearly countable as a non-vote.

3 Comments

Comment by Carolyn Coggins (Voting System Test Laboratory)

Creating the conditions to functionally test this requirement may not be possible. What are alternative acceptable methods for testing?

Comment by Frank Padilla (Voting System Test Laboratory)

Is this requirement testable?

Comment by Electronic Privacy Information Center (Advocacy Group)

This subsection is important and should be strengthened. Addition: "…the DRE SHALL so notify the voter…" [STRIKE] "and" [INSERT] "by visual and audible means and only then "provide clear instructions as to the steps the voter should take to cast the ballot." Discussion: Care should be taken to selecting an appropriate time interval to allow voters to review their ballot. The goal is to inform the voter that they have not completed the process so that they may do so.
3.2.2.2-A Notification of overvoting

The voting system SHALL be capable of providing feedback to the voter that identifies specific contests for which the voter has made more than the allowable number of votes (i.e.,. overvotes).

Applies To: PCOS

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.2.2-B Notification of undervoting

The voting system SHALL be capable of providing feedback to the voter that identifies specific contests for which the voter has made fewer than the allowable number of votes (i.e., undervotes). The system SHALL provide a means for an authorized election official to deactivate this capability entirely and by contest.

Applies To: PCOS

Test Reference: Part 3: 3.2 "Functional Testing"

2 Comments

Comment by ted selker (Academic)

Manually-marked paper ballots (MMPB) do not have the same flexibility Should be changed to: Manually-marked paper ballots (MMPB) do not have the same ability to give feedback.

Comment by Electronic Privacy Information Center (Advocacy Group)

This subsection provides greater transparency to voters who cast ballots using optical scan precinct count systems. Precinct count systems should also provide access voters who are minority language speakers or persons with disabilities.
3.2.2.2-C Notification of blank ballots

The voting system SHALL be capable of notifying the voter that he or she has submitted a paper ballot that is blank on one or both sides. The system SHALL provide a means for an authorized election official to deactivate this capability.

Applies To: PCOS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

One purpose of this feature is to detect situations in which the voter might be unaware that the ballot is two-sided. This feature is distinct from the ability to detect and warn about undervoting.

1 Comments

Comment by ted selker (Academic)

If the voting system has notified the voters that a potential error condition … SHALL then allow the voter… Should be changed to: If the voting system has detected that a potential error condition …SHALL then notify and allow the voter…
3.2.2.2-D Ballot correction or submission following notification

If the voting system has notified the voter that a potential error condition (such as an overvote, undervote, or blank ballot) exists, the system SHALL then allow the voter to correct the ballot or to submit it as is.

Applies To: PCOS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement mandates that the equipment be capable of allowing either correction or immediate submission. For instance, a questionable paper ballot might be physically ejected for possible correction. This requirement does not constrain the procedures that jurisdictions might adopt for handling such situations (e.g., whether poll worker intervention is required).

3 Comments

Comment by ted selker (Academic)

"gray zone" needs to be removed from the recommendations.

Comment by Carolyn Coggins (Voting System Test Laboratory)

Does a system fail if reads and counts a marginal mark?

Comment by Frank Padilla (Voting System Test Laboratory)

Are marginal marks allowed or not allowed? If the voter can still be allowed to cast a ballot with them without correction, how do these count? An interpretation would have to made to determine what the mark is for and then judged against how it was counted? This is not really testable.
3.2.2.2-E Handling of marginal marks

Paper-based precinct tabulators SHOULD be able to identify a ballot containing marginal marks. When such a ballot is detected, the tabulator SHALL:

  1. Return the ballot to the voter;
  2. Provide feedback to the voter that identifies the specific contests for which a marginal mark was detected; and
  3. Allow the voter either to correct the ballot or to submit the ballot "as is" without correction.

Applies To: Precinct tabulator

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The purpose of this requirement is to provide more certainty about the handling of poorly-marked ballots. If a given candidate or option is clearly marked as chosen, or left completely unmarked, then there is no ambiguity to resolve. However, each manufacturer should define a "gray zone" (with respect to location, darkness, etc.) in which marks will be actively flagged as ambiguous.

1 Comments

Comment by Richard Carback (Academic)

How can you verify that the system can always do this? What if there is a hardware failure? For example, what if the screen fails and goes black? How long is a "freeze"? If it sticks for a second or two, does that constitute a "freeze"? At some level this requirement is untestable. A more general requirement that can be tested would be more useful. For instance, It would be better to simply require that machines have logic that indicates failure (e.g. a blue screen of death that does not say "YOUR VOTE WAS NOT COUNTED" would fail the test, as would a system who's screen went blank and did not have logic to BEEP or turn an LED RED to indicate machine failure).
3.2.2.2-F Notification of ballot casting failure (PCOS)

If the voter takes the appropriate action to cast a ballot, but the system does not accept and record it successfully, including failure to read the ballot or to transport it into the ballot box, the PCOS SHALL so notify the voter.

Applies To: PCOS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement means that PCOS systems must detect and report electrical and mechanical failures within the system itself. It does not require the detection of errors on the part of the voter. See also Requirement Part 1: 7.7.4-B.

3.2.3 Privacy

The voting process must preclude anyone else from determining the content of a voter's ballot without the voter's cooperation. Privacy ensures that the voter can cast votes based solely on his or her own preferences without intimidation or inhibition.

6 Comments

Comment by Fernando Morales (Manufacturer)

Add to comply with HAVA02 Section 301. 3.2.3.2 Privacy out-side the polls 3.2.3.2-A System support of privacy The absentee voting system SHALL prevent others from determining for who an individual vote. The following three minutes video show how to do it: http://www.havabreach.com/video1.html

Comment by Fernando Morales (Manufacturer)

Change must for shall to complay with HAVA02 Section 301 The voting process SHALL preclude anyone else from determining the content of a voter's ballot without the voter's cooperation. Privacy ensures that the voter can cast votes based solely on his or her own preferences without intimidation or inhibition.

Comment by Alan A. Jorgensen, Ph.D., for the Association for Software Testing Special Interest Group on eVoting (Advocacy Group)

The requirement that the voting system offer the voter the chance to verify that the cast ballot matches the ballot they selected goes in opposition to the privacy requirement. The requirement that the voting system offer audible cues goes against the privacy requirement. The fact that it is impossible to test for a negative ("shall prevent" is a negative) goes against the privacy requirement. This requirement though, crucial and worthy, will be very problematic to test and enforce. We recommend creating an expert team of exploratory testers to test the private-ness of the system, in the mold of Open Ended Vulnerability Testing (OEVT). We recommend including on the team one or more criminologists who study vote-influencing tactics. . . . . . . In any case, testing with pre-determined tests will not suffice for this requirement, because someone will always be able to come up with a theory about how a voter's privacy could be invaded which is not addressed in these tests. The only option in this case of essentially infinite problem space is to employ skilled experts and let them test until their considered opinion is that the system is sufficiently private.

Comment by Richard Carback (Academic)

Intimidation would constitute a voter's cooperation. A coercer could intimidate you to cooperate to violate your privacy. Every effort should be made to make sure, even with cooperation from the voter, that privacy is maintained. Also, what about vote selling, where the voter wishes to cooperate in order to prove their intent for money or other gain? The text here is contradictory. If you can force me to cooperate, I can be intimidated..

Comment by David Cary, Californians for Electoral Reform (Advocacy Group)

This section should distinguish that the important goal is to reduce the opportunity to associate ballot content with a voter. Limiting knowledge of ballot content is one method of achieving that goal, but it is a method that can not be relied upon exclusively without creating conflict with other goals. For example, a VVPAT is designed to allow others to view the content of a ballot while reducing the availability of information about who cast the ballot. The guideline language should clarify and distinguish these concepts more carefully.

Comment by Frank Padilla (Voting System Test Laboratory)

"The voting process shall..."? (now states "must").

3.2.3.1 Privacy at the polls

3.2.3.1-A System support of privacy

The voting system SHALL prevent others from determining the contents of a ballot.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The voting system itself provides no means by which others can "determine" how one has voted. Of course voters could simply tell someone else for whom they voted, but the system provides no evidence for such statements, and therefore voters cannot be coerced into providing such evidence.

It is assumed that the system is deployed according to the installation instructions provided by the manufacturer. Whether the configuration of the voting system protects privacy may well depend on proper setup.

5 Comments

Comment by Fernando Morales (Manufacturer)

Add to comply with HAVA02 Section 301. The voting system SHALL prevent others from determining the contents of a ballot and SHALL prevent the voter to take a picture of the ballot.

Comment by ted selker (Academic)

This privacy point precludes the recommendation endorsing no-excuse absentee ballots as being private or secure.

Comment by David Cary, Californians for Electoral Reform (Advocacy Group)

This requirement should be modified to reflect that what needs to be controlled is the ability to associate ballot contents with a voter. The guidelines should clarify that this requirement does not preclude the collection and examination of ranked order voting ballot contents for the purposes of tabulating and verifying results. In particular, the guidelines should clearly allow for existing practices of publishing of ballot contents, for example as has been practiced in San Francisco. Publishing ballot content in this context improves the transparency and public confidence in elections. It may also be appropriate to add requirements that provide restrictions on the publishing of ballot contents, for example: -- not providing an association between the contents of a ballot for different contests, -- hiding the order in which ballots were cast, -- assuring a minimum number of ballots in a reported subgroup, or -- restricting which rankings are published.

Comment by Kitty Garber (Advocacy Group)

Use of different types of equipment at the polls for disabled and nondisabled voters allows the privacy of the ballot to be violated. This is true both for DREs and for blended systems that use DREs for accessibility. Only electronic ballot markers with PCOS permit accumulation of votes at the precinct without the potential for revealing the votes those who choose to use the disabled accessible equipment.

Comment by Electronic Privacy Information Center (Advocacy Group)

3.2.3.1 Overall Performance Measure for Privacy of Ballot Casting Process • Voter’s ability to navigate the ballot casting process without assistance
3.2.3.1-A.1 Visual privacy

The ballot, any other visible record containing ballot information, and any input controls SHALL be visible only to the voter during the voting session and ballot submission.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement may involve different approaches for electronic and paper interfaces. In both cases, appropriate shielding of the voting station is important. When a paper record with ballot information needs to be transported by the voter, devices such as privacy sleeves may be necessary. This requirement applies to all records with information on votes (such as a vote verification record) even if that record is not itself a ballot.

4 Comments

Comment by Marybeth Kuznik (Local Election Official)

This needs to be tested more thoroughly. Our current system (ES&S iVotronic, 2002 VVSG) has caused MANY complaints in my community regarding voter privacy despite our best efforts to position the machines in a manner that does not allow viewing by others while a voter is voting. But why should local election officials have to drive ourselves nuts moving and angling machines just so? Better design and better testing is needed. Marybeth Kuznik Elected Majority Inspector of Elections Penn Township, Westmoreland County PA

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

USACM Comment #10. Section 3.2.3.1-A.1 — Visual Privacy USACM recommends that the text of this requirement be amended to the following: "The ballot, any other visible record, containing ballot information, and any input controls SHALL be visible only to the voter during the voting session and ballot submission, independent of whether the voter is seated or standing." DISCUSSION: The additional clause, "independent of whether the voter is seated or standing," would clarify that appropriate shielding of the voting station must take into account variations in height of the voter, many of which may result from a voter who is seated while voting compared to one who is standing while voting.

Comment by Cem Kaner (Academic)

In practice, insertion of ballots into optical scanners has been confusing enough for voters that poll workers often end up handling the ballot. I know this from voting in several elections in Brevard County, Florida and from observing the conduct of one election. In one election, poll workers insisted on inserting ballots themselves (and occasionally commented on the choices of the voter). In others, voters were allowed to insert their ballots themselves but a poll worker was stationed beside the scanner and could easily see what was on the ballot. The rationale for the stationing (I asked) was that people often had trouble with the scanner and so the poll worker was there to help quickly--speed being important when there were (as there were in fact) long lines of voters waiting to cast their vote. .......... Assessment of the privacy actually afforded by a voting system must include observation of the use of the equipment in actual elections. There should also be a public reporting system for members of the public to identify system-level privacy issues (like this one) and any other usability issues. .......... Without a well-publicized, easy to access, feedback loop from the actual users of the system to the system developers, we will never understand the actual usability characteristics of a system or how it needs to be improved. .......... (Affiliation Note: IEEE representative to TGDC)

Comment by ted selker (Academic)

Privacy sleeves needs design and process specification work here. They have not been very successful. In my experience the only ones that get used correctly must be demonstrated: Privacy sleeves SHALL be collated with ballots before handing them to the voter. This avoids inevitable problems of voters not using them or not using them in a way that allows the ballot to be read.
3.2.3.1-A.2 Auditory privacy

During the voting session, the audio interface of the voting system SHALL be audible only to the voter.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Voters who are hard of hearing but need to use an audio interface may also need to increase the volume of the audio. Such situations require headphones with low sound leakage.

3 Comments

Comment by Robert A. Wolsey (Academic)

To ensure that votes are cast the way the voter intended, voters should be able to request a paper receipt showing how the voting machine recorded their selections. Without the paper receipt, there is absolutely no way a voter can make sure their vote was counted properly in the event of tampering or mechanical failure. As long as printing the receipt is optional, this has no effect on the voter's privacy. Without the option of a voting "paper trail", large-scale voting fraud becomes easier and voter confidence in the election system declines. I will leave no other comments on this document, since I feel that this point is several orders of magnitude more important than the myriad technical requirements put forth.

Comment by Richard Carback (Academic)

"No Receipts" is misleading, it would be better to say "No Privacy Leaking/Revealing Receipts". This requirement says that it should not violate your privacy, not that you should not get a receipt.

Comment by Electronic Privacy Information Center (Advocacy Group)

"No receipt" may be too broad the goal is not to issue to a voter physical evidence of how they cast a ballot in an election. Many jurisdictions provide "I voted" stickers to voters as they leave voting locations. These are not considered a threat to the secrecy of the ballot or voter privacy. If a digital equivalent of the "I voted" sticker is provided they should not be prohibited by the standard. This "I voted" equivalent should provide no more or no less than this information to preserve voter privacy and ballot secrecy
3.2.3.1-A.3 Privacy of warnings

The voting system SHALL issue all warnings in a way that preserves the privacy of the voter and the confidentiality of the ballot.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

HAVA 301 (a)(1)(C) mandates that the voting system must notify the voter of an attempted overvote in a way that preserves the privacy of the voter and the confidentiality of the ballot. This requirement generalizes that mandate.

3.2.3.1-A.4 No receipts

The voting system SHALL NOT issue a receipt to the voter that would provide proof to another of how the voter voted.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.3.2 No recording of alternative format usage

When voters use non-typical ballot interfaces, such as large print or alternative languages, their anonymity may be vulnerable. To the extent possible, only the logical contents of their ballots should be recorded, not the special formats in which they were rendered. In the case of paper ballots, where the interface is the record, some format information is unavoidably preserved.

4 Comments

Comment by Brian V. Jarvis (Local Election Official)

Need to elaborate on "to the extent possible." This phrase leaves too much wiggle-room for manufacturers. Recommend specifying a list of permissible items.

Comment by ted selker (Academic)

"To the extent possible" must be quantified and enumerated.

Comment by Jim Tobias (Voting System Test Laboratory)

In the case of alternative interfaces that produce a paper ballot clearly indicating that it was produced alternatively, a polling place may support privacy by requesting other, non-disabled voters to use that alternate interface.

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

What is permitted to be recorded within the voting system? For example it is valuable to record general paging information and session times as the basis of measuring that voters did not struggle with the machine. Granted this recording should not capture the language chosen, font size or anything else which could identify a voter.
3.2.3.2-A No recording of alternative languages

No information SHALL be kept within an electronic CVR that identifies any alternative language feature(s) used by a voter.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

1 Comment

Comment by Premier Election Solutions (Manufacturer)

This requirement prevents the ability to re-print the VVPAT in the same form (i.e. language) as originally printed. Proposed Change: Remove this requirement.
3.2.3.2-B No Recording of Accessibility Features

No information SHALL be kept within an electronic CVR that identifies any accessibility feature(s) used by a voter.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.4 Cognitive issues

The features specified in this section are intended to minimize cognitive difficulties for voters. They should always be able to operate the voting system and understand the effect of their actions.

2 Comments

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

General comment - the brevity of instructions is important but the standards should allow for referendum information to be rendered as well as candidate and party information, subject to length limitations for platforms. Note that referenda supporting text is often very large.

Comment by U.S. Public Policy Committee of the Association for Computing Machinery (USACM) (None)

USACM Comment #11. Section 3.2.4 Cognitive Issues [incomplete] USACM recommends that the following requirement be added to the list of cognitive requirements: Non-threatening language: Warnings and alerts issued by the voting system SHALL use non-threatening language. Terms such as ‘abort’, ‘die’, or ‘fatal error’ should be avoided. "Clear and direct relationship between selection mechanism and candidate: Any mechanism that is used to select a candidate must be spatially near the candidate being selection, and positioned in such a way as to make a clear and unambiguous correspondence between the mechanism and candidate. DISCUSSION: Threatening language may intimidate some voters and cause them to lose faith in their operation of the machine. Appropriately crafted messages can indicate the nature of problems without scaring voters. A clear and direct relationship between selection mechanism and candidates would prevent situations where, for example, the candidate name is on the far left side of the screen and the selection mechanism is on the far right side of the screen.
3.2.4-A Completeness of instructions

The voting station SHALL provide instructions for all its valid operations.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

If an operation is available to the voter, it must be documented. Examples include how to change a vote, how to navigate among contests, how to cast a straight party vote, how to cast a write-in vote, and how to adjust display and audio characteristics.

2 Comments

Comment by ted selker (Academic)

When are instructions adequate? Also, the phrase, "It must be documented" must be quantified; where, how, and how tested?

Comment by Frank Padilla (Voting System Test Laboratory)

Who must these instructions be available to? Part of the TDP? Posted on the system?
3.2.4-B Availability of assistance from the system

The voting system SHALL provide a means for the voter to get help directly from the system at any time during the voting session.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The voter should always be able to get help from the system if needed. The purpose is to minimize the need for poll worker assistance. VEBD voting systems may provide this with a distinctive "help" button. Any type of voting system may provide written instructions that are separate from the ballot.

3 Comments

Comment by ted selker (Academic)

Provide written instructions Should change to: provide written instructions, readable in the formats interpretable by the voter.

Comment by Carolyn Coggins (Voting System Test Laboratory)

"Get help" is vague. Define the types of help required to be available at all times.

Comment by Frank Padilla (Voting System Test Laboratory)

What type of help? This term is to vague and is not testable.
3.2.4-C Plain Language

Instructional material for the voter SHALL conform to norms and best practices for plain language.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Although part of general usability, the use of plain language is also expected to assist voters with cognitive disabilities. The plain language requirements apply to instructions that are inherent to the voting system or that are generated by default. To the extent that instructions are determined by election officials designing the ballot, they are beyond of the scope of this requirement.

4 Comments

Comment by Gail Audette (Voting System Test Laboratory)

"Plain language" is not defined. It the tester must define "plain language", it is then a qualitative test and not consistent between test labs. How is compliance determined and who sets these guidelines?

Comment by Alan A. Jorgensen, Ph.D., for the Association for Software Testing Special Interest Group on eVoting (Advocacy Group)

"Making Ballot Language Understandable to Voters" by Sharon J. Laskowski and Janice (Ginny) Redish reports that "The wording and placement of instructions can help or hinder voters from successfully voting for their choices. ... A recent study ... shows that typical ballots in the United States violate many of the best practices drawn from research." (https://www.usenix.org/events/evt06/tech/full_papers/laskowski/laskowski.pdf) . . . . . . We recommend that the "plain language" requirement of this section be extended to all materials handed off from the manufacturer and test lab, to testers, to poll workers, to manufacturers, as well as to voters.

Comment by Frank Padilla (Voting System Test Laboratory)

Plain Language is not defined. This would mean something different to every person asked.

Comment by Janice Redish (Advocacy Group)

The requirements for plain language are very important. Best practices do exist for giving instructions clearly. Plain language is required for many other types of documents. For example, writers of government documents in the state of Washington are required by executive order to use plain language (http://www.accountability.wa.gov/plaintalk/). Writers of government documents in the state of Oregon are required by law to use plain language (http://www.leg.state.or.us/07orlaws/sess0100.dir/0142.htm). A bill requiring writers in federal government agencies to use plain language passed the House of Representatives in March by a vote of 376 for; 1 opposed. A similar bill is being considered in the Senate. It was voted out of committee with a unanimous vote in favor and should come up soon for a vote of the full Senate. Several states also have laws requiring plain language in non-government consumer documents. Plain language for voting systems makes excellent sense. Plain language helps everyone. In the VVSG, plain language comes under the requirements for making voting accessible to people with cognitive disabilities. It should in fact apply to all ballots in all situations for all people. Plain language is an example, like so many others, of how accessibility and usability go together: helping people with special needs helps everyone. For example, we all read common and frequent words faster than we read less common and less frequent words. If we write in plain language for people with low literacy, we find that people with high literacy are grateful, too.

Warnings and alerts issued by the voting system SHOULD clearly state:

  1. The nature of the problem;
  2. Whether the voter has performed or attempted an invalid operation or whether the voting equipment itself has malfunctioned in some way; and
  3. The set of responses available to the voter.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, "You have not interacted with the system for the past three minutes. Please press the ‘Need more time’ button right away to tell the system that you’re still here – Thank you." rather than "System detects imminent timeout condition." In case of an equipment failure, the only action available to the voter might be to get assistance from a poll worker.

2 Comments

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

These apply when the machine is operating correctly but also the system should present error modes and degraded functionality modes as much as possible which consider that they are most likely to be seen by the voter before a PO sees them. Failure messages therefor should consider that the voter wonders if their vote has been recorded or not or whether the system is safe. It would be wise to assume voters report problems first and so error modes need to perhaps hide information and merely ask for the PO. Note that http://vote.nist.gov/meeting-08172007/Usability-Benchmarks-081707.pdf tests do not include any test names with foreign characters in them nor party names of considerable length nor candidates or parties with close similarity among them. These things affect voter comprehension.

Comment by ted selker (Academic)

"Warnings and alerts issued by the voting system SHOULD clearly state:" Should change to: "Warnings and alerts issued by the voting system SHALL be distinguishable from other information. They must be interpretable and accessible." "Past 3 minutes" needs reference and justification
3.2.4-C.2 Context before action

When an instruction is based on a condition, the condition SHOULD be stated first, and then the action to be performed.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, use "In order to change your vote, do X", rather than "Do X, in order to change your vote."

The system SHOULD use familiar, common words and avoid technical or specialized words that voters are not likely to understand.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, "... there are more contests on the other side ..." rather than "...additional contests are presented on the reverse ..."

1 Comment

Comment by Frank Padilla (Voting System Test Laboratory)

Familiar, common words: these should be defined or state a reference.
3.2.4-C.4 Start each instruction on a new line

The system SHOULD start the visual presentation of each new instruction on a new line.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This implies not "burying" several unrelated instructions in a single long paragraph.

1 Comment

Comment by ted selker (Academic)

The system should start the visual presentation of each new instruction on a new line Should change to: The system SHALL separate the instructions spatially for visual interfaces, and temporally for tactile or auditory communication.
3.2.4-C.5 Use of positive

The system SHOULD issue instructions on the correct way to perform actions, rather than telling voters what not to do.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, "Fill in the oval for your write-in vote to count" rather than "If the oval is not marked, your write-in vote cannot be counted."

1 Comment

Comment by ted selker (Academic)

The system should issue Should change to: The system SHALL issue
3.2.4-C.6 Use of imperative voice

The system's instructions SHOULD address the voter directly rather than use passive voice constructions.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, "remove and retain this ballot stub" rather than "this ballot stub must be removed and retained by the voter."

3 Comments

Comment by ted selker (Academic)

This needs a testing approach and metric. Also, "Certain differences in presentation are mandated by law…" Should change to: The VVSG does not support biased ballots. While there is a history of laws concerning ballot design and incumbent races in some jurisdictions, these laws are not in compliance with measurable fairness in selection ease in elections.

Comment by Carolyn Coggins (Voting System Test Laboratory)

Introduction of bias is subjective and vague. List methods to mitigate bias that do not conflict with any state election law.

Comment by Craig Burton, CTO, EveryoneCounts.com (Manufacturer)

Is this relevant to randomization of choices, and if so, does randomization have to be limited to party accepted orders or can it be truly random? In some ballot rotation systems (admittedly outside the USA) ballot rotations are a small, acceptable set, not "truly random".
3.2.4-C.7 Gender-based pronouns

The system SHOULD avoid the use of gender-based pronouns.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, "...write in your choice directly on the ballot..." rather than "... write in his name directly on the ballot..."

3.2.4-D No bias among choices

Consistent with election law, the voting system SHALL support a process that does not introduce bias for or against any of the contest choices to be presented to the voter. In both visual and aural formats, the choices SHALL be presented in an equivalent manner.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

Certain differences in presentation are mandated by state law, such as the order in which candidates are listed and provisions for voting for write-in candidates. However, comparable characteristics such as font size or voice volume and speed must be the same for all choices.

3.2.4-E Ballot design

The voting system SHALL provide the capability to design a ballot with a high level of clarity and comprehensibility.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.4-E.1 Contests split among pages or columns

The voting system SHOULD NOT visually present a single contest spread over two pages or two columns.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Such a visual separation poses the risk that the voter may perceive one contest as two, or fail to see additional choices. If a contest has a large number of candidates, it may be infeasible to observe this guideline.

3.2.4-E.2 Indicate maximum number of candidates

The ballot SHALL clearly indicate the maximum number of candidates for which one can vote within a single contest.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.4-E.3 Consistent representation of candidate selection

The relationship between the name of a candidate and the mechanism used to vote for that candidate SHALL be consistent throughout the ballot.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, the response field where voters indicate their votes must not be located to the left of some candidates' names, and to the right of others'.

3.2.4-E.4 Placement of instructions

The system SHOULD display instructions near to where they are needed.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, only general instructions should be grouped at the beginning of the ballot; those pertaining to specific situations should be presented where and when needed.

3.2.4-F Conventional use of color

The use of color by the voting system SHOULD agree with common conventions: (a) green, blue or white is used for general information or as a normal status indicator; (b) amber or yellow is used to indicate warnings or a marginal status; (c) red is used to indicate error conditions or a problem requiring immediate attention.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.4-G Icons and language

When an icon is used to convey information, indicate an action, or prompt a response, it SHALL be accompanied by a corresponding linguistic label.

Applies To: Voting device

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

While icons can be used for emphasis when communicating with the voter, they must not be the sole means by which information is conveyed, since there is no widely accepted "iconic" language and therefore not all voters may understand a given icon.

3.2.5 Perceptual issues

The requirements of this section are designed to minimize perceptual difficulties for the voter. Some of these requirements are designed to assist voters with poor reading vision. These are voters who might have some difficulty in reading normal text, but are not typically classified as having a visual disability and thus might not be inclined to use the Accessible Voting Station.

3.2.5-A Screen flicker

No voting system display screen SHALL flicker with a frequency between 2 Hz and 55 Hz.

Applies To: VEBD-V

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

Aside from usability concerns, this requirement protects voters with epilepsy.

3.2.5-B Resetting of adjustable aspects at end of session

Any aspect of the voting station that is adjustable by the voter or poll worker, including font size, color, contrast, audio volume, or rate of speech, SHALL automatically reset to a standard default value upon completion of that voter's session. For the Acc-VS, the aspects include synchronized audio/video mode and non-manual input mode.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This ensures that the voting station presents the same initial appearance to every voter.

3.2.5-C Ability to reset to default values

If any aspect of a voting system is adjustable by the voter or poll worker, there SHALL be a mechanism to reset all such aspects to their default values.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The purpose is to allow a voter or poll worker who has adjusted the system into an undesirable state to reset all the aspects and begin again.

3.2.5-D Minimum font size

Voting systems SHALL provide a minimum font size of 3.0mm (measured as the height of a capital letter) for all text intended for voters or poll workers.

Applies To: Voting device

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.5-E Available font sizes

A voting station that uses an electronic image display SHALL be capable of showing all information in at least two font sizes, (a) 3.0-4.0 mm and (b) 6.3-9.0 mm, under control of the voter. The system SHALL allow the voter to adjust font size throughout the voting session while preserving the current votes.

Applies To: VEBD-V

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

While larger font sizes may assist most voters with poor vision, certain disabilities such as tunnel vision are best addressed by smaller font sizes. Larger font sizes may also assist voters with cognitive disabilities. This requirement mandates the availability of at least two font sizes, but additional choices (including continuous variability) are allowed.

3.2.5-F Use of sans serif font

Text intended for the voter SHOULD be presented in a sans serif font.

Applies To: Voting device

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Research has shown that users prefer such fonts.

3.2.5-G Legibility of paper ballots and verification records

Voting systems using paper ballots or paper verification records SHALL provide features that assist in the reading of such ballots and records by voters with poor reading vision.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

While this requirement may be satisfied by one of its sub-requirements, other innovative solutions are not precluded.

3.2.5-G.1 Legibility via font size

The system MAY achieve legibility of paper records by supporting the printing of those records in at least two font sizes, 3.0 - 4.0mm and 6.3 - 9.0mm.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Although the system may be capable of printing in several font sizes, the use of various font sizes in an actual election may be governed by local or state laws and regulations.

3.2.5-G.2 Legibility via magnification

The system MAY achieve legibility of paper records by supporting magnification of those records. This magnification MAY be done by optical or electronic devices. The manufacturer MAY either: 1) provide the magnifier itself as part of the system, or 2) provide the make and model number of readily available magnifiers that are compatible with the system.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The magnifier(s) either provided or cited must, of course, provide legibility for the paper as actually presented on the system. For instance, if the paper record is under a transparent cover to prevent the voter from touching it, the means of magnification must be compatible with this configuration.

3.2.5-H Contrast Ratio

The minimum figure-to-ground ambient contrast ratio for all text and informational graphics (including icons) intended for voters or poll workers SHALL be 3:1.

Applies To: Voting device

Test Reference: Part 3: 3.1 "Inspection"

3.2.5-I High contrast for electronic displays

The voting station SHALL be capable of showing all information in high contrast either by default or under the control of the voter. The system SHALL allow the voter to adjust contrast throughout the voting session while preserving the current votes. High contrast is a figure-to-ground ambient contrast ratio for text and informational graphics of at least 6:1.

Applies To: VEBD-V

Test Reference: Part 3: 3.1 "Inspection"

3.2.5-J Accommodation for color blindness

The default color coding SHALL support correct perception by voters with color blindness.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

There are many types of color blindness and no color coding can, by itself, guarantee correct perception for everyone. However, designers should take into account such factors as: red-green color blindness is the most common form; high luminosity contrast will help colorblind voters to recognize visual features; and color-coded graphics can also use shape to improve the ability to distinguish certain features.

3.2.5-K No reliance solely on color

Color coding SHALL NOT be used as the sole means of conveying information, indicating an action, prompting a response, or distinguishing a visual element.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

While color can be used for emphasis, some other non-color mode must also be used to convey the information, such as a shape or text style. For example, red can be enclosed in an octagon shape.

3.2.6 Interaction issues

The requirements of this section are designed to minimize interaction difficulties for the voter.

3.2.6-A No page scrolling

Voting systems SHALL NOT require page scrolling by the voter.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

That is, the page of displayed information must fit completely within the physical screen presenting it. Scrolling is not an intuitive operation for those unfamiliar with the use of computers. Even those experienced with computers often do not notice a scroll bar and miss information at the bottom of the "page." Voting systems may require voters to move to the next or previous "page."

3.2.6-B Unambiguous feedback for voter's selection

The voting system SHALL provide unambiguous feedback regarding the voter’s selection, such as displaying a checkmark beside the selected option or conspicuously changing its appearance.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.6-C Accidental Activation

Input mechanisms SHALL be designed to minimize accidental activation.

Applies To: Voting device

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

There are at least two kinds of accidental activation. One is when a control is activated as it is being "explored" by the voter because the control is overly sensitive to the touch. A second issue is the problem of having a control in a location where it can easily be activated unintentionally. An example would be a button in the very bottom left corner of the screen where a voter might hold the unit for support.

3.2.6-C.1 Size and separation of touch areas

On touch screens, the sensitive touch areas SHALL have a minimum height of 0.5 inches and minimum width of 0.7 inches. The vertical distance between the centers of adjacent areas SHALL be at least 0.6 inches, and the horizontal distance at least 0.8 inches.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.6-C.2 No repeating keys

No key or control on a voting system SHALL have a repetitive effect as a result of being held in its active position.

Applies To: Voting device

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This is to preclude accidental activation. For instance, if a voter is typing in the name of a write-in candidate, depressing and holding the "e" key results in only a single "e" added to the name.

3.2.6.1 Timing issues

These requirements address how long the system and voter wait for each other to interact. This section uses the following terms (also defined in Appendix A: Definitions of Words with Special Meanings):

  • Initial system response time: the time taken from when the voter performs some detectible action (such as pressing a button) to when the voting system begins responding in some obvious way (such as an audible response or any change on the screen).
  • Completed system response time: the time taken from when the voter performs some detectible action to when the voting system completes its response and settles into a stable state (e.g., finishes "painting" the screen with a new page).
  • Voter inactivity time: the amount of time from when the system completes its response until there is detectible voter activity. In particular, note that audio prompts from the system may take several minutes and that this time does not count as voter inactivity.
  • Alert time: the amount of time the equipment will wait for detectible voter activity after issuing an alert before going into an inactive state requiring poll worker intervention.
3.2.6.1-A Maximum initial system response time

The initial system response time of the voting system SHALL be no greater than 0.5 seconds.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This is so the voter can very quickly perceive that an action has been detected by the system and is being processed. The voter never gets the sense of dealing with an unresponsive or "dead" system. Note that this requirement applies to VEBD-A (audio) as well as to VEBD-V (visual) systems.

3.2.6.1-B Maximum completed system response time for vote confirmation

When the voter performs an action to record a single vote, the completed system response time of the voting system SHALL be no greater than one second in the case of a visual response, and no greater than five seconds in the case of an audio response.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, if the voter touches a button to indicate a vote for a candidate, a visual system might display an "X" next to the candidate's name, and an audio system might announce, "You have voted for Smith for Governor".

3.2.6.1-C Maximum completed system response time for all operations

The completed system response time of the voting system for visual operations SHALL be no greater than 10 seconds.

Applies To: VEBD-V

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Even for "large" operations such as initializing the ballot or painting a new screen, the system must never take more than 10 seconds. In the case of audio systems, no upper limit is specified, since certain operations may take longer, depending on the length of the text being read (e.g., reading out a long list of candidates running in a contest).

3.2.6.1-D System response indicator

If the system has not completed its visual response within one second, it SHALL present to the voter, within 0.5 seconds of the voter's action, some indication that it is preparing its response.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, the system might present an hourglass icon indicating that it is "busy" processing the voter's request. This requirement is intended to preclude the "frozen screen" effect, in which no detectible activity is taking place for several seconds. There need not be a specific "activity" icon, as long as some visual change is apparent (such as progressively "painting" a new screen).

3.2.6.1-E Voter inactivity time

The voting system SHALL detect and warn about lengthy voter inactivity during a voting session. Each system SHALL have a defined and documented voter inactivity time, and that time SHALL be between two and five minutes.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Each type of system must have a given inactivity time that is consistent among and within all voting sessions. This ensures that all voters are treated equitably.

3.2.6.1-F Alert time

Upon expiration of the voter inactivity time, the voting system SHALL issue an alert and provide a means by which the voter may receive additional time. The alert time SHALL be between 20 and 45 seconds. If the voter does not respond to the alert within the alert time, the system SHALL go into an inactive state requiring poll worker intervention.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.7 Alternative languages

HAVA Section 301 (a)(4) states that the voting system shall provide alternative language accessibility pursuant to the requirements of Section 203 of the Voting Rights Act of 1965 (42 U.S.C. 1973aa-1a). Ideally every voter would be able to vote independently and privately, regardless of language. As a practical matter, alternative language access is mandated under the Voting Rights Act of 1975, subject to certain thresholds (e.g., if the language group exceeds 5% of the voting age population). Thus, election officials must ensure that the voting system they deploy is capable of handling the languages meeting the legal threshold within their districts.

While the following requirements support this process, it should be noted that they are requirements only for voting systems to be certified. It is anticipated that jurisdictions will apply additional requirements appropriate for their particular circumstances for procurement and deployment.

3.2.7-A General support for alternative languages

The voting system SHALL be capable of presenting the ballot, contest choices, review screens, vote verification records, and voting instructions in any language declared by the manufacturer to be supported by the system.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, if the manufacturer claims that a given system is capable of supporting Spanish and Chinese, then it must do so.

3.2.7-A.1 Voter control of language

The system SHALL allow the voter to select among the available languages throughout the voting session while preserving the current votes.

Applies To: VEBD

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, a voter may initially choose an English version of the ballot, but then wish to switch to another language in order to read a referendum question.

3.2.7-A.2 Complete information in alternative language

Information presented to the voter in the typical case of English-literate voters (including instructions, warnings, messages, contest choices, and vote verification information) SHALL also be presented when an alternative language is being used, whether the language is written or spoken.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Therefore, it may not be sufficient simply to present the ballot per se in the alternative language, especially in the case of VEBD systems. All the supporting information must also be available in the alternative language.

3.2.7-A.3 Auditability of records for English readers

Any records, including paper ballots and paper verification records, SHALL have sufficient information to support auditing by poll workers and others who can read only English.SHALL

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Even though the system must be easily available to voters without a command of English, any persistent records of the vote must also be fully available to English-only readers for auditing purposes. In the case of paper, this does not imply a fully bi-lingual ballot. For instance, the full text of a referendum question might appear only in the alternative language, but the content of the vote (e.g., "yes" on ballot question 106) needs to be readable by English-only readers.

3.2.7-A.4 Usability testing by manufacturer for alternative languages

The manufacturer SHALL conduct summative usability tests for each of the system's supported languages, using subjects who are fluent in those languages but not fluent in English and SHALL report the test results, using the Common Industry Format, as part of the TDP.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

3.2.8 Usability for poll workers

Voting systems are used not only by voters to record their votes, but also by poll workers who are responsible for set-up, operation while polls are open, light maintenance, and poll closing. Because of the wide variety of implementations, it is impossible to specify detailed design requirements for these functions. The requirements below describe general capabilities that all systems must support. Also, note that Maintainability of the voting system is covered in Part 1: 6.4.5 "Maintainability".

3.2.8-A Clarity of system messages for poll workers

Messages generated by the system for poll workers in support of the operation, maintenance, or safety of the system SHALL adhere to the requirements for clarity in Part 1: 3.2.4 "Cognitive issues".

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

3.2.8.1 Operation

Poll workers are responsible for opening polls, keeping the polls open and running smoothly during voting hours, and closing the polls afterwards. Operations may be categorized in three phases:

Setup includes all the steps necessary to take the system from its state as normally delivered to the polling place, to the state in which it is ready to record votes. It does not include ballot definition.

Polling includes such functions as:

  • voter identification and authorization;
  • preparing the system for the next voter;
  • assistance to voters who wish to change their ballots or need other help;
  • system recovery in the case of voters who abandon the voting session without having cast a ballot; and
  • routine hardware operations, such as installing a new roll of paper.

Shutdown includes all the steps necessary to take the system from the state in which it is ready to record votes to its normal completed state in which it has captured all the votes cast and the voting information cannot be further altered.

3.2.8.1-A Ease of normal operation

The procedures for system setup, polling, and shutdown, as documented by the manufacturer, SHALL be reasonably easy for the typical poll worker to learn, understand, and perform.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement covers procedures and operations for those aspects of system operation normally performed by poll workers and other "non-expert" operators. It does not address inherently complex operations such as ballot definition or system repair. While a certain amount of complexity is unavoidable, these "normal" procedures should not require any special expertise. The procedures may require a reasonable amount of training.

3.2.8.1-B Usability testing by manufacturer for poll workers

The manufacturer SHALL conduct summative usability tests on the voting system using individuals who are representative of the general population and SHALL report the test results, using the Common Industry Format, as part of the TDP. The tasks to be covered in the test SHALL include setup, operation, and shutdown.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

3.2.8.1-C Documentation usability

The system SHALL include clear, complete, and detailed instructions and messages for setup, polling, and shutdown.

Applies To: Voting system

DISCUSSION

This requirement covers documentation for those aspects of system operation normally performed by poll workers and other "non-expert" operators. It does not address inherently complex operations such as ballot definition. The instructions would usually be in the form of a written manual, but could also be presented on other media, such as a DVD or videotape. In the context of this requirement, "message" means information delivered by the system to the poll worker as he or she attempts to perform a setup, polling, or shutdown operation.

Source: New requirement

3.2.8.1-C.1 Poll Workers as target audience

The documentation required for normal system operation SHALL be presented at a level appropriate for non-expert poll workers.

Applies To: Voting system

DISCUSSION

For instance, the documentation should not presuppose familiarity with personal computers.

Source: New requirement

3.2.8.1-C.2 Usability at the polling place

The documentation SHALL be in a format suitable for practical use in the polling place.

Applies To: Voting system

DISCUSSION

For instance, a single large reference manual that simply presents details of all possible operations would be difficult to use, unless accompanied by aids such as a simple "how-to" guide.

Source: New requirement

3.2.8.1-C.3 Enabling verification of correct operation

The instructions and messages SHALL enable the poll worker to verify that the system

  1. Has been set up correctly (setup);
  2. Is in correct working order to record votes (polling); and
  3. Has been shut down correctly (shutdown).

Applies To: Voting system

DISCUSSION

The poll worker should not have to guess whether an operation has been performed correctly. The documentation should make it clear what the system "looks like" when correctly configured.

Source: New requirement

3.2.8.2 Safety

All voting systems and their components must be designed so as to eliminate hazards to personnel or to the equipment itself. Hazards include, but are not limited to:

  • fire hazards;
  • electrical hazards;
  • potential for equipment tip-over (stability);
  • potential for cuts and scrapes (e.g., sharp edges);
  • potential for pinching (e.g., tight, spring-loaded closures); and
  • potential for hair or clothing entanglement.
3.2.8.2-A Safety certification

Equipment associated with the voting system SHALL be certified in accordance with the requirements of UL 60950-1, Information Technology Equipment – Safety – Part 1 [UL05] by a certification organization accredited by the Department of Labor, Occupational Safety and Health Administration’s Nationally Recognized Testing Laboratory program. The certification organization’s scope of accreditation SHALL include UL 60950-1.

Applies To: Voting system

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

UL 60950 is a comprehensive standard for IT equipment and addresses all the hazards discussed above under Safety.

3.3 Accessibility requirements

HAVA Section 301 (a) (3) [HAVA02] reads, in part:


ACCESSIBILITY FOR INDIVIDUALS WITH DISABILITIES.--The voting system shall--

(A) be accessible for individuals with disabilities, including nonvisual accessibility for the blind and visually impaired, in a manner that provides the same opportunity for access and participation (including privacy and independence) as for other voters;

(B) satisfy the requirement of subparagraph (A) through the use of at least one direct recording electronic voting system or other voting system equipped for individuals with disabilities at each polling place;


The voting process is to be accessible to voters with disabilities through the use of a specially equipped voting station. A machine so equipped is referred to herein as an Accessible Voting Station (Acc-VS).

The requirements in this section are intended to address this HAVA mandate. Ideally, every voter would be able to vote independently and privately. As a practical matter, there may be some number of voters who, because of the nature of their disabilities, will need personal assistance with any system. Nonetheless, these requirements are meant to make the voting system independently accessible to as many voters as possible.

This section is organized according to the type of disability being addressed. For each type, certain appropriate design features are specified. Note, however, that a feature intended primarily to address one kind of disability may very well assist voters with other kinds.

There are many other requirements that apply to the Acc-VS besides those in this section. Please see Part 1: 3.1.3 "Interaction of usability and accessibility requirements" for a full explanation.

3.3.1 General

The requirements of this section are relevant to a wide variety of disabilities.

3.3.1-A Accessibility throughout the voting session

The Acc-VS SHALL be integrated into the manufacturer’s complete voting system so as to support accessibility for disabled voters throughout the voting session.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement ensures accessibility to the voter throughout the entire session. Not only must individual system components (such as ballot markers, paper records, and optical scanners) be accessible, but also they must work together to support this result.

3.3.1-A.1 Documentation of Accessibility Procedures

The manufacturer SHALL supply documentation describing 1) recommended procedures that fully implement accessibility for voters with disabilities and 2) how the Acc-VS supports those procedures.

Applies To: Acc-VS

DISCUSSION

The purpose of this requirement is for the manufacturer not simply to deliver system components, but also to describe the accessibility scenarios they are intended to support.

3.3.1-B Complete information in alternative formats

When the provision of accessibility involves an alternative format for ballot presentation, then all information presented to non-disabled voters, including instructions, warnings, error and other messages, and contest choices, SHALL be presented in that alternative format.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

3.3.1-C No dependence on personal assistive technology

The support provided to voters with disabilities SHALL be intrinsic to the Accessible Voting Station. It SHALL NOT be necessary for the Accessible Voting Station to be connected to any personal assistive device of the voter in order for the voter to operate it correctly.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement does not preclude the Accessible Voting Station from providing interfaces to assistive technology. (See definition of "personal assistive devices" in Appendix A..) Its purpose is to assure that disabled voters are not required to bring special devices with them in order to vote successfully. The requirement does not assert that the Accessible Voting Station will eliminate the need for a voter’s ordinary non-interfacing devices, such as eyeglasses or canes.

3.3.1-D Secondary means of voter identification

If a voting system provides for voter identification or authentication by using biometric measures that require a voter to possess particular biological characteristics, then the system SHALL provide a secondary means that does not depend on those characteristics.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, if fingerprints are used for voter identification, another mechanism must be provided for voters without usable fingerprints.

3.3.1-E Accessibility of paper-based vote verification

If the Acc-VS generates a paper record (or some other durable, human-readable record) for the purpose of allowing voters to verify their votes, then the system SHALL provide a means to ensure that the verification record is accessible to all voters with disabilities, as identified in Part 1: 3.3 "Accessibility requirements".

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

While paper records generally provide a simple and effective means for technology-independent vote verification, their use can present difficulties for voters with certain types of disabilities. The purpose of this requirement is to ensure that all voters have a similar opportunity for vote verification. Note that this requirement addresses the special difficulties that may arise with the use of paper. Verification is part of the voting process, and all the other general requirements apply to verification, in particular those dealing with dexterity (e.g. 3.3.4-C "Ballot Submission and Vote Verification"), blindness (e.g. 3.3.3-E "Ballot Submission and Vote Verification"), and poor vision issues (e.g. 3.2.5-G "Legibility of Paper Ballots and Verification Records").

3.3.1-E.1 Audio readback for paper-based vote verification.

If the Acc-VS generates a paper record (or some other durable, human-readable record) for the purpose of allowing voters to verify their votes, then the system SHALL provide a mechanism that can read that record and generate an audio representation of its contents.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Sighted voters can directly verify the contents of a paper record. The purpose of this requirement is to allow voters with visual disabilities to verify, even if indirectly, the contents of the record. It is recognized that the verification depends on the integrity of the mechanism that reads the record to the voter. The audio must be generated via the paper record and therefore not depend on any electronic or other "internal" record of the ballot. Note that the paper record and its audio representation may be rendered in an alternative language. See also Requirements Part 1: 4.2.4-A, B.

3.3.2 Low vision

These requirements specify the features of the accessible voting station designed to assist voters with low vision.

Low (or partial) vision includes dimness of vision, haziness, film over the eye, foggy vision, extreme near-sightedness or far-sightedness, distortion of vision, color distortion or blindness, visual field defects, spots before the eyes, tunnel vision, lack of peripheral vision, abnormal sensitivity to light or glare and night blindness. For the purposes of this discussion low vision is defined as having a visual acuity worse than 20/70.

People with tunnel vision can see only a small part of the ballot at one time. For these users it is helpful to have letters at the lower end of the font size range in order to allow them to see more letters at the same time. Thus, there is a need to provide font sizes at both ends of the range.

People with low vision or color blindness benefit from high contrast and from a selection of color combinations appropriate for their needs. Between 7% and 10% of all men have color vision deficiencies. Certain color combinations in particular cause problems. Therefore, use of color combinations with good contrast is required. Note also the general Requirement Part 1: 3.2.5-J.

However, some users are very sensitive to very bright displays and cannot use them for long. An overly bright background causes a visual white-out that makes these users unable to distinguish individual letters. Thus, use of non-saturated color options is an advantage for some people.

It is important to note that some of the requirements in Part 1: 3.2.5 "Perceptual issues" also provide support for voters with certain kinds of vision problems.

3.3.2-A Usability testing by manufacturer for voters with low vision

The manufacturer SHALL conduct summative usability tests on the voting system using individuals with low vision and SHALL report the test results, using the Common Industry Format, as part of the TDP.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.2-B Adjustable saturation for color displays

An Accessible Voting Station with a color electronic image display SHALL allow the voter to adjust the color saturation throughout the voting session while preserving the current votes. At least two options SHALL be available: a high and a low saturation presentation.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

It is not required that the station offer a continuous range of color saturation. "High saturation" refers to bright, vibrant colors. "Low saturation" refers to muted (or grayish) colors.

3.3.2-C Distinctive buttons and controls

Buttons and controls on Accessible Voting Stations SHALL be distinguishable by both shape and color. This applies to buttons and controls implemented either "on-screen" or in hardware. This requirement does not apply to sizeable groups of keys, such as a conventional 4x3 telephone keypad or a full alphabetic keyboard.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

The redundant cues assist those with low vision. They also help individuals who may have difficulty reading the text on the screen.

3.3.2-D Synchronized audio and video

The voting station SHALL provide synchronized audio output to convey the same information as that which is displayed on the screen. There SHALL be a means by which the voter can disable either the audio or the video output, resulting in a video-only or audio-only presentation, respectively. The system SHALL allow the voter to switch among the three modes (synchronized audio/video, video-only, or audio-only) throughout the voting session while preserving the current votes.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This feature may also assist voters with cognitive disabilities.

3.3.3 Blindness

These requirements specify the features of the Accessible Voting Station designed to assist voters who are blind.

3.3.3-A Usability testing by manufacturer for blind voters

The manufacturer SHALL conduct summative usability tests on the voting system using individuals who are blind and SHALL report the test results, using the Common Industry Format, as part of the TDP.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.3-B Audio-tactile interface

The Accessible Voting Station SHALL provide an Audio-Tactile Interface (ATI) that supports the full functionality of the visual ballot interface, as specified in Part 1: 6.2 "Voting Variations".

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Note the necessity of both audio output and tactilely discernible controls for voter input. Full functionality includes at least:

  1. Instructions and feedback on initial activation of the ballot (such as insertion of a smart card), if applicable;
  2. Instructions and feedback to the voter on how to operate the Accessible Voting Station, including settings and options (e.g., volume control, repetition);
  3. Instructions and feedback for navigation of the ballot;
  4. Instructions and feedback for contest choices, including write-in candidates;
  5. Instructions and feedback on confirming and changing votes; and
  6. Instructions and feedback on final submission of ballot.
3.3.3-B.1 Equivalent functionality of ATI

The ATI of the Accessible Voting Station SHALL provide the same capabilities to vote and cast a ballot as are provided by its visual interface.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, if a visual ballot supports voting a straight party ticket and then changing the vote for a single contest, so must the ATI.

3.3.3-B.2 ATI supports repetition

The ATI SHALL allow the voter to have any information provided by the voting system repeated.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This feature may also be useful to voters with cognitive disabilities.

3.3.3-B.3 ATI supports pause and resume

The ATI SHALL allow the voter to pause and resume the audio presentation.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This feature may also be useful to voters with cognitive disabilities.

3.3.3-B.4 ATI supports transition to next or previous contest

The ATI SHALL allow the voter to skip to the next contest or return to previous contests.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This is analogous to the ability of sighted voters to move on to the next contest once they have made a selection or to abstain from voting on a contest altogether.

3.3.3-B.5 ATI can skip referendum wording

The ATI SHALL allow the voter to skip over the reading of a referendum so as to be able to vote on it immediately.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This is analogous to the ability of sighted voters to skip over the wording of a referendum on which they have already made a decision prior to the voting session (e.g., "Vote yes on proposition #123").

3.3.3-C Audio features and characteristics

Voting stations that provide audio presentation of the ballot SHALL do so in a usable way, as detailed in the following sub-requirements.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

These requirements apply to all voting system audio output, not just to the ATI of an Accessible Voting Station.

3.3.3-C.1 Standard connector

The ATI SHALL provide its audio signal through an industry standard connector for private listening using a 3.5mm stereo headphone jack to allow voters to use their own audio assistive devices.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

3.3.3-C.2 T-Coil coupling

When a voting system utilizes a telephone style handset or headphone to provide audio information, it SHALL provide a wireless T-Coil coupling for assistive hearing devices so as to provide access to that information for voters with partial hearing. That coupling SHALL achieve at least a category T4 rating as defined by [ANSI01] American National Standard for Methods of Measurement of Compatibility between Wireless Communications Devices and Hearing Aids, ANSI C63.19.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Note that Requirement Part 1: 3.3.6-C protects the use of hearing devices.

3.3.3-C.3 Sanitized headphone or handset

A sanitized headphone or handset SHALL be made available to each voter.

Applies To: VEBD-A

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

This requirement can be achieved in various ways, including the use of "throwaway" headphones, or of sanitary coverings.

3.3.3-C.4 Initial volume

The voting system SHALL set the initial volume for each voting session between 40 and 50 dB SPL.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

A voter does not "inherit" the volume as set by the previous user of the voting station. See Requirement Part 1: 3.2.5-B.

3.3.3-C.5 Range of volume

The audio system SHALL allow the voter to control the volume throughout the voting session while preserving the current votes. The volume SHALL be adjustable from a minimum of 20dB SPL up to a maximum of 100 dB SPL, in increments no greater than 10 dB.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

3.3.3-C.6 Range of frequency

The audio system SHALL be able to reproduce frequencies over the audible speech range of 315 Hz to 10 KHz.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

The required frequencies include the range of normal human speech. This allows the reproduced speech to sound natural.

3.3.3-C.7 Intelligible audio

The audio presentation of verbal information SHOULD be readily comprehensible by voters who have normal hearing and are proficient in the language. This includes such characteristics as proper enunciation, normal intonation, appropriate rate of speech, and low background noise. Candidate names SHOULD be pronounced as the candidate intends.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement covers both recorded and synthetic speech. It applies to those aspects of the audio content that are inherent to the voting system or that are generated by default. To the extent that the audio presentation is determined by election officials designing the ballot, it is beyond of the scope of this requirement.

3.3.3-C.8 Control of speed

The audio system SHALL allow the voter to control the rate of speech throughout the voting session while preserving the current votes. The range of speeds supported SHALL include 75% to 200% of the nominal rate.

Applies To: VEBD-A

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Many blind voters are accustomed to interacting with accelerated speech. This feature may also be useful to voters with cognitive disabilities.

3.3.3-D Ballot activation

If the voting station supports ballot activation for non-blind voters, then it SHALL also provide features that enable voters who are blind to perform this activation.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, smart cards might provide tactile cues so as to allow correct insertion.

3.3.3-E Ballot submission and vote verification

If the voting station supports ballot submission or vote verification for non-blind voters, then it SHALL also provide features that enable voters who are blind to perform these actions.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, if voters using this station normally perform paper-based verification, or if they feed their own optical scan ballots into a reader, blind voters must also be able to do so.

3.3.3-F Tactile discernability of controls

Mechanically operated controls or keys on an Accessible Voting Station SHALL be tactilely discernible without activating those controls or keys.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Note also the more general Requirement Part 1: 3.2.5-C against accidental activation of controls.

3.3.3-G Discernability of key status

The status of all locking or toggle controls or keys (such as the "shift" key) SHALL be visually discernible, and also discernible through either touch or sound.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

3.3.4 Dexterity

These requirements specify the features of the Accessible Voting Station designed to assist voters who lack fine motor control or use of their hands.

3.3.4-A Usability testing by manufacturer for voters with dexterity disabilities

The manufacturer SHALL conduct summative usability tests on the voting system using individuals lacking fine motor control and SHALL report the test results, using the Common Industry Format, as part of the TDP.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.4-B Support for non-manual input

The Accessible Voting Station SHALL provide a mechanism to enable non-manual input that is functionally equivalent to tactile input. All the functionality of the Accessible Voting Station (e.g., straight party voting, write-in candidates) that is available through the conventional forms of input, such as tactile, SHALL also be available through the non-manual input mechanism.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement ensures that the Accessible Voting Station is operable by individuals who do not have the use of their hands. Examples of non-manual controls include mouth sticks and "sip and puff" switches. While it is desirable that the voter be able to independently initiate use of the non-manual input mechanism, this requirement guarantees only that the voter can vote independently once the mechanism is enabled.

3.3.4-C Ballot submission and vote verification

If the voting station supports ballot submission or vote verification for non-disabled voters, then it SHALL also provide features that enable voters who lack fine motor control or the use of their hands to perform these actions.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For example, if voters using this station normally perform paper-based verification, or if they feed their own optical scan ballots into a reader, voters with dexterity disabilities must also be able to do so. Note that the general requirement for privacy when voting (Requirement Part1: 3.2.3.1-A) still applies.

3.3.4-D Manipulability of controls

Keys and controls on the Accessible Voting Station SHALL be operable with one hand and SHALL NOT require tight grasping, pinching, or twisting of the wrist. The force required to activate controls and keys SHALL be no greater 5 lbs. (22.2 N).

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Controls are to be operable without excessive force.

3.3.4-E No dependence on direct bodily contact

The Accessible Voting Station controls SHALL NOT require direct bodily contact or for the body to be part of any electrical circuit.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This requirement ensures that controls are operable by individuals using prosthetic devices.

3.3.5 Mobility

These requirements specify the features of the Accessible Voting Station designed to assist voters who use mobility aids, including wheelchairs. Many of the requirements of this section are based on the ADA Accessibility Guidelines for Buildings and Facilities (ADAAG).

3.3.5-A Clear floor space

The Accessible Voting Station SHALL provide a clear floor space of 30 inches (760 mm) minimum by 48 inches (1220 mm) minimum for a stationary mobility aid. The clear floor space SHALL be level with no slope exceeding 1:48 and positioned for a forward approach or a parallel approach.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5-B Allowance for assistant

When deployed according to the installation instructions provided by the manufacturer, the voting station SHALL allow adequate room for an assistant to the voter. This includes clearance for entry to and exit from the area of the voting station.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Disabled voters sometimes prefer to have an assistant help them vote. The setup of the voting station should not preclude this.

3.3.5-C Visibility of displays and controls

Labels, displays, controls, keys, audio jacks, and any other part of the Accessible Voting Station necessary for the voter to operate the voting system SHALL be easily legible and visible to a voter in a wheelchair with normal eyesight (no worse than 20/40, corrected) who is in an appropriate position and orientation with respect to the Accessible Voting Station.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

There are a number of factors that could make relevant parts of the Accessible Voting Station difficult to see, such as: small lettering; controls and labels tilted at an awkward angle from the voter's viewpoint; and glare from overhead lighting.

3.3.5.1 Controls within reach

The requirements of this section ensure that the controls, keys, audio jacks and any other part of the Accessible Voting Station necessary for its operation are within easy reach. Note that these requirements have meaningful application mainly to controls in a fixed location. A hand-held tethered control panel is another acceptable way of providing reachable controls.

3.3.5.1-A Forward approach, no obstruction

If the Accessible Voting Station has a forward approach with no forward reach obstruction then the high reach SHALL be 48 inches maximum and the low reach SHALL be 15 inches minimum. See Part 1: Figure 3-1.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-B Forward approach, with obstruction

If the Accessible Voting Station has a forward approach with a forward reach obstruction, the following sub-requirements SHALL apply (See Part 1: Figure 3-2).

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-B.1 Maximum size of obstruction

The forward obstruction SHALL be no greater than 25 inches in depth, its top no higher than 34 inches and its bottom surface no lower than 27 inches.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-B.2 Maximum high reach over obstruction

If the obstruction is no more than 20 inches in depth, then the maximum high reach SHALL be 48 inches, otherwise it SHALL be 44 inches.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-B.3 Toe clearance under obstruction

Space under the obstruction between the finish floor or ground and 9 inches (230 mm) above the finish floor or ground SHALL be considered toe clearance and SHALL comply with the following provisions:

  1. Toe clearance depth SHALL extend 25 inches (635 mm) maximum under the obstruction;
  2. The minimum toe clearance depth under the obstruction SHALL be either 17 inches (430 mm) or the depth required to reach over the obstruction to operate the Accessible Voting Station, whichever is greater; and
  3. Toe clearance width SHALL be 30 inches (760 mm) minimum.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-B.4 Knee clearance under obstruction

Space under the obstruction between 9 inches (230 mm) and 27 inches (685 mm) above the finish floor or ground SHALL be considered knee clearance and SHALL comply with the following provisions:

  1. Knee clearance depth SHALL extend 25 inches (635 mm) maximum under the obstruction at 9 inches (230 mm) above the finish floor or ground;
  2. The minimum knee clearance depth at 9 inches (230 mm) above the finish floor or ground SHALL be either 11 inches (280 mm) or 6 inches less than the toe clearance, whichever is greater;
  3. Between 9 inches (230 mm) and 27 inches (685 mm) above the finish floor or ground, the knee clearance depth SHALL be permitted to reduce at a rate of 1 inch (25 mm) in depth for each 6 inches (150 mm) in height. (It follows that the minimum knee clearance at 27 inches above the finish floor or ground SHALL be 3 inches less than the minimum knee clearance at 9 inches above the floor.); and
  4. Knee clearance width SHALL be 30 inches (760 mm) minimum.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-C Parallel approach, no obstruction

If the Accessible Voting Station has a parallel approach with no side reach obstruction then the maximum high reach SHALL be 48 inches and the minimum low reach SHALL be 15 inches. See Part 1: Figure 3-3.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-D Parallel approach, with obstruction

If the Accessible Voting Station has a parallel approach with a side reach obstruction, the following sub-requirements SHALL apply. See Part 1: Figure 3-4.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

DISCUSSION

Since this is a parallel approach, no clearance under the obstruction is required.

3.3.5.1-D.1 Maximum size of obstruction

The side obstruction SHALL be no greater than 24 inches in depth and its top no higher than 34 inches.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

3.3.5.1-D.2 Maximum high reach over obstruction

If the obstruction is no more than 10 inches in depth, then the maximum high reach SHALL be 48 inches, otherwise it SHALL be 46 inches.

Applies To: Acc-VS

Test Reference: Part 3: 3.1 "Inspection"

Table 3-1 Unobstructed reach measurements

 

image042

image044

Figure 3-1 Unobstructed forward reach

Figure 3-2 Obstructed forward reach

(a) for an obstruction depth of up to 20 inches (508 mm)
(b) for an obstruction depth of up to 25 inches (635 mm)

image046

image048

Figure 3-3 Unobstructed side reach with an allowable obstruction less than 10 inches (254 mm) deep

Figure 3-4 Obstructed side reach

(a) for an obstruction depth of up to 10 inches (254 mm)
(b) for an obstruction depth of up to 24 inches (610 mm)

3.3.6 Hearing

These requirements specify the features of the Accessible Voting Station designed to assist voters with hearing disabilities.

3.3.6-A Reference to audio requirements

The Accessible Voting Station SHALL incorporate the features listed under Requirement Part 1: 3.3.3-C for voting equipment that provides audio presentation of the ballot.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Note especially the requirements for volume initialization and control.

3.3.6-B Visual redundancy for sound cues

If the voting system provides sound cues as a method to alert the voter, the tone SHALL be accompanied by a visual cue, unless the station is in audio-only mode.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

For instance, the voting equipment might beep if the voter attempts to overvote. If so, there would have to be an equivalent visual cue, such as the appearance of an icon, or a blinking element. If the voting system has been set to audio-only mode, there would be no visual cue.

3.3.6-C No electromagnetic interference with hearing devices

No voting equipment SHALL cause electromagnetic interference with assistive hearing devices that would substantially degrade the performance of those devices. The voting equipment, considered as a wireless device, SHALL achieve at least a category T4 rating as defined by [ANSI01] American National Standard for Methods of Measurement of Compatibility between Wireless Communications Devices and Hearing Aids, ANSI C63.19.

Applies To: Voting device

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

"Hearing devices" include hearing aids and cochlear implants.

3.3.7 Cognition

These requirements specify the features of the Accessible Voting Station designed to assist voters with cognitive disabilities.

3.3.7-A General support for cognitive disabilities

The Accessible Voting Station SHOULD provide support to voters with cognitive disabilities.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

Because of the highly varied nature of disabilities falling within the "cognitive" category, there are no design features uniquely aimed at helping those with such disabilities. However, many of the features designed primarily for other disabilities and for general usability are also highly relevant to these voters:

  1. the synchronization of audio with the displayed screen information (Requirement Part 1: 3.3.2-D);
  2. the general cognitive usability requirements ( Part 1: 3.2.4) and, in particular, the use of plain language (Requirement Part 1: 3.2.4-C);
  3. large font sizes and legibility of paper (Requirement Part 1: 3.2.5-E and Part 1: 3.2.5-G); and
  4. the ability to control various aspects of the audio presentation (Requirement Part 1: 3.3.3-B and Part 1: 3.3.3-C) such as pausing, repetition, and speed.

3.3.8 English proficiency

These requirements specify the features of the Accessible Voting Station designed to assist voters who lack proficiency in reading English.

3.3.8-A Use of ATI

For voters who lack proficiency in reading English, the voting equipment SHALL provide an audio interface for instructions and ballots as described in Part 1: 3.3.3-B.

Applies To: Acc-VS

Test Reference: Part 3: 3.2 "Functional Testing"

3.3.9 Speech

3.3.9-A Speech not to be required by equipment

No voting equipment SHALL require voter speech for its operation.

Applies To: Voting system

Test Reference: Part 3: 3.2 "Functional Testing"

DISCUSSION

This does not preclude voting equipment from offering speech input as an option, but speech must not be the only means of input.