United States Election Assistance Comittee

Register to Vote!

Use the National Mail Voter Registration Form to register to vote, update your registration information with a new name or address, or register with a political party.

Note: If you wish to vote absentee and are a uniformed service member or family member or a citizen living outside the U.S., contact the Federal Voting Assistance Program to register to vote.

EAC Newsletters
and Updates

Sign up to receive information about EAC activities including public meetings, webcasts, reports and grants.

Give Us Your Feedback

Share your feedback on EAC policy proposalsElection Resource Library materials, and OpenEAC activities. Give feedback on general issues, including the Web site, through our Contact Us page.

Military and Overseas Voters

EAC has several projects under way to assist states in serving military and overseas citizens who register and vote absentee under the Uniformed and Overseas Citizens Absentee Voting Act. Learn more

Chapter 3: Terminal Data Package

3.1 Scope

This section contains a description of manufacturer documentation relating to the voting system that must be submitted with the system as a precondition of conformity assessment. These items are necessary to define the product and its method of operation; to provide technical and test data supporting the manufacturer's claims of the system's functional capabilities and performance levels; and to document instructions and procedures governing system operation and field maintenance. Any other items relevant to the system evaluation, such as media, materials, source code, object code, and sample output report formats, must be submitted along with this documentation.

This documentation is used by the test lab in constructing the test plan. Testing of systems submitted by manufacturers that consistently adhere to particularly strong and well-documented quality assurance and configuration management practices will generally be more efficient than for systems developed and maintained using less rigorous or less well-documented practices.

Both formal documentation and notes of the manufacturer's system development process must be submitted for conformity assessment. Documentation describing the system development process permits assessment of the manufacturer's systematic efforts to develop and test the system and correct defects. Inspection of this process also enables the design of a more precise test plan. The accredited test lab must design and conduct the appropriate tests to cover all elements of the system and to ensure conformance with all system requirements.

2 Comments

Comment by Harry VanSickle (State Election Official)

The requirements outlined in Chapter 3 relating to the technical data package submitted by a manufacturer are more than adequate. It also appears that the standards will appropriately focus more attention on reviewing the usability of user documentation (operations manuals and maintenance manuals). The lack of clear and concise user documentation has led to some of the user error problems in the past.

Comment by Cem Kaner (Academic)

The Technical Data Package shall be a public record. .......... (Affiliation Note: IEEE representative to TGDC)

3.1.1 Content and format

The content of the Technical Data Package (TDP) is intended to provide clear, complete descriptions of the following information about the system:

  1. Overall system design, including subsystems, modules and the interfaces among them;
  2. Specific functional capabilities provided by the system;
  3. Performance and design specifications;
  4. Design constraints, applicable standards, and compatibility requirements;
  5. Personnel, equipment, and facility requirements for system operation, maintenance, and logistical support;
  6. Manufacturer practices for assuring system quality during the system's development and subsequent maintenance; and
  7. Manufacturer practices for managing the configuration of the system during development and for modifications to the system throughout its life cycle.

3.1.1.1 Required content for initial conformity assessment

3.1.1.1-A TDP, identify full system configuration

The manufacturer SHALL submit to the test lab documentation necessary for the identification of the full system configuration submitted for evaluation and for the development of an appropriate test plan by the test lab.

Applies To: Voting system

Source: [VSS2002] I.9.2

3.1.1.1-B TDP, documents list

The manufacturer SHALL provide a list of all documents submitted controlling the design, construction, operation, and maintenance of the system.

Applies To: Voting system

Source: [VSS2002] II.2.1.1

1 Comment

Comment by C. Coggins (Voting System Test Laboratory)

Cross Reference this to 3.1.1.3.A.
3.1.1.1-C TDP contents

At minimum, the TDP SHALL contain the following documentation:

  1. Implementation statement;
  2. The voting equipment user documentation (Part 2: Chapter 4: "Voting Equipment User Documentation (manufacturer)");
  3. System hardware specification;
  4. Application logic design and specification;
  5. System security specifications;
  6. System test specification;
  7. Configuration management plan;
  8. Quality assurance program;
  9. System change notes; and
  10. Configuration for testing.

Applies To: Voting system

Source: [VSS2002] II.2.1.1.1

1 Comment

Comment by C. Coggins (Voting System Test Laboratory)

Add the list from 3.1.1.1-B to the TDP contents.

3.1.1.2 Required content for system changes and reassessment

3.1.1.2-A TDP, change notes

For systems seeking reassessment, manufacturers SHALL submit system change notes as described in Part 2: 3.7 "System Change Notes", as well as current versions of all documents that have been updated to reflect system changes.

Applies To: Voting system

DISCUSSION

Manufacturers may also submit other information relevant to the evaluation of the system, such as test documentation, and records of the system's performance history, failure analysis, and corrective actions.

Source: [VSS2002] II.2.1.1.2

2 Comments

Comment by Brian V. Jarvis (Local Election Official)

Recommend that 3.1.1.2-A (TDP, Change Notes) be satisfied by what industry refers to as a Version Description Document (VDD).

Comment by C. Coggins (Voting System Test Laboratory)

Cross Reference this to 3.1.1.1.B. Clarify that this requirement applies to the entire TDP and not the individual documents of the TDP.

3.1.1.3 Format

The requirements for formatting the TDP are general in nature; specific format details are of the manufacturer's choosing.

3.1.1.3-A TDP, table of contents and abstracts

The TDP SHALL include a detailed table of contents for the required documents, an abstract of each document, and a listing of each of the informational sections and appendices presented.

Applies To: Voting system

Source: [VSS2002] II.2.1.1.3

3.1.1.3-B TDP, cross-index

A cross-index SHALL be provided indicating the portions of the documents that are responsive to documentation requirements enumerated in Requirement Part 2: 3.1.1.1-C.

Applies To: Voting system

Source: [VSS2002] II.2.1.1.3

3.1.2 Other uses for documentation

Although all of the TDP documentation is required for conformity assessment, some of these same items may also be required during the state certification process and local level acceptance testing. Therefore, it is recommended that the technical documentation required for conformity assessment and acceptance testing be deposited in escrow.

3.1.3 Protection of proprietary information

3.1.3-A TDP, identify proprietary data

The manufacturer SHALL identify all documents, or portions of documents, containing proprietary information that is not releasable to the public.

Applies To: Voting system

DISCUSSION

This requirement was added to make it easier for test labs to identify information that the manufacturer considers proprietary. In current practice, test labs accepting proprietary information about a voting system from the manufacturer normally agree to use that information solely for the purpose of analyzing and testing the system, and agree to refrain from otherwise using the proprietary information or disclosing it to any other person or agency. While the content of any agreement between the test lab and manufacturer is outside of the scope of the VVSG, this requirement is intended to provide support for that practice.

An accredited test lab may reject a TDP if it is so encumbered by intellectual property claims as to obstruct the lab's delivery of the Test Plan (Part 2: Chapter 5) or Test Report (Part 2: Chapter 6).

An overuse of trade secret and patent protection may prevent certification by a certification authority (e.g., [KS05] 3.42: "The Manufacturer's entire proposal response package shall not be considered proprietary.").

Source: [VSS2002] II.2.1.3

3 Comments

Comment by C. Coggins (Voting System Test Laboratory)

This requirement should be a "should" not a "shall". The test lab can't determine what content is proprietary information. Discussion: The test lab has no criteria by which to assess information that should or should not be proprietary. If specific documents need to be public, the standard needs to identify what "shall not" be marked proprietary.

Comment by ACCURATE (Aaron Burstein) (Academic)

The limitations on manufacturers' leeway to mark certain materials as confidential are a salutary change from VSS II.2.1.3 and would help to bring the TDP requirements into line with the EAC's Voting System Testing and Certification Manual. This requirement should be adopted.

Comment by ACCURATE (Aaron Burstein) (Academic)

[NOTE: An incomplete version of this comment was previous submitted in error. Please disregard that comment.] The limitations on manufacturers' leeway to mark certain materials as confidential are a salutary change from VSS II.2.1.3 and would help to bring the TDP requirements into line with the EAC's Voting System Testing and Certification Manual. This requirement should be adopted. However, the mention of patent protection in the final paragraph of requirement 2:3.1.3-A should be removed. A patent gives the patentholder a right to restrict others from practicing the invention described in the patent. A patent does not restrict disclosures of information about a system containing a patented invention. To the contrary, the grant of a patent is conditional upon publication of how to practice the invention covered by the patent. Thus, a patent will not protect information submitted by the manufacturer from public disclosure. A more appropriate focus for requirement 2:3.1.3-A is any material designated as confidential or as a trade secret. Requirements 2:3.1.3-A and B would be clearer if they eliminated references to "intellectual property" and "proprietary information" and replaced them with confidential or trade secret information."
3.1.3-B TDP, consolidate proprietary data

The manufacturer SHOULD consolidate proprietary information to facilitate its removal from the Public Information Package.

Applies To: Voting system

Source: New requirement

3.2 Implementation Statement

2 Comments

Comment by C. Coggins (Voting System Test Laboratory)

This requirement should be a "shall" not a "should". The standard should provide direction on the method of consolidate proprietary information to facilitate it's removal.

Comment by ACCURATE (Aaron Burstein) (Academic)

This requirement would be clearer if it eliminated references to "intellectual property" and "proprietary information" and replaced them with confidential or trade secret information." See also ACCURATE's comment for 2:3.1.3-A.
3.2-A TDP, implementation statement

The TDP SHALL include an implementation statement as defined in Part 1: 2.4 "Implementation Statement".

Applies To: Voting system

DISCUSSION

Manufacturers may wish to contact their intended testing labs in advance to determine if those labs can supply them with an implementation statement pro forma to facilitate meeting this requirement.

Source: New requirement

2 Comments

Comment by C. Coggins (Voting System Test Laboratory)

Remove or change the Discussion. The suggestion that the lab provide a pro forma for development of an implementation statement violates EAC rules regarding consulting. The lab's role is to confirm that the documentation meets the requirements of the standard and not instruct the manufacturers in how to prepare their TDP. If a standard pro forma is required it needs to be identified in the standard.

Comment by Diane Gray (Voting System Test Laboratory)

Discussion gives the option to the manufacturer to contact intended test labs to determine if the labs can provide them with an implementation statement pro forma….Based on the required content of the Implementation Statement and that it must be approved by the EAC, suggest a pro forma statement be provided so that all vendors and VSTLs can be consistent.

3.3 System Hardware Specification

3.3-A TDP, system hardware specification

The manufacturer SHALL expand on the system overview included in the user documentation by providing detailed specifications of the hardware components of the system, including specifications of hardware used to support the telecommunications capabilities of the system, if applicable.

Applies To: Voting system

Source: [VSS2002] II.2.4

3.3.1 System hardware characteristics

3.3.1-A TDP, system hardware characteristics

The manufacturer SHALL provide a detailed discussion of the characteristics of the system, indicating how the hardware meets individual requirements defined in Part 1, including:

  1. Performance characteristics: Basic system performance attributes and operational scenarios that describe the manner in which system functions are invoked, describe environmental capabilities, describe life expectancy, and describe any other essential aspects of system performance;
  2. Physical characteristics: Suitability for intended use, requirements for transportation and storage, health and safety criteria, security criteria, and vulnerability to adverse environmental factors;
  3. Reliability: System and component reliability stated in terms of the system's operating functions, and identification of items that require special handling or operation to sustain system reliability;
  4. Maintainability: Ease with which maintenance actions can be performed based on the design characteristics of equipment and software and the processes the manufacturer and election officials have in place for preventing failures and for reacting to failures. Maintainability includes the ability of equipment and software to self-diagnose problems and make non-technical election workers aware of a problem. Maintainability also addresses a range of scheduled and unscheduled events; and
  5. Environmental conditions: Ability of the system to withstand natural environments, and operational constraints in normal and test environments, including all requirements and restrictions regarding electrical service, telecommunications services, environmental protection, and any additional facilities or resources required to install and operate the system.

Applies To: Voting system

Source: [VSS2002] II.2.4.1

1 Comment

Comment by Brian V. Jarvis (Local Election Official)

Recommend adding the following quantitative quality factors in this list: - availability (the ability to be accessed and operated when needed), - flexibility (the ability to be easily adapted to changing requirements), - portability (the ability to be easily modified for a new environment), - testability (the ability to be easily and thoroughly tested), and - usability (the ability to be easily learned and used).

3.3.2 Design and construction

3.3.2-A TDP, identify system configuration

The manufacturer SHALL provide sufficient data, or references to data, to identify unequivocally the details of the system configuration submitted for testing.

Applies To: Voting system

Source: [VSS2002] II.2.4.2

3.3.2-A.1 TDP, photographs for hardware validation

The manufacturer SHALL provide photographs of the exterior and interior of devices included in the system to identify the hardware of the system configuration submitted for testing.

Applies To: Voting system

Source: New requirement

3.3.2-B TDP, list of materials

The manufacturer SHALL provide a list of materials and components used in the system and a description of their assembly into major system components and the system as a whole.

Applies To: Voting system

Source: [VSS2002] II.2.4.2

3.3.2-C TDP, design and construction miscellany

Text and diagrams SHALL be provided that describe:

  1. Materials, processes, and parts used in the system, their assembly, and the configuration control measures to ensure compliance with the system specification;
  2. Electromagnetic environment generated by the system;
  3. Operator and voter safety considerations, and any constraints on system operations or the use environment; and
  4. Human factors considerations, including provisions for access by disabled voters.

Applies To: Voting system

Source: [VSS2002] II.2.4.2

3.3.3 Hardwired logic

3.3.3-A TDP, hardwired and mechanical implementations of logic

For each non-COTS hardware component (e.g., an Application-Specific Integrated Circuit or a manufacturer-specific integration of smaller components), the manufacturer SHALL provide complete design and logic specifications, such as Computer Aided Design and Hardware Description Language files.

Applies To: Voting system

Source: New requirement

3.3.3-B TDP, PLDs, FPGAs and PICs

For each Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), or Peripheral Interface Controller (PIC) that is programmed with non-COTS logic, the manufacturer SHALL provide complete logic specifications, such as Hardware Description Language files or source code.

Applies To: Voting system

Source: New requirement

3.4 Application Logic Design and Specification

3.4-A TDP, application logic design and specification

The manufacturer SHALL expand on the system overview included in the user documentation by providing detailed specifications of the application logic components of the system, including those used to support the telecommunications capabilities of the system, if applicable.

Applies To: Programmed device

Source: [VSS2002] II.2.5

1 Comment

Comment by Premier Election Solutions (Manufacturer)

This section is requesting "detailed specifications of the application logic components. Since this is a general statement and the specific are listed in the subsections this probably should not be a testable entry but just an informative description of what follows.

3.4.1 Purpose and scope

3.4.1-A TDP, describe application logic functions

The manufacturer SHALL describe the function or functions that are performed by the application logic comprising the system, including that used to support the telecommunications capabilities of the system, if applicable.

Applies To: Programmed device

Source: [VSS2002] II.2.5.1

3.4.2 Applicable documents

3.4.2-A TDP, list documents controlling application logic development

The manufacturer SHALL list all documents controlling the development of application logic and its specifications.

Applies To: Programmed device

Source: [VSS2002] II.2.5.2

3.4.3 Application logic overview

3.4.3-A TDP, application logic overview

The manufacturer SHALL provide an overview of the application logic.

Applies To: Programmed device

Source: [VSS2002] II.2.5.3

3.4.3-A.1 TDP, application logic architecture

The overview SHALL include a description of the architecture, the design objectives, and the logic structure and algorithms used to accomplish those objectives.

Applies To: Programmed device

Source: [VSS2002] II.2.5.3.a, reworded

3.4.3-A.2 TDP, application logic design

The overview SHALL include the general design, operational considerations, and constraints influencing the design.

Applies To: Programmed device

Source: [VSS2002] II.2.5.3.b

3.4.3-A.3 TDP, application logic overview miscellany

The overview SHALL include the following additional information for each separate software package:

  1. Package identification;
  2. General description;
  3. Requirements satisfied by the package;
  4. Identification of interfaces with other packages that provide data to, or receive data from, the package; and
  5. Concept of execution for the package.

Applies To: Programmed device

Source: [VSS2002] II.2.5.3.d

3.4.4 Application logic standards and conventions

3.4.4-A TDP, application logic standards and conventions

The manufacturer SHALL provide information on application logic standards and conventions developed internally by the manufacturer as well as published industry standards that have been applied by the manufacturer.

Applies To: Programmed device

Source: [VSS2002] II.2.5.4

3.4.4-B TDP, application logic standards and conventions, checklist

The manufacturer SHALL provide information that addresses the following standards and conventions related to application logic:

  1. Development methodology;
  2. Design standards, including internal manufacturer procedures;
  3. Specification standards, including internal manufacturer procedures;
  4. Coding conventions, including internal manufacturer procedures;
  5. Testing and verification standards, including internal manufacturer procedures, that can assist in determining the correctness of the logic; and
  6. Quality assurance standards or other documents that can be used to examine and test the application logic. These documents include standards for logic diagrams, program documentation, test planning, and test data acquisition and reporting.

Applies To: Programmed device

Source: [VSS2002] II.2.5.4

3.4.4-C TDP, justify coding conventions

The manufacturer SHALL furnish evidence that the selected coding conventions are "published" and "credible" as specified in Requirement Part 1: 6.4.1.3-A.

Applies To: Programmed device

Source: New requirement

3.4.5 Application logic operating environment

3.4.5-A TDP, application logic operating environment

The manufacturer SHALL describe or make reference to all operating environment factors that influence the design of application logic.

Applies To: Programmed device

Source: [VSS2002] II.2.5.5

3.4.5.1 Hardware environment and constraints

3.4.5.1-A TDP, hardware environment and constraints

The manufacturer SHALL identify and describe the hardware characteristics that influence the design of the application logic, such as:

  1. Logic and arithmetic capability of the processor;
  2. Memory read-write characteristics;
  3. External memory device characteristics;
  4. Peripheral device interface hardware;
  5. Data input/output device protocols; and
  6. Operator controls, indicators, and displays.

Applies To: Programmed device

Source: [VSS2002] II.2.5.5.1

3.4.5.2 Application logic environment

3.4.5.2-A TDP, identify operating system

The manufacturer SHALL identify the operating system and the specific version thereof, or else clarify how the application logic operates without an operating system.

Applies To: Programmed device

Source: [VSS2002] II.2.5.5.2

3.4.5.2-B TDP, identify compilers and assemblers

For systems containing compiled or assembled application logic, the manufacturer SHALL identify the COTS compilers or assemblers used in the generation of executable code, and the specific versions thereof.

Applies To: Programmed device

DISCUSSION

See Requirement Part 1: 6.4.1.7-A.3. Although compiled code should not be very sensitive to the versioning of the compiler, this information should be documented in case complications arise.

Source: [VSS2002] II.2.5.5.2

3.4.5.2-C TDP, identify interpreters

For systems containing interpreted application logic, the manufacturer SHALL specify the COTS runtime interpreter that SHALL be used to run this code, and the specific version thereof.

Applies To: Programmed device

DISCUSSION

See Requirement Part 1: 6.4.1.7-A.4.

Source: New requirement

3.4.6 Application logic functional specification

3.4.6-A TDP, application logic functional specification

The manufacturer SHALL provide a description of the operating modes of the system and of application logic capabilities to perform specific functions.

Applies To: Programmed device

Source: [VSS2002] II.2.5.6

3.4.6.1 Functions and operating modes

3.4.6.1-A TDP, functions and operating modes

The manufacturer SHALL describe all application logic functions and operating modes of the system, such as ballot preparation, election programming, preparation for opening the polls, recording votes and/or counting ballots, closing the polls, and generating reports.

Applies To: Programmed device

DISCUSSION

The word "function" here has the meaning suggested by the list of voting activities and should not be interpreted in the sense of callable unit.

Source: [VSS2002] II.2.5.6.1

3.4.6.1-B TDP, functions and operating modes detail

For each application logic function or operating mode, the manufacturer SHALL provide:

  1. A definition of the inputs to the function or mode (with characteristics, limits, tolerances or acceptable ranges, as applicable);
  2. An explanation of how the inputs are processed; and
  3. A definition of the outputs produced (again, with characteristics, limits, tolerances, or acceptable ranges, as applicable).

Applies To: Programmed device

Source: [VSS2002] II.2.5.6.1

3.4.6.2 Application logic integrity features

3.4.6.2-A TDP, application logic integrity features

The manufacturer SHALL describe the application logic's capabilities or methods for detecting or handling:

  1. Exception conditions;
  2. System failures;
  3. Data input/output errors;
  4. Error logging for audit record generation;
  5. Production of statistical ballot data;
  6. Data quality assessment; and
  7. Security monitoring and control.

Applies To: Programmed device

Source: [VSS2002] II.2.5.6.2

3.4.7 Programming specifications

3.4.7-A TDP, programming specifications

The manufacturer SHALL provide in this section an overview of the application logic's design, its structure, and implementation algorithms and detailed specifications for individual modules.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7

3.4.7.1 Programming specifications overview

3.4.7.1-A TDP, programming specifications overview

The programming specifications overview SHALL document the architecture of the application logic.

Applies To: Programmed device

Source: Summary of [VSS2002] II.2.5.7.1

1 Comment

Comment by Premier Election Solutions (Manufacturer)

It appears that this requirement is already accommodated in requirement 3.4.3-A.1. Proposed Change: Remove this requirement
3.4.7.1-A.1 TDP, programming specifications overview, diagrams

This overview SHALL include such items as UML diagrams, data flow diagrams, and/or other graphical techniques that facilitate understanding of the programming specifications.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.1

3.4.7.1-A.2 TDP, programming specifications overview, function

This section SHALL be prepared to facilitate understanding of the internal functioning of the individual modules.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.1

 
3.4.7.1-A.3 TDP, programming specifications overview, content

Implementation of the functions SHALL be described in terms of the architecture, algorithms, and data structures.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.1

 

3.4.7.2 Programming specifications details

3.4.7.2-A TDP, programming specifications details

The programming specifications SHALL describe individual application logic modules and their component units, if applicable.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.2

 
3.4.7.2-B TDP, module and callable unit documentation

For each application logic module and callable unit, the manufacturer SHALL document:

  1. Significant module and unit design decisions, if any, such as algorithms used;
  2. Any constraints, limitations, or unusual features in the design of the module or callable unit; and
  3. A description of its inputs, outputs, and other data elements as applicable with respect to communication over system interfaces (see Part 2: 3.4.9 "Interfaces").

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.2.a, b, and e

 

1 Comment

Comment by Premier Election Solutions (Manufacturer)

This type of information is valid for modules but not for callable units. In modern programming a callable unit can, and often is, a single line of code that is encapsulating and protecting a data element. Requiring this level of documentation for callable units is not realistic and is an impediment to good programming practices. Proposed Change: Change this requirement to read as follows: 3.4.7.2-B TDP, module documentation For each application logic module, the manufacturer SHALL document: a. Significant module and unit design decisions, if any, such as algorithms used; b. Any constraints, limitations, or unusual features in the design of the module; and c. A description of its inputs, outputs, and other data elements as applicable with respect to communication over system interfaces (see Part 2:3.4.9 "Interfaces").
3.4.7.2-C TDP, justify mixed-language software

If an application logic module is written in a programming language other than that generally used within the system, the specification for the module SHALL indicate the programming language used and the reason for the difference.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.2.c

3.4.7.2-D TDP, references for foreign programming languages

If a module contains embedded border logic commands for an external library or package (e.g., menu selections in a database management system for defining forms and reports, on-line queries for database access and manipulation, input to a graphical user interface builder for automated code generation, commands to the operating system, or shell scripts), the specification for the module SHALL contain a reference to user manuals or other documents that explain them.

Applies To: Programmed device

Source: [VSS2002] II.2.5.7.2.d

1 Comment

Comment by Premier Election Solutions (Manufacturer)

This requirement needs to be clarified. Please clarify if the "user manual" referred to in this requirement is the user manual for the "foreign programming language".
3.4.7.2-E TDP, source code

For each callable unit (function, method, operation, subroutine, procedure, etc.) in application logic, border logic, and third-party logic, the manufacturer SHALL supply the source code.

Applies To: Programmed device

Source: [VSS2002] II.2.1

3.4.7.2-F TDP, inductive assertions

For each callable unit (function, method, operation, subroutine, procedure, etc.) in core logic, the manufacturer SHALL specify:

  1. Preconditions and postconditions of the callable unit, formally stated using the terms defined in Part 1: 8.3.1 "Domain of discourse" and possibly other terms defined by the manufacturer, including any assumptions about capacities and limits within which the system is expected to operate; and
  2. A sound argument (possibly, but not necessarily, a formal proof) that the preconditions and postconditions of the callable unit accurately represent its behavior, assuming that the preconditions and postconditions of any invoked units are similarly accurate.

Applies To: Programmed device

DISCUSSION

Sufficient invariants and assertions should be provided to make it possible to perform the verification of Part 3: 4.6 "Logic Verification" through purely local checks (i.e., using the callable unit itself, the pre- and postconditions of any invoked units, and the invariants of any global data accessed by the callable unit, but not the source code of the invoked units nor any other logic).

The use of preconditions and postconditions as inductive assertions derives primarily from [Hoare69], but a list of relevant work predating [Hoare69] can be found in [Morris84]. As a pragmatic compromise to avert "analysis paralysis," the verification described here is considerably less rigorous than was envisioned in the literature.

A sound argument need not be complicated. In cases where the relationship between preconditions and postconditions and the behavior of the callable unit is completely obvious or trivial, it may suffice to state as much. The acceptance of such a statement is at the discretion of the test lab.

Postconditions that impact something outside the domain of discourse are not of interest unless that thing impacts the behavior of some function with respect to the domain of discourse. The manufacturer must define such terms as are necessary to state any and all dependencies and assumptions that may impact the behavior and use them consistently in all affected preconditions and postconditions. An excess of extraneous dependencies may negatively impact the test lab's ability to verify the system's correctness and thereby preclude a positive finding of conformance.

A callable unit that has no impact on anything in the domain of discourse and no dependency on anything in the domain of discourse is not core logic.

Source: New requirement

3 Comments

Comment by Kevin Wilson (Voting System Test Laboratory)

The domain of discourse and consequently core logic does not appear to include the concept of "ballot generation", but only of voting. Consider the case when a ballot is erroneously generated with the same candidate appearing twice in the virtual list of candidates. This is Candidate A and Candidate B who are actually the same person, but due to a ballot generation error appear virtually to be different but not different enough that both are displayed during voting. In the voting portion of the software, the Candidate A appears to the voter, but when counting votes Candidate B is counted. In this situation Candidate A would not appear to have received any votes. Therefore ballot generation errors can cause undetermined errors in core logic regardless of the attempted proof given in 1-8.3. Although this situtation is unlikely in IVVR systems, it is otherwise still possible. In the case of a VVPAT system this error is observable by audit but could still occur if the "image" of the ballot that the counting system has is erroneous compared to the "image" of the ballot that indicates voter intent. Is ballot generation part of "core logic" or could the document specify more clearly that ballot generation "impacts" the "domain of discourse" Furthermore "core logic" as defined in Appendix A does not include ballot generation, but only voting and tabulation.

Comment by Kevin Wilson (Voting System Test Laboratory)

As defined in Appendix A, third-party logic could include all of the source code to Windows CE. However, as of the writing of this question, only 3.9 million lines are available for VSTL review. Windows CE is used in a variety of non-voting devices, which provides a strong argument that it is COTS. Perhaps a reasonable compromise is that as in the 2002 version there is trust in the non-modified portions of any COTS software and any modified portions are incorporated into the solution those portions must meet EAC standards and be delivered to the VSTL for review.

Comment by Diane Gray (Voting System Test Laboratory)

What constitutes "a sound argument..."?
3.4.7.2-G TDP, high-level constraints

The manufacturer SHALL specify a sound argument (possibly, but not necessarily, a formal proof) that the core logic as a whole satisfies each of the constraints indicated in Part 1: 8.3 "Logic Model (normative)" for all cases within the aforementioned capacities and limits, assuming that the preconditions and postconditions of callable units accurately characterize their behaviors.

Applies To: Programmed device

Source: New requirement

1 Comment

Comment by Diane Gray (Voting System Test Laboratory)

What constitutes a "sound argument (possibly, but not necessarily, a formal proof) that the core logic satsfies each constraint?
3.4.7.2-H TDP, safety of concurrency

The manufacturer SHALL specify a sound argument (possibly, but not necessarily, a formal proof) that application logic is free of race conditions, deadlocks, livelocks, and resource starvation.

Applies To: Programmed device

DISCUSSION

If application logic does not perform any sort of concurrent computing (e.g., multiple processes or threads), it suffices to note this fact.

Source: New requirement

2 Comments

Comment by Alan A. Jorgensen, Ph.D., for the Association for Software Testing Special Interest Group on eVoting (Advocacy Group)

"Race conditions" commonly occur because of unprotected "critical sections." Commonly, unprotected critical sections occur where there is a timer and timeout occurs. "Vote-Capture Devices" should have timers that can expire during a normal voting session. Coincidence of timer expiration and the event being timed must be tested. Assurance from a vendor, no matter what form, is insufficient evidence that race conditions do not exist. . . . . . . . . . . Unprotected critical sections are difficult to find using code inspection methods. . . . . . . . . . . Testing for race conditions involves searching for the coincidence of events and testing for the appropriate results (correct timeout response or correct timed event response). Since there is a randomness interjected by other events in the software, repetitive automated testing is required. . . . . . . . . . . We recommend that the VVSG require testing for "Unsafe concurrency."

Comment by Diane Gray (Voting System Test Laboratory)

If the software does perform concurrent computing, what constitutes a "sound argument (possibly, but not necessarily, a formal proof) that the application logic is free of the listed condidtions?
3.4.7.2-I TDP, justify long units

The manufacturer SHALL justify any callable unit lengths that violate Requirement Part 1: 6.4.1.4-B.1.

Applies To: Programmed device

Source: [VSS2002] II.5.4.2.i

3.4.8 System database

3.4.8-A TDP, system database

The manufacturer SHALL identify and provide a diagram and narrative description of the system's databases and any external files used for data input or output.

Applies To: Programmed device

Source: [VSS2002] II.2.5.8

 
3.4.8-B TDP, database design levels

For each database or external file, the manufacturer SHALL specify the number of levels of design and the names of those levels (e.g., conceptual, internal, logical, and physical).

Applies To: Programmed device

Source: [VSS2002] II.2.5.8.a

 
3.4.8-C TDP, database design conventions

For each database or external file, the manufacturer SHALL specify any design conventions and standards (which may be incorporated by reference) needed to understand the design.

Applies To: Programmed device

Source: [VSS2002] II.2.5.8.b

 
3.4.8-D TDP, data models

For each database or external file, the manufacturer SHALL identify and describe all logical entities and relationships and how these are implemented physically (e.g., tables, files).

Applies To: Programmed device

DISCUSSION

This requirement calls for a data model but a specific modeling language is no longer mandated. ([VSS2005] II.2.5.8 required an E-R diagram.)

Source: [VSS2002] II.2.5.8.c and d

3.4.8-E TDP, schemata

The manufacturer SHALL document the details of table, record or file contents (as applicable), individual data elements and their specifications, including:

  1. Names/identifiers;
  2. Data type (alphanumeric, integer, etc.);
  3. Size and format (such as length and punctuation of a character string);
  4. Units of measurement (meters, seconds, etc.);
  5. Range or enumeration of possible values (0–99, etc.);
  6. Accuracy (how correct) and precision (number of significant digits);
  7. Priority, timing, frequency, volume, sequencing, and other constraints, such as whether the data element may be updated and whether business rules apply;
  8. Security and privacy constraints; and
  9. Sources (setting/sending entities) and recipients (using/receiving entities).

Applies To: Programmed device

DISCUSSION

The majority of this requirement may be satisfied by supplying the source of the database schema if it is in a widely recognized and standardized language.

Source: [VSS2002] II.2.5.8.e

3.4.8-F TDP, external file maintenance and security

For external files, manufacturers SHALL document the procedures for file maintenance, management of access privileges, and security.

Applies To: Programmed device

Source: [VSS2002] II.2.5.8.f

3.4.9 Interfaces

3.4.9-A TDP, identify and describe interfaces

Using a combination of text and diagrams, the manufacturer SHALL identify and provide a complete description of all major internal and external interfaces.

Applies To: Programmed device

DISCUSSION

"Major" interfaces are at the level of those identified in the system overview (Part 2: 4.1 "System Overview"). These are interfaces between subsystems and components, not callable units.

Source: [VSS2002] II.2.5.9

3.4.9.1 Interface identification

3.4.9.1-A TDP, interface identification details

For each interface identified in the system overview, the manufacturer SHALL:

  1. Provide a unique identifier assigned to the interface;
  2. Identify the interfacing entities (systems, configuration items, users, etc.) by name, number, version, and documentation references, as applicable; and
  3. Identify which entities have fixed interface characteristics (and therefore impose interface requirements on interfacing entities) and which are being developed or modified (thus having interface requirements imposed on them).

Applies To: Programmed device

Source: [VSS2002] II.2.5.9.1

3.4.9.2 Interface description

3.4.9.2-A TDP, interface types

For each interface identified in the system overview, the manufacturer SHALL describe the type of interface (e.g., real-time data transfer or data storage-and-retrieval) to be implemented.

Applies To: Programmed device

Source: [VSS2002] II.2.5.9.2.a

3.4.9.2-B TDP, interface signatures

For each interface identified in the system overview, the manufacturer SHALL describe characteristics of individual data elements that the interfacing entity(ies) will provide, store, send, access, receive, etc., such as:

  1. Names/identifiers;
  2. Data type (alphanumeric, integer, etc.);
  3. Size and format (such as length and punctuation of a character string);
  4. Units of measurement (meters, seconds, etc.);
  5. Range or enumeration of possible values (0–99, etc.);
  6. Accuracy (how correct) and precision (number of significant digits);
  7. Priority, timing, frequency, volume, sequencing, and other constraints, such as whether the data element may be updated and whether business rules apply;
  8. Security and privacy constraints; and
  9. Sources (setting/sending entities) and recipients (using/receiving entities).

Applies To: Programmed device

Source: [VSS2002] II.2.5.9.2.b

3.4.9.2-C TDP, interface protocols

For each interface identified in the system overview, the manufacturer SHALL describe characteristics of communication methods that the interfacing entity(ies) will use for the interface, such as:

  1. Communication links/bands/frequencies/media and their characteristics;
  2. Message formatting;
  3. Flow control (e.g., sequence numbering and buffer allocation);
  4. Data transfer rate, whether periodic/aperiodic, and interval between transfers;
  5. Routing, addressing, and naming conventions;
  6. Transmission services, including priority and grade; and
  7. Safety/security/privacy considerations, such as encryption, user authentication, compartmentalization, and auditing.

Applies To: Programmed device

Source: [VSS2002] II.2.5.9.2.c

 
3.4.9.2-D TDP, protocol details

For each interface identified in the system overview, the manufacturer SHALL describe characteristics of protocols the interfacing entity(ies) will use for the interface, such as:

  1. Priority/layer of the protocol;
  2. Packeting, including fragmentation and reassembly, routing, and addressing;
  3. Legality checks, error control, and recovery procedures;
  4. Synchronization, including connection establishment, maintenance, termination; and
  5. Status, identification, and any other reporting features.

Applies To: Programmed device

Source: [VSS2002] II.2.5.9.2.d

3.4.9.2-E TDP, interface etceteras

For each interface identified in the system overview, the manufacturer SHALL describe any other pertinent characteristics, such as physical compatibility of the interfacing entity(ies) (dimensions, tolerances, loads, voltages, plug compatibility, etc.).

Applies To: Programmed device

Source: [VSS2002] II.2.5.9.2.e

3.4.10 Appendices

The manufacturer may provide descriptive material and data supplementing the various sections of the body of the logic specifications. The content and arrangement of appendices are at the discretion of the manufacturer. Topics recommended for amplification or treatment in appendix form include:

  • Glossary: A listing and brief definition of all module names and variable names, with reference to their locations in the logic structure. Abbreviations, acronyms, and terms should be included, if they are either uncommon in data processing and software development or are used with an unorthodox meaning;
  • References: A list of references to all related manufacturer documents, data, standards, and technical sources used in logic development and testing; and
  • Program Analysis: The results of logic configuration analysis algorithm analysis and selection, timing studies, and hardware interface studies that are reflected in the final logic design and coding.

3.5 System Security Specifications

This section defines the documentation requirements for voting systems. These recommendations apply to the full scope of voting system functionality, including functionality for defining the ballot and other pre-voting functions, as well as functions for casting and storing votes, vote reporting, system logging, and maintenance of the voting system. User documentation includes all public information that is provided to the end users. The Technical Data Package (TDP) includes the user documentation along with other private information that is viewed only by the test labs.

3.5.1 General

3.5.1-A TDP, overall security

Manufacturers SHALL document in the TDP all aspects of system design, development, and proper usage that are relevant to system security. This includes, but is not limited to the following:

  1. System security objectives;
  2. All hardware and software security mechanisms;
  3. Development procedures employed to ensure absence of malicious code;
  4. Initialization, usage, and maintenance procedures necessary to secure operation;
  5. All attacks the system is designed to resist or detect; and
  6. Any security vulnerabilities known to the manufacturer.

Applies To: Voting system

Source: [VVSG2005] I.8.7

1 Comment

Comment by Diane Gray (Voting System Test Laboratory)

The source stated is Volume I Section 8.7, which is Quality Assurance Requirements, Documentation?
3.5.1-B TDP, high level security

Manufacturers SHALL provide at a minimum the high-level documents listed in Part 2: Table 3-1 as part of the TDP.

Applies To: Voting system

Source: [VVSG2005] I.8.7

Table 3-1 High level voting system documentation

Document

Description

Security Threats Controls

This document shall identify the threats the voting system protects against and the implemented security controls on voting system and system components.

Security Architecture

This document shall provide an architecture level description of how the security requirements are met, and shall include the various authentication, access control, audit, confidentiality, integrity, and availability requirements.

Interface Specification

This document shall describe external interfaces (programmatic, human, and network) provided by each of the computer components of the voting system (examples of components are DRE, Central Tabulator, Independent Audit machine).

Design Specification

This document shall provide a high-level design of each voting system component.

Development Environment Specification

This document shall provide descriptions of the physical, personnel, procedural, and technical security of the development environment including configuration management, tools used, coding standards used, software engineering model used, and description of developer and independent testing.

Security Testing and Vulnerability Analysis Documentation

These documents shall describe security tests performed to identify vulnerabilities and the results of the testing. This also includes testing performed as part of software development, such as unit, module, and subsystem testing.

2 Comments

Comment by Diane Gray (Voting System Test Laboratory)

Please provide strong guidelines on what security information can be considered Proprietary.

Comment by ACCURATE (Aaron Burstein) (Academic)

See ACCURATE's comment to Part 2:4.3.1.

3.5.2 Access Control

3.5.2-A TDP, general user

Manufacturers SHALL provide user and TDP documentation of access control capabilities of the voting system.

Applies To: Voting system

Source: [VVSG2005] I.7.2.1.2

3.5.2-B TDP, general access control technical specification

Manufacturers SHALL provide descriptions and specifications of all access control mechanisms of the voting system including management capabilities of authentication, authorization, and passwords in the TDP.

Applies To: Voting system

DISCUSSION

Access control mechanisms include those that are designed to permit authorized access to the voting system and prevent unauthorized access to the voting system. Specific examples of access control measures include but are not limited to: use of data and user authorization, security kernels, computer-generated password keys, and special protocols.

Source: [VVSG2005] I.7.2.1.2

3.5.2-C TDP, unauthorized access technical specification

Manufacturers SHALL provide descriptions and specifications of methods to prevent unauthorized access to the access control mechanisms of the voting system in the TDP.

Applies To: Voting system

Source: [VVSG2005] I.7.2.1.2

 
3.5.2-D TDP, access control dependant voting system mechanisms

Manufacturers SHALL provide descriptions and specifications of all other voting system mechanisms that are dependent upon, support, and interface with access controls in the TDP.

Applies To: Voting system

Source: [VVSG2005] I.7.2.1.2

3.5.2-E TDP, voting operations and roles

Manufacturers SHALL provide a list of all of the operations possible on the voting device and list the default roles that have permission to perform each such operation as part of the TDP.

Applies To: Voting system

Source: [VVSG2005] I.7.2.1.2

3.5.3 System Event Logging

3.5.3-A TDP, general user

Manufacturers SHALL provide TDP documentation of event logging capabilities of the voting devices.

Applies To: Voting system

Source: [VVSG2005] I.5.4

1 Comment

Comment by Diane Gray (Voting System Test Laboratory)

This requirement encompasses many requirements, such as Part 1 Chapter 5.7 table 5-5. Other applicable sections should be referenced.
3.5.3-A.1 TDP, event logging design and implementation

Manufacturers SHALL provide a technical data package that describes system event logging design and implementation.

Applies To: Voting system

Source: [VVSG2005] I.5.4

3.5.4 Software Installation

3.5.4-A TDP, software list technical data package

The manufacturer SHALL provide a list of all software related to the voting system in the technical data package (TDP).

Applies To: Voting system

DISCUSSION

This requirement establishes a list of the software used by the voting system. All software related to a voting system includes application logic, border logic, third party logic, COTS software, and installation software. Installation software is used to install and configure the software on non-volatile storage of programmed devices of the voting system. Software may be in the form of source code, executable code, or both.

1 Comment

Comment by Diane Gray (Voting System Test Laboratory)

Part I Section 5.3 discusses more TDP requirements.
3.5.4-B TDP, software information

The manufacturer SHALL provide at a minimum in the TDP the following information for each piece of software related to the voting system: software product name, software version number, software manufacturer name, software manufacturer contact information, type of software (application logic, border logic, third party logic, COTS software, or installation software), list of software documentation, component identifier(s) (such as filename(s)) of the software, type of software component (executable code, source code, or data).

Applies To: Voting system

3.5.4-B.1 TDP, software location information

As part of the TDP, the manufacturer SHALL provide the location (such as full path name or memory address) and storage device (such as type and part number of storage device) where each piece of software is installed on programmed devices of the voting system.

Applies To: Programmed device

DISCUSSION

This requirement applies to software installed on programmed devices of the voting system. The full directory path is the final destination of the software when installed in non-volatile storage with a file system.

3.5.4-B.2 TDP, software functionality for programmed devices

As part of the TDP, the manufacturer SHALL document the functionality provided to the voting system by the software installed on programmed devices.

Applies To: Programmed device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

3.5.4-B.3 TDP, software dependencies and interaction

As part of the TDP, the manufacturer SHALL map the dependencies and interactions between software installed on programmed devices of the voting system.

Applies To: Programmed device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

3.5.4-C TDP, build environment software and hardware

As part of the TDP, the manufacturer SHALL provide a list of all software and hardware required to assemble the build environment used to create voting system software executable code including application logic, border logic, and third party logic.

Applies To: Voting system

DISCUSSION

Third party software (such as operating systems, compilers, and libraries) required to build voting system software are captured by this requirement.

3.5.4-D TDP, build environment assembly procedures

As part of the TDP, the manufacturer SHALL document the procedures to assemble the build environment(s) used to create voting system software executable code including application logic, border logic, and third party logic.

Applies To: Voting system

Source: [EAC06] Section 5.6.1.2

3.5.4-E TDP, voting system software build procedures

As part of the TDP, the manufacturer SHALL document the procedures used to build the voting system software executable code including application logic, border logic, and third party logic.

Applies To: Voting system

3.5.4-F TDP, original certified voting system software identification

As part of the TDP, the manufacturer SHALL provide the certification number associated with the voting system software to be updated.

Applies To: Voting system

3.5.4-G TDP, updated voting system software build procedure

As part of the TDP, the manufacturer SHALL document the procedures used to build the updated voting system software including application logic, border logic, and third party logic using the post build environment associated with the previously built voting system software.

Applies To: Voting system

3.5.4-H TDP, build environment software and hardware

As part of the TDP, the manufacturer SHALL provide a list of all software and hardware required to assemble the build environment used to create voting system software executable code including application logic, border logic, and third party logic.

Applies To: Voting system

DISCUSSION

Third party software (such as operating systems, compilers, and libraries) required to build voting system software are captured by this requirement.

1 Comment

Comment by Premier Election Solutions (Manufacturer)

Items H-L seem like duplicates of C-G. Proposed Change: Remove items H-L or remove items C-G.
3.5.4-I TDP, build environment assembly procedures

As part of the TDP, the manufacturer SHALL document the procedures to assemble the build environment(s) used to create voting system software executable code including application logic, border logic, and third party logic.

Applies To: Voting system

Source: [EAC06] Section 5.6.1.2

1 Comment

Comment by Premier Election Solutions (Manufacturer)

Items H-L seem like duplicates of C-G. Proposed Change: Remove items H-L or remove items C-G.
3.5.4-J TDP, voting system software build procedures

As part of the TDP, the manufacturer SHALL document the procedures used to build the voting system software executable code including application logic, border logic, and third party logic.

Applies To: Voting system

1 Comment

Comment by Premier Election Solutions (Manufacturer)

Items H-L seem like duplicates of C-G. Proposed Change: Remove items H-L or remove items C-G.
3.5.4-K TDP, original certified voting system software identification

As part of the TDP, the manufacturer SHALL provide the certification number associated with the voting system software to be updated.

Applies To: Voting system

1 Comment

Comment by Premier Election Solutions (Manufacturer)

Items H-L seem like duplicates of C-G. Proposed Change: Remove items H-L or remove items C-G.
3.5.4-L TDP, updated voting system software build procedure

As part of the TDP, the manufacturer SHALL document the procedures used to build the updated voting system software including application logic, border logic, and third party logic using the post build environment associated with the previously built voting system software.

Applies To: Voting system

1 Comment

Comment by Premier Election Solutions (Manufacturer)

Items H-L seem like duplicates of C-G. Proposed Change: Remove items H-L or remove items C-G.

3.5.5 Physical Security

3.5.5-A TDP, unauthorized physical access

The manufacturer SHALL provide a list of all voting device components to which access must be restricted and a description of the function of each said component.

Applies To: Voting device

DISCUSSION

This list may be included in the technical data package a well as in the user documentation.

3.5.5-B TDP, physical port and access point

As part of the TDP, the manufacturer SHALL provide a listing of all ports and access points.

Applies To: Voting device

3.5.5-C TDP, physical lock documentation of use

For each lock used on a voting device, manufacturer SHALL document whether the lock was installed to secure an access point.

Applies To: Voting device

DISCUSSION

Locks on voting devices may be used to secure access points such as doors and panels or they may be used simply to fasten a segment of the voting device’s encasement. In the former case, testing labs must verify that the lock does indeed provide a measure of security. In the latter case, the testing lab must verify that bypassing the lock does not put the security of the system in jeopardy. Manufacturer attestation should be included in User documentation, and in the TDP.

3.5.5-D TDP, power usage

Manufacturer SHALL provide a list of all physical security countermeasures that require power supplies.

Applies To: Voting device

3.5.5-E TDP, physical security

Manufacturer SHALL provide a technical data package that documents the design and implementation of all physical security controls for the voting device and its components.

Applies To: Voting device

3.5.6 System Integrity Management

3.5.6-A TDP, binaries per voting system mode

As part of the TDP, manufacturers SHALL provide a list of the binaries that are required to be executed on the electronic device for each voting system mode.

Applies To: Electronic device

DISCUSSION

This requirement supports requirements in Part 1: 5.5 "System Integrity Management".

Source: [VVSG2005] I.7.4.6

3.5.7 Setup Inspection

3.5.7-A TDP, installed software identification

The manufacturer SHALL provide the technical specifications of how programmed devices of voting systems identifies installed software in the TDP.

Applies To: Programmed device

DISCUSSION

The requirement provides implementation information for test labs to support the testing of the voting system.

Source: [VVSG2005] I.7.4.6 (c)

3.5.7-B TDP, software integrity verification

The manufacturer SHALL provide a technical specification of how the integrity of software installed on programmed devices of the voting system is verified as part of the TDP.

Applies To: Programmed device

DISCUSSION

The requirement provides implementation information for test labs to support the testing of the voting system.

Source: [VVSG2005] I.7.4.6 (c)

3.5.7-B.1 TDP, software integrity verification technique software non-modification

Software integrity verification techniques SHALL prevent the modification of software installed on programmed devices of the voting system.

Applies To: Programmed device

Source: [VVSG2005] I.7.4.6 (b) (iii)

1 Comment

Comment by Premier Election Solutions (Manufacturer)

Verification techniques cannot prevent changes but rather can only detect changes. Proposed Change: Change the word "prevent" to "detect".
3.5.7-C TDP, register and variable value inspection

The manufacturer SHALL provide a technical specification of how the inspection of all the voting device registers and variables is implemented by the voting device in the TDP.

Applies To: Voting device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

Source: [VVSG2005] I.7.4.6 (f)(i)

1 Comment

Comment by Premier Election Solutions (Manufacturer)

A clear definition of registers and variables need to be provided. Software uses variables and their contents change all the time (that's why they are called variables) so it is unclear as to what these registers and variables are being insepected for. Please clarify what the variables are being inspected for.
3.5.7-D TDP, backup power inspection

The manufacturers SHALL provide a technical specification of how the inspection of the remaining charge of the backup power sources is implemented by the voting device in the TDP.

Applies To: Voting device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

 
3.5.7-E TDP, cabling connectivity inspection

The manufacturers SHALL provide a technical specification of how the inspection of the connectivity of cabling attached to a voting device is implemented by the voting device in the TDP.

Applies To: Voting device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

3.5.7-F TDP, communication operational status inspection

The manufacturers SHALL provide a technical specification of how the inspection of the operational status of the communications capability is implemented by the voting device in the TDP.

Applies To: Voting device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

1 Comment

Comment by Carolyn Coggins (Voting System Test Laboratory)

Clarify what is meant by "communications capability".
3.5.7-G TDP, communication on/off inspection

The manufacturer SHALL provide a technical specification of how the inspection of the on/off status of the communications capability is implemented by the voting device in the TDP.

Applies To: Voting device

DISCUSSION

This requirement provides implementation information for test labs to support the testing of the voting system.

1 Comment

Comment by Carolyn Coggins (Voting System Test Laboratory)

Clarify what is meant by "communications capability".
3.5.7-H TDP, consumable inspection

The manufacturer SHALL provide a technical specification of how the inspection of the remaining amount of each consumable is implemented by the voting device in the TDP.

Applies To: Voting device

3.5.7-I TDP, calibration of voting device components inspection

The manufacturer SHALL provide a technical specification of how the inspection of the calibration for each component is implemented by the voting device in the TDP.

Applies To: Voting device

3.5.7-J TDP, calibration of voting device components adjustment

The manufacturers SHALL provide a technical specification of how the adjustment to the calibration of each component is implemented by the voting device in the TDP.

Applies To: Voting device

3.5.8 Cryptography

3.5.8-A TDP, cryptography

The manufacturer documentation SHALL include a precise definition of the fields in the Device Certificate, Election Certificate, the naming supported in certificates, the algorithms supported, and the format of the Election Closeout Record in the TDP.

Applies To: Voting system

3.6 System Test Specification

3.6-A TDP, development and system tests

The manufacturer SHALL provide test specifications for:

  1. Development test specifications; and
  2. System test specifications.

Applies To: Voting system

Source: [VSS2002] II.2.7

3.6.1 Development test specifications

3.6.1-A TDP, development test specifications

The manufacturer SHALL describe the plans, procedures, and data used during development and system integration to verify system logic correctness, data quality, and security. This description shall include:

  1. Test identification and design, including test structure, test sequence or progression, and test conditions;
  2. Standard test procedures, including any assumptions or constraints;
  3. Special purpose test procedures including any assumptions or constraints;
  4. Test data, including the data source, whether it is real or simulated, and how test data are controlled;
  5. Expected test results; and
  6. Criteria for evaluating test results.

Applies To: Voting system

DISCUSSION

Documentation that is already required under the life cycle process adopted by the manufacturer may satisfy this requirement.

Previous iterations of these VVSG cited MIL-STD-498, Software Test Plan and Software Test Description. That standard was cancelled in 1998. Currently applicable standards include [IEEE97] and [IEEE98].

Source: [VSS2002] II.2.7.1

3.6.2 System test specifications

Note: Part 1: Chapter 3: "VVSG Background" contains several requirements for usability testing by the manufacturer and that each of these requirements also mandates that the manufacturer report the test results as part of the TDP. These requirements are not present in this section but need to be considered as part of the system test specifications.

3.6.2-A TDP, functional test specifications

The manufacturer SHALL provide specifications for verification and validation of overall system performance. These specifications shall cover:

  1. Control and data input/output;
  2. Processing accuracy;
  3. Data quality assessment and maintenance;
  4. Ballot interpretation logic;
  5. Exception handling;
  6. Security;
  7. Production of audit trails and statistical data;
  8. Expected test results; and
  9. Criteria for evaluating test results.

Applies To: Voting system

Source: [VSS2002] II.2.7.2

3.6.2-B TDP, demonstrate fitness for purpose

The specifications SHALL identify procedures for assessing and demonstrating the suitability of the system for election use.

Applies To: Voting system

Source: [VSS2002] II.2.7.2

1 Comment

Comment by Carolyn Coggins (Voting System Test Laboratory)

What does "suitability of the system for election use" mean?

3.7 System Change Notes

3.7-A TDP, system change notes

Manufacturers submitting modifications for a system that has been tested previously SHALL submit system change notes.

Applies To: Voting system

DISCUSSION

These will be used by the accredited test lab to assist in developing and executing the test plan for the modified system.

Source: [VSS2002] II.2.13

3.7-B TDP, system change notes content

The system change notes SHALL include the following information:

  1. Summary description of the nature and scope of the changes, and reasons for each change;
  2. Listing of the specific changes made, citing the specific system configuration items changed, and providing detailed references to the documentation sections changed;
  3. Specific sections of the documentation that are changed (or completely revised documents, if more suitable to address a large number of changes); and
  4. Documentation of the test plan and procedures executed by the manufacturer for testing the individual changes and the system as a whole, and records of test results.

Applies To: Voting system

Source: [VSS2002] II.2.13

3.8 Configuration for Testing

Configuration of hardware and software, both operating systems and applications, is critical to proper system functioning. Correct test design and sufficient test execution must account for the intended and proper configuration of all system components. If the voting system can be set up in both conforming and nonconforming configurations, the configuration actions necessary to obtain conforming behavior must be specified.

3.8-A TDP, photographs illustrating hardware set-up

The manufacturer SHALL provide photographs illustrating the proper set-up of the voting system hardware.

Applies To: Voting system

Source: New requirement

3.8-B TDP, provide answers to installation prompts

The manufacturer SHALL provide a record of all user selections that must be made during software/firmware installation for the voting system to meet the requirements of the VVSG.

Applies To: Voting system

DISCUSSION

Screen shots showing the installation actions may be helpful.

Source: [VSS2002] I.4.1.1

3.8-C TDP, post-install configuration

The manufacturer SHALL also submit a record of all configuration changes that must be made to the software/firmware following its installation for the voting system to meet the requirements of the VVSG.

Applies To: Voting system

DISCUSSION

Screen shots showing the configuration actions may be helpful.

Source: [VSS2002] I.4.1.1

3.8-D TDP, configuration data

The manufacturer SHALL submit all configuration data needed to set up and operate the voting system.

Applies To: Voting system

Source: New requirement