How will the program be evaluated to determine if the money was effectively utilized?

1. Quality of data.  One approach would be to randomly select some precincts in each of the participating states and to verify the accuracy of the data with phone and mail follow-ups.

This evaluation is primarily focused on the completeness of the data and how well states were able to perform with respect to their data collection efforts. Data quality is important and information about quality will likely surface, as states will be expected to consider this as they attempt to generate complete sets of data.

2. Completeness of data.  Have the states provided all of the information that they have committed to provide?

The completeness of the states’ data is a part of the evaluation plan. Please see page 1, 4, and 5 of the evaluation plan.

3. Speed.  How long did it take for the states to collect and process the data?

  This will be addressed through various aspects of the evaluation design (e.g., achievements and challenges). Please see page 2, 4, and 5 of the evaluation plan.

4. Follow-up.  While the focus of the project is data collection, one would hope that individual states would follow up if they observed anomalies.   For example, if an abnormally large number of undervotes were registered in a particular precinct, will the state investigate to determine if those undervotes might have been caused by a technical failure of the voting machines, a poorly designed ballot, or something else?  If not, what is the value in obtaining this information?

These issues are beyond the scope of this evaluation; however, it is possible that such issues will surface in discussions about the achievements and challenges of data collection. Please see page 2, 4, 5, and 6 of the evaluation plan

Will states track voting machine failures, both in terms of numbers and times that the machines were unavailable?  Will states do the same for poll books and other technologies related to voter registration databases?  If not, how will we be able to determine if the information that is collected has been impacted by technological failures?

These issues are beyond the scope of this grant program evaluation.

It would be very useful if all of the data that is collected could be presented in a uniform fashion to facilitate analysis and evaluation.  This has been recognized by some of the proposals that discuss the use of XML (Extensible Markup Language).  XML is a general purpose markup language that allows people to create tailored versions for their specific needs.

However, there already is a widely recognized markup language for elections, namely the Election Markup Language (EML).  EML was created by an international task group operating under OASIS (Organization for the Advancement of Structured Information Standards).  This task group included election officials, representatives of vendors, and academics.  I've been told that at least some vendors, and perhaps all of them, would like to have a common language.  Using a common language would facilitate analysis by election officials and academics, as well as concerned citizens.

In addition to ease of use and interoperability, using EML would save states the cost of developing yet another markup language.  This money could be better put to other uses.

EAC is aware of the XML/EML formats available to states. Information regarding the states’ chosen formats will occur as part of the discussion around data collection methods. Please see page 4 of the evaluation plan.

Once the analysis of the data is complete, what is the next step? Will the EAC compile best practices or guidelines?

 The EAC will issue a report to Congress by June 30, 2009. The report will include recommendations and best practices.

Does the EAC or Congress plan to propose legislation based on the results to make the State’s data collection processes uniform?

 The EAC does not propose legislation. However, the EAC will closely consider the findings of this grant program as it contemplates recommendations and suggestions for data collection.

Is there a specific approach to data collection that is preferred by the EAC?

 At the present, there is not. However, the EAC will closely consider the findings of this grant program as it contemplates recommendations and suggestions for future election data collection.

What are the consequences for test States that stray from the recommended course of action?

 This is a grant program in which one of the goals is to learn which of the various approaches being “tested” work and which do not. There are no consequences. The EAC will offer recommendations based upon the grant program findings.

In reviewing the timeline, it seems unreasonable that the states would be able to implement such ambitious goals (in some cases, overhauling the elections management system, or, in Minnesota’s case, 11 different sub-projects) in time for the November 2008 election. Is this a concern? Has work already begun on these projects? According to the timeline, initial contact with grantees is to be completed by September 19, 2008.  Perhaps I am missing this in the documents and if so please disregard.  However, the timing is still a concern if the states must conduct an overhaul of their elections management system.

All of the states that applied for the grant program and were eventually awarded a grant were aware of the time constraints and limitations. The EAC will use what the states proposed in their applications and monitor their successes or challenges based on their indicated timelines. The nature of this pilot program is to learn what works and what does not work. Thus, if a state’s timelines turn out to be too ambitious, the results of the evaluation will indicate this.