Ciber Not Given Interim Accreditation, Part 2

The story in the New York Times that Ciber Labs has been barred from certifying election equipment under the EAC interim certification rules simultaneously stuns me and frustrates me. That Ciber has not been performing the work it was contracted to do is very old news to me. It is somewhat frustrating to see this breathlessly reported as something new. The thing that stunned me, though, was that the Ciber portion of the Ciber/Wyle team failed to meet the requirements of the EAC interim accreditation. How can that be?

In this, the second of two articles, I will discuss the poor workmanship of the Ciber/Wyle team, which I documented during the summer of 2005 here in Wisconsin. The first of these two articles discussed the interim certification process of the EAC and questions how it was even possible for Ciber to fail in its bid for certification.

NASED/EC Independent Test Authorities

Ciber labs is part of a network of three testing laboratories called Independent Test Authorities (ITA). For 10 years, under the auspices of the Election Center (EC) and then later under the those of the National Association of State Election Directors (NASED), the vendor-funded research conducted by the ITA labs has been cited as evidence that voting machinery sold in the United States is fit for use in election administration. The statement has been repeated many, many times but two specific examples are illustrative.

November 4, 2003 of Market Watch from Down Jones.

Mark Radke, director of the Diebold Election Systems unit, rejects charges that the company's systems are flawed or that politics could affect how it writes software. The company's equipment is certified by election officials before use, he said, and is subject to rigorous testing before, during and after the voting process

January 2005 from News Forge:

North Carolina's Board of Elections relied on reports from the Independent Testing Authority (ITA), which has OKed systems and software that have been problematic in the past.

The primary complaint with the independent test authorities has been that they are not. Vendor-funded labs operating under strict non-disclosure agreements (NDA’a) are not independent. The qualifications and authority of Shawn Southworth of Ciber (a company previous known as Nichols Research, PSI Net, and Metamor) has been repeatedly questioned. And there are years of evidence showing that the expected testing has not been done. Not independent, Not an authority. Not doing the testing, this is why Independent Test Authorities are not.

A History of Failing to Test

As I stated there has been much evidence dating back to 1997 that the "ITA" labs have not been testing voting software well. Here is an abbreviated list.

November 1, 2004 from Wired News

Last year [2003], computer scientists found that the Diebold system still possessed the same flaws Jones had flagged six years earlier [in 1997], despite subsequent rounds of testing.

December 22, 2005 from S&R News

One study this July tested 96 Diebold TSx DREs with AccuView printers and logged 34 separate system failures. The machines were tested for “5.33 hours in a setting designed to emulate a real election,” according to a report by the Voting Systems Technology Assessment Advisory Board. “The 34 failures broke down into 14 printer jams and 20 software failures. ... For some of the failures, the machine reported a fatal error and was unable to proceed. Other failures left the machine stuck, hung, or frozen in some state and unresponsive to voter input.”

Research performed on behalf of the Secretary of State of Ohio in 2003 and later in 2005 found numerous problems, all of which were missed by the vendor-funded "ITA" labs.

Research by the State of California in September, 2004 also found problems missed by the Ciber/Wyle team of "ITA" labs. A later study published on February 14, 2006 by the California Voting Systems Technical Advisory and Assistance Board (VSTAAB) stated:

“[The system] had not been subjected to thorough testing and review by” the national ITA which had approved the system in 2005.

No discussion of the "ITA" labs failure to test voting machine properly would be complete without mentioning ‘interpreted code’ -- which allows uninspected, ad-hoc programming to control the software. Interpreted code is expressly banned by both the 1990 and 2002 standards for voting systems, yet for years the test labs filed to detect, or failed to report ,the presence of this prohibited code. The dangers of the presence and use of this prohibited form of programming were demonstrated in Leon County under election night conditions on December 13, 2005 by Harri Hursti, testing the system at the request of the Leon county Supervisor of Elections. In this demonstration, the interpreted code misrepresented the election results so that the correct result of 6 No and 2 Yes was misreported as 1 No and 7 Yes.

Vote Trust USA has proposed a different, more effective testing frame work because of the failure of the NASED ITA testing process.

The Wisconsin Experience

While it is interesting to note the "ITA" testing failures from around the country, the main question is how does this affect us, the voters? The Elections Board of the State of Wisconsin (WI SEB) relies heavily on the assurances from the ITA labs. This reliance is so heavy that Wisconsin does no state-based testing other than to hold a mock election for each of the three types of Wisconsin elections: Partisan Primary, General Election, and Presidential Primary Election. There is no independent review such as performed by California, Ohio, or Pennsylvania. The reason is quite simple. The WI SEB does not have the funds to have such testing done on its behalf.

Because of the experiences with the Ciber/Wyle team of "ITA" labs and NASED system N-01-06-22-22-001, several members of the board mentioned in the November 30, 2005 meeting that while the ITA process was clearly suspect, the board was in no position to rectify the situation.

What led them to their conclusion? It was my attempts to verify that the Wisconsin certification process was being followed. The state certification process is documented in the administrative rules found in ElBd 7. Here is a short chronology of my investigation from 2005.

I requested the "ITA" reports delivered by the vendor to the State Elections Board as required by ElBd 7.01.

Details here

I was told "ITA" reports are a “trade secret” and thus exempt from disclosure under Wisconsin’s Open Records act.

Details here, here, here, and here

A redacted version is made available and it is discovered the ITA report is incomplete and is missing elements required by the 2000 VVSG.

Details here and here

The issues regaring the incomplete and missing reports were taken up by the board.

Details here, and here.

While the State Election Board denied Diebold's application on November 30, 2005, the system was later approved for use during 2006.

What I discovered in the "ITA" reports submitted to the State Elections Board by the State of Wisconsin were:

1. Neither Ciber nor Wyle provided in any of the reports the system identification as required under section 8.7.1 of Volume I of the 2002 VVSG. This system identification is called a physical configuration audit (PCA) and the inclusion of the PCA is required by Appendix B of the 2002 VVSG. This means it is impossible to tell which system the reports apply to.
2. Wyle labs created the ITA report for the application called "VC Programmer". The problem with this is that Wyle labs is only authorized to test hardware and the firmware executed on that hardware. VC Programmer is an application program which runs on a standard desktop computer.
3. Neither Wyle nor Ciber produced a report for the application called "JResultsClient".
4. The Ciber report was so short and incomplete it is impossible to tell what if any testing was done or more importantly how such testing was done.
5. The NASED number, N-1-06-22-22-001, for the systems was issued on June 27, 2005 but the ITA reports were not completed until August 4, 2005. This means the NASED number was issued before the NASED Voting Systems Board had recieved the test results.

In conclusion, the failure of the NASED ITA system in general and the particular failures of Ciber Labs (in all of its prior incarnations) to perform adequate testing of voting machines is a problem stretching back for more than a decade. This means that voting systems have been approved by "ITA" labs and state election officials have relied upon those approvals. Because of this elections have been held using voting equipment which has never been adequately tested.

How this actually may have affected us, the voters, is anybody's guess.