|Truth, Lies, & Politics
When is a coincidence too much of a
coincidence to be one?
LANI: "Testing voting systems should and can be reliable and
ELLEN: Should, yes, but not can.
Bear in mind that testing would have to be done, reliably and
extensively, before every election on every machine that was newly
programmed for a new election.
LANI (responding): When you say "newly programmed," are you
talking of the ballot parameters input into the machine that define the
ballot for each election? If so, the software program remains the same
as that tested and certified by the Secretary of State. The state's
certification process should be intensive enough to test for the most
complex ballot configurations and voter behavior.
ELLEN (responding): Some emails that Jody Holder got hold of in an
open records request revealed that, with ES&S optical scanners (at
least), the ballot programming is merged into the basic executable code
to generate brand new software for the election. It isn't just putting in
John Washburn wrote a terrific piece about it where he used a paint
analogy - which fits perfectly. He pointed out that the source code is like
yellow paint and the ballot programming is blue paint. In the ES&S
system, the two are inextricably mixed, and the final software is green.
So it isn't the same software as the certified software at all. And if
there's different ballot programming for different precincts, then it isn't
even same software on all the county's machines.
LANI: Another, "Yes, but." The base line is the same. The base source
program code they started out with SHOULD have been the same base
program code that was certified. So in effect the only new piece is the
ballot, not the base program code that counts the votes. In other words,
some computer programmer somewhere is not writing new source code
for every election.
ELLEN: What if ES&S does the ballot programming? - as they do all
over the country? Aren't they producing new software for every
ELLEN (continuing): And the testing of the new program would have to
be done by an election administrator who has no experience in testing
any type of equipment -- let alone software -- and no understanding of
what such testing involves. (Perhaps there are a couple of exceptions
out there somewhere.)
LANI: I do not believe election administrators should have to be IT
experts. They're administrators not programmers. If they wanted to be
computer programmers, they would have gone to geek school. (And
even if they were IT experts, they would never know the election
program code and would not initially be familiar with their election
system, until they became familiar with it. Hmm.)
However, election administrators are indeed users of the system and
they should know their system from a user perspective. If their system
is too complex, too cumbersome to know and use effectively, it's a fault
of the system.
Your statement, "While it may be their jobs, they don't have the
necessary qualifications," is troubling. In a perfect world none of us
would be assigned jobs for which we lack access to the tools, budget,
experts necessary to be successful in that job. Easy to say, "Hire the
best people." Not so easy to do when you've no budget, training,
ELLEN (continuing): AND, if the jurisdiction has DREs, for the testing
to be reliable and extensive, every single machine would have to have a
huge number of ballots input by hand -- not the automated scripting that
only tests that the software can read the other software. To test DREs
correctly would be prohibitive. Especially, when you consider that an
error in the results on one machine would have to be investigated, which
means reviewing the video of the person inputting the hundreds of
ballots by hand on the touch screen (or push button).
LANI: Back to "same hardware, same software." If the election
administrator PROVES the ballots work on a sampling of like DREs,
they "should" work on other like DREs. However, all machines should
be functional, whatever process the election administrator uses to prove
the screens are calibrated, the correct ballot is loaded, the vote count is
zero, and ...
Yes, a huge number of ballots should be input by hand testing all
possible voter behavior, page forward/backward, single/double click
(GAO's word not mine).
ELLEN (continuing): Of course, it's more feasible to test the opscans,
but even there each one would have to be tested -- not just a
representative sample. You could use many of the same ballots in each
deck, but some large counties have hundreds of scanners. Even to run
the same deck through all these scanners and compare the results with
the expected results would take quite a chunk of time. So, they don't do
LANI: Same as above.
ELLEN: But just because the read heads in one opscan are working
fine, how do you know the read heads in all of them are? (LANI:
Exactly. You don't.) Maybe they got dirty in a few and not the others.
Maybe a sensor or a light is wearing out in some and not the others.
Maybe the rollers or the ballot alignment thingy ..... (Actually, I don't
know the critical parts, so I'm sort of making up these part names.)
IF the state certification is sound, then using John Washburn's testing
guidelines to build a test deck and executing it on every opscan would
be pretty close to reliable.
LANI: I like this idea.
ELLEN (continuing): But using his guidelines to test every DRE would
be virtually impossible -- especially when you add the
page-forward/page-back aspects you pointed out.
I think my real point is that IT NEVER IS "same hardware, same
software." (I got a terrible Dell computer once, but other people love
LANI: And components break at different rates, which is why it is so
troubling to me that we don't have better "it's a bad election" processes
in place. You get a tainted read head and so goes the election.
The certified software should be the same on all machines, as should the
ELLEN (continuing): Election administrators don't do the necessary
testing, partly because they have no idea how to do it right, and partly
because if they did know how, they would realize they didn't have the
Instead, they fall back on faith in federal testing and state certification
-- with ABSOLUTELY NO UNDERSTANDING that their particular
election makes everything new and different, that all previous testing is
virtually irrelevant, and the testing has to be started and done
thoroughly all over again.
LANI: IF the state certification is sound, all other things being equal
only their ballot(s) changes. I don't intend to minimize the job of defining
and testing the ballot(s). It's enormous. But, if you're starting on a level
playing, one in which you know your machines work, then proving your
ballot is less the impossible dream and more within the realm of your
expectations and control.
ELLEN (continuing): This is exactly the reason why it is indisputable
that every election is a beta test of the election equipment, with no
reporting mechanism in place to make it a respectable beta test.
LANI: Your comment, "every election is a beta test," is indeed the core
of the problem.
Two identical computers containing identical hardware and identical
software will perform identically until something breaks or is altered.
The Secretary of State certifies those computers (hardware and
software) as working and acceptable to the task of correctly counting
votes, using variable ballot input parameters. That means every time:
when the ballot is intrinsically complex, when there is record turnout,
when there's power fluctuation or failure, and so on. In the case of the
DREs it means even when the voter changes the ballot and pages
forward and backward innumerable times. It is the job of the Secretary
of State to prove the machines will not botch the votes.
The election supervisor/election administrator "should" be able to be
confident that rigorous, comprehensive acceptance testing has occurred.
The election supervisor "should" be able to assume the machines
he/she acquires perform to the quality certified by the Secretary of
However, the election supervisor (ELLEN: More often, the vendor)
creates and inputs his/her ballot parameters for every election. The
election supervisor must scrupulously test that ballot. In Sarasota 2006,
97% percent of the possible Election Day ballot data combinations were
not tested. Testing absolutely every ballot data combination would not
be possible. Testing the more likely combinations is reasonable and
responsible and absolutely doable.
Should every machine be load tested prior to each election? If all
computers are identical, I don't believe this is necessary. However, one
extreme is Miami-Dade's SoE in 2000 who didn't clean the chads out of
his ballot stands (one obstacle to punching the chads out cleanly).
Another is Sarasota 2006 with the (un)calibrated touchscreens and the
smoothing filter and the warning memo that went unheeded.
The election supervisor should expect the state to do its acceptance
testing job during certification. However he/she cannot and should not
expect that nothing will ever go wrong with the ballots or the machines.
Each and every machine should be proved working prior to being used
ELLEN: Hmm, it feels like we're having an argument -- or a debate. But
I'm not sure what it's about. We seem to agree on an awfully lot. Is our
basic point of disagreement in the subject line ["Testing should and can
be reliable"]? I agree it should be reliable, but not that it can be. Or do
we even disagree?
LANI: Ellen, Absolutely not, no, neither. Neither argument nor debate,
more like peeling the proverbial onion. There is layer upon layer of
complexities and systems and processes and people responsibilities. On
any given day, we might all of us say something completely different but
all be absolutely correct.
I don't believe we disagree. On the "testing should and can be reliable,"
I believe, I know we can and should do more. I look at this from the
technical perspective, knowing that in a "perfect world" all systems are
quality tested to a fault before being shipped out the door. Also knowing
that, depending on the corporation and circumstances (your Dell), some
glitches fall through the cracks.
"Cracks" not canyons. Now comes Florida. Consider what might have
been IF Florida's Secretary of State had performed a thoughtful
analysis of voting machine requirements before sanctioning the
touchscreens in the first place. You know, the part about counting all
votes, reliability, stability, auditability, proof.
Consider what might have been IF Florida's Secretary of State had
performed rigorous, comprehensive testing before certifying the
touchscreens. (Remember the GAO reported they didn't even touch the
touchscreens when performing load testing. Plus they used ES&S test
Consider, in the event Florida's Secretary of State's scrupulous testing
failed to uncover the touchscreen calibration problem, what might have
been in 2006 had Sarasota's supervisor of elections performed a more
thorough testing of her ballot and performed a cursory sniff-test on the
ELLEN: Yes, peeling the onion. I feel like I've been doing that for the
past five years. Waaay too slowly.
Five years ago, I worked really, really hard to get additional
co-sponsors to HR.2239 - Holt's first vvpat bill, which required a
whoppin' 0.5% spot check (which the bill called an "audit"). And we got
a LOT of co-sponsors -- nearly enough to call the bill out of committee
for a floor vote. Hurrah for us!
But since then I've been reading, learning, stopping and thinking, and
absorbing information passed on by others:
Teresa Hommel on the necessity of observability. ("Oh My Gosh, of
course," say I)
Bev Harris on the real meaning of audits ("OMG, of course," say I)
Nancy Tobi on the EAC ("OMG, of course," say I)
Pokey Anderson on the incomparable incentive for stealing an election
("OMG, of course," say I)
... and the list could go on and on.
Fiction Stops Here
An email dialog between Lani Brown (author of "A Margin of Error, Ballots
of Straw") and Ellen Theisen (founder and Co-Director of VotersUnite.Org).