The WinVote video

Just after the November 2008 election, a video appeared on YouTube showing someone playing with an AVS WinVote DRE voting machine in my home county (Fairfax Virginia), and showing that under certain circumstances it records incorrect votes, and even flips selected candidates.

I was recently asked for comments on the video, so I thought I'd post them here too.

My reaction (aside from the amateur filmmaking) is that while it tooka bunch of fiddling before the machine failed, the fact that the failures occurred show that under some circumstances the WinVote will fail to get the voter's intent - i.e., there's unquestionably one or more bugs in the software that can trigger under certain circumstances and cause incorrect candidate selection. The question is whether it would also fail under normal circumstances - once you've established that the bug occurs in an unrealistic situation, one has to ask can it also occur in a realistic situation. Given the minimal level of testing done in the Federal and state certification processes, it would be highly unlikely to be detected as part of certification. I don't know whether Fairfax County's testing is thorough enough that it might have been found there.

The more interesting question is whether a voter (other than the one who made this video) ever encountered this problem by accident, and believing it to be their mistake never reported it (or reported it to a pollworker, who never reported it to the Fairfax County office). [Or perhaps it got reported in another state that used the WinVote, but the word never got to the Fairfax County Board of Elections.

And the other interesting question is whether this bug, when triggered, causes incorrect recording of the candidate choices.... if the voter doesn't notice that it switched candidates on them, does the vote recorded in memory reflect what is being shown on the screen or what the voter had actually selected before the flip? The answer isn't obvious...

Cloud Computing - all that's old is new again

Cloud computing is the buzzword du jour. What amazes me most is how little people realize that it's nothing new.

From about 1978 until ultimate cancellation in 1986, AT&T ran a project called "Net 1000" (codename: ACS or Advanced Computing System). This was the first product AT&T released as part of the deregulatory process.

Managing projects in telecommunication services by Mostafa Hashem Sherif describes Net 1000 as follows: "The service consisted in providing customers with the capacity to develop, install, and manage applications software to run on AT&T's owned processors. The architecture was based on having a large number (100-200) of dispersed data centers (caled "service points"). These were interconnected using an X.25 packet switched network from the regulated part of AT&T. Initially, data centers were built in New York, Chicago, Los Angeles, Greensboro, Salt Lake City, Camden, Kansas City, and San Antonio [...] A Network Operations Center was constructed in Somerset, NJ. [...] The idea of Net 1000 was for users to pay for what they use. They wer charged for network terminations (ports), disk storage, transmission bandwidth, connection time, and communications process." (Page 79)

Sherif continues on page 81 that problems with the business included "the absence of application software and overlook[ing] the time needed to develop, test, and deploy software applications, particularly in a new operating environment."

Later on, AT&T changed the direction for Net 1000, and it ceased to be an application hosting infrastructure. But that's another story.

AT&T lost more than $1B on Net 1000. Yes, that's billion.

Certainly there are significant differences between cloud computing and Net 1000 - AT&T was trying to sell both the network communications and the applications platform, while cloud vendors are using the existing network infrastructure. And of course computer equipment is much cheaper - at the time I worked on Net 1000, the VAX 11/780 computers used as the application hosting platforms cost about $200,000 each, and operated at a speed comparable to about 1 MHz (vs. 2+GHz for a typical laptop today). Databases are a lot more mature too - the Net 1000 product was built on a DBMS called "Seed", which I think was written in FORTRAN, with a COBOL layer built on top of that. (We looked at a startup called Oracle but their products weren't mature enough to use for a nationwide offering!)

I'm not predicting that cloud computing will go the way of Net 1000. Just saying that all that's old is new again.

Election Day 2009 - report from the trenches

What, it’s election day again? Yes, Virginia, there is an election this year (state and most local candidates are elected in odd numbered years). Today was the Democratic primary– it’s an open primary to select Governor and Lieutenant Governor candidates, and in some places to select candidates for the House of Delegates. (The Republicans picked their candidates at a convention last month – in Virginia, it’s up to the parties whether to select candidates by convention or primary. The third statewide office, Attorney General, only had one candidate on the Democratic side so it wasn’t on the ballot.)

My precinct in Fairfax County has 1975 registered voters (small for this county), of whom 146 showed up over the 13 hours the polls were open, and 6 others voted absentee. We had four pollworkers – a chief and three assistants (including me). We were using two AVS WinVote DREs, neither of which had an apparent problems (after the special election a few months ago with strange results, I checked the zero and end of day tapes carefully). I found the election interesting because it was so slow that I had a chance to observe all of the weird things that happen in nearly any election, but in a general election we’re too busy to notice.

1. One of the three candidates for Lieutenant Governor had withdrawn before the election, but after the ballots were approved. We had signs everywhere telling voters that, but he still got four votes in our precinct.

2. Several voters didn’t know it was only a Democratic primary, and wanted to vote for Republican candidates. I presume they undervoted.

3. One voter left without pressing the final “Vote” button. Local rules say that the vote is voided rather than cast.

4. Several voters seemed surprised that there were just two races on the ballot (as noted above, some areas also had a third race for Delegate).

5. One voter who didn’t have a driver’s license or similar ID tried to use a Visa card with a photo. Luckily, Virginia allows an affidavit as an alternative to an ID, so we didn’t have to decide whether a credit card is a valid ID.

6. One voter was listed as a permanent overseas voter who gets an absentee ballot automatically, so she had to vote a provisional ballot until the county can verify that she hadn’t already voted absentee.

7. One voter needed to vote curbside; the DRE was very easy to handle for that use. However, the rules in Virginia are such that I could carry it to the car by myself (without a second pollworker coming along), so I could have (theoretically) cast extra votes without anyone noticing – except that the count would have mismatched. We discovered when it was time to close the polls and fill out the final reports that we forgot to note the protective counter when the machine was carried out to the curb and back again – most likely because none of the pollworkers had ever done curbside voting before.

8. One voter said he had registered to vote in his high school in the past few weeks (which was probably after the deadline). I wanted to allow him to cast a provisional ballot (if for no other reason than to give him the feeling that his vote might be counted), but the chief for the precinct called the county which said they didn’t have him listed, so she sent him away.

9. One voter had trouble getting the touchscreen to respond to him. The problem seemed to be that he was balling up his fist and pushing the screen with his thumb, which probably caused his other fingers to touch the screen at the same time.

10. No one asked about the paper optical scan ballots we used in the fall general election, nor did anyone express concern about the reliability/accuracy of the DREs (other than my wife). Just a statement of facts, ma’am!

All in all, a thoroughly ordinary election, but one that reinforced the range of “unusual” activities.

Metric to English measurements - no more precision

Here's an excerpt from a CNN report about the airplane crash: "The part of the ocean where the debris and bodies have been found ranges between 19,685 and 26,247 feet (6,000 and 8,000 meters) deep. The search area covers 77,220 square miles (200,000 square km), an area nearly as big as the country of Romania."

Converting 6000 meters to feet doesn't change from 1 to 5 digits of precision, and similarly for the other numbers.

This happens in all sorts of reporting. Why isn't it taught as part of Journalism 101?

A little knowledge is a dangerous thing

Last week the Washington Post reported that a web site belonging to the Virginia Department of Health Professions was broken into, and that millions of records regarding use of controlled drugs had been at least potentially accessed by an attacker. The attacker claims to have encrypted the records with a key only s/he knows, and will not release the key without being paid a ransom.

Clearly this was a bad thing.

But here's where we get into "a little knowledge". The Washington Post reports that "Del. Joe May, an electrical engineer by profession and the House's resident expert on technology issues, wanted to know what security measures the hacker had to overcome to access the records." So far, so good. They then quote May as saying "It doesn't sound like the proper firewalls, the proper backing up, the proper security measures were in place, ... and the question is why didn't we go ahead and have VITA do it."

Unfortunately, the problem almost certainly wasn't an issue of firewalls or similar security measures - it's much more subtle than that, probably an application security flaw.

I served with Del May on a state commission on electronic voting issues some years ago, and learned that he's got a great understanding of the big picture, but doesn't understand the details. As an example, he insisted that it was impossible for someone to break into a voting machine because there's no source code that's publicly available. I'm sure that will come as a great surprise to the black hats who routinely reverse engineer products to find vulnerabilities and develop attacks, to the white hats at companies like Symantec and McAfee who reverse engineer the attacks to come up with protections, and the hundreds of millions of users who have to install patches to protect against the vulnerabilities that, in Del May's mind, cannot exist.

It's great that the legislature has technical members - this is very much in keeping with Thomas Jefferson's view of a citizen legislature. However, those members need to be aware enough of their breadth of knowledge to understand when it's time to call an expert. You don't ask an oncologist for an expert opinion on brain surgery, or vice versa. Del. May and the legislature need to ask for help when they need it.

Ironically, the article concludes "Paquette [the state technology director] said DHP had one of the most secure systems in state government, and that firewall systems and backups were operational at the time of the attack". If this is one of the "most secure" systems, I'd hate to see the others....

Too much information (for too little value)

I rarely go to the movies - I usually just wait until whatever I want to see comes out on DVD. But recently I wanted to see the This American Life live simulcast. When I bought my tickets, I was disappointed that I had to register in order to make the purchase - and very surprised that a required field in the registration is a birthdate.

I can see where if I were under 18 they might want a birthdate to verify what types of movies I'm allowed to see (although it should really control admission, not ticket purchase). But for those of us over 18, it's only useful for marketing purposes - and it's an unnecessary piece of personal information for them to have. Just another bit of data to put identities at risk...

The privacy policy says "Through customer surveys, subscriptions, and newsletter registration, our site may request users to give us contact, demographic and/or financial information (such as
their name, locale, gender, age, income level and email address). The demographic information is used, among other things, to enhance user experience so we can be more content specific." I guess they consider the birthdate to be demographic, although IMHO if that were the case they could do just fine with the year alone.

I've filed a complaint with their customer service department. Not surprisingly, I haven't heard back. I may file a complaint with their privacy person next.

How to guarantee bad passwords (part 2)

As I described in a recent post, overly complex password rules lead to hard-to-remember passwords that get written down. Well, I tried not to write it down, and promptly forgot it. So I called the helpful help desk person, who reset it for me to a random value, and had me reset the password. Other than the cost to the organization of having to have a person involved in password resets, so far so good.

But then the kicker: your new password can't have any two character sequence the same as any of your previous 9 passwords. Makes sense to some extent - you shouldn't be able to switch from "myDOGspot!!" to "myDOGspot??". However, it means that in order to do this check, they're almost certainly not storing one-way hashes of the passwords (as good security engineers do), but rather the original passwords. The help desk person assured me that the passwords are encrypted, so there's nothing to worry about. So even if someone breaks into their site, all they'd get is the encrypted passwords.... Of course, if someone figures out how to get in below the level of encryption, then game over.

Moral of the story: "improving" security by strong passwords can backfire in many ways, by causing users to write them down, having servers that store the original passwords, etc.

Disaster recovery

Bruce Perens has an excellent analysis of the recent California bay area telecom outage, which showed the level of interdependency among systems. Anyone who thinks cloud computing is a panacea, especially for government services (some of which are needed in emergencies) needs to understand the implications of this outage, and what attackers may have learned from it.

Well worth reading.