The calls started flooding in from hundreds of irate North Carolina voters just after 7 a.m. on Election Day in November.
Dozens were told they were ineligible to vote and were turned away at the polls, even when they displayed current registration cards. Others were sent from one polling place to another, only to be rejected. Scores of voters were incorrectly told they had cast ballots days earlier. In one precinct, voting halted for two hours.
Susan Greenhalgh, a troubleshooter at a nonpartisan election monitoring group, was alarmed. Most of the complaints came from Durham, a blue-leaning county in a swing state. The problems involved electronic poll books — tablets and laptops, loaded with check-in software, that have increasingly replaced the thick binders of paper used to verify voters’ identities and registration status. She knew that the company that provided Durham’s software, VR Systems, had been penetrated by Russian hackers months before.
“It felt like tampering, or some kind of cyberattack,” Greenhalgh said about the voting troubles in Durham.
There are plenty of other reasons for such breakdowns — local officials blamed human error and software malfunctions — and no clear-cut evidence of digital sabotage has emerged, much less a Russian role in it. Despite the disruptions, a record number of votes were cast in Durham, following a pattern of overwhelming support for Democratic presidential candidates, this time Hillary Clinton.
But months later, for Greenhalgh, other election security experts and some state officials, questions still linger about what happened that day in Durham as well as counties in North Carolina, Virginia, Georgia and Arizona.
After a presidential campaign scarred by Russian meddling, local, state and federal agencies have conducted little of the type of digital forensic investigation required to assess the impact, if any, on voting in at least 21 states whose election systems were targeted by Russian hackers, according to interviews with nearly two dozen national security and state officials and technology specialists.
The assaults on the vast back-end election apparatus — voter-registration operations, state and local election databases, e-poll books and other equipment — have received far less attention than other aspects of the Russian interference, such as the hacking of Democratic e-mails and spreading of false or damaging information about Clinton. Yet the hacking of electoral systems was more extensive than previously disclosed, the New York Times found.
Beyond VR Systems, hackers breached at least two other providers of critical election services well before the 2016 voting, said current and former intelligence officials, speaking on condition of anonymity because the information is classified. The officials would not disclose the names of the companies.
Intelligence officials in January reassured Americans that there was no indication that Russian hackers had altered the vote count on Election Day, the bottom-line outcome. But the assurances stopped there.
Government officials said they intentionally did not address the security of the back-end systems, whose disruption could prevent voters from even casting ballots.
That’s partly because states control elections; they have fewer resources than the federal government but have long been loath to allow even cursory federal intrusions into the voting process.
That, along with legal constraints on intelligence agencies’ involvement in domestic issues, has hobbled any broad examination of Russian efforts to compromise U.S. election systems. Those attempts include combing through voter databases, scanning for vulnerabilities or seeking to alter data, which have been identified in multiple states. Current congressional inquiries and the special counsel’s Russia investigation have not focused on the matter.
“We don’t know if any of the problems were an accident, or the random problems you get with computer systems, or whether it was a local hacker, or actual malfeasance by a sovereign nation-state,” said Michael Daniel, who served as the cybersecurity coordinator in the Obama White House. “If you really want to know what happened, you’d have to do a lot of forensics, a lot of research and investigation, and you may not find out even then.”
In interviews, academic and private election security experts acknowledged the challenges of such diagnostics but argued that the effort is necessary. They warned about what could come, perhaps as soon as next year’s midterm elections, if the existing mix of outdated voting equipment, haphazard election-verification procedures and array of outside vendors is not improved to build an effective defense against Russian or other hackers.
North Carolina went for Donald Trump in a close election. But in Durham County, Clinton won 78 percent of the 156,000 votes, winning by a larger margin than President Barack Obama had against Mitt Romney in 2012.
While only a fraction of voters were turned away because of the e-poll book difficulties — more than half the county cast their ballots days earlier — plenty of others were affected when the state mandated that the entire county revert to paper rolls on Election Day. People steamed as everything slowed. Voters gave up and left polling places in droves — there’s no way of knowing the numbers, but they include more than 100 North Carolina Central University students facing four-hour delays.
At a call center operated by the monitoring group Election Protection, Greenhalgh was fielding technical complaints from voters in Mississippi, Texas and North Carolina. Only a handful came from the first two states.
Her account of the troubles matches complaints logged in the Election Incident Reporting System, a tracking tool created by nonprofit groups. As the problems mounted, the Charlotte Observer reported that Durham’s e-poll book vendor was Florida-based VR Systems, which Greenhalgh knew had been hacked earlier by Russians.
“Chills went through my spine,” she recalled.
The vendor does not make the touch-screen equipment used to cast or tally votes and does not manage county data. But without the information needed to verify voters’ identities and eligibility, which county officials load onto VR’s poll books, voters cannot cast ballots at all.
Details of the breach did not emerge until June, in a classified National Security Agency report leaked to the Intercept news site. That report found that hackers from Russia’s military intelligence agency, the GRU, had penetrated the company’s systems as early as August 2016, then sent “spear-phishing” e-mails from a fake VR Systems account to 122 state and local election jurisdictions. The e-mails sought to trick officials into downloading malicious software to take over their computers.
The NSA analysis did not say whether the hackers had sabotaged voter data.
“It is unknown,” the agency concluded, whether Russian phishing “successfully compromised the intended victims, and what potential data could have been accessed.”
VR Systems’ chief operating officer, Ben Martin, said he did not believe that Russian hackers were successful. He acknowledged that the vendor was a “juicy target,” given that its systems are used in battleground states. But he said that the company blocked access from its systems to local databases, and employs security protocols to bar intruders and digital triggers that sound alerts if its software is manipulated.
The idea of subverting the U.S. vote by hacking election systems is not new. In an assessment of Russian cyberattacks released in January, intelligence agencies said Kremlin spy services had been collecting information on election processes, technology and equipment in the U.S. since early 2014.
The Russians shied away from measures that might alter the “tallying” of votes, the report added, a conclusion drawn from U.S. spying and intercepts of Russian officials’ communications and an analysis by the Department of Homeland Security, according to the current and former government officials.
The January intelligence assessment implied that the Russian hackers had achieved broader access than has been assumed. The report said that the Russians had “obtained and maintained access to multiple U.S. state and local election boards.”
Two previously acknowledged strikes in June 2016 hint at Russian ambitions. In Arizona, Russian hackers successfully stole a username and password for an election official in Gila County. And in Illinois, Russian hackers inserted a malicious program into the Illinois State Board of Elections’ database. According to Ken Menzel, the board’s general counsel, the program tried unsuccessfully “to alter things other than voter data” — he declined to be more specific — and managed to illegally download registration files for 90,000 voters.
On Election Day last year, a number of counties reported problems similar to those in Durham. In North Carolina, e-poll book incidents occurred in the counties that are home to the state’s largest cities. Three of Virginia’s most populous counties and Fulton County, Ga., which includes Atlanta, and Maricopa County, Ariz., which includes Phoenix, reported difficulties. All were attributed to software glitches.
Sen. Mark Warner, D-Va., vice chairman of the Senate Intelligence Committee, argued for more scrutiny of suspicious incidents. “We are not making our elections any safer by withholding information about the scope and scale of the threat,” he said.