Keeping your vote safe and secure: A story from inside the 2020 election
The concept of the dark web has become a trope in TV shows and movies, with villains conducting their business in the shadowy corners of the internet.
It turns out that fiction isn’t far from reality.
The heroes of the 2020 election were officials on the front lines, and teams at Microsoft wanted to do all they could to support them.
Security analysts are constantly monitoring the dark web for potential threats. At Microsoft, scans run by teams like Cloud and AI Security and M365 Security reach into those places on the dark web known to be frequented by hackers and cybercriminals, searching for mentions of domain names related to the company and its customers. These can include comments referring to customers by name or security credentials being sold; these scans return millions of possible hits.
The Defending Democracy Program was announced by Microsoft in the lead up to the 2018 midterms to support customers who work on election-related technology. And this team knew scanning for threats would be a crucial capability for the U.S. elections community in 2020.
Several times a day, this team would scrutinize reports, filtering them against key words and criteria related to election preparations. Mostly, they were false positives: old, out-of-use credentials that posed no concern.
That was regular work for the team, until one day what looked like another false positive turned out to be anything but.
The login details for a well-connected and highly visible government staffer working on elections in a key battleground state were for sale on the dark web. To cybercriminals, usernames and passwords like these are the keys to the castle, allowing them to take control of social media or email accounts to steal information or spread disinformation.
Security analysts, alerted to the potential dangers, alerted the Microsoft team, who quickly rallied to verify the nature of the threat, identify the fastest way to contact the state’s chief information officer, and neutralize the problem – all within an hour.
This is just one example of the work across the company to help protect those on the front lines of the 2020 election.
An election amid a pandemic
Next to the global Covid-19 pandemic, few stories dominated headlines in 2020 like the U.S. election.
It may seem that every story about Americans going to the polls last November has been told. But this is another, lesser-known tale, about work that was undertaken to help protect those who were on the front lines of planning elections and reporting results. It’s a story that begins more than a year before polling day and involves months of behind the-scenes preparation undertaken by hundreds of people across the country.
Teams of state officials, providers of election technology and Microsoft’s experts worked together to fend off cyberattacks and ensure that the company supported those responsible for running the elections. They kept voter registration websites safe and enabled a reliable exchange of vital information. And they safeguarded reporting of the final results.
The smooth running of the election went further than the vote itself, according to Frank LaRose, Secretary of State for Ohio. “There’s a success story here,” LaRose says.
Lessons from the past
In the summer of 2016, the Democratic party suffered high-profile hackings and email leaks. In 2018, the city of Atlanta was hit by a ransomware attack that demanded a payment of Bitcoins to the value of around $52,000. Efforts to remedy the attack are believed to have cost $2.6 million. In 2019, the IT systems of local government offices in 22 towns in Texas were hacked and subjected to ransomware attacks.
Cyberattacks like these raised alarms within government circles and private-sector companies. Ginny Badanes, Director of Strategic Projects at Microsoft and a member of the Defending Democracy Program, watched the interest in cybersecurity grow after 2016. “The conversations that elections customers wanted to have with us changed from using data visualization and machine-learning to the importance of things like multi-factor authentication,” Badanes says.
Building a network, one connection at a time
Fast forward to December 2019: That is when planning for the 2020 U.S. election began in earnest at Microsoft. Ethan Chumley, a Senior Cybersecurity Strategist and early member of the Defending Democracy Program, was tasked with considering the avenues adversaries might explore to undermine the 2020 elections in the United States – and to find people who could fight those threats.
With threats that include using ransomware to lock vote reporting systems and hacks that change electronically stored voter rolls, Chumley realized he would need a diverse team with a variety of skills to be successful. Early meetings brought together threat-hunters, the Microsoft Threat Intelligence Center (MSTIC), who track nation-states’ activities, and recovery and response teams who help customers quickly bounce back from attacks and setbacks.
Chumley also quickly realized that his team couldn’t do this alone. While working with other large technology companies and the Cybersecurity and Infrastructure Security Agency (CISA), might seem of obvious importance when securing the election, some of the avenues Chumley considered could be considered unconventional. He worked with the Xbox team to ensure comments in Xbox chat were monitored for violations – calls for violence at the polls, for example – and that any concerning comments were reported back to the Defending Democracy Program so appropriate action could be taken.
“We needed to work closely with federal and state election officials across the country, plus the many smaller technology vendors these officials employ,” Jan Neutze, Head of the Defending Democracy Program, says. “It also meant creating open lines of communication with other technology companies such as Google, Facebook and Twitter.”
Identifying and supporting customers
In many cases, the front line of defense against state-sponsored hacking and highly organized cybercrime is a local government official operating under extreme pressure. Identifying these officials and related customers was a priority. It would allow threat-hunting teams to better monitor for actors specifically targeting election infrastructure.
Many election-related activities are built on Azure, Microsoft’s cloud platform that, among other things, hosts critical activities, including voter registration portals. Being an Azure customer requires no formal relationship or contact with Microsoft, which meant important election-related Azure customers were effectively anonymous.
The discovery process involved teams within the company pooling their knowledge and contacts. It also meant calling upon relationships with external bodies like CISA to put the call out to the elections community across the U.S.
Another route to identifying relevant customers was through Microsoft’s extensive partner network. One example is BPro – an elections software and services business, which deals with voter registration, election management, information portals, campaign finance solutions and election reporting systems for states and counties, that runs on Azure.
That’s a long reach for a business of that size, putting BPro in a position of great responsibility. Ensuring the safety and security of customer systems during a time of unprecedented importance is no small undertaking.
“At a moment’s notice, during the election, just having Microsoft on the phone, helping us address any issues that come up is something that gives our customers peace of mind,” says George Munro, Government Outreach Director at BPro. “They know it’s not just these folks from South Dakota who are working on the issue, it’s those folks as well.”
In total, the elections team identified more than 2,000 Azure customers that were running election-related workloads on their systems, several which were then given a detailed security and resiliency audit by Microsoft. The audits generated reports outlining recommendations to boost performance in those two key areas.
Karen Intrachat, a Principal Program Manager within the Azure Customer Experience Team (Azure CXP), explains, “In our resiliency reviews, we might discover a customer is running their election reporting application on a single server,” she says.
Security in a world of infinite storage
Name: BPro Inc
Location: Pierre, South Dakota
Specialization: A software services company providing on-premise and cloud-based election solutions to states and localities across the U.S.
Aspects of everyday life require individuals to verify their identity with a signature. Over their lifetime, a typical U.S. citizen may sign dozens of official documents.
Those signatures are an important part of the identity validation process, and the more examples of an individual’s signature there are, the better.
Having a lifetime’s worth of signatures available can be a logistical challenge. It would effectively mean keeping scanned copies of potentially hundreds of official documents per citizen. The storage requirements for such an undertaking are significant.
“The systems we host in Azure have virtually unlimited storage capacity,” says George Munro, Government Outreach Director at BPro. “That means any official records with a signature can be captured by our systems and all those signatures can be used to verify a voter’s absentee ballot to ensure it is counted.”
This company also creates systems that pull data from sources like death registrations and help streamline the process of updating voter lists. The systems run on Azure to offer scalability to its clients.
Bogus sites, spam and disinformation
Not all threats were hidden away in the dark corners of the internet. An incident in the summer of 2020 showed that some threats existed in plain sight.
The elections team was alerted to a spoofed voter registration website that had been built to look genuine. It was being promoted through search engines and social media marketing, indicating the sophistication of the group behind it. Voters could easily have mistaken this site for the real thing.
As soon as the site was detected, the team members drew on months of careful preparation, and first worked with teams at Bing and LinkedIn to limit the spread of this bogus webpage. Its details were eventually incorporated into algorithms that scan the web for disinformation and used in Outlook spam filters so it could not be sent over email.
Training customers and election officials to recognize and respond to threats like fake voter registration sites was key to the smooth running of the election across the country.
Since 2018, CISA had been organizing trainings for elections officials to supplement their cybersecurity knowledge and skills, using a mix of tabletop exercises and in-person training. But the pandemic meant such activities had to take place virtually.
In early 2020, Microsoft and the Brennan Center for Justice at NYU School of Law created a series of online training sessions with CISA for election officials. This training ultimately reached election officials in 40 of the 50 states and helped them stay on track with their preparations.
“Something that’s unique, in terms of American elections, is that they are largely administered at a very local level. It’s very decentralized,” says Gowri Ramachandran, counsel with the Election Security Team at the Brennan Center. The center is headquartered in New York and has an additional presence in Washington, D.C.
“Over the past decade, and especially since the 2016 election,” she explains, running an election, “has turned into a job that encompasses cybersecurity, physical security, all of those sorts of additional things.”
Preparing for every eventuality
Location: New York; Washington, DC
Mission: “The Brennan Center for Justice works to build an America that is democratic, just and free – for all.”
With CISA and Microsoft, the Brennan Center delivered a series of training courses to county and municipal elections officials from across the U.S.
They were focused training activities that covered crisis planning, scenario training and confidence building, covering scenarios like what to do if voting machines stopped working, or if a ransomware attack happened on Election Day.
Liz Howard, Senior Counsel with the Election Security team at the Brennan Center, is a former election official who has seen first-hand the way security has become a key issue.
“This has been a tough year for many people for a variety of reasons. But our election officials have been an incredible source of inspiration and hope for so many of us,” Howard says. “When I was an election official, security was mostly about knowing who had the keys to the room in the basement where the voting machines were kept.”
She continues, “Through grit and amazing determination, they have been able to conduct one of the safest and most secure elections in our history, in spite of a pandemic, in spite of being under-resourced and in spite of any number of other challenges at the state and local level.”
Shared responsibility – the importance of patches, updates and customer awareness
Upholding cybersecurity isn’t a one-way street. All users have to assume their share of responsibility for maintaining the integrity of their system’s security infrastructure, particularly on a governmental level.
As Matthew Masterson, then Senior Cybersecurity Advisor at CISA, said when testifying in front of the House of Representatives Committee on Homeland Security in October 2019: “It will take continual investment from all levels of government to ensure that election systems across the nation are upgraded, patched and better secured, with older more vulnerable systems retired. These efforts require a whole-of-government approach.”
This was brought into sharp focus less than a year later with what is known in security circles as the ZeroLogon exploit. Hackers used it to gain elevated rights to people’s networks.
Sean Ensz is a Crisis Response Manager in the Microsoft Security Response Center (MSRC), and part of his role is to spot problems before they affect customers. “We issued warnings about this to state and local governments,” Ensz says. “To ensure they understood it was important to install the patch.”
But, despite Ensz’s warnings, not all of them took immediate action.
“A few weeks before the election, MSRC was engaged to help run a daily effort around threat-hunting which involved keeping an eye out for any potential vulnerabilities around the systems the identified election customers were using,“ Ensz explains.
Although Masterton’s testimony proved to be prophetic, Ensz’s team were able to spot system vulnerabilities and then collaborate with CISA to amplify their message about the need to activate patches and updates, thus ensuring systems were secure in the run-up to Election Day.
The Covid-19 effect
With the pandemic raging, Election Day was like nothing that had taken place before. Hoping to avoid crowds at polling places, more than 100 million ballots were cast during the early voting period. Several states, already familiar with processing mail-in ballots, found themselves facing a huge increase in their number. In 2016, Idaho had less than 200,000 absentee ballots; in November 2020, that number rose to almost 450,000.
The virus was also taking its toll on the state’s workforce, many of whom found themselves working remotely.
Brett Brandon, Idaho Secretary of States’s Cybersecurity Strategist, explains, “Old practices for cybersecurity regarded your firewall as the perimeter of the network.” Remote working led to people outside the firewall, outside of that perimeter, needing access to the network.
“We had to facilitate communications and make sure people and processes still worked, while enabling next-level security,” he says.
Remote working also meant that far reaching cultural and behavioral changes were needed, according to Foster Cronyn, Deputy Secretary of State for Idaho.
“We had to get people thinking differently about security, and the part they played in it,” he says. “Being more careful about their email, for example, or ensuring their computer would automatically go into sleep mode if left unattended.”
Idaho was just one of 50 states grappling with similar issues.
Adapting to change and embracing the future
Population: 1.78 million
Counties: 44 in total, some with just a few hundred voters and some with thousands.
In early 2020, while adjusting to remote working and coping with the impact of the pandemic on the health of some of its staff, Idaho embarked on a major IT upgrade project.
The state was migrating from its on-premise elections platform to a cloud-based system using Azure. With the election coming and the pandemic disrupting workflows, the Idaho team faced pressing deadlines.
“We had to find a way to go through that full migration, understand the security risks, and make sure they were remediated,” Brett Brandon, Cybersecurity Strategist in the state’s IT department, says. “This is where Microsoft had a huge part to play.”
The elections team examined the infrastructure and network in detail, supplying the Idaho team with details of any necessary improvements.
“It certainly helped us focus on what we needed to concentrate on,” says Foster Cronyn, the Idaho Deputy Secretary of State. “As smart and as educated as we can possibly be, it is essential to partner with individuals and companies that focus on specific areas of technology and to take their expert advice when it is offered in a critical situation.”
“We’d certainly participate in that kind of operation again, without even giving it a second thought.”
‘I don’t think we’ve faced a more complex or difficult election’
In Ohio, Sec. LaRose, who served in the 101st Airborne Division and the U.S. Special Forces, approaches cybersecurity with the mindset of being and staying prepared. In 2019, he was behind a security directive for Ohio’s election officials to keep cybersecurity top-of-mind.
“We’re always preparing for the next thing as it relates to cybersecurity,” he says. “I’m a big believer in checklists. It’s the kind of mentality that pilots have – even if you have thousands of flight-hours, you won’t take off unless you’ve completed your pre-flight checklist. Every pilot knows that. Something as important as the cybersecurity of our elections should be handled with that same level of care.”
Just as it had in Idaho, the pandemic caused further complications. “Every aspect of running an election was made more difficult. In many ways it was kind of the worst-case scenario,” LaRose says. “I don’t think we’ve ever faced a more complex or difficult election.
“One of my key priorities was to maximize early and absentee voting,” he explains. “The more Ohioans we could serve during the month before the election, through early voting and absentee voting, the more we could reduce crowding at the polling stations.”
That meant, LaRose says, that by the morning of Nov. 3, 59% of Ohio’s ballots had already been cast before the polls opened. By way of a comparison, in 2016 it was around 34%.
A security mindset with technical capability
Population: 11.69 million
Counties: 88, ranging in population size from a few thousand to over one million.
On Jan. 12, 2019, Frank LaRose was sworn in as Secretary of State for Ohio. A few months later, he was observing international elections in an official capacity. Along with his distinguished military career, this gave him a real insight into just how important it was to maintain public confidence in elections.
“I served for 10 years in the armed forces and I travelled all over the world,” he says. “In some of the countries where I served, people don’t have a high level of faith in their elections and that’s very troubling. It’s easy to lose that trust and it’s very hard to rebuild it.”
“Our democracy works better when we’re all involved in it and being a voter is a really powerful way to impact the future of our nation. The good news is that elections routinely run so smoothly that people feel like they can take it for granted,” he says, referring to the U.S. elections.
But LaRose did not take the smooth running for granted and remains committed to safeguarding public trust in the institutions and processes of a flourishing democracy. Because of the security directive initiated in 2019, Ohio now has a network of intrusion detection alarms protecting its County Board of Elections.
Ohio has further embraced the idea that cybersecurity is everyone’s responsibility with the creation of the Ohio Cyber Reserve, which LaRose likens to the National Guard. It’s a reserve force of civilian cybersecurity experts that volunteer as a rapid response team in the event of a hack.
“I think that it’s fair to say that cybersecurity is about creating the right culture as much as it is about the right technology,” LaRose says. “It’s that human firewall, so to speak.
“You can have the best technology in the world, but if you don’t have people trained to do the right thing, who take it seriously, who place an emphasis on its importance, the technology may not be all that helpful.”
A battle against increasingly sophisticated foes
Despite the threats of nation-state interference and the challenges of responding to a global pandemic, from election officials’ perspective the 2020 election was a success.
But this doesn’t mean adversaries were not active. A March 2021 report by the U.S. intelligence community concluded that several foreign adversaries did attempt to interfere. However, the report also said there were no indications that any foreign actor attempted to alter any technical aspect of the voting process, including voter registration, casting ballots, vote tabulation or the reporting of results.
While the election took time, energy and resources from all across the company, there were other teams within the organization with hundreds of cybersecurity experts that continued to focus on keeping other customers and products secure.
In the weeks following the election, it was revealed that hackers inserted malicious code into the update process of a software vendor called SolarWinds, infecting government and corporate networks. Weeks after that, an exploit targeting Microsoft’s Exchange Server software was discovered.
These attacks and incidents further demonstrate the need for a broader understanding of risks across the board – along with an acceptance that combating them is everyone’s responsibility.
“The work done around the 2020 election shows the value of participation, collaboration and training,” says Jan Neutze.
“Cybersecurity is just one element – but it remains a crucial one,” continues Neutze. “By the time campaigning for 2024 is underway, the technology hackers rely on will have grown more powerful and the hackers themselves will have become more sophisticated. The wider societal and geopolitical landscape will have evolved, too.”
Defenders like Ohio Sec. LaRose plan to be ready. “We had a saying when I was in the Army that the bad guys only have to be right once, but the good guys and gals have to be right every single day. It’s all about vigilance. It’s about constantly being on guard.”