Apple Mac Firmware Updates Are Quietly Failing and No One Knows Why

Even if you keep your software up to date, your computer may be hiding vulnerable, outdated code within its deepest recesses that hackers can exploit to totally compromise your machine—leaving you none the wiser.

The issue has to do with firmware, programming written directly onto the metal of a machine that controls hardware. Firmware sits beneath the operating system at a level of privilege that, when accessed by an attacker, grants free-ranging, undetectable hacking powers.

This layer is so deep that even installing a new operating system or replacing a hard disk will not rescue an affected machine. A computer thus compromised is effectively unsalvageable.

Security researchers at Duo Labs gathered three years worth of data across 73,000 Apple (AAPL) Mac computers used in organizations spanning a variety of industries—some data were from customers, others were contributed by admins friendly to the research community—to see whether the machines were running the proper firmware, or extensible firmware interface (EFI), code that handles a computer’s pre-boot processes. (EFI firmware is the first part of a Mac’s programming that runs after a computer is turned on.)

The researchers made a surprising discovery. In a significant number of cases, computers running the latest versions of the macOS operating system lag when it comes to firmware—potentially leaving a core part open to compromise.

Of the tens of thousands of machines examined, roughly 54,000 computers were actively maintained by Apple. Of this subset, the researchers found on average a 4.2% deviation from the expected norm, meaning thousands of machines were running unexpected versions of EFI firmware. The iMac 16,2 with a 21.5-inch screen released in late 2015 had the highest occurrence of incorrect firmware at 43%, followed by three versions of the MacBook Pro with a13-inch screen released in late 2016, which deviated between 35% and 25%. (For the full rundown, read the team’s blog post, which contains a link to the full research report.)

“There shouldn’t be any deviance ever,” says Rich Smith, Duo’s director of research and development.But there is and in some cases it is quite significant.”

Since 2015, Apple has bundled firmware updates in with updates to its operating system—a move the researchers applaud for taking some of the onus off users for keeping their systems up to date. But there’s a problem; should a firmware update fail, users aren’t warned.

“There’s no notification that an EFI update failed—no retry, it’s just a silent failure,” Smith says. This means your machine could be vulnerable and you would have no idea.

In contrast, when something goes wrong during an operating system upgrade, an alert typically pops up.

“You’re software secure, but firmware vulnerable,” Smith says.

[fortune-brightcove videoid=5460338133001]

The researchers identified 16 Apple computer models—including iMacs, MacBooks, MacBook Pros, Macbook Airs, Macminis, and MacPros—that receive support for operating system security updates, yet no longer appear to receive them for their EFI firmware. The inconsistencies raise questions about the quality assurance Apple has been applying to firmware updates.

The findings also present a mystery. “From the data we could see what was happening, but not say why it was happening,” Smith tells Fortune. “We don’t have data to look inside why there was a failure.”

The researchers are set to unveil their research at the annual Ekoparty computer security conference in Buenos Aires, Argentina on Friday, where they hope to raise people’s interest in firmware security. The topic popped into the news earlier this year when the anti-secrecy website WikiLeaks posted an alleged dump of CIA files called Vault 7 that detailed a trove of hacking tools, including one, “Sonic Screwdriver,” that allowed spies to subvert Mac firmware.

Duo has been in talks with Apple about the new research since late July, Smith says. (“We’ve been pleased with way they’ve worked with us.”)

The feeling is mutual. When an abstract for the presentation appeared on the Ekoparty conference website earlier this month, an Apple security engineer, Xeno Kovah, posted on Twitter a since-deleted note of praise. “They were nice enough to share their report with us beforehand,” Kovah wrote. “I agree with their conclusions, that we’ve got things we can do better.”

Reached for comment, an Apple spokesperson told Fortune that “We appreciate Duo’s work on this industry-wide issue.” The spokesperson continued: “Apple continues to work diligently in the area of firmware security and we’re always exploring ways to make our systems even more secure.”

In the latest version of macOS, also known as 10.13 or “High Sierra,” Apple included a tool that validates the authenticity of the firmware running on a given Mac computer on a weekly basis, the spokesperson said. While the tool does not check whether a machine is running the latest version, it does reveal whether the firmware has been tampered with.

Duo said it chose Apple because the company, which controls its own hardware, firmware, and software, offered the most consistent data—unlike, say, the fragmented ecosystem of Microsoft (MSFT) Windows PCs running on Intel (INTC) chips. Despite the unnerving findings, Smith says he suspects that Apple is “doing best job of all the major vendors.”

Get Data Sheet, Fortune’s technology newsletter

The people who should be most concerned about the findings of this research are those running corporate IT programs or organizations with large fleets of computers, like corporations or governments, Smith says. Everyday consumers should be more concerned about keeping their operating systems and software up to date, since flaws in them are more common entry points for hackers.

Even if you are running the latest Apple operating system—macOS 10.13, or High Sierra—you are not guaranteed to be running the latest version of EFI firmware, as Duo’s research shows. If you wish to check whether you’re running the latest version of EFI firmware, you can use these open source tools Duo released on its Github page that help determine which vulnerabilities might be exposed on your machine.

If you are running IT for an organization that might be at risk of being targeted by nation state actors or industrial espionage perpetrators, Duo suggests considering scrapping affected computers and buying new ones, or repurposing vulnerable machines for less critical duties.

Subscribe to the Eye on AI newsletter to stay abreast of how AI is shaping the future of business. Sign up for free.