Researchers Demonstrate Subliminal Smart Device Commands That Have Potential for Malicious Attacks

Researchers in the United States and China have been performing tests in an effort to demonstrate that "hidden" commands, or those undetectable to human ears, can reach AI assistants like Siri and force them to perform actions their owners never intended. The research was highlighted in a piece today by The New York Times, suggesting that these subliminal commands can dial phone numbers, open websites, and more potentially malicious actions if placed in the wrong hands.

A group of students from the University of California, Berkeley and Georgetown University published a research paper this month, stating that they could embed commands into music recordings or spoken text. When played near an Amazon Echo or Apple iPhone, a person would just hear the song or someone speaking, while Siri and Alexa "might hear an instruction to add something to your shopping list." Or, more dangerous, unlock doors, wire money from your bank, and purchase items online.

siri iphone x
The method by which the students were able to accomplish the hidden commands shouldn't be a concern for the public at large, but one of the paper's authors, Nicholas Carlini, believes malicious parties could already be making inroads with similar technology.

“We wanted to see if we could make it even more stealthy,” said Nicholas Carlini, a fifth-year Ph.D. student in computer security at U.C. Berkeley and one of the paper’s authors.

Mr. Carlini added that while there was no evidence that these techniques have left the lab, it may only be a matter of time before someone starts exploiting them. “My assumption is that the malicious people already employ people to do what I do,” he said.

Last year, researchers based at Princeton University and Zheijiang University in China performed similar tests, demonstrating that AI assistants could be activated through frequencies not heard by humans. In a technique dubbed "DolphinAttack," the researchers built a transmitter to send the hidden command that dialed a specific phone number, while other tests took pictures and sent text messages. DolphinAttack is said to be limited in terms of range, however, since it "must be close to the receiving device."

DolphinAttack could inject covert voice commands at 7 state-of-the-art speech recognition systems (e.g., Siri, Alexa) to activate always-on system and achieve various attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile.

In yet another set of research, a group at the University of Illinois at Urbana-Champaign proved this range limitation could be increased, showing off commands received from 25 feet away. For the most recent group of researchers from Berkeley, Carlini told The New York Times that he was "confident" his team would soon be able to deliver successful commands "against any smart device system on the market." He said the group wants to prove to companies that this flaw is a potential problem, "and then hope that other people will say, 'O.K. this is possible, now let's try and fix it.'"

For security purposes, Apple is stringent with certain HomeKit-related Siri commands, locking them behind device passcodes whenever users have passcodes enabled. For example, if you want to unlock your front door with a connected smart lock, you can ask Siri to do so, but you'll have to enter your passcode on an iPhone or iPad after issuing the command. The HomePod, on the other hand, purposefully lacks this functionality.

Top Rated Comments

nwcs Avatar
77 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Score: 18 Votes (Like | Disagree)
daveschroeder Avatar
77 months ago
That is NOT "subliminal".

I think you're looking for another word.
Score: 12 Votes (Like | Disagree)
bluespark Avatar
77 months ago
In fairness, Apple has made great strides in tackling this issue already through Siri's continued uselessness.
Score: 12 Votes (Like | Disagree)
amaier1986 Avatar
77 months ago
HomePod directs me to use my phone to unlock my front door or open my garage doors. This potential issue seems to be somewhat under control with iOS.
Score: 7 Votes (Like | Disagree)
w5jck Avatar
77 months ago
That is NOT "subliminal".

I think you're looking for another word.
My thought is that the word should be "inaudible" and NOT "subliminal". As in, "The devices can react to inaudible commands."
Score: 7 Votes (Like | Disagree)
jarman92 Avatar
77 months ago
This is really clever. I wouldn’t have thought that the AIs would respond to non-vocal frequencies as they’re intended to listen to humans only. I would think that checking the frequency range of the command would be enough to counteract this problem fairly simply.
Agreed. But why wouldn't Apple have foreseen this and limited the frequency range in the first place? There's literally no need for phone mics to detect anything below/above human voice frequencies.
Score: 6 Votes (Like | Disagree)

Popular Stories

reset password request iphone

Warning: Apple Users Targeted in Phishing Attack Involving Rapid Password Reset Requests

Tuesday March 26, 2024 4:34 pm PDT by
Phishing attacks taking advantage of Apple's password reset feature have become increasingly common, according to a report from KrebsOnSecurity. Multiple Apple users have been targeted in an attack that bombards them with an endless stream of notifications or multi-factor authentication (MFA) messages in an attempt to cause panic so they'll respond favorably to social engineering. An...
iPhone Home Screen Gradient Blank Spaces 1

Sources: iOS 18 Lets Apps Be Placed Anywhere on Home Screen Grid

Sunday March 24, 2024 1:33 pm PDT by
iOS 18 will give iPhone users greater control over Home Screen app icon arrangement, according to sources familiar with the matter. While app icons will likely remain locked to an invisible grid system on the Home Screen, to ensure there is some uniformity, our sources say that users will be able to arrange icons more freely on iOS 18. For example, we expect that the update will introduce...
iPad Pro 2024 Landscape Camera Feature

New iPad Pro Again Rumored to Feature Landscape Front-Facing Camera

Monday March 25, 2024 5:43 am PDT by
The next-generation iPad Pro will feature a landscape-oriented front-facing camera for the first time, according to the Apple leaker known as "Instant Digital." Instant Digital reiterated the design change earlier today on Weibo with a simple accompanying 2D image. The post reveals that the entire TrueDepth camera array will move to the right side of the device, while the microphone will...
sonoma desktop wwdc

Apple Releases macOS Sonoma 14.4.1 With Fix for USB Hub Bug

Monday March 25, 2024 10:10 am PDT by
Apple today released macOS Sonoma 14.4.1, a minor update for the macOS Sonoma operating system that launched last September. macOS Sonoma 14.4.1 comes three weeks after macOS Sonoma 14.4. The ‌‌‌‌macOS Sonoma‌‌ 14.4‌.1 update can be downloaded for free on all eligible Macs using the Software Update section of System Settings. There's also a macOS 13.6.6 release for those who...
Generic iOS 18 Feature Purple

iOS 18 Will Finally Bring This Android Feature to iPhone

Monday March 25, 2024 6:42 am PDT by
iOS 18 will allow iPhone users to place app icons anywhere on the Home Screen grid, according to sources familiar with development of the software update. This basic feature has long been available on Android smartphones. While app icons will likely remain locked to an invisible grid system on the Home Screen, our sources said that users will be able to arrange icons more freely on iOS 18....
maxresdefault

Apple Announces WWDC 2024 Event for June 10 to 14

Tuesday March 26, 2024 10:02 am PDT by
Apple today announced that its 35th annual Worldwide Developers Conference is set to take place from Monday, June 10 to Friday, June 14. As with WWDC events since 2020, WWDC 2024 will be an online event that is open to all developers at no cost. Subscribe to the MacRumors YouTube channel for more videos. WWDC 2024 will include online sessions and labs so that developers can learn about new...
apple maps 3d feature

Apple Maps May Gain Custom Routes With iOS 18

Tuesday March 26, 2024 3:10 pm PDT by
Apple may be planning to add support for "custom routes" in Apple Maps in iOS 18, according to code reviewed by MacRumors. Apple Maps does not currently offer a way to input self-selected routes, with Maps users limited to Apple's pre-selected options, but that may change in iOS 18. Apple has pushed an iOS 18 file to its maps backend labeled "CustomRouteCreation." While not much is revealed...
iphone reasons to upgrade

Apple Outlines 'Reasons to Upgrade' Your iPhone on New Website

Monday March 25, 2024 12:53 pm PDT by
Apple today added a "Why Upgrade" section to its website, which is aimed at encouraging customers with older iPhones to upgrade to a newer model. The website allows customers to compare the iPhone 11, 11 Pro, 11 Pro Max, 12, 12 mini, 12 Pro, and 12 Pro Max to the iPhone 15, iPhone 15 Plus, iPhone 15 Pro, or iPhone 15 Pro Max. Each comparison shows what new features someone with an older...