Google Keystone update breaks Macs’ ability to boot if System Integrity Protection is disabled
On the evening of Monday, September 23rd, a number of film and TV editors started reporting that their workstations were not rebooting successfully. The problem was initially blamed on the Media Composer software sold by Avid.
On September 24th, more instances were reported and it became clear that this was not an issue restricted to Macs with Media Composer installed. After extensive checking and testing, the folks in the MacAdmins Slack were able to narrow down the issue to an update to Google’s Keystone software, which Google uses to update Google Chrome and other Google products on macOS.
The now-pulled Keystone update attempts to remove the /var symlink, which is usually protected by Apple’s System Integrity Protection (SIP) security feature.
On Macs where SIP was disabled, this protection did not apply and the Keystone update was able to remove the /var symlink. This symlink is not a directory itself, but points to another directory (/private/var) which contains software necessary for the operating system to boot and function correctly, so removing the /var symlink rendered the affected Macs unbootable.
As mentioned previously, Google has pulled the problematic Keystone update and has published instructions on how to remediate affected Macs. For more details, please see below the jump.
Note: Remediation is only needed on Macs experiencing problems resulting from this Keystone update. If your Mac is able to boot, remediation is not needed.
To remediate on Macs running OS X 10.9.x – OS X 10.10.x, where SIP is not available:
1. Boot to Recovery
2. Identify the Mac’s boot drive using the following steps:
A. Go to the Apple menu.
B. Select Startup Disk…
C. Look for the highlighted drive. This is the startup disk.
D. Note the name of the startup disk.
Note: By default, Apple sets the name of the boot drive to Macintosh HD.
3. Quit the Startup Disk application.
4. Go to the Utilities menu
5. Select Terminal.
6. Enter the following commands:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chroot "/Volumes/Macintosh HD" | |
rm -rf Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle | |
mv var var_back | |
ln -sh private/var var | |
chflags -h hidden var |
Note: If the name of your boot drive is different from Macintosh HD, the first command should be the following:
chroot "/Volumes/Different Name Goes Here"
Use the actual name of the boot drive in the command where I have Different Name Goes Here.
7. Reboot your Mac and see if it now boots successfully.
To remediate on Macs running OS X 10.11.x – macOS 10.14.x, where SIP is available:
1. Boot to Recovery
2. Identify the Mac’s boot drive using the following steps:
A. Go to the Apple menu
B. Select Startup Disk…
C. Look for the highlighted drive. This is the startup disk.
D. Note the name of the startup disk.
Note: By default, Apple sets the name of the boot drive to Macintosh HD.
3. Quit the Startup Disk application.
4. Go to the Utilities menu
5. Select Terminal.
6. Enter the following commands:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chroot "/Volumes/Macintosh HD" | |
rm -rf Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle | |
mv var var_back | |
ln -sh private/var var | |
chflags -h restricted var | |
chflags -h hidden var | |
xattr -sw com.apple.rootless "" var |
Note: If the name of your boot drive is different from Macintosh HD, the first command should be the following:
chroot "/Volumes/Different Name Goes Here"
Use the actual name of the boot drive in the command where I have Different Name Goes Here.
7. Reboot your Mac and see if it now boots successfully.
Hi! this worked fine the first time, now the problem is back and the solution is not working
Yup, same here. It works until reboot.
Ditto. I ran the exact same process again though, and it worked again. Any ideas how to make it permanent? What a crazy thing for a Google update to take down thousands of machines… Imagine all the people who weren’t lucky enough to stumble upon this solution.
This didn’t work on my infected machines.
`/Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle` doesn’t exists because is in `~/Library/Google/` (under the home/user directory, not disk root)
so `/Users//Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle`
Because of that, what worked instead was:
“`
chroot /Volumes/Macintosh\ HD # “Macintosh HD” is the default
rm -rf /Users//Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle
mv var var_back # var may not exist, but this is fine
ln -sh private/var var
chflags -h restricted /var
chflags -h hidden /var
xattr -sw com.apple.rootless “” /var
“`
Where username is the user with the Google services installed
This didn’t work on my infected machines.
`/Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle` doesn’t exists because is in `~/Library/Google/` (under the home/user directory, not disk root)
so `/Users/USERNAME/Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle`
Because of that, what worked instead was:
“`
chroot /Volumes/Macintosh\ HD # “Macintosh HD” is the default
rm -rf /Users/USERNAME/Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle
mv var var_back # var may not exist, but this is fine
ln -sh private/var var
chflags -h restricted /var
chflags -h hidden /var
xattr -sw com.apple.rootless “” /var
“`
Where USERNAME is the user with the Google services installed
Hello, has anyone solved that problem permanently?