Sal Soghoian Reveals macOS’s Hidden Custom Control Panels

In “TipBITS: How to Mirror Selected Screens in a Multiple Monitor Setup” (22 October 2019), I teased readers with a talk that automation guru Sal Soghoian gave at MacTech Conference a few weeks ago. Sal has now finished the videos that explain what he did in detail—which you’ll want to watch in their entirety—so I can now share what he said at MacTech.

In essence, Sal discovered and described how to use an accessibility feature hidden deep within macOS that lets you turn an iPad into a completely customizable control panel for a Mac. Tap a button on your iPad, and things happen on your Mac. Sal demoed this in macOS 10.15 Catalina, but it should also work in at least 10.14 Mojave, and possibly earlier versions of macOS as well.

To make the connection between the Mac and the iPad, you need a Luna Display dongle (see “Luna Display Turns an iPad into a Responsive Mac Screen,” 7 December 2018). Apple’s Sidecar technology won’t work because it doesn’t allow touch input on the iPad when it’s acting as a Mac screen (“Catalina’s Sidecar Turns an iPad into a Second Mac Monitor,” 21 October 2019). Luna Display works with any iPad running iOS 9.1 or later, which makes this a perfect use for an old iPad 2 or later. Of course, being able to tap the control panel buttons on the iPad connected by Luna Display is just a bonus—you can also click the buttons with a mouse on a standard Mac screen.

The next important piece of the puzzle is a built-in macOS accessibility app called Panel Editor, which lets you create custom accessibility keyboard panels. You can open Panel Editor from System Preferences > Accessibility > Keyboard > Accessibility Keyboard > Panel Editor. But before you open Panel Editor, be sure to select the Enable Accessibility Keyboard checkbox—which displays an on-screen keyboard—and in System Preferences > Keyboard > Keyboard, be sure to select “Show keyboard and emoji viewers in menu bar.”

Once you’ve enabled all those settings, click the Panel Editor button in System Preferences > Accessibility > Keyboard > Accessibility Keyboard to open the app and create a new panel. These panels are technically “keyboards,” but Panel Editor lets you create buttons that can perform a variety of actions. You can create back buttons, show or hide the toolbar, enter text, press keys on the keyboard, open apps, trigger system events, and get typing suggestions. And—you probably knew this was coming as soon as I mentioned that Sal is behind this hack—a button can trigger an AppleScript.

In my example above, I’ve created five buttons (which are shown floating above the Panel Editor window as well). Open iTunes uses the Open App action, Play/Pause and Mute use System Events actions, Identify Desktop runs an AppleScript, and Border Image invokes a Keyboard Maestro macro via the Press Keys action. That’s right, you aren’t limited to AppleScript. Along with Keyboard Maestro, you should be able to integrate Automator workflows via the Open App action. Nearly any action can become a custom button in Panel Editor.

To open your panel, click the Custom button at the upper-right corner of the Accessibility Keyboard and then click your panel. You can also click the gear icon and choose your panel from the pop-up menu. It’s a dark panel that floats above all other windows, and you can move it anywhere you like, such as that second screen provided by the iPad and Luna Display.

(One tip: if you’re doing this in Mojave, don’t close your panel because getting it back requires a trip to System Preferences > Accessibility > Keyboard > Accessibility Keyboard > Panel Editor and selecting Enable Accessibility Keyboard again. In Catalina, you can just choose Show Keyboard Viewer from the keyboard icon in the menu bar.)

My example panel has only a few buttons, but when you watch Sal’s video, you’ll see that he has created so many buttons that his custom panel fills the entire iPad screen. Many of them are generalized buttons for launching apps, changing system settings, and so on, but all the buttons in the middle trigger scripts that control Keynote. You may not be interested in automating Keynote, but if you’re an automation aficionado, just imagine all the actions you can attach to buttons.

I’ve glossed over some important details that Sal covers in his video, such as setting security permissions for the Assistive Control app and how to use an AppleScript Library to hold all your scripts, which you then call from the individual buttons.

However, there are two final important points to make:

• You can share your custom panels with other people, within a version of macOS (the Mojave version of Panel Editor can’t open panels saved from the Catalina version). That means you could develop a custom panel for company-specific actions and then give it to everyone in your office. To do this, in Panel Editor, choose File > Duplicate, then File > Move To to save an .ascconfig file. Then, to import it into Panel Editor on another Mac, choose File > Import Panels.
• Sal has shared both his custom Keynote panel (for Catalina) and the AppleScript Library containing all his scripts, so you can use them as is or to get started building custom panels.

Let us know in the comments if you’re thinking about creating your own custom panels and for what purpose!