Yesterday I explained how the combination of a quarantine flag and the OpenWith extended attribute is lethal to attempts to open a document via the Finder. This article details the mechanism by which macOS Mojave accomplishes this act of self-immolation.

My first example is from a simple and commonplace situation: dragging and dropping a document onto an app in the Dock. The app in question has already cleared its first run checks with Gatekeeper, and is notarized “for added security”. The document is a Rich Text file (RTF), hardly a major malware vector on macOS, and isn’t executable. Times are given in decimal seconds at the start of each log entry.

Dropping the document on the app icon triggers LaunchServices to process the user action:
03.744281 Dock LaunchServices LaunchApplication: appToLaunch={ "ApplicationType"="Foreground", "CFBundleExecutablePath"="/Applications/DelightEd.app/Contents/MacOS/DelightEd" […]

Less that 0.04 seconds later, XProtect kicks in with a series of signposts used internally by Apple to assess its performance:
03.781679 signposts XprotectService XprotectService
03.781684 signposts XprotectService XprotectService <private>
03.781851 signposts XprotectService XprotectService 3

Then XProtect is called on to perform its assessment of the document:
03.782701 XprotectService Calling SecAssessmentCreate with URL <private>, context <private>

About 0.02 seconds later, it returns an error:
03.802353 Security MacOS error: -67062

This error code is quite inappropriate: it is errSecCSUnsigned, which means a “code object is not signed”. There is, of course, no code object in the document in question, it is a simple RTF document.

The security system appears to try a ‘temporary signing’ of type 3, then the error is repeated, and the redacted results are returned:
03.802471 Security temporarySigning type=3 matchFlags=0x0 path=/Volumes/External1/Documents/0newDownloads/DelightEd20b1/ReadMeStyles.rtf
03.803060 Security MacOS error: -67062
03.804557 SecAssessment results: <private> (null)

After the user has dismissed the security error alert, macOS takes apart its preparations for opening the offending document:
19.313035 LaunchServices Killing process 93084, sending it signal 9
19.313296 LaunchServices Got exit notification for application App:"DelightEd" asn:0x0-27cf7cd pid:93084 refs=5 @ 0x7fd3ef5311b0, pid=93084, from port=0x18657:MACH_PORT_RIGHT_RECEIVE=1, fExitStatus=9.
19.313330 LaunchServices ADDING: App:"DelightEd" asn:0x0-27cf7cd pid:93084 refs=4 @ 0x7fd3ef5311b0 exitStatus=9
19.313372 LaunchServices ADDING { "ApplicationType"="Foreground", "CFBundleExecutablePath"="/Applications/DelightEd.app/Contents/MacOS/DelightEd", "CFBundleIdentifier"="co.eclecticlight.DelightEd", "DeathTime"=now-ish 2019/05/02 09:58:19, "LSBundlePath"="/Applications/DelightEd.app", "LSDisplayName"="DelightEd", "LSExitStatus"=9, "pid"=93084 } to sDeadApplicationInformation for 93084

My second example is more complex. In this case, the Get Info dialog for the same Rich Text document has been set to open the file using a copy of an app which bears a quarantine flag and hasn’t yet undergone its first run. Rather than making a cleaner start using the Dock, the document has simply been double-clicked in a Finder window.

This initial entry marks the second of the double clicks:
00.504754 Finder AppKit sendAction:

Over 0.07 seconds later, LaunchServices becomes engaged in starting to launch the quarantined app:
00.579813 LaunchServices LaunchApplication: appToLaunch={ "ApplicationType"="Foreground", "CFBundleExecutablePath"="/private/var/folders/sz/qb23fyss56v96vmh60p8ft7r0000gn/T/AppTranslocation/E1520051-849F-4751-B862-CA6DE4673AE7/d/DelightEd.app/Contents/MacOS/DelightEd" […]

One odd log entry which may be unrelated relates to the sandbox. Note that the app being opened doesn’t run in a sandbox, but it is hardened and notarized:
00.587735 kernel Sandbox syscall_extension_issue failed: 2

Because the app being launched is in quarantine and being run from an unusual location, it undergoes translocation as it normally would for its first run:
00.609427 LaunchServices LaunchedApplication: "/private/var/folders/sz/qb23fyss56v96vmh60p8ft7r0000gn/T/AppTranslocation/E1520051-849F-4751-B862-CA6DE4673AE7/d/DelightEd.app/Contents/MacOS/DelightEd", psn=[ 0x0/0x27ca7c8]

Again, XProtect first appears in the log thanks to its signpost entries:
00.620067 signposts XprotectService XprotectService

Following which XProtect appears to start its examination not of the document to be opened, but of the quarantined app:
00.622407 XprotectService Calling SecAssessmentCreate with URL <private>, context <private>
00.639953 XprotectService Bundle is not apple signed
00.640426 XprotectService Bundle size result: 14709332 (YES)
00.640427 XprotectService Always scan: YES
01.203631 XprotectService Starting malware scan for: <private>

A second set of signposts are then written by XProtect:
01.258279 signposts XprotectService XprotectService
01.258282 signposts XprotectService XprotectService <private>
01.258503 signposts XprotectService XprotectService 3

After which it examines the document, generating identical log entries to those seen above in the first case:
01.259754 XprotectService Calling SecAssessmentCreate with URL <private>, context <private>
01.261065 Security MacOS error: -67062
01.261178 Security temporarySigning type=3 matchFlags=0x0 path=/Volumes/External1/Documents/0newDownloads/DelightEd20b1/ReadMeStyles.rtf
01.261743 Security MacOS error: -67062
01.262635 SecAssessment results: <private> (null)

This returns the same error -67062, which is reflected in the resulting alert, and the user action is aborted.

I think that these log extracts demonstrate how the failure to open these documents is a behaviour determined by XProtect as a result of its scan not of the document contents (which were entirely innocent), but on discovery of the quarantine flag and the OpenWith extended attribute. The error returned, -67062, is incorrect, and results in the wrong alert being displayed to the user.

As far as I can discover, this behaviour and its use of this alert is undocumented by Apple in either its user documentation or that for developers. The only reference that I have been able to find to the alert is in this article, which explains its use with respect to app quarantine. The advice given is not only irrelevant to its use with documents, but leads to users blaming not macOS or the document, but the app developer:
“These messages don’t necessarily mean that something is wrong with the app. For example, some apps were written before Developer ID registration. If you see a warning, it means that the app has not been signed by the developer, so macOS can’t check whether the app has been modified or broken since it was released.
You may want to look for a later version of the app in the App Store or look for an alternative app.”

Apple needs to correct this immediately: blaming its third-party developers for an undocumented feature in macOS is plain wrong.