Every September we go through the same thing. Apple needs to bring its autumn/fall harvest of iPhones to market. What it sowed just three months ago at WWDC now suddenly has to be ripened, polished, and put out on the stalls for users to buy.

For those iPhones to sell, they need new hardware and software to bring zingy new features; that means a new release of iOS, which drags macOS with it. And to enable Apple’s engineers and third-party developers to show the user how wonderful the new models are, that requires a new version of the development tools, centred on Xcode.

In those three months, Apple has to beta-test and bug-fix new major versions of iOS, macOS, and Xcode. All at the same time. Any change in the operating systems might require changes in the SDK within Xcode, and keeping all three in step is clearly not possible. As I write, iOS 12 is at beta 12, macOS Mojave at beta 9, Xcode 10 still at beta 6, with watchOS and tvOS different again. By the end of September, Apple’s engineers must be utterly frazzled, as they next have to face making urgent fixes to those bugs that weren’t picked up during the beta phase.

This year’s testing phase has highlighted the gulf that remains between iOS and macOS, with their commendable changes in Transparency Consent and Control – TCC, better known simply as Privacy.

iOS apps have always existed in their sandbox; when they want to do anything beyond it, they enter a jungle of entitlements and barriers. Many macOS apps and tools come from a world in which Unix permissions are their only real limit; most users run most of the time as admins, and it only takes the magic word sudo to change even that. macOS apps commonly use these features to empower the user, and suddenly constraining them can stop important apps from working as users expect.

So far, two main areas in Mojave’s Privacy have been identified as particular problems.

Many apps use AppleEvents to work with other apps, and now face quite severe constraints in what they can do. An early article by Felix Schwarz explained how damaging this would be, and led to changes in Mojave’s approach. Most recently, Felix has explained in careful detail how Apple’s response still poses serious problems.

Last week, I explained how Mojave’s privacy protection is complex and will crash innocent apps user controls in Mojave beta-releases, coupled with their response to unexpected attempts to access protected data, could lock apps from inviting user consent. Even when apps built against the new SDK were pre-authorised to give them Full Disk Access, TCC was expecting them to contain a ‘usage description’. It turned out that this behaviour was unintended, and I was encouraged to report it as a bug.

Full credit to Apple’s engineers: no sooner had I filed the bug report than they were working on reproducing it. I confess that in the past I have often not reported bugs direct, but felt that the time and effort involved was unlikely to bring any useful result for the user. In this case, I’m highly optimistic that this will be fixed in the release version of Mojave which millions of us will be installing later this month.

I’m amazed, though, that I seem to have been the first developer to encounter the problem. This probably stems from the fact that the bug only manifests itself in apps built using the 10.14 SDK; as that remains in a beta-release of Xcode, and relatively few developers seem to have built much with Xcode 10 yet, it hasn’t really been noticed. All the recent app updates that I have installed here (outside of the Mojave betas) have still been built with the 10.13 or earlier SDKs in Xcode 9.

That in itself is worrying, as making good use of Mojave’s features, most obviously its new Dark Mode, relies to a greater extent on building against the 10.14 SDK. As it stands, apart from Apple’s apps, very little of the software which I will take with me to Mojave looks able to use it, and quite a lot may prove ugly or even unusable when run in Dark Mode. That’s bad for those products, and bad for Mojave. But commercial developers are understandably reluctant to risk shipping products which have been built with a beta-release of Xcode.

So why haven’t these problems become apparent to Apple’s engineers when they have been developing its own Mojave apps, which are generally built against the 10.14 SDK, albeit using a special ‘internal’ version of Xcode?

The answer brings understanding to several of the issues arising with TCC: Apple’s apps don’t (and never have) followed the rules which apply to third-party software. They have their own private entitlements, which among other things let them get away with accessing protected data without declaring usage descriptions. Now it becomes clear how dangerous AppleEvents and scripting can be: all a malicious app would have to do is use one of Apple’s apps as a surrogate for its access to private data. Closing that potential vulnerability is not so simple after all.

How have I come to learn about the entitlements of apps, their usage declarations, and even the version of the SDK used to build them?

I earlier made the case for users being better-informed about the capabilities of the apps which they use. Having spent some time rummaging through this information in Sierra, High Sierra, and Mojave, I now realise that I was wrong. These issues are too complex to summarise in the Finder’s Get Info dialog, so I have put together an app named Taccy (from TCC) which will do the job for you. Its results are fascinating, revealing, and essential reading for the advanced user. I’ll be explaining more, and providing an early beta-release, tomorrow.

In the meantime, where does this leave Mojave?

I’m still convinced that this month’s market stalls are going to be offering us a good crop. Some users will still encounter problems in Mojave, and I think that TCC will continue to evolve before it is really polished, as will third-party apps. But I don’t see this being a re-run of the High Sierra fiasco, and I know that a lot of Apple engineers have been putting a huge and sustained effort into getting Mojave right

In these last few days before going to Golden Master, I wish them success. We all need it.