Home » Learning Curve
Apple and Open SystemsA rise, a peak, a fall?
All good systems are open systems. This is without exception today.
Things weren't always this way, but they are today. More and more it's become obvious that the only systems that can be trusted are the open systems, and Ken Thompson showed us how even that may not always be enough, but 'open' at least gives users a fighting chance, where 'closed' is tantamount to totalitarian. Close the system - push to the gates of the walled garden - and people will run for it. Run for their freedom. As they should. As they must.
The original Unix, although not open source at the time, had a complete user manual that fit comfortably in an attaché case. The total number of lines of code was 13,500 - about the same as Microsoft's WINFILE for Windows 3.1 twenty years later. Graphical user interfaces are exorbitant, adding 400% more code to a project.
But that doesn't explain everything. A study of call graphs for comparative web servers showed us the rest, with Microsoft exposed as the most ridiculous software vendor in the world.
Apple had a great advantage twenty years ago. Right now they're betting their floricultural system on things continuing as before, even as they try to change the rules like never before. Yes, push will lead to shove. And, if history teaches us anything, it'll end in a backlash that history would rather forget.
IBM, the venerable 'Snow White' of the industry, found open source years ago. Their mainframes still run proprietary code, but that code can cradle Linux. Microsoft's foray into open source, after harassing both IBM and Linux for so many years, is feeble in the extreme, and the epithet 'fanboy', for those who still defend the Redmond monster, fits nicely.
Apple inherited a FreeBSD system from NeXT. The decision by NeXT to go with FreeBSD was a good one. Why waste time maintaining a system that others already maintain? But then Redwood City came to Cupertino and things changed radically.
It doesn't take five years to redo a system that's already being successfully marketed and used by corporations like Dell and WorldCom. It doesn't take even one night.
Apple made a feeble attempt to keep their new OS 'open' - it would have made sense if it had been done correctly and honestly (which it was not). FreeBSD already has a team to maintain the code. And FreeBSD is so good that Linus Torvalds admitted he wouldn't have embarked on his own project if he'd known about it.
That Unix is relatively stable and secure is the result of open vetting and years of testing in the real world. Apple's skunkworks don't qualify in that regard, as we've seen time and again. It's an added cost, opening the system to vulnerabilities that should not exist, and it's an additional expense that can't be defended.
Add to that the resulting hostility in the wider community.
Apple's iPhone changed everything. A company weeks away from receivership, they morphed into the richest corporation in the world, able to change the name of the game and make their own set of rules for this new game - up to a point at least, as monopolies too suffer from market constraints.
Open Darwin fell apart because no one at Apple gave a shit. Curator Rob Braun declared it had only been a PR stunt anyway. Parts of Safari went open, as the code came from from another open source project, but the system itself remains closed, and now it's increasingly antagonistic. The low-hanging fruit of a future where even third-party software is under their total control: that's the wet dream that's obsessed them for more than ten years.
But all good things must come to an end, mustn't they? Apple created the smartphone market, and naturally led it to start with, but the competition soon took over. Still, thanks to planned obsolescence and high prices, they were able to profit handsomely.
Until the market finally peaked.
The market was saturated and started to dry up. Apple saw this moment coming long before the pundits, despite its inevitability. Sales statistics are no longer published. Apple approached Hollywood. And they returned to their base OS, the one they'd relegated to picking up crumbs from the iOS table of feast. The time had come to start exploiting the desktop OS, the goal being to recuperate a steady flow of billions in annual revenue by turning free developers into indentured serfs.
Locking the gates to the walled garden may be a step too far, it may result in a backlash that ruins the remains of a crumbling empire, but can't people be induced to come inside of their own volition?
The technical details of Apple's 'gambit' are easy to understand. They deliver the OS, their OS controls the launching of software, and so forth. That's the gates, that's the 'kill switch'. The gates can be closed at any time - the 'kill switch' can be delivered at any time - through an otherwise innocent-looking OS update. But the repercussions are perhaps too dire for this option to even be considered.
Operating systems and file systems need ways to protect themselves, their users, and their resources. Almost all systems give such matters due consideration, save for Microsoft Windows of course, where things are mostly being done as an afterthought to save face and financial loss.
Access control is the name of the game. In broad terms, there are two types of access control models: discretionary and mandatory. Microsoft's is of the former, meaning that their systems may impose access control but don't need to. Unix systems are the latter - or the opposite - meaning that access control is always imposed. It's of course possible to grant unlimited access to an object, but this occurs only within the access control model, not in spite of it.
Unix was designed to be able to protect itself and its users. Unix gives users the means to protect themselves from one another and from themselves, and Unix sees that it can protect itself from its users as well - and, by extension, from itself.
Unix isn't in the business of hiding things. Unix has a more than adequate security model. Concealment isn't part of that model, nor should it be, nor can it ever be, in the long run.
Malware writers on Windows, and to a lesser extent on Unix systems, can try 'camouflage' as a means of protection. The infamous Windows Registry, unfathomable to mere mortals and even to some entry-level programmers, is a great place to hide stuff. Popular applications such as Paint Shop Pro have used the Windows Registry in this manner. Windows configuration (INI) files are another good hiding place. Opera's web browser for Windows used WIN.INI to hide things.
Software vendors can use 'camouflage' on Unix systems as well. A file 'browser' uses a number of concealment methods. And preferences files can be used for more than preferences - by vendors of FTP clients. Other attempts at camouflage can be found here, here, here, and here.
'Camouflage' isn't protection, and it will ultimately fail, but perhaps it's regarded more as a speed bump than a full barrier.
Aside from instances such as the above, neither Unix nor software running on Unix tries to hide anything, and the Unix system never tries to hide. For hiding is futile. It's always possible, and often necessary, to restrict access to system resources, but hiding the fact that such resources exist isn't part of the game, nor was it ever meant to be.
The dot file - a file whose name begins with a 'dot' - isn't really trying to hide. It's just politely kept out of the way in default file listings, unless a command is invoked with special switch. Apple's Finder has a similar option, where everything, including dot files, can be exposed issuing the command:
defaults write com.apple.Finder AppleShowAllFiles -bool YES
How Windows Protects
Can Windows protect its users? Can Windows protect itself? The short version: no it cannot. Standard file attributes say nothing about ownership or permissions. Access control lists, introduced by the Dave Cutler team for Windows NT, provide discretionary access control only, and are an arcane exercise.
How Unix Protects
Unix uses a base system of three permissions for three groups of users. In almost all cases, this is more than sufficient, and was used for years as the only protection scheme, even as the popularity of the platform grew into what it is today, all-pervasive on the Internet and the 'WorldWideWeb'.
The three permissions are Read, Write, and eXecute. The three user groups are owner or user, one of the owner's groups, and everyone else - 'other'.
Because execute permissions also apply to directories, directory owners can effectively restrict access to disk areas and thereby prevent malicious attacks. Check closely where your Unix system has its 'bin' directories and what permissions they and their parent directories have.
Unix systems of today can also have so-called 'file flags' to further augment control. Your root directory, for example, should be marked with a 'sunlnk' flag, meaning that your root directory cannot be unlinked, even by the root account, other than in single-user mode.
Going through a Mojave root reveals an assortment of 'sunlnk', 'hidden', 'uchg', and 'restricted' flags on subdirectories. The flags 'sunlnk' and 'uchg' are designed to protect. The flags 'restricted' and 'hidden'? Not so much.
The flags 'restricted' and 'hidden' are not 'Unix'. They're 'Apple'.
ls -@aCeilO
total 37
2 drwxr-xr-x@ 31 root wheel sunlnk 992 Aug 2 22:11 .
com.apple.backupd.unlockrecordUUIDs 84
2 drwxr-xr-x@ 31 root wheel sunlnk 992 Aug 2 22:11 ..
com.apple.backupd.unlockrecordUUIDs 84
647151 d--x--x--x 9 root wheel uchg 288 Mar 11 2019 .DocumentRevisions-V100
19 dr-xr-xr-t@ 2 root wheel uchg,hidden 64 Jul 15 2017 .HFS+ Private Directory Data?
com.apple.FinderInfo 32
4320874042 -rw-r--r-- 1 root wheel - 1188 Aug 2 22:11 .OSInstallerMessages
440239 drwxr-xr-x 2 root admin - 64 Jul 15 2017 .PKInstallSandboxManager
618010 drwxr-xr-x@ 3 root admin - 96 Sep 20 14:45 .PKInstallSandboxManager-SystemSoftware
com.apple.rootless 0
624104 drwx------ 5 root wheel - 160 Jul 15 2017 .Spotlight-V100
4310250579 ---------- 1 root admin - 0 Aug 17 2018 .file
624255 drwxr-xr-x 2 root wheel uchg 64 Mar 21 2019 .fseventsd
4310250580 drwxr-xr-x 2 root wheel hidden 64 Sep 24 2018 .vol
4310250581 drwxrwxr-x+ 44 root admin sunlnk 1408 Aug 2 22:10 Applications
0: group:everyone deny delete
4310361351 drwxr-xr-x+ 63 root wheel sunlnk 2016 Sep 24 2018 Library
0: group:everyone deny delete
4310396148 drwxr-xr-x 2 root wheel hidden 64 Sep 24 2018 Network
4310396149 drwxr-xr-x@ 5 root wheel restricted 160 Sep 21 2018 System
com.apple.rootless 0
0: group:everyone deny delete
420280 drwxr-xr-x 4 root admin - 128 May 1 2010 Users
420284 drwxr-xr-x 3 root wheel hidden 96 Sep 21 07:12 Volumes
4310945599 drwxr-xr-x@ 37 root wheel restricted,hidden 1184 Jul 24 23:08 bin
com.apple.rootless 0
4310945670 drwxrwxr-t 2 root admin hidden 64 Sep 24 2018 cores
305 dr-xr-xr-x 3 root wheel hidden 4308 Sep 15 22:11 dev
4310945672 lrwxr-xr-x@ 1 root wheel restricted,hidden 11 Sep 24 2018 etc -> private/etc
com.apple.rootless 0
5 dr-xr-xr-x 2 root wheel hidden 1 Sep 21 23:31 home
4310945673 -rw-r--r-- 1 root wheel hidden,compressed 313 Aug 18 2018 installer.failurerequests
3 dr-xr-xr-x 2 root wheel hidden 1 Sep 21 23:31 net
4310250552 drwxr-xr-x 8 root wheel sunlnk,hidden 256 Sep 24 2018 private
4310946809 drwxr-xr-x@ 64 root wheel restricted,hidden 2048 Jul 24 23:08 sbin
com.apple.rootless 0
4310946911 lrwxr-xr-x@ 1 root wheel restricted,hidden 11 Sep 24 2018 tmp -> private/tmp
com.apple.rootless 0
4310946912 drwxr-xr-x@ 9 root wheel restricted,hidden 288 Sep 21 2018 usr
com.apple.rootless 0
4310982149 lrwxr-xr-x@ 1 root wheel restricted,hidden 11 Sep 24 2018 var -> private/var
com.apple.rootless 0
How Apple's Unix Hides
But as can be seen already above, Apple's Unix does more than protect - it hides. And hiding is not a good thing. Not ever.
AppleShowAllFiles isn't available as a menu option. You have to know this possibility exists, and it's not exactly being broadcast across the Internet. It obviously is there so Apple's people can get at it. If they wanted their users to get at it, they'd have it available as a menu option. But they don't.
Apple's 'Finder' (it's supposed to be a file manager) won't even reveal the third 'execute' permission bit, nor will it reveal access control entries or file flags - nor is it capable of reaching all destinations in the file system, 'AppleShowAllFiles' or no.
The directory listed above as '.HFS+ Private Directory Data?' doesn't have a question mark ('?') in its name, but is rendered as such by the Unix command line, because the final character in its name is the unprintable '\r', the 'carriage return' character, dating back to the days of teletype machines.
The directory is used by the system backup facility. In this case, Apple took things a step further, trying to make access impossible (at least for mere mortals).
This is akin to the system in use for many years to hide 'multi-linked files', a system that went even further to obfuscate: in that case, .DS_Store was employed to keep the display coordinates in the tens of thousands so no computer screen anywhere would ever find it.
The same goes for directories found under /private/var/folders. The names here used to begin with a 'dash' character ('-') in another attempt to thwart access, even for those who have access rights and perhaps reasons for being there.
Apple go even farther by perverting the low-level API getdirentries(). This API, a standard part of Unix, is supposed to be used to prune directory enumerations for the higher-level API readdir. What Apple did was - under file system level - mark files with the bogus inode value '0' ('zero') used in Unix to mark files scheduled for unlinking by the 'lazy write' system.
They did this so you wouldn't know that certain files actually existed. (The 64-bit version of getdirentries() is no longer available to user-land code, another attempt to hide things.)
The user directory ~/Library is today marked with the Apple-specific file flag 'Hidden'.
The file flag 'Hidden' is specific to Apple. This is a replacement for the old '.hidden' file used in directories on HFS file systems. The file flag variant works on all file systems, even the new APFS.
The file flags 'Restricted' and 'Tracked' are also specific to Apple. Their exact portent hasn't been broadcast either, but neither of them bode for more 'user-friendly'.
Access control lists can be used to augment protection but so far don't contain mechanisms to hide anything. File flags seem to rank higher than access control lists anyway.
Apple's extended attributes (XAs), on the other hand, can indeed be used to hide (and more).
Previous experience with their iPod showed Apple that, even if some things that have been hidden are suddenly exposed, and even if the barrier is no longer insurmountable, the overwhelming majority of users will either never learn how to gain access or never have the guts to try.
A key part of Apple's proposal to get record companies to sign off on iTunes was convincing the companies that file-sharing would not be a problem. Apple had to make it difficult for iPod owners to find their downloaded songs on their own devices. This was accomplished by making the core directory iPod_Control invisible.
And so forth. If history has anything else to teach us about Apple and their customers, it's that those customers can be fooled into almost anything, and that Apple are seldom adverse to taking advantage.
We will never do music. We will never do video iPod. We will never do phone. We will never go Intel.
- '9toX'
For you Apple apologists claiming Apple will never lock down the Mac, step one is in place and you all let it happen.
- Ron Gilbert
About RixstepStockholm/London-based Rixstep are a constellation of programmers and support staff from Radsoft Laboratories who tired of Windows vulnerabilities, Linux driver issues, and cursing x86 hardware all day long. Rixstep have many years of experience behind their efforts, with teaching and consulting credentials from the likes of British Aerospace, General Electric, Lockheed Martin, Lloyds TSB, SAAB Defence Systems, British Broadcasting Corporation, Barclays Bank, IBM, Microsoft, and Sony/Ericsson.
Rixstep and Radsoft products are or have been in use by Sweden's Royal Mail, Sony/Ericsson, the US Department of Defense, the offices of the US Supreme Court, the Government of Western Australia, the German Federal Police, Verizon Wireless, Los Alamos National Laboratory, Microsoft Corporation, the New York Times, Apple Inc, Oxford University, and hundreds of research institutes around the globe. See here.
All Content and Software Copyright © Rixstep. All Rights Reserved.
|