What Apple’s Forthcoming APFS File System Means to You
Among the tidbits Apple revealed to its developer audience at the recently completed Worldwide Developers Conference was a new file system for the whole range of its products (see “macOS 10.12 Sierra to Succeed OS X 10.11 El Capitan,” 13 June 2016). Dubbed “APFS” (an acronym that Apple doesn’t completely spell out even in its developer documentation), the file system is meant to replace HFS+, the file system that in turn replaced 1985’s HFS (Hierarchical File System) in 1998. (HFS+ has received numerous updates since 1998, so don’t get the impression that it’s completely obsolete.) Apple released a developer preview of APFS with macOS 10.12 Sierra, and the company says APFS
will become the default file system in all of its operating systems — macOS, iOS, watchOS, and tvOS — by late 2017.
Changing the default file system for an operating system is a big deal, since the file system is responsible for keeping track of all of the data on the device. But what does such a change mean for users?
The Finder Is Not the File System — Unless you’re one of those rare individuals who lives on the Terminal command line and who can type ls -la
faster than you can double-click a folder icon, the Finder, along with the Mac’s Open and Save File dialogs, is normally how you see what’s on your Mac and how you navigate among your files and folders. This won’t change when APFS takes over from HFS+.
That’s because the Finder is a client of the file system. The Finder shows you a view of the items you have stored on your Mac and lets you arrange files and folders in a manner that works for you. The Finder, along with its folder and file icons, has been on the Mac since before even HFS came around, and has worked more or less the same from the user’s point of view since the first Mac came out of a bag and said “Hello” back in 1984.
The file system works behind the scenes, providing information to the Finder and to applications about the files and directories that are stored in some fashion on a device connected to your Mac — whether in magnetic fields recorded on a spinning platter or charged cells in a solid-state device. The file system keeps track of how much storage capacity files take up on the device, where on the device the data that make up the files are stored, and all sorts of metadata about those files, such as their names, when they were created and last changed, which users are allowed to open them, and a great deal of other stuff.
The Finder is thus an intermediary. Its job is to present us ordinary mortals with an easily understandable view of the data that the file system actually manages, and to instruct the file system about what you want to do with that data. As long as the Finder can communicate adequately with a file system to enable that view and pass along your instructions, the actual file system being used on a storage device doesn’t matter much.
In fact, if you have ever used a thumb drive or an SD card with your Mac, you may have already seen the Finder working with a file system other than HFS+. Most thumb drives and SD cards come formatted for Windows computers and use the FAT (File Allocation Table) file system, and yet you can still see and manipulate the files and folders on them with the Finder.
APFS is designed to understand nearly all of the same instructions and information requests that the Finder, or any other Mac application, issues regarding storage devices managed by HFS+. You’ll still be able to move files, rename files, copy files, open files, delete files, tag files, and so on just as you always did.
But You Will Notice Some Differences — However, APFS does provide some benefits over HFS+, and they’re significant enough that you will likely notice them in action.
HFS+ came along well before large storage devices containing gigabytes, let alone terabytes, were common, well before flash memory was commonly used for file storage, well before file encryption was something that ordinary users cared about, and well before Mac OS was replaced with OS X and its virtual-memory-enabled multi-tasking capabilities.
Support for large data volumes, encryption, flash drives, and virtual memory was more or less bolted onto HFS+ instead of being integrated into it. APFS has that support, and more, built in and will thus have an impact on your experience as a user.
- Faster file copying and saving: Saving files and duplicating them on the same device will happen much more quickly and without using as much space. That’s because APFS has been designed to reduce data duplication by sharing data between files as necessary. When you copy a file to another folder in APFS, no data is moved. Instead the copy points to the original file’s data on the device. If one of those copies is later changed, only the changes are stored: APFS handles which bits belong to which files. APFS’s capability to change only parts of files also speeds up file saving, and, especially, automatic saving using versions. Viewing and restoring from a previous version in an app should be faster too.
Apple’s goal of optimizing APFS for flash storage and solid-state storage devices (SSD for short) is behind this shared data approach. When you save a file repeatedly over time, the actual data making up the file can’t be stored in one contiguous chunk on the drive, and thus it becomes “fragmented.” That’s normal, but on a spinning disk, file fragmentation reduces performance by forcing the read/write heads to seek more in order to access the widely separate parts of the file. On a spinning disk, the shared data approach will mean more disk fragmentation.
But fragmentation on an SSD does not affect speed nearly as much. On an SSD, rewriting memory cells is much more costly, both in terms of speed and in terms of the cells’ usable life: rewriting SSD cells gradually wears them out. Writing just the changed file contents, and doing so in scattered physical locations on an SSD, is an efficient way to increase wear-leveling as well as to reduce the number of write operations. That’s a win-win now that most Apple devices, including Macs, rely on flash storage.
However, APFS is designed to work with spinning disk drives as well and should be smart enough to take disk location and rotational latency into account when choosing locations to store file changes on such devices.
- Snappier backups and restores: APFS can create read-only snapshots of a volume’s file contents. Once again, this capability comes courtesy of the shared data approach. Since file changes are scattered in different locations on a device, APFS can keep track of the data locations that comprise a file at any given time and make sure they are preserved. New versions of Time Machine, as well as other backup utilities, should be able to take advantage of APFS snapshots, making incremental backups that much more snappy.
-
Flexible partitions: APFS creates physical containers on a device, which can then be subdivided, or partitioned, into individual storage volumes. Unlike the partitions managed by HFS+, APFS partitions share their space with each other within the same container. This means that if you have a 500 GB APFS container, you can create multiple volumes within it that each will show up in the Finder as having 500 GB capacities. As a consequence, the amount of free space shown for each volume reflects the free space available in the container, not the individual volumes.
Although this approach could lead to user confusion, it might be mitigated by one other APFS feature: fast directory sizing. Under APFS you will no longer have to wait for seconds or even minutes while the Finder shows you the exact amount of storage used by a folder in a Get Info window. Fast directory sizing makes that information much more quickly available — useful if you want to know if your photo library will fit on the portable drive you’re bringing on your vacation.
-
More encryption options: macOS Sierra, like versions of OS X since 10.7 Lion, offers full disk encryption via FileVault 2. Similarly, although this isn’t obvious to the user, iOS has offered individual file encryption since iOS 4 and has encrypted all user files created by third-party apps since iOS 7. APFS provides both full-volume and individual file encryption: you can use full volume encryption on one volume in an APFS container and use individual file encryption on another volume in the same container.
In addition, APFS provides not only single-key file encryption but multi-key encryption, allowing a file’s data and its metadata to be encrypted separately. For example, you could have a volume that allowed a file cataloging utility to decrypt metadata about a file, such as its name and creation date, but not to decrypt the actual file contents.
Regardless, although it’s difficult to know how Apple and other developers will take advantage of these capabilities, it’s safe to say that you’ll have more encryption options available to you when APFS is in charge of your device storage. Plus, since Apple built encryption into APFS, encryption should be snappy.
Transition to APFS — Apple intends the transition to be as painless as possible. When you buy a new Mac once APFS is the default file system, you should have no transition hassle at all: Setup Assistant should move your data from your old Mac to the new one pretty much as it has all along. And, since macOS will continue to support HFS+ as well as APFS (just as it supports other file systems), you’ll be able to mount and use older HFS+-formatted external drives with no problems.
Apple also plans to provide an in-place APFS migration utility for users who upgrade older Macs to a macOS version that uses APFS as its default. Most likely, the migration process will take some time (minutes or hours) to convert an existing HFS+ device to APFS, but Apple’s goal is to make the process as simple and as safe as possible.
Let’s all hope Apple meets that goal: APFS is a big deal, offering notable performance, space-saving, and security benefits to users while promising to work its magic unobtrusively behind the scenes — as any good file system should.
Please note that I'm quite possibly wrong here. I'm reading APFS as Apple File System. That should be AFS, but that collides with Carnegie's Andrew File System, which has several years of seniority.
And thanks for the reminder of MFS (Macintosh File System) in which folders (subdirectories) didn't exist: Finder manufactured the folders it presented. You couldn't have file test in more than one folder.
I've been told by a colleague that the "P" is for "parve" which is a Hebrew term for a kosher food item that is neither milk nor meat and is hence considered neutral.
If so, APFS should be kosher to use… ?
Apple Preferred File System?
'Apple says APFS will become the default file system in all of its operating systems — macOS, iOS, watchOS, and tvOS — by late 2017'
Does this mean that iOS may get a file system addressable outside apps?
I don't think so. It simply means developers will have a common set of file system calls they can use in their apps regardless of the device.
P is for Parvenu
As in "Mademoiselle from Armentieres, Parvenu"?
Sorry, but the P stands for "parley-vous"; do you speak. Has nothing to do with parvenu, which means nouveau rich!!!
Will it be possible to have APFS and HFS+ partitions on the same hard drive?
This would allow booting to older OS versions without requiring an external drive.
Probably. I haven't dug into APFS materials but I wouldn't expect it to require a new partition scheme. The PowerPC era scheme is called Apple Partition Map (APM) and the current one is called GUID Partition Table (GPT), along with the Windows world Master Boot Record, choosing a partition scheme is an option when re-partitioning a drive.
ZFS partitions can exist within a GPT scheme and even APM so I don't see what APFS wouldn't.
Since the underlying OS kernel is UNIX-based, the file system has a heritage that dates back to roughly 1970 in Bell Labs. That's where the concepts of '.' meaning "the current directory," and "any name starting with '.' is hidden," and "root," and "inode," and many others come from ... and even those may have been inherited from an even older project, MIT's Multics.
Also, the Terminal (or bash to those of us on a first name basis) is also just a client. It doesn't change (much) with changes of the file-system type, either.
Almost every Mac is running off an HFS+ volume, and that was released in 1998. Mac didn't inherit the concept of .hidden files, etc., until OS X.
Where_have_all_my_files_gone?
APFS = APple File System. It's that simple. I hate acronyms and I figured it out.
Michael wrote: "A developer preview of APFS is baked into the forthcoming macOS Sierra."
NO it's NOT. Please revise this sentence. APFS is supplied as a PREVIEW alongside the preview of macOS Sierra. APFS is not 'baked into' anything. It is supplied for testing purposes only. You CANNOT run macOS Sierra on it. Apple is explicit in pointing out that APFS is NOT to be used at this point in its development on ANY startup disk.
If you can't tell, I'm miffed that these facts were mangled here at beloved TidBITS. Bad show Michael. Please correct your misinformation.
You may have figured out the acronym but Apple remains silent on it.
"Baked in" means "included" and, as I understand it, APFS is included in the current developer preview of Sierra for testing purposes and will continue to be in subsequent Sierra releases for developers to work with as Apple refines it. APFS won't be exposed to Mac users, but it will very much be available.
"Baked in" does not mean "included". It means "built in" or "native" or "included as an inseparable or permanent part". (I know nothing about APFS.)
APFS is not a separate developer download. It is available in the developer preview of Sierra. If your Mac is running the Sierra preview and someone hands you an external drive formatted with APFS, your Mac will be able to mount it and the Finder will display the files on the device and let you manipulate them as you would any other files.
Based on that, I'd say APFS is "built in" and "included as an inseparable part."
Why so harsh with the people you love? Is it because life itself is turning out to be half-baked?
HFS+ came out in 1998, at which time multi-gigabyte drives were standard on all Macs.
HFS came out in 1985, back when people didn't even know how to pronounce "giga" (see "one point twenty-one Jigawatts!" in Back to the Future). :)
Wait. How do *you* pronounce GIF?
"If"—the "G" is silent, just like in "right" and "knight".
This could be the best answer to that question.
Game, set, and match to Michael!
Unless you speak Pythonese, in which case "knight" is pronounced "Kanigget."
You just won the internet!
I'm intrigued about the intersection of my Documents folder on iCloud and APFS. Could that mean that I can access files in the Documents folders on, say, my MB Pro via apps on my idevices?
APFS is a red herring for your use case :] Currently there's a master copy of your iCloud folder on Apple's servers, which is synced across all your devices. I've already moved almost all my documents there. In Sierra Apple will also be (optionally) syncing the actual Documents folder, and making downloading of contents optional.
So the short answer is yes, plus you can do it currently if you move the documents to the iCloud folder.
Thanks. The only problem with moving docs to iCloud now is that the docs, though still on your local hard drive, effectively disappear from view through the Finder, except through iCloud. IOW, the move to iCloud inexplicably mangles the filenames beyond recognition and destroys the folder structure. So, I expect that, thru APFS, I'll be able to look at documents in my Documents folder both thru the Finder, as if I'd not moved them to iCloud, and via apps on my idevices, navigating the same organizational structure as I have in my Documents folder. That's way cool!
Next, we'll have to see if idevice apps can manipulate those documents -- copy, move, revise, etc. -- so that it syncs back to my local filesystem, and whether those idevice apps can actually create new documents that sync back to the local filesystem.
Not true, and again, nothing to do with APFS. The iCloud folder the Finder (currently) presents is actually stored in ~/Library/Mobile\ Documents/com~apple~CloudDocs. Under that directory, all file and folder names are exactly as you enter them. And yes iDevices can see and manipulate those files and folders, though with limitations.
Thanks again, Tom. You've pointed me in some good directions. When I moved parts of my Documents folder into iCloud a while back, the experience was dismal. Then, moving the documents from iCloud back into the Docs folder was brutal - I lost lots of files, probably due to my clumsiness, and had to painstakingly restore them from backups. Maybe I'll try again, now that it seems that the file/folder names remain intact somewhere on the local hard drive. I'll also try to learn more about APFS to understand how sharing a file system across devices won't better support handling a given file by each of those devices.
No prob :] The only scenario I know of where files are (or ever have been) renamed is when moving ePubs & PDFs into iBooks.
What impact, if any, will this have on users of SoftRAID?
You'd need to ask SoftRAID's developers that.
The notion of saving only changes to documents that have been duplicated is a spectacularly bad decision, particularly in an era with huge amounts of storage available (thereby eliminating the need for such a false economy). First, for large or complex documents that change frequently, this will progressively and continuously increase the time required to open and reconstruct the document. Second, and more seriously, this is exactly what Microsoft did for many years with its "fast save" feature for Microsoft Word, which was a guarantee of file corruption and data loss -- to the point Microsoft finally admitted defeat, stopped trying to solve the problem, and eliminated the "feature". Someone please talk Apple out of this design. Or make it a difficult-to-select option rather than the default.
The world has come a long way since Word's Fast Save feature. :-) Services like CrashPlan and Dropbox has shown that you can move bits of files around separately from the file itself and still put it all back together when it matters.
I think the performance improvements on flash-based storage devices in particular are more important than any potential space saving.
Yes, MS's system was a nasty hack. Apple's approach is different: 1. if the user duplicates a file, present them as two files but only store them on disk once, until one of them is changed. 2. for large files like virtual disk images, APFS gives the ability to overwrite just the part of the file which changed — the end result will be identical to re-saving the entire file.