iPhone XS benchmarks: The fastest gets faster (and lasts longer, too)

The A11 Bionic processor in the iPhone X and iPhone 8 made those phones the most powerful smartphones of 2017. While it’s difficult to directly compare performance between Android and iOS devices, even the fastest Android phones of today haven’t quite caught up to Apple’s best from last year.

Now, the A12 Bionic makes the iPhone XS even faster, widening the performance gap between Apple and everyone else. The iPhone XS and iPhone XS Max are clearly the fastest iPhones ever made, which also makes them the fastest phones ever made. Apple has managed to wring a little more battery life out of them, too.

Here are the results of our benchmarking and performance testing of the iPhone XS and XS Max. We run every test multiple times, allowing the phone to cool off between runs. Tests are run in airplane mode where possible and with no other applications loaded. We do all of this to help eliminate inconsistencies and allow each device the opportunity to achieve its best result.

Apple’s claims and our predictions

Before we dive into our test results, let’s go over  Apple’s own performance claims. The company says that the A12 Bionic offers up to 15 percent faster performance than the A11 Bionic on its two high-performance cores, while using up to 40 percent less power. The four high-efficiency cores, while not faster, use up to 50 percent lower power. The graphics processor is claimed to be up to 50 percent faster.

There’s also a faster image processor, faster storage controller, and the Neural Engine (which speeds up machine learning tasks) is up to nine times as fast—five trillion operations per second compared to 600 billion on the A11 Bionic. 

That’s not too far off from our earlier predictions about the A12 Bionic. Apple opted to push energy efficiency a bit more than we expected, so our single-core performance predictions are a few percent higher than the real A12 Bionic’s. And since Apple didn’t really speed up the energy-efficient cores at all, the multi-core benchmarks are a little further behind our predicted results.

It was the Neural Engine performance where we greatly mis-predicted. We expected it to be the biggest speed improvement in the chip, and it is, but we thought that would mean making it about twice as fast (over 1 trillion operations per second). Unfortunately, there are no quality benchmarks to test that part of the chip yet.

Geekbench 4

Geekbench is a mainstay of performance benchmarking in part because it is available for so many platforms:  macOS, Windows, Linux, iOS, and Android. There’s no one synthetic benchmark that is perfectly comparable across platforms, but Geekbench 4 comes about as close as you can get.

Jason Cross/IDG

The iPhone XS and XS Max offer essentially the same performance. Single-threaded CPU performance is about 13 percent faster, very close to Apple’s claimed 15 percent speedup. That helps contribute to a very small improvement in multi-core performance, but since the four energy-efficient cores aren’t really any faster, the difference is minimal.

Geekbench’s GPU test uses Metal to perform computational tasks, so it’s a pretty good indicator of the graphics processor’s ability to do math without actually rendering 3D graphics on your screen. It’s almost 40 percent faster, which is impressive, though not quite the “up to 50 percent” that Apple claims.

AnTuTu v7

The AnTuTu benchmark is best known in the Android world, but it is also available for iOS. We had quite a few problems getting a stable score out of the latest version, with frequent crashes and inconsistent results. We don’t suggest putting a ton of stock into them, as it’s not always clear exactly what’s being tested or how tests differ between platforms. Nonetheless, we present the results to satisfy the curious.

Jason Cross/IDG

AnTuTu’s CPU score jumps up 11 percent on the iPhone XS and 18 percent on the iPhone XS Max. The GPU score is 28 and 40 percent faster, respectively, with an overall score about 25 percent faster. 

We wouldn’t take these results to mean that the iPhone XS Max is significantly faster than the XS. As we said, AnTuTu delivered often wildly different results and crashed frequently. If we could run the test a hundred times on each phone and average the best results, we’d probably see very similar scores. Chalk up the difference to benchmark inconsistency or iOS instability rather than any meaningful difference in hardware between the regular iPhone XS and the XS Max.

3DMark

Our favorite 3D graphics performance test is 3DMark. The Sling Shot version of the test runs a graphically intensive game-like scene before delivering a final performance score.

Jason Cross/IDG

There are two versions of the test here. Sling Shot Extreme runs the benchmark using Apple’s Metal API at a resolution of 2560×1440, which is then scaled to the output resolution of the test device. Sling Shot Extreme Unlimited renders the same test off-screen to avoid any speed limits from display scaling or vsync.

Really, the results are almost identical to that of the iPhone X. What happened to the “up to 50 percent” GPU speed improvement of the A12 Bionic? Well, it’s quite possible that this test, with all of its large art assets, is entirely bottlenecked by memory bandwidth and cache performance rather than the GPU’s ability to perform computations.

That theory is somewhat validated if we run the older Ice Storm Unlimited test.

Jason Cross/IDG

This test runs a simpler game-like scene using OpenGL ES 2.0 at a resolution of 1280×720 with a fixed time-step between frames (so all devices render the exact same frames).

This less strenuous test, with comparatively low-res art assets and a lower resolution, puts less demand on memory bandwidth. It runs around 18 percent faster on the new hardware.

Battery life

It’s difficult to get a realistic picture of battery life on a smartphone. Battery drain is more than just a function of processor efficiency—the brightness and efficiency of the display plays a major factor, as does wireless networking, RAM, and a host of other factors. Everyone uses their phone differently and in different conditions, so it’s all but impossible for a battery benchmark test to tell you what your everyday experience will be.

We test battery life using Geekbench 4’s “full rundown” battery life test, which continually loops Geekbench tests from a full charge until the device drains completely and shuts off. We run the tests with display brightness calibrated to 200 nits on a full-white screen, with auto-brightness and True Tone disabled. The “dim display” option in Geekbench is turned off, too.

Jason Cross/IDG

What you’re looking at is nearly a worst-case scenario—screen always on at comfortable “bright indoor office lighting” setting, running very CPU and GPU intensive tests nonstop. In the real world, you’re going to get a lot longer battery life than this, especially if you perform mostly simple tasks. It’s not a great analog to everyday use, but it’s a consistent and repeatable test that compares fairly between devices.

Despite having a roughly 3 percent smaller battery than the iPhone X (2658 mAh versus 2716 mAh), the iPhone XS lasted 7 minutes longer. That’s not the “up to 30 minutes longer” that Apple advertises, but this is a worst-case torture test. In regular use, the difference would be exaggerated, and it’s easy to believe the average user will indeed see a good half-hour longer battery life.

The iPhone XS Max lasts 50 minutes longer than the XS, and 57 minutes longer than last year’s iPhone X. That’s to be expected, with its comparatively big 3174 mAh battery (the largest ever in an iPhone). That’s more than 20 percent longer battery life under these extreme test conditions. Apple claims up to 1.5 hours longer battery life over the iPhone X, and if it lasts almost an hour longer while running Geekbench non-stop, it’s easy to believe you’ll get at least 1.5 hours more doing simpler activities.

It’s especially enlightening to see how far Apple has come in the two years since the introduction of the iPhone 7 Plus, formerly the iPhone with the largest battery (at 2900 mAh). The iPhone XS, with a smaller battery, lasts over 30 minutes longer in this torture test. The iPhone XS Max lasts 83 minutes longer. That’s 36 percent longer battery life with a 10 percent larger battery and a much larger display.

Overall performance notes

The most important aspect of a device’s performance are not the benchmark results, but how fast and responsive the device feels. We’d take a phone that has smooth scrolling and snappy touch response but poor benchmark results over one that tops the benchmark charts but stutters, hitches, and drops frames all the time.

Frankly, the iPhone 8 and X are already so fast that most of the things you do just won’t feel any faster with an iPhone XS. In particular, iOS 12’s focus on performance (the stuff you feel rather than higher benchmark scores) makes everything so snappy and smooth on last year’s phones that it sometimes seems like there’s just nowhere left to go.

However, we did notice a few areas where we think you’ll notice a significant improvement with the iPhone XS, even compared to the iPhone X or iPhone 8. Augmented Reality apps, for example, lean heavily on the Neural Engine to process the environment. AR games in particular, which stress the CPU, GPU, and Neural Engine, feel smoother and more responsive. Apps, too, seem to launch faster than on the iPhone X, sometimes shaving 3 to 5 seconds off the load times of big apps like games. And of course, all that processing power is used to help produce better photos.

Apple also keeps improving battery life, all while giving you hands-down the fastest mobile processors anywhere. It’s worth noting that a number of Android phones last a good deal longer; this appears to be largely due to battery size. The Pixel 2 XL has a 3520 mAh battery, and the Huawei P20 Pro crams in a 4000 mAh battery (26 percent more juice than the iPhone XS Max). Apple’s latest devices will easily last all day, but their battery life falls firmly in the middle of the pack among premium smartphones.