How Apple and Facebook bring tech accessibility to the masses

This is part of CNET's "Tech Enabled" series about the role technology plays in helping the disability community.

Imagine being excluded from an inside joke shared among all your best friends simply because you can't see the photo they're laughing at.

It's a scenario Facebook engineer Matt King understands all too well.

"Even the smallest and most trite of conversations can make you feel like you're not part of it," King said in a briefing at the company's London headquarters.

King, who went blind while he was a student at the University of Notre Dame, was keen to improve the accessibility of the world's largest social network. Specifically, he was keen to give blind and visually impaired people a better experience of images.

Two billion photos are uploaded to Facebook every single day, contributing to the very visual way users communicate with one another -- but as a form of communication, they don't speak to everyone. "They may be worth a thousand words to you, but they're worth zero to me," said King.

Thanks to King, Facebook in May introduced a new tool that generates a rich list-like description of every photo using object recognition. It replaced an earlier format, which just read out that an image was there and the name of the person who uploaded it.

King's work is an example of how large tech companies like Facebook and Apple are employing different perspectives -- sometimes from their own staff -- to make their products more broadly accessible. Some of these features are hidden in plain sight. Browse through the accessibility section of your settings menu on your phone and you will probably discover a whole bunch of tools that you've never taken advantage of. Other features intended to improve accessibility end up being popular enough to be used by everyone.

A massive minority

It was a combination of Facebook's commitment to accessibility and its size that persuaded King, who spent most of his career working at IBM, to join the company. "There were well over 1 billion people using the product," he said. "The reason that struck me so hard was because I knew the demographics of disability."

Enlarge Image

The number of people with a specific disability tend to make up a small percentage of the total population. But when that is a percentage of billions, it frequently puts the number of people affected in the hundred millions. Nearly 40 million people globally are totally blind, and another 246 million have a severe visual impairment.

Facebook's tool employs a specific artificial intelligence technology, image recognition, to tell blind users in detail what is in each picture. "There is no way we could have an army of people describing photos," King said. "There's not enough human resources in the world."

Image-recognition technology can make a huge difference to visually impaired Facebook users, but its usefulness extends beyond simply improving accessibility.

Engineering for accessibility

The same is true of other emerging technologies being built into mainstream products -- tech like haptic feedback and the NFC wireless technology that makes mobile payments possible. Touchscreens need screen-reading tech and braille adapters to make them usable for people with visual impairments, but these newer methods of interacting with and getting feedback from technology are by their nature more accessible. That helps bridge the divide between mainstream tech and accessible tech.

On the iPhone 7 and the Apple Watch , the built-in taptic engine -- a mechanism that vibrates to deliver tactile responses to the user -- is a major piece of the experience. It plays a part in almost every feature built into Apple's WatchOS software.

Enlarge Image

But these features play a truly critical role in benefiting those who struggle to interact with technology. The latest timepiece from the company has a feature called Taptic Time, which informs users of the hour and minutes using a series of vibrations, something that was directly requested by blind users.

Apple designed the Watch Series 2 so that it would be usable for people with disabilities straight out of the box, something that surprised journalist Ian Macrae, who is visually impaired, when he experienced it for the first time.

"During a long career in disability media, I've seen any number of self-proclaimed life-changing, life-enhancing or life-affirming access technology gizmos," Macrae wrote in his review of the Apple Watch for his site Disability Now. "But as I said to someone on the day that I was introduced for the first time to Apple Watch Series 2, I have never fallen so immediately, so heavily and so completely for anything else."

In a follow-up interview, Macrae added that he found the extra-large digital face and the simple analog face particularly helpful because they were easier to see, along with the dictation features that allowed him to reply to messages directly from his wrist.

There are features on the Apple Watch Series 2 that were designed for disabled users first, and able-bodied users second. A prime example are the Minnie and Mickey Mouse voices, which tell you the time if you are using the relevant watch face. Apple worked with the Walt Disney Co. to bring these voices to its Voiceover feature for visually impaired users. It was so popular among staffers that Apple made it into a mainstream feature.

A "universal" culture

Innovation in the accessibility field isn't just conjured out of thin air. For a tech company, it means prioritizing accessibility and making it a key part of its culture -- another reason King said he was keen to work for Facebook. "Here was a company that has this baked into the mission," he said.

Apple takes a similar approach, using a principle known as Universal Design to ensure designers and engineers think holistically. The company says that it doesn't concoct products in a vacuum and that its employees with disabilities are also daily users of the tech themselves.

That has spawned features including Magnifier, a camera built into iOS 10, the latest version of the company's mobile software, that allows people to see an enlarged view of anything around them. Similarly, the ability to invert colors on the iPhone -- available as far back as the 3GS, introduced in 2009 -- is great for anyone who is reading at night, but it's crucial to visually impaired users.

Similarly, Microsoft kicked off an event in New York on Wednesday with a video on accessibility, showing what the company is doing for blind users of Windows 10 and encouraging developers to embrace its accessibility tools. One key feature it's introduced is the ability to change the speed of the screen-reading technology, Narrator. Many blind users are familiar with listening to and understanding speech that seems very fast to the untrained ear, and it's something Apple says blind users wanted to be sure was included in its own Voiceover technology.

"It probably sounds like gibberish, but once you get used to it it's pretty quick," said Microsoft Senior Program Manager Kelly Ford in the video.

Sometimes building accessibility features is a case of playing catch-up, and at other times the tech is ahead of the game. Facebook began implementing broad support for screen readers in 2010. Apple has built increasingly sophisticated accessibility features into every iPhone since the 3GS. Google has been a little slower to introduce a full suite of accessibility features into Android, but has largely caught up now.

Companies can still do better.

"As a blind person -- and I'm sure this is true for many other disabled people -- buying a piece of kit can be a bit of a lottery and often you just accept that you're going to have to compromise or ask for assistance in some contexts," said Macrae.

For engineers like King, banishing these feelings of trepidation is part of his job, and he's confident that tech is on the right the track.

"We're getting closer and closer to that world where we can make everyone feel much more included," he said.

First published October 26 at 5:43 a.m. PT.
Updated at 7:31 a.m. PT: Added information and video from Microsoft's Wednesday event.