The Analysts Recording Your Screen Say It’s for Your Own Good
Secretly watching users navigate an app can help companies fix bugs. But it can also be a tool to manipulate customers’ behavior.
Apple issued a warning to developers last week after a TechCrunch report revealed that a number of banking and travel companies had contracted with an analytics firm, Glassbox, to record customers’ screens as they used their apps. Apple’s App Store requires developers to obtain consent and inform users if they’re being recorded. Glassbox never required its customers to alert users to screen recordings.
“We have notified the developers that are in violation of these strict privacy terms and guidelines, and will take immediate action if necessary,” an Apple spokesperson said via email.
Glassbox’s software records video of users’ screens as they use apps, then compresses and plays back the footage for analysis. This “session replay” becomes a record of every keyboard press, everything they type, the error messages they see, the amount of time they spend on each page, and so on. Session replays are versatile documents. Glassbox claims that companies could use them in legal disputes, for example, as proof that a litigious user signed a contract. But the replays also risk capturing sensitive information such as credit-card or passport numbers. Glassbox touts them as providing extremely valuable perspective on consumer behavior.
“Imagine what you could do with the insight you would get from being able to follow every customer around your store,” a Glassbox white paper reads, “knowing not only what they bought, but also the exact route around the aisles they took, what they stopped to look at and for how long, what they picked up and put down again, etc.”
Glassbox has contracts with major U.S. banking companies, including Bank of America and Wells Fargo, according to its site, in addition to major retailers and travel sites such as Abercrombie & Fitch, Hotels.com, Air Canada, and Singapore Airlines. Because banks are so large, a single banking customer can generate as many as 3 billion recorded sessions a month, Glassbox’s marketing vice president, Audelia Boker, claims in a promotional video. Other analytics firms using session replays include Appsee, FullStory, and UXCam. After a 2017 Wired report, Walgreens promised to end its contracts with FullStory.
Glassbox CEO Yaron Morgenstern explained over the phone that session replays can, in addition to offering consumer insight, identify difficult-to-spot problems in apps.
“We’re using machine-learning capabilities that look at abnormal activity so they can identify where something is wrong,” Morgenstern said. “A system process or [user’s behavior] is not behaving as expected, and based on that we identify struggles. What we do is we help our customers to find those struggles.”
Morgenstern discussed the example of an application form in a banking mobile app. The bank may notice, for example, that customers get to page eight of a 12-page application, then suddenly stop. Or, conversely, that page eight takes the longest to fill out. Glassbox could flag this for the bank and view a session replay to find the issue: an undiscovered software bug or poorly worded question.
“It’s not only that we know that at some point or on some page or some area within the app customers are failing; we actually point them to the reason the customers were struggling: a problem with the app, something with the way the application was designed, a problem with the query or UX, etc.,” Morgenstern said. “All this is embedded within our … machine-learning capabilities, which point to the very specific elements that cause the struggles.”
Glassbox uses machine-learning analysis and real-time session replays to suggest changes to its customers and improve apps. These suggestions aren’t only useful for solving user-experience problems, but can also be exploited to calibrate site layouts so customers spend more time and money while using an app. The industry term for this is journey management, maximizing revenue by analyzing how customers make their way through an app. This is better known as the “nudge,” and it’s a very old concept. A grocery store may “nudge” shopper behavior by rearranging its store so that healthier foods are prominently displayed in the front while snacks and junk food are farther toward the back. It isn’t a regulatory change or even a direct announcement, but it still affects consumer behavior.
Since Apple’s warning, Morgenstern said Glassbox is working with customers to ensure compliance, advising its clients to only record screens if they have each user’s permission first. “We’ve taken a proactive approach and are increasing our team that is working under our director of compliance, and expanding the services we offer to our customers to make sure they’re getting all the support they require,” Morgenstern said.
While developers are moving to ensure that they aren’t booted from the App Store, session replays are still built on the idea of observing customers in their “natural” state, unaware of the suite of tools analysts use to quantify their every move. Consent is essential, but inherent to the session replay is the idea that customers are most valuable when they don’t know they’re being watched.