The Privacy Project: Save us from dystopia

January 21, 2020

By Charlie Warzel

Opinion writer at large

Hoan Ton-That testing the Clearview AI app.Amr Alfiky for The New York Times

Over the long weekend, my newsroom colleague Kashmir Hill had a blockbuster article about a facial recognition company “that might end privacy as we know it.” It charts the rise of Clearview AI, a company that scrapes images from social networks like Facebook, YouTube, Venmo and millions of other sites to create a repository of billions of images. Using Clearview’s app, law enforcement and government agencies can upload a photo of a person and the database will return matches to more photos and links to where the pictures came from.

You should read the whole article but one part that’s really stayed with me comes from one of Clearview’s early investors, David Scalzo:

“I’ve come to the conclusion that because information constantly increases, there’s never going to be privacy,” Mr. Scalzo said. “Laws have to determine what’s legal, but you can’t ban technology. Sure, that might lead to a dystopian future or something, but you can’t ban it.”

Mr. Scalzo’s quotation is helpful because he’s saying the quiet part out loud. His reasoning is alarming: Privacy is dead and nothing should halt the march of technological progress — not even the possibility of dystopia.

Clearview’s founder, Hoan Ton-That, also seemed caught off guard when asked to imagine the negative externalities of his tech. “There’s always going to be a community of bad people who will misuse it,” he told The Times. And when faced with the bigger question — How do you feel about effectively eroding the ability to be anonymous in a crowd? — Mr. Ton-That was hesitant. “I have to think about that,” he said. “Our belief is that this is the best use of the technology.”

Continue reading the main story

Mr. Ton-That and Mr. Scalzo give a master class in what the writer Rose Eveleth calls “the myth of inevitable technological progress.” Technologists decide to compare the creep of new tools to evolution — a natural process. Of course, this isn’t true. Tech doesn’t evolve naturally; it’s the result of calculated decisions by people motivated by any number of factors: ambition, greed, curiosity or even boredom.

A better, more depressing explanation comes from Al Gidari, a professor specializing in privacy issues at Stanford Law School, who argues that companies like Clearview will proliferate because “there is no monopoly on math.” He then utters the most ominous line in the article: “Absent a very strong federal privacy law, we’re all screwed.”

Professor Gidari is right. But what does a very strong federal privacy law look like? It’s hard to know. In reporting out our series on location data, I kept running into examples of companies using loopholes to skirt privacy laws like Europe’s General Data Protection Regulation.

Location data is declared technically “anonymous” even though it’s easily tied to advertiser IDs or deanonymized. And so the companies didn’t have to classify it as “personally identifiable information” and don’t have to provide it to citizens who request it. In other cases, I’ve heard anecdotes about companies simply claiming falsely that they complied with privacy laws because they knew full well that regulators wouldn’t scrutinize them.

Continue reading the main story

Without teeth and tough enforcement, a federal privacy law won’t stop techno-evolutionists like Mr. Ton-That and Mr. Scalzo. Which means the law would have to be comprehensive. This brings me to a Privacy Project essay on Monday by Bruce Schneier, a Harvard Kennedy School fellow and computer security professional. In it, he discusses the scourge of facial recognition but suggests that simply banning it isn’t enough. Instead, he argues, mass surveillance has three important components: identification, correlation and discrimination. Any privacy law needs to tackle all three.

Here’s Mr. Schneier’s key point:

A ban on facial recognition won’t make any difference if, in response, surveillance systems switch to identifying people by smartphone MAC addresses. The problem is that we are being identified without our knowledge or consent, and society needs rules about when that is permissible.

Think of it this way: Facial recognition apps like Clearview AI are an engine. But for the engine to run, it needs some kind of fuel, which is provided by other areas of the surveillance economy. In Clearview’s case, it’s the social networks like Facebook, which with lax privacy settings and default-to-public profiles, allowed its users’ photos to be scraped against the platforms’ own terms of service. None of these companies operated in a vacuum and, as Mr. Schneier notes, even random bits of information can tie anonymous data back to your true identity.

Perhaps most important, Mr. Schneier argues for “better rules about when and how it is permissible for companies to discriminate.” This is absolutely crucial as it is more about societal norms than any particular line of code or piece of technology. Setting clear rules about when technology can single us out and treat us differently based on unique identification requires that we all pause and do the difficult work of imagining the world we want to build. It means not hiding behind the false premise that privacy-obliterating technology is inevitable, as Mr. Ton-That and Mr. Scalzo have chosen to do.

Continue reading the main story

ADVERTISMENT

We don’t have to resign ourselves to dystopia. It’s not inevitable. But it will require something. We have to envision the world we want and lobby for it. We have to demand that lawmakers create a framework that allows technology to operate inside those constraints. And if we don’t, Professor Gidari is likely right. We’re all screwed.

Policing the Porch

I absolutely loved this article on Ring doorbell cameras by my newsroom colleague John Herrman. In it, he describes how Ring is changing our relationships to the places we live in ways that feel subtle now, but may soon feel drastic. “Ring is something like a home-security counterpart to the work email account on your personal phone, or the scheduling app buzzing you about a shift, ensuring you can never truly clock out,” he writes. “Home surveillance means you’re never quite home, but you’re never completely away from home, either.”

Here Mr. Herrman gets at something important. Privacy-eroding technology warps our attention by allowing us to pay attention to things (like our front steps) that we previously couldn’t. In Ring’s case there’s a personal effect, which is that you’re constantly tethered to your home. But there’s also a public element to Ring’s surveillance, as the article notes:

The presence of a camera at the door creates peculiar new forms of interaction. The father who half-seriously interrogated his daughter’s date — in a video publicized by the company and later covered by national news outlets — was at work when his phone buzzed. He conducted his grilling remotely, using the doorbell’s voice function. Ring cameras themselves are now being stolen, leaving their owners with a final few seconds of footage — a hand, a face, a mask — before losing their connections.

Already we’re seeing ways that Ring is commandeering and warping our attention. Ring’s social network, Neighbors, feeds a steady stream of porch videos onto the internet. They fuel local news coverage, sometimes depicting funny or silly moments but, more often than not, showing a petty thief absconding with a box. The clips go viral, and sometimes they’re picked up by national news outlets.

In this way, Ring makes the hyperlocal national. Our attention is directed to a porch in a small town we’ve never heard of, watching footage of somebody we don’t know that’s been recorded by somebody who isn’t home. And yet maybe we feel violated just watching those clips. The world feels less safe. We think about our own porches and the packages that may or may not be waiting for us. Who’s on our stoop? Who might show up?

The type of constant surveillance offered by technology like Ring has many obvious downsides. It can fuel our paranoia or our worst, discriminatory instincts. But, just as important, it can begin to chip away at our sense of place. As Mr. Herrman writes, it places us in a strange tech-fueled purgatory. We’re never quite in the moment, but we’re never quite removed from it, either.

I want to hear from you

Send me your pressing questions about tech and privacy. Each week, I’ll select one to answer here. And if you’re enjoying what you’re reading, please consider recommending it to friends. They can sign up here.