Privacy Project: How much school surveillance is too much?

View in browser|nytimes.com

AUGUST 27, 2019

Florida Department of Education

By Charlie Warzel

opinion writer at large

Back-to-school season is upon us and the future of child surveillance may soon be underway in Florida.

Last month I referenced a proposed school surveillance program in the state, where lawmakers were planning to introduce a statewide database "that would combine individuals' educational, criminal-justice and social-service records with their social media data, then share it all with law enforcement."

This month we know a bit more about what that program will look like, thanks to a 19-slide PowerPoint presentation released by the Florida Department of Education. Turns out, it's quite extensive. You can look at the full presentation here and read a local news article on the report here.

A few highlights:

Officials are calling it a "safety portal" because the software will pull information from multiple databases, rather than host it all itself. Much of the information inside will be pulled from platforms like Twitter, Facebook, Instagram, YouTube, Reddit, Flickr, Google+ and Pinterest. Software deployed by the state will scan social media and general websites. It will monitor and store students' social media posts as well as the geolocation, showing where the posts were made.

According to the PowerPoint, the software will monitor keywords on five topics: "gun," "bomb," "bullying," "mental health" and "general," which is vague enough to leave room for virtually any topic. Recorded information will go into the state database (portal) and school districts, and specific "Threat Assessment Teams" will get alerts if it's determined there's a threat.

In the age of mass school shootings, such a program may seem comforting to some parents. But the program's sweeping collection parameters and the combination of information from multiple state databases mean that the portal will ultimately be home to a great deal of sensitive information. And the portal remains an excellent example of how programs designed to protect may have serious unintended privacy consequences.

For instance, according to the PowerPoint, the portal will not "store information about students' race, religion, disability or sexual orientation." However, information in the portal will contain School Environmental Safety Incident Reporting tags, which could reveal information about students who were bullied because of race, sexual orientation or disability. This type of de-anonymizing, even if accidental, is common when cross-referencing different databases.

For whatever safety it could add, automating systems to catch school shooters and other threats also increases the likelihood of technical errors. One slide from the PowerPoint notes that the portal will assess potential threats and monitored social media posts using "programmatical scoring that can help in determining relevancy of each returned record."

The slide is vague on what exactly "relevancy" means in this case, and without algorithm transparency, it's unclear whether the scoring system will bias students who may be vulnerable because of, say, a disclosed mental health issue. And since relevancy is left vague, it's unclear how often a threat will trigger the portal's flagging mechanisms. Depending on algorithmic calibration, teams could be inundated with false alarms or, perhaps, not receive them at all.

But as with any database, the biggest concern is who will have access to the data of hundreds of thousands of students. Here's how Amelia Vance, who directs the Future of Privacy Forum's Education Privacy Project, put it in an email:

There are over 4,000 schools in Florida; if we assume each school has a three-person minimum threat assessment team, that equals over 12,000 people who will have access to this portal. Even if Threat Assessment Teams are only at the district level, that is a minimum of 200 people for the 67 districts. With that many people able to access the system, it is highly likely that there will be multiple security vulnerabilities."

Perhaps the system works flawlessly (though I'm quite skeptical). Early reports suggest that the database may have serious limitations. Still, it's hard not to think about the impact on the students themselves. The surveillance state seems to be encroaching on every aspect of minors' lives lately. It comes from all angles — from overeager parents, facial recognition in summer camps and, increasingly, in school.

And it shows no signs of letting up. Near the end of the PowerPoint presentation is a slide that says, "What's Next?," which includes an action item on collecting School Environmental Safety Incident Reporting data (including categories from arson to tobacco use).

"It is currently collected 3 times a year," the slide reads. "A weekly collection is being considered."

From the Archives: "The Candid Picture"

As we continue to have the facial recognition debate, this piece from 1950 by Jacob Deschin is a great read:

The camera is, inescapably, an intimate instrument. It records for a fraction of a second a fellow being as he is. The man who operates it for public dissemination then, must have a deep sense of respect for all human life. I detest and always have detested the sneak snapshot. The photograph should never be used as a sort of modern revival of medieval bear-baiting or a Roman holiday designed to bring amusement to the masses at the expense of depreciating any human being.

Deschin would not have liked the iPhone.

Candid photography should not be used as a tool for ridicule or from malicious motives. It is a valuable and useful medium; let it not be abused. Let us picture man as we would have him picture us, with understanding and sympathy; at his best, not at his worst. The true candid worker is a thoughtful, serious individual, and the pictures he turns out are valued for their human qualities, not their vindictiveness.

Tip of the Week: Your Phone Number Is a Security Risk

A few weeks ago my newsroom colleague Brian X. Chen wrote an unnerving piece about how our phone numbers are a huge privacy vulnerability. It's an excellent bit of reporting (he got security researchers to dig up surprising amounts of personal information on him just by giving them his digits).

You should read the entire piece — it's not only important but also super captivating. And at the end, Brian has some helpful advice, which I thought I'd excerpt right here:

You might also want to set up a second phone number to cloak your personal digits altogether. You could share this second phone number with people and brands you don't entirely trust. Apps like Google Voice and Burner let you create a different number that you can use for calls and texts.

As for two-factor authentication, most tech companies offer other verification options. They include apps that generate temporary security codes or a physical security key that can be plugged in. Generally, those are safer to use than a phone number.

Here's a bonus piece of advice. If you have business cards with your personal number printed on them, shred them and order new ones with just your office line.

I want to hear from you

Send me your pressing questions about tech and privacy. Each week, I'll select one to answer here. And if you're enjoying what you're reading, please consider recommending it to friends. They can sign up here.

What I'm Reading

What happens when a technology correspondent buys a banana with a credit card? Check it out.

Footage on Twitter of protesters in Hong Kong felling a facial recognition tower.

"For decades, we handled all of your drug deals, love affairs, runaways, pimps, crime tips, cranks, heavy breathers, and emergencies. You name it, we heard it all." What's that? Public pay phones, of course. This brilliant short film takes a look at their numbered days.