Ever since the Middle Eastern terrorist group known variously as ISIS, ISIL or Daesh first started intruding on the public consciousness, the organization’s savvy use of social media has been one of its calling cards. With YouTube videos of prisoners being beheaded or burned alive, and Twitter accounts that spread its message of radical Islam, ISIS has adopted social networks and the web as a distribution method a lot faster than many Western media organizations and governments.
Given that, it’s not surprising that governments have been trying to convince online services like Google and social platforms such as Twitter and Facebook to help cripple ISIS in any way possible. The latest in that effort was a series of recent meetings in Washington, where about 50 different digital companies met with various members of the U.S. government’s anti-terrorism agencies to talk about cutting the flow of digital oxygen to ISIS.
The problem is that these attempts, as laudable as their goals may be, involve security agencies asking Google, Twitter, and Facebook to suppress and/or promote certain kinds of speech. And this is one of the slipperiest of slippery slopes imaginable. Who decides which speech is to be tolerated and which isn’t? The Pentagon? Facebook? The same secret FISA court that decides who the NSA can spy on?
As a number of observers like Electronic Frontier Foundation director Jillian York have pointed out repeatedly, we have already surrended much of what used to be public discourse to for-profit social platforms. While it’s true that free-speech laws technically don’t apply to corporations, it becomes a critical public issue when those platforms are being asked by the government to favor certain kinds of speech over others. Especially when the connection between a Twitter account or Facebook page and acts of terrorism is tenuous at best.
As a story that BuzzFeed wrote about the latest meeting makes obvious (it was the only media organization that was asked to attend, for unknown reasons), the Obama administration is not only unsure of how to accomplish its goal, but also seems unaware of just how far across the line it is asking companies like Google and Twitter (TWTR) to go. Or perhaps it doesn’t care.
For example, at one point the story notes that executives from a number of the technology companies involved have found some of the requests by the Pentagon and other government agencies to be “jarring.” Why? In at least one case, BuzzFeed says, “the Pentagon spoke with several companies—who asked not to be named as a condition of discussing the meeting with BuzzFeed News—about tweaking their algorithms to promote certain types of content.” According to a Google source:
That’s something that is always brought up in meetings. And it shows how little they understand us. This is a Pandora’s box we won’t open, because if we answer a request by the U.S. government to feature one search result over another, what’s to stop other countries from requesting the same? What’s to stop each country from tailoring the search results of their citizens to their agenda?
Google recently agreed to promote certain positive anti-terrorist messages, but only in the space at the top of its search results that is usually devoted to ads. But even this triggered fears that Google (GOOG) might also be tweaking its regular search results behind the scenes to promote or suppress certain content for political purposes, even though the company has said repeatedly that it would never agree to do this.
Evan Williams: Twitter is primarily a news system
It’s not clear what Twitter has agreed or not agreed to do. In the past, Twitter staked much of its reputation on being the “free-speech wing of the free-speech party,” and it fought repeated attempts by both the U.S. government and other governments to force the company to provide user information or to squash discussion on the network. And it couldn’t use an algorithm to do much of anything, because the main Twitter feed wasn’t filtered algorithmically.
More recently, however, Twitter has added more algorithmic features, and says it is planning to make these the default experience. In addition, the company has been much more active about banning users and accounts for various kinds of behavior including harassment (something many users have been calling for). And in a tangible sign of the pressure on it from the government, Twitter recently removed more than 125,000 accounts that were associated with violent extremists.
The problem is that once Twitter agrees to remove specific user accounts the government doesn’t like, or Facebook (FB) removes user pages, or Google promotes some search results over others, where will security agencies stop? It’s a little like the fight Apple is currently in with the FBI over unlocking a terrorist’s phone. It’s not just about that phone, it’s about the precedent that such a move would set for future cases.
Get Data Sheet, Fortune‘s technology newsletter.
There’s an additional argument against this kind of strategy that isn’t free-speech based: Watching what ISIS and other groups do through Twitter and Facebook and YouTube can actually provide great intelligence on their activities. Journalists like Eliot Higgins and others have used these kinds of accounts to great effect in tracking who is doing what. But by far the biggest threat is that social networks will become an arm of government.
Not that long ago, the Justice Department was convinced that WikiLeaks and Julian Assange were an imminent threat to U.S. freedom, so the government put pressure on Amazon to remove leaked documents that WikiLeaks had put on its cloud servers, and pressure on Visa and PayPal to block people from using their payment systems to donate to WikiLeaks. Did it try to get Twitter or Facebook to block accounts or pages? We don’t know. But it wouldn’t be surprising.
There’s a big difference between WikiLeaks and ISIS, obviously. But the next time the government tries to block or ban specific people or groups from social networks, the differences might not be so obvious. The gray area between what the U.S. government defines as terrorist behavior and the normal flow of political speech on Twitter or Facebook might become a little grayer. And if we are letting the White House decide which social accounts should be blocked or banned, we are on a very slippery slope indeed.
