Is The Internet Now Just One Big Human Experiment In Manipulation?

Reprinted from the Guardian, by Dan Gillmore

It’s Not Only Facebook Treating Us Like Lab Rats – Dating Sites Can Manipulate Our Emotions, Too – And Blame It On User Testing – The Possibilities Are Endlessly Scary

‘I understand … why the anger is there,’ OKCupid’s co-founder said. ‘But people also need to understand that … nobody launches a redesign without testing on different users.’

Table of Contents

If you thought the internet industry was chastened by the public firestorm after Facebook revealed it had manipulated the news feeds of its own users to affect their emotions, think again: OKCupid.com, the dating site, is now bragging that it deliberately arranged matches between people whom its algorithms determined were not compatible – just to get data on how well the site was working.

In a Monday blog post entitled – I’m not making this up – “We Experiment On Human Beings!” the site’s co-founder, Christian Rudder, essentially told us to face the facts of our modern world … at least as he sees them:

[G]uess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.”

Human experimentation is definitely part of how websites work, in a way, because all online services of considerable size do something called A/B testing – seeing how users respond to tweaks, then adjusting accordingly. But that doesn’t mean sites can, do or should routinely and deliberately deceive their users or customers.

Yet Rudder – whose observations about data on his site’s “OKTrends” blog were almost always fascinating when he was posting regularly – acknowledges that OKCupid wasn’t merely A/B testing when it recently tried to figure out whether its human recommendation algorithm was actually correct:

“To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)”

Where I come from, we call this deception, and the Washington Post’s Brian Fung asks, reasonably, “If you’re lying to your users in an attempt to improve your service, what’s the line between A/B testing and fraud?”

But when BuzzFeed’s Charlie Warzel queried Rudder [SEE BELOW], the OKCupid chief was unrepentant – and he largely took Facebook’s side on the emotion-manipulation issue. Perhaps online date-seekers are more forgiving of such experimentation than the rest of us seem to have been – even though Facebook is probably harder to give up – but I have to wonder if they will continue to trust a service that misleads them, even in the name of getting better data.

If this kind of experimentation is becoming the norm, we can only imagine what other companies will feel free to do as they, too, “experiment on human beings” as part of their business models.

Because they will experiment, and because they’re so hungry for more page views and “engagement”, news organizations could conceivably deliver two versions of stories: one version of an article or video could faithfully report what the outlet’s reporters have learned; the other could tailor that story according to the algorithmically assumed biases of the reader, with wording, photo and video selections designed to raise or lower blood pressure depending on what editors wanted their audiences to feel. You can imagine how this could play out in coverage of, say, the crisis in Gaza. Tomorrow’s power-hungry media barons, like William Randolph Hearst a century ago, must be overjoyed that human manipulation is simply the way things work.

Or try online shopping: I hate to give anyone ideas, but stores might figure out how to generate the best profits by manipulating the placement and visibility of friendly and unfriendly product reviews from other customers.

Ooh, and imagine what fun your healthcare providers could have testing unproven new medicines on you. Hooray, we can all be part of the world’s biggest drug trial!

These are merely hypotheticals, and testing and experimentation are not bad – they serve a valuable purpose. But let’s not get too comfortable as we unwittingly become lab rats. Let’s not get comfortable at all. And let’s especially not let experimenters conduct their tests in the dark. When disclosure and consent aren’t part of the process, it’s deeply wrong – and in some cases, like pharmaceutical trials, illegal.

In the way they operate, the internet companies hold almost all the cards, and their users hold almost none. We – members of the public and academics alike – should not just let it happen, argues the University of North Carolina’s Zeynep Tufekci:

“To me, this resignation to online corporate power is a troubling attitude because these large corporations (and governments and political campaigns) now have new tools and stealth methods to quietly model our personality, our vulnerabilities, identify our networks, and effectively nudge and shape our ideas, desires and dreams. These tools are new, this power is new and evolving.”

If it doesn’t clean up its lab-rat act, the internet industry is just begging for regulatory intervention beyond the obvious need to require users’ specific permission ahead of time, with full disclosure of what’s being done. I hope these companies will decide to conduct their research the right way. I’m not keen on being anyone’s virtual Frankenstein.

From The BuzzFeed Article:

Today, in his first blog post in years, OkCupid co-founder and data scientist Christian Rudder posted a tongue-in-cheek response to the Facebook study backlash titled, “We Experiment On Human Beings!” The post documents some of the dating site’s algorithmic experiments, including one where OkCupid intentionally deceived users on the quality of their matches. According to the post:

“We took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)† Not surprisingly, the users sent more first messages when we said they were compatible. After all, that’s what the site teaches you to do.”

“But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does.”

“When we tell people they are a good match, they act as if they are. Even when they should be wrong for each other.”