[G]uess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work.”
Human experimentation is definitely part of how websites work, in a way, because all online services of considerable size do something called A/B testing – seeing how users respond to tweaks, then adjusting accordingly. But that doesn’t mean sites can, do or should routinely and deliberately deceive their users or customers.
Yet Rudder – whose observations about data on his site’s “OKTrends” blog were almost always fascinating when he was posting regularly – acknowledges that OKCupid wasn’t merely A/B testing when it recently tried to figure out whether its human recommendation algorithm was actually correct:
“To test this, we took pairs of bad matches (actual 30% match) and told them they were exceptionally good for each other (displaying a 90% match.)”
Where I come from, we call this deception, and the Washington Post’s Brian Fung asks, reasonably, “If you’re lying to your users in an attempt to improve your service, what’s the line between A/B testing and fraud In law, fraud is intentional deception to secure unfair or unlawful gain (money or other assets), or to deprive a victim of a legal right. Fraud can violate civil law (e.g., a fraud victim may sue the fraud perpetrator to avoid the fraud or recover monetary compensation) or criminal law (e.g., a fraud perpetrator may be prosecuted and imprisoned by governmental authorities), or it may cause no loss of money, property, or legal right but still be an element of another civil or criminal wrong. The purpose of fraud may be monetary gain or other benefits, for example by obtaining a passport, travel document, or driver's license, or mortgage fraud, where the perpetrator may attempt to qualify for a mortgage by way of false statements.
A fraud can also be a hoax, which is a distinct concept that involves deliberate deception without the intention of gain or of materially damaging or depriving a victim.?”
But when BuzzFeed’s Charlie Warzel queried Rudder, the OKCupid chief was unrepentant – and he largely took Facebook’s side on the emotion-manipulation issue. Perhaps online date-seekers are more forgiving of such experimentation than the rest of us seem to have been – even though Facebook is probably harder to give up – but I have to wonder if they will continue to trust a service that misleads them, even in the name of getting better data.
If this kind of experimentation is becoming the norm, we can only imagine what other companies will feel free to do as they, too, “experiment on human beings” as part of their business models.
Because they will experiment, and because they’re so hungry for more page views and “engagement”, news organizations could conceivably deliver two versions of stories: one version of an article or video could faithfully report what the outlet’s reporters have learned; the other could tailor that story according to the algorithmically assumed biases of the reader, with wording, photo and video selections designed to raise or lower blood pressure depending on what editors wanted their audiences to feel. You can imagine how this could play out in coverage of, say, the crisis in Gaza. Tomorrow’s power-hungry media barons, like William Randolph Hearst a century ago, must be overjoyed that human manipulation is simply the way things work.
Or try online shopping: I hate to give anyone ideas, but stores might figure out how to generate the best profits by manipulating the placement and visibility of friendly and unfriendly product reviews from other customers.
Ooh, and imagine what fun your healthcare providers could have testing unproven new medicines on you. Hooray, we can all be part of the world’s biggest drug trial!
These are merely hypotheticals, and testing and experimentation are not bad – they serve a valuable purpose. But let’s not get too comfortable as we unwittingly become lab rats. Let’s not get comfortable at all. And let’s especially not let experimenters conduct their tests in the dark. When disclosure and consent aren’t part of the process, it’s deeply wrong – and in some cases, like pharmaceutical trials, illegal.
In the way they operate, the internet companies hold almost all the cards, and their users hold almost none. We – members of the public and academics alike – should not just let it happen, argues the University of North Carolina’s Zeynep Tufekci:
“To me, this resignation to online corporate power is a troubling attitude because these large corporations (and governments and political campaigns) now have new tools and stealth methods to quietly model our personality, our vulnerabilities, identify our networks, and effectively nudge and shape our ideas, desires and dreams. These tools are new, this power is new and evolving.”
If it doesn’t clean up its lab-rat act, the internet industry is just begging for regulatory intervention beyond the obvious need to require users’ specific permission ahead of time, with full disclosure of what’s being done. I hope these companies will decide to conduct their research the right way. I’m not keen on being anyone’s virtual Frankenstein.