regarding the amazon "glitch"

[trochee]

There seems to be a debate running about whether to attribute the sudden de-ranking of a huge number of LGBT books from Amazon (known as "#amazonfail") to malice or a dumb algorithm at Amazon.

ETA: this post contains wild speculation, some of which turns out to be wrong. See my following post about what we can learn despite my wrong guess.

Wikipedia's page on Hanlon's razor suggests that the original adage may have been a Heinlein quote (hah, now there's a hotbed of homophobia) with a form something like this:

Never attribute to malice that which can be adequately explained by stupidity, but don't rule out malice.

My take on this is to lean in the direction of malice -- some fairly well-organized darknet of Amazon rankers who raised objectionable content flags on huge numbers of queer and feminist-themed books, aided by stupidity, in that Amazon's own data-oriented review/moderation techniques are vulnerable to being swung by collaborating (or sock-puppeted) vocal single-issue factions. It might have been books about abortion, but for the relative lack thereof.

The Malice of Neighborhoods
When I say " fairly-well-organized darknet" I don't mean some malicious crowd of hackers. I mean some a mailing list of pissed-off angry white sexist homophobes, defending their turf: shopping. They are throwing their own sort of Tea Party and fighting what they believe to be a culture war over the territory that matters to them-- in this case, the territory is commercial space.

In particular, I suspect that this "darknet" minority (and this is the word we should use; outraged white sexist homophobe is, thank god, a minority in this country⁰) is feeling particularly hemmed-in, with the recent election of a black man as American President and several public affirmations of queer rights (Iowa, DC, Vermont), not to mention outrage and anger around Prop. 8. Their organization may not even be deliberate, but they are acting out in the way they feel they own -- they're going shopping, and they're going to keep Those People out of My Space; I mean Do They Really Have To Be So Public About It?

And they have done their organizing in relative private, but are acting in a relatively concerted way, using the "flag inappropriate comment". They are agitating to have Queers Not Welcome, the same way they might organize among the PTA and neighborhood associations to have a Good Vibrations storefront run out of the local mall by filing every single piece of irritating paperwork, double-checking their tax records, asking mall security to "keep an eye on them, please", etc.

The Stupidity of Crowds:
One reason that the Amazon business has scaled up as far as it has is that they are able to treat the purchase and browsing history of millions as clouds in massive data-aggregation¹. This means treating many many users as datapoints. I suspect that the stupidity on Amazon's part was two-part:

1. deciding that the cost of presenting an "objectionable" book is extremely high
2. trusting the count of "objectionable" reports to be relatively unbiased, rather than swung by a co-ordinated minority

My best guess is that Amazon did both of these. It's also possible, of course, that they put a new keyword list in², but this is far less likely than a weighting error (#1 above) or a borked independence assumption (#2).

All this is not to defend Amazon -- they screwed up, whether by malice or stupidity, and as-yet they have not come clean on what happened or why. But if I'm correct - Amazon doesn't yet know why this happened -- it's the interaction of a change in models and (possibly) an unexpectedly unified minority challenging Amazon's data-mining models.

⁰After some discussion in the comments, I want clarify: pointing out that the 'darknet get-out-of-my-mall ban trolls' are a numerical minority is meant not to take the heat off the rest of us. We still hang out in this mall [Amazon, and other online commerce] and if mall policy makes it easy to be homophobic, homophobia will stay ther. I am not asking anybody to calm down. I would like those of us thinking about homophobia, about sexism, about racism to (please!) rattle our cages when we see injustice -- even if the injustice is in our favor. it is just as bad as you have been led to believe, even if it was an accident. Our culture usually not only overlooks homophobia, but perpetuates it -- straight couples can hold hands anywhere, but gay couples have to look over their shoulder in most places. Amazon's failure this weekend -- even if it's a keyword list screwup, as Daisey's link suggests in note 2 -- shows just how close we are, culturally and algorithmically, to declaring gayness -- and sexuality -- something we must be protected from.

¹a tiny point-and-laugh, for the other NLP/datamining readers: some computer people don't get statistical models at all.

²a new keyword list is exactly what Mike Daisey reports, so I may be wrong. The question still stands, of course, about how to cope with the malice and stupidity described above. Stupidity we can work with. Malice is harder.

oo, i'm not the only person to think this:

• is it an architectural vulnerability?
• it's a symptom
• I don't think it was a single hacker (and I don't believe the ones who claim it was)
• why it won't be fixed tonight
• Patrick Nielsen Hayden has a similar thought, and one of his commenters has virtually the same analysis as me

Comments

[firinel.livejournal.com]

I'd say "Well said" except that I very much have to disagree with your statement that white sexist homophobic people are a minority in this country.

[trochee.livejournal.com]

note i did not say that they were disempowered.

it's a numerical fact, though: white people are somewhat less than 60% of the people, and thus it would require that only one in six white people is not violently sexist and homophobic for the restrictive category to sum to less than 50%.

[firinel.livejournal.com]

Ah, well 1) I wasn't thinking violently, I must've missed if it you'd specified that, but 2) I think of it like how there's systemic racism, there's systemic homophobia that affects almost all straight people, and certainly a fair deal of queer people as well, in which case "one in six" people not being homophobic would be.. well, optimistic.

Unrelated to this, but just occurred to me - was there a reason you stated 'white'? I don't recall a reason to assume anything about race at all.

thank you for asking

[trochee.livejournal.com]

good points all.

You are absolutely right: I think that lots of us -- myself included -- qualify as white, sexist, and homophobic due to my privileges of being white, male, and straight in this culture. I was specifically referring to outraged white sexist homophobic people, who are the ones who I see as having a violent, active reaction to the appearance in public of non-white, non-male, or non-straight voices. And, like q_pheevr below (who's another white dude, so caveat lector), I believe that violent reaction is in the minority and (slowly) waning. (The rest of us still allow their silencing: this is not to say we're not racist, sexist, or homophobic; at best we are actively battling our own condition.)

I could be wrong about race being a factor here, but I feel like the whole "malice of neighbors" thought is given shape by the insidious notion of defense of "whitebread normal America", and race is involved.

I call out "white" because I think that these nasty outraged "I find this objectionable" attack-you-with-the-mall-cop maneuvers are done in the kind of falsely "[color/gender/queer]blind" way that characterizes white outrage. ("I don't know why those people [black people/queers/women] have to be so aggressive about being [black/queer/female]", which not-too-subtly sets white/straight/male as the default. I believe that most people who have lived the experience of these things will still have blind spots, but are not quite so quick to declare themselves the defenders of "normal". It's much easier to spit on others if you're standing at the top of the hill.

Re: thank you for asking

[firinel.livejournal.com]

Thanks for explaining, thanks makes sense.

It was interesting because I digested your comment while I had to be afk (showering, a good time I find to think/reflect) and realised that I'd taken something about your original post to imply "it's important to remember that it's not as bad as it may seem", so I thought when I was done that I'd come back and see if you'd actually said that. On a rereading, I don't believe you've said that all all, but I can see how I took it that way, too. Whilst I am absolutely not intending to 'blame' you for the way I interpreted something, I do think that there is a long standing of people in the privileged groups saying to those not "calm down, it's not bas as you'd lead the rest of us to believe", and it may be something that if you're not aware of, you'd want to be, so could make it abundantly clear that what you're trying to address is NOT meant to imply that.

Re: thank you for asking

[trochee.livejournal.com]

thank you for the perspective. I think I will make a few edits in this direction.

[marnanel.livejournal.com]

I think that

1) while it may be true that | white ∩ sexist ∩ homophobic | < |people|/2, you don't need to be ALL of the three to do this kind of thing; it may be closer to the mark to talk of unions rather than intersections.

2) "minority" is a difficult word because its meaning in statistics is so different from its meaning when discussing privilege (in some sense, women in .us and black people in .za are and have been minorities even though they both comprise more than half the population.)

[trochee.livejournal.com]

good point, it is tricky. I agree with (1), actually; it's possible that some of the 'malice of neighbors' group is not white, or perhaps not sexist (though I'd find that hard to believe). But see my comments to firinel above.

(2) I will stand behind my use of 'minority'; in discussions of privilege we need to be clear about oppression and privilege and not try to cover the word with 'minority'.

[marnanel.livejournal.com]

outraged white sexist homophobe is, thank god, a minority in this country

I think you may be over-optimistic in this belief. :(

[trochee.livejournal.com]

see my comment to firinel above.

[eldan.livejournal.com]

Interesting. I think your explanation makes more sense than the others I've read (i.e. Amazon censorship or the "haxxor" who looks more like a common-or-garden troll), though if you're right then its a very sad instance of how a small and motivated minority get to look far more numerous than they are.

[trochee.livejournal.com]

thanks! I don't claim to have a monopoly on fact; just giving my perspective on the whole kerfuffle -- not the response to it (which is about ten seconds from winning the new best link for "Tone Argument"), but the original screwup on Amazon's part.

The link to Mike Daisey in footnote 2 suggests that it might not even be that complicated a story, but the main issues are just as important anyway -- how to assist the cloud ("web 2.0") in understanding how a vocal, directed minority could sabotage things. Think Bolshevist vanguardism, but led by James Dobson.
(edited to fix link)

[q-pheevr.livejournal.com]

Me, I really don't know how it happened. Your hypothesis seems pretty plausible, and so does Patrick Nielsen Hayden's, which is somewhat similar in spirit (careless implementation at Amazon + homophobia elsewhere = #amazonfail), though different in details. But I figure that whatever caused it, I might as well give Amazon further incentive to make sure it doesn't happen again, so I cancelled an open order and gave "#amazonfail" as a one-word explanation for why I was doing so.

And I agree with you that angry white sexist homophobes are a minority. It's very hard for anyone to be entirely free of latent sexism and homophobia in this society, but I don't think it's overly optimistic to suggest that the people who are sexist and homophobic enough to want books about gay people to disappear from Amazon search results are in the minority (let alone the ones who are angry enough to actually try to make that happen).

[trochee.livejournal.com]

of course, i have no idea how it happened either.

PNH's own screwups lately have made him extra sensitive to words ending in -fail , but he seems to have basically the same thought.

I am overjoyed that there is egg on Amazon's face right now. I know that -- in addition -- all the other big web 2.0 people are taking a good hard, erm, look at their choice of "safe search" and how it interacts with homophobia and feminism and queerness, and I am glad to see this.

I hear you about the 'minority'. Of course, a minority of eligible voters elected GWB president, and the result that was sexy-fun-times for almost no one.

[evan.livejournal.com]

http://community.livejournal.com/brutal_honesty/3168992.html

[evan.livejournal.com]

nm, i see someone else already dismissed it.

[trochee.livejournal.com]

yah; i think he's a fairly-well-informed BSer. But it *could* have been something like that; my best guess ATM is that Mike Daisey's story is right: some ontology had a semantic merge between "porn" and "gay" (and maybe "sex") and then it all went to hell.

Griefers would *love* to have pulled that off, but I'm not convinced.

I still think the questions about user-facing data-driven censorship are quite complicated and need more thinking than Amazon (obviously) did when these data got relabeled.

[chachachana.livejournal.com]

interesting! i want to share this ~

[chachachana.livejournal.com]

you know those moments when you just want to bound out of your seat and jump up and hug someone? well, consider yourself accosted as such. :)