Notes to: Yvain's "Priming" Sequence

This is commentary. And this is really good. 

A four-part "sequence" or series on "priming" by Yvain (AKA Scott Alexander).

1. Never Leave Your Room
2. Bogus Pipeline, Bona Fide Pipeline
3. The Implicit Association Test
4. Fight Biases, or Route Around Them?

"Never Leave Your Room"

• "Psychologists define 'priming' as the ability of a stimulus to activate the brain in such a way as to affect responses to later stimuli. If that doesn't sound sufficiently ominous, feel free to reword it as 'any random thing that happens to you can hijack your judgment and personality for the next few minutes." 
• The article then discusses priming in the following situations: an increase in competitiveness in the Ultimatum Game by seeing a briefcase; children being more likely to share after seeing an image of St. Nicholas' hat and less likely when shown the Toys 'R' Us logo; rating Bush's policies at 42% in control (random alphanumeric combinations) but 75% after seeing "alphanumeric combinations that recalled the 9/11 WTC attacks (ie '911' or 'WTC'); effect on voting results according to polling location, such as people voting at school being more likely to support education-friendly policies, by an amount of 3 percentage points. All cases are drawn from existing studies.
• "Avoid exposure to any salient stimuli in the few minutes before making an important decision. Everyone knows about the 9-11 terrorist attacks, but the War on Terror only hijacked the decision-making process when the subjects were exposed to the related stimuli directly before performing the rating task." 
• "Second, try to make decisions in a neutral environment and then stick to them. The easiest way to avoid having your vote hijacked by the location of your polling place is to decide how to vote while you're at home." 
• "Instead of never leaving your room, you can make decisions in your room and then carry them out later in the stimulus-laden world." 
• "I can't help but think of the long tradition of master rationalists 'blanking their mind' to make an important decision... Your grandmother telling you to 'sleep on it' before you make an important life choice." 
• "Waiting a few minutes in a stimulus-free environment before a big decision might be a good idea." 
• "Priming is one of the phenomena behind all the hype about subliminal advertising and other subliminal effects. The bad news is that it's real: a picture of popcorn flashed subliminally on a movie screen can make you think of popcorn. The good news is that it's not particularly dangerous: your thoughts of popcorn aren't any stronger or any different than they'd be if you just saw a normal picture of popcorn." 
• Cannibal Smith, commenting: "Can we use this to our advantage? Maybe tattoo 'Less Wrong' on one's palm and look at it when in distress?"
• Yvain, commenting: "That's a very interesting idea. Again, it reminds me of certain forms of magic - for example, of magic practitioners who used to surround themselves with imagery of the God of Music before a music recital on the theory it would help them play better...I suspect that saying a prayer to God for help resisting temptation may work in the same way--you're activating your concept of God and with it the entire religious system of morality and self-control. In the same way, giving yourself a reminder of rationalism or even reciting one of those rationalist litanies might make the stuff you learn here more salient." 
• Matt_Simpson, commenting: "A friend of mine is interested in chaos magic which, from what he has told me, is essentially a collection of psychological tricks to perform on oneself. There even seems to be a tacit awareness that it's not actually magic, though I don't think the awareness ever becomes explicit." 
• ciphergoth, commenting: "AFAICT most chaos magicians believe in the supernatural, but at least some have been thoroughgoing materialists who believe only in the psychological power of suspension of disbelief." 
• Yvain, commenting: "The few times I experimented with similar techniques (usually expressed as self-help rather than magic), I found it impossible to suspend disbelief, and ended up laughing at myself and giving up. I wonder if other rationalists would have the same experience, and if this would be different between self-selected rationalists and random people put through a course in a rationality dojo. I'd also like to see whether this is mediated by the hypnotizability trait."
• eirenicon, commenting: "The most interesting thing I've read on it says you ought to laugh and make fun of yourself, because chaos magic is ridiculous and can't possibly be real. Then you forget about it, and then it works." 
• CJewell, commenting: "It would not be a stretch to say that the Yudkowsky story alluded to in the main article exhibits some of the same psychological tricks that 'chaos magicians' practice. The pure white rationality dojo, the symbolic plaques, the importance invested in each of the rooms, the sort of cleansing ritual and meditation techniques the main character uses. And most importantly to Chaos Magicians: 'Symbols could be made to stand for anything; a flexibility of visual power that even the Bardic Conspiracy would balk at admitting outright.' The power of symbols is super important. Even the more supernatural aspects of Chaos Magic could be looked upon as a practice in belief annihilation, much like Eliezer's story. By examining the mechanism of belief through practice and meditation, one starts to exhibit more control over one's beliefs. More control, the more likely one will catch irrational beliefs. When one has complete control over beliefs, then rationality can control the monkey brain...btw, laughter is a haos magic dispelling/banishing technique."
• pjeby, commenting: "Suspension of disbelief merely means that you withhold your rejection of input as false. In order to understand something at all, you have to temporarily represent it in your mind as true... which is why there are studies that show you can convince people of things by telling them the idea and then distracting them before they have a chance to analyze or reject it. Disbelief is an active, conscious process; belief is the default... An ability to logically disbelieve probably had to evolve as a defensive weapon against human liars -- it doesn't make much sense to have it until you also have language and imagination. [ellipsis original]"
• pjeby, commenting: "Commitment rituals aren't very useful for telling whether you're actually committed. The real test is whether, for all the worst-case scenarios you imagine possible, you feel you can accept and handle that as the outcome... And that's not something that 'taking the red pill' [or red smartie] is going to make happen, since it most likely induced you to suppress your doubts, rather than face them." 

Whenever you're called on to make up your mind,

and you're hampered by not having any,

the best way to solve the dilemma, you'll find,

is simply by spinning a penny.

No -- not so that chance shall decide the affair

while you're passively standing there moping,

but at the moment the penny is up in the air,

you suddenly know what you're hoping.

--Piet Hein

"Bogus Pipeline, Bone Fide Pipeline"

• The Bogus Pipeline is a means of checking lying by having test subjects take a survey, surreptitiously copying that survey, hooking them up to a fake liar detector, asking them questions from the survey, pinging them whenever the answer diverges from the survey, and then eventually asking new questions. "Since its invention in the 70s, several different studies demonstrate that its victims will give significantly less self-enhancing answers to a wide variety of questions than will subjects not connected to the machinery. In cases where facts can be checked, Pipeline subjects' answers tend to be more factually correct than normal subjects'." 
• The Bona Fide Pipeline... "uses a complicated process to disguise itself as an ordinary study on distraction or face recognition or somesuch, but the active ingredient is this: the subjects play a game where they must hit one key... if the screen displays a good word... and a different key... if the screen displays a bad word."
• "But before it gives you the word, it shows a picture of a white person or a black person. Remember priming? That picture of a black person is going to prime your brain's concept of 'black person' and any concepts you associate with 'black person'. If you have racist attitudes,.. you're going to have a very easy time recognizing 'ugly' as a bad word, because your 'bad' concept is already activated. But you're going to have a harder time recognizing 'wonderful' as a good concept, because your brain is already skewed in the opposite direction. It's not impossible, it's just going to take a few hundred more milliseconds. Each of which the Bona Fide Pipeline is recording and processing. At the end, it spits out a score telling you that you took an average of three hundred milliseconds longer to recognize good words when primed with black people's pictures than white people's pictures." 

"The Implicit Association Test"

• "If you're never taken the Implicit Association Test before, try it now." 
• Article goes on to list data supporting the IAT. 
• "Now comes the prosecution. A common critique of the test is that the same individual often gets two completely different scores taking the same test twice. As far as re-test reliability goes, .6 correlation is pretty good from a theoretical point of view, but more than enough to be frequently embarrassing. It must be admitted: this test, while giving consistent results for populations, is of less use for individuals wondering how much bias they personally have." 
• "I myself think racism is a bad word. Not in the way 'shit' is a bad word, but in the way 'wiggin' is a bad word. It divides experience in a perverse way, drawing a boundary such that Adolf Hitler ends up in the same category as the guy who feels a pang of guilty fear late at night when he sees a big muscular black guy walking toward him. Taboo the word 'racism', 'prejudice', and any other anti-applause-light, and a lot of the IAT debate loses its meaning."
• "I think that the IAT is about much more than who is or isn't racist. The IAT is a tool for measuring distances in thingspace." 
• "Thingspace, remember, is the sort of space in which we draw categories. 'Chair' is a useful category because it describes a cluster of things that are close together in concept-space in a certain way... Quok, where a 'quok' is defined as either a chair or Vladimir Lenin, is a useless category, because Lenin isn't anywhere near all the other members." 
• "Speaking of communists, remember back when East and West Germany got reunited? And remember a little further back, when North and South Vietnam got reunited too? Those reunifications, no matter how you feel about them politically, were natural links between culturally and historically similar regions. But imagine trying to unite East Germany with South Vietnam, and West Germany with North Vietnam. The resulting countries would be ungovernable and collapse in a matter of weeks." 
• "If you provoke a war between the reunified Germany and Southeast Vietnermany, and watch which side coordinates its forces better, you get the Implicit Association Test." 
• "There's also fear someone might use it for, say, evaluating applicants for a job. Due to its weakness as an individual measurement and the uncertainty about how well it predicts behavior, this would be a terrible idea." 
• "Full disclosure: Despite strongly opposing prejudice on a conscious level and generally getting along well with minorities in my personal life, I get assessed as moderately biased on the racism IAT. I had some memorable bad experiences with certain black people in my formative years, so this doesn't much surprise." 

"Fight Biases, or Route Around Them?"

• "I've not yet seen it pointed out that we use 'bias' to mean two different things. Sometimes we use 'bias' to mean a hard-coded cognitive process that results in faulty beliefs... Other times, we use 'bias' to mean a specific faulty belief generated by such a process, especially one that itself results in other faulty beliefs."
• "You don't have to believe something for it to be a belief; consider again the skeptic who flees the haunted house. She claims she doesn't belief [sic] in ghosts, and she's telling the truth one hundred percent. She's still going to be influenced by her belief in ghosts. She's not secretly supernaturalist any more than someone who gets 'strongly biased' on the IAT is secretly racist. But she needs to know she's still going to run screaming from haunted houses, and IAT-takers should be aware they're still probably going to discriminate against black people in some tiny imperceptible way." 
• "By making a group take the Implicit Association Test, applying a technique to them, giving them the test again, and seeing how their score changed, we gain the ability to test bias-fighting techniques. I wouldn't want to do this on one person, because the test only has moderate reliability at the individual level. But a group of a few dozen, all practicing the same technique, would be quite sufficient. If another group learns a different technique, we can compare their IAT score improvement and see which technique is better, or if different techniques are better in different circumstances." 
• "In one of the IAT experiments, subjects evaluated essays written by black or white students. This is a fiendishly difficult task upon which to avoid bias. A sneaky researcher can deliberately select essays graded as superior by a blind observer and designate them 'white essays', so anyone trying to take the easy way out by giving all essays the same grade can be caught immediately. I like this essay task. It's utterly open to any technique you want to use to reduce bias." 
• "This, then, is one solution to schools [of psychology] proliferating without evidence. With enough research, it could be turned into one of the missing techniques of rationality verification." 
• "Much of what is in this essay would work poorly (though probably still better than nothing) with a simple IAT. But having someone taking the IAT ten times over ten days and averaging the results might give a more accurate picture... And in any case the IAT is quite good at comparing groups of people with sample size >1."
• Stuart_Armstrong, commenting: "Anchoring may still be a problem here. Your Isaac may decide to try and debiase [sic] himself through the methods you suggest, but when will he stop these efforts? When he's reached a point where he feels reasonable debiased - which will be an improvement, but still on the Israeli side of 'reasonable'. If Sayed Muhammad had done the same thing from the opposite side, he would have ended up on the Palestinian side of 'reasonable'. And the span of reasonable may be quite large. Actual debiasing will require efforts 'beyond the call of duty', to avoid the anchoring effect. Another approach, if both Isaac and Sayed Muhammad are honest in their quest, is to lock them together in a room and only let them out when they agree (not when they 'have negotiated a reasonable compromise', but actually agree on the issues)." 
• Yvain, commenting: "The score you should get on the IAT should be correlated to your conscious opinion. If you consciously think Palestinians are inferior, then you should be happy with an IAT score showing you think Palestinians are inferior. If you consciously think Palestinians are equal to Israelis, you should be trying to get an IAT score reflecting that equality. It's all about trying to get the unconscious mind to correspond to your rational beliefs... My conscious, rational brain believes that most Palestinians are probably decent people who have been driven to extremes by their situation. My conscious mind also believes that the best Middle East peace plan is one where everyone, Israeli or Palestinian, is considered equally deserving of happiness simply because they are human. That's my moral system, and yours may differ. The point is, that is my moral system, and I would like to be able to operate on it. If I'm going around subconsciously thinking that Palestinians are bad and don't deserve happiness, I can't enact my goals." 
• Thomblake, commenting: "'Hating Hitler doesn't mean you're biased against Hitler.' Doesn't it? Does it make it less likely that you'll give Hitler a job? Even one that he might be very good at? Okay, Hitler's not a very good example--he's dead, and that makes him pretty bad at most jobs. Am I biased against dead people? I think so." 
• Vladimir_Nesov: "You are biased against a person only if you devalue her too much, compared to the extent that you should. You are biased against Hitler if you e.g. are expected to systematically don't [sic 'not'?] give him a job that you should've given."