@slatestarcodex

I don't know, I think it really does damage the credibility of a movement with 60 year old voters in Ohio when it's hosting Slutcon and asking questions about sex with dead cows on its application forms. I do pretty regularly encounter people who at least claim that it's some tangential lifestyle thing - whether it's polyamory, or the Harry Potter fanfic, or my treating neoreactionaries as reasonable blogging foils - which has made them hate us and ignore everything we say. All of those things are real costs, and the benefit of intellectual freedom and resistance to ideological movements that take themselves too seriously may counterbalance those costs but it's a genuine tradeoff.

(though maybe this is changing - I'm hopeful that when people debate whether to acknowledge a fringe cause, then things like lifestyles are compelling counterarguments, but when people are deciding their opinion on a large cause that is thrust in front of them whether they like it or not, it's a little less tenable to reject it based on one member's sex life).

My impression is that EA has optimized for power and for the sort of strategy you need to have in order to survive if you're going to be powerful, and rationalists have optimized for other things, like intellectual freedom. This has given EA the power to affect the discourse, eg by METR writing about the Hugging Face hack or Dario publishing the Pacing The Frontier letter. This makes our mutual enemies - A16Z, NVIDIA, etc - reasonably want to bring down the powerful people who are affecting the discourse, and prove that METR is biased or Dario has a conflict of interest. EA correctly expected this and correctly limited their attack surface, and in the end, the worst that our opponents' stooges at the New York Post could manage was the actually-bad-and-not-just-a-PR-misstep FTX fraud, and a cover suggesting that METR might be woke because Beth Barnes has short hair and Ajeya is Indian. If EA had given them a bigger attack surface against METR, the New York Post would have taken it, and then their plan to use their power for good would have worked less well.

I think the reason the rationalists are confused about this is that they've gotten into seeing everything as some great EA vs. rationalist war where everything that happens must be processed into yet another piece of proof that EA has always been wrong about everything, and rationalists right. Therefore, the fact that politicians and the public are waking up must be proof that waking up the public and politicians was always easy, that MIRI's mass persuasion strategy was brilliant, and that EA was foolish for ever doing anything else. But my impression is that MIRI (and me, and many others) tried waking up the public and politicians for years, without result. What worked now wasn't that someone finally pressed a button marked TRY MASS PERSUASION, it was a special moment created by the combination of the Hugging Face hack, the Jacob Coxon resignation, and the Pacing The Frontier saga (open letter, Dario's essay, all the other labs endorsing). The Hugging Face hack is to nobody's credit, but it's to METR's credit that they did such a good job investigating and communicating it. The Coxon resignation is to nobody's credit but Coxon's, but there are swirling rumors that he got some publicity help from EAs, and of course it was very helpful that Hubinger and lots of other lab researchers immediately spoke up in his favor. The Pacing The Frontier saga is to the credit of the various lab company employees who signed it, the various EA-connected people who helped organize it, and Dario.

I think this is to the credit of EA's overall strategy, and my guess is that our opponents think the same. So when they try to cram it back in the box, they're attacking EA, who they interpret as being a dangerous and successful enemy, and not the rationalists, who they think of as being some other group of weirdos operating on the sidelines. Sure, they can manage an attack of opportunity against Nate for dating you, but if they made this the front page of every newspaper in the country, people would reasonably ask "Who's this Nate guy and why aren't they going after the bigshots like METR and Dario?" I think this is hard for rationalists in particular to appreciate because of their insistence on seeing everything through the lens of how it can be turned into yet another argument for how EA has failed and they have succeeded.

I continue to think the right answer is the one we've been doing for the past many years; effective altruism and rationality as two socially-entangled and allied movements which are distinct enough that they can pursue different strategies without sabotaging the other's strategy too badly. The rationalists' comparative advantage is freedom, which lets them attract more people, and pursue crazy controversial ideas that can then compete against each other and maybe cross the semipermeable membrane into EA after a while. The EAs' comparative advantage is respectability, which means they can interface with all the parts of society - corporations, politicians, funders, journalists - who will only interface with respectable people. I don't think either movement should be trying to force the other to assimilate, and I think a conclusion that one strategy or the other has failed would require analysis of a level much higher than I've seen people perform.

(unrelatedly, I think you're wrong about how many EAs were working in x-risk. You should look through someone's grant slush pile sometime; if nothing else, it teaches you that there are hundreds of very earnest organizations you've never heard of!)