@GavinSBaker

Post 1 of 2

Ok. Becky just brought me another gin and tonic and is watching our daughter. So: I think the framing of cyber as defense dominant and bio as offense dominant is sensible and the bio point is probably the best pro-regulation argument. And the USG should absolutely have all information necessary to make informed choices as cards are turned over. Agreed! Likewise agree that the compute shortage plus high DRAM prices (which effectively reduce compute per capex $) may be really civilizationally useful in giving us time to adapt to what might be a fast takeoff. And in a world that is likely compute constrained for a long, long time, I do think the computational efficiency of humans (I’m impressed that your brain runs on only 15-20 watts tbh) means that humans will be economically useful for the foreseeable future \even\ in a fast takeoff, AGI maximalist scenario. And compute inequality is absolutely going to be a massive, massive issue just like income inequality. But my argument would be the same: rather than worrying about inequality we should focus on growing the pie so fast that even the bottom of the income/compute distribution are much better off here in America than almost anywhere else. This has been the case with income and hopefully will be the same with compute. Tokens per intelligence density per token per watt for the win - hopefully America wins that race! If you’re ever in Boston, love to take you to dinner unless you are part of this terrible trend of not drinking in which case perhaps we could go for a walk. Anyways, have really appreciated everything about this dialogue so sincere thanks!

Post 2 of 2

Sholto, thank you for setting the record straight. Larger issue is that multiple very serious people in Silicon Valley have heard some variation of this and believe it to be true. And the reason it is believable to so many is that it is consistent with Dario’s public messaging and what he outlined in the essay you shared: this technology \might\ be dangerous for humans in multiple ways, could lead to extreme concentration of economic power (as outlined in the essay) and therefore needs to be regulated thoughtfully. I agree with the potential risks and I believe Dario makes all of these arguments in good faith. As discussed on the pod, if one agrees that AI \might\ be dangerous, there are two ways to address this potential risk. Either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely. Essentially boils down to whether one believes AI is too dangerous to concentrate or too dangerous to distribute. There are reasonable arguments on both sides, but I profoundly agree with Zuckerberg’s statement that: “The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic. Historically, hoping that an absolute power will benevolently provide for humanity if sufficiently enlightened has not led to safe or positive outcomes.” And as Dario says in the aforementioned essay, “some may object that we can simply keep AIs in check with a balance of power between many AI systems, as we do with humans.” I believe this is the best path forward: I want as many AIs as possible to maximize the odds that one shares my own particular values. And as Dario notes, no human has ever been able to take over the world. At this point, I think safe to say that Dario has lost the argument. His messaging has failed to result in his preferred regulatory path. The fact that the only solution to the recent incident where an unreleased advanced OpenAI model hacked Hugging Face was an open-source model likely ended any chance of strict near-term regulation. Essentially every major company other than Anthropic has signed Jensen’s letter. However, Dario’s messaging has been massively helpful to efforts to ban datacenters here in America. I suspect we will see anti-datacenter advocacy groups runnings ads using clips of Dario warning about how dangerous AI could be for humans. His good faith efforts in favor of regulation are now increasing the odds that AI will not be beneficial for Americans and humans everywhere. I believe that there is a reasonable chance AI might help us cure most forms of disease such that we have extended lifespans and can enjoy these long lives in an abundant Star Trek like future. That is the future that I want and I think Dario is decreasing the odds of that future at this point. He is about to be the CEO of one of the most important public companies in the world and given that the pro-regulatory effort has failed (at least for now), I respectfully think he should make an effort to be a more positive advocate for his own industry. And if I am wrong and we do need to regulate this technology, he will be a more effective advocate for this in the future having been open-minded to the alternative. And for the sake of clarity and as I outlined on the pod, I think Anthropic has deep competitive advantages and is an amazing company. Ironically, the main risk I saw to Anthropic a few months ago was nationalization as a result of Dario’s own rhetoric and behavior.