MTS@MTSliveAug 15

Awesome response - thank you for coming back with such good faith. So much I agree with here. Also a lot of things I think the picture is more nuanced and I want to justify our takes more A few quick ones, but I might post more here later. I think that cyber is ultimately defence dominant, and if we put in the work over the next 2 years (tldr every financial institution and critical infrastructure provider trying to white hat hack themselves), then I have 0 concern for any open coding model. In the interim, a totally reasonable choice is ‘we’ll take substantially increased cyber attack risk for no imposition on our freedom’ - but the USG should be in a position to actively make that choice having measured and evaluated the risks for themselves. Bio is offence dominant, and fixing that will probably take well into the 2030s - this means that we’re stuck with genuinely huge risks there which society needs to decide whether it wants to accept (I.e. world leading virologist in your pocket). It’s entirely possible that society says ‘yes those risks are worth it’, but I think society/the gov should have an arm stood up to take that seriously (e.g. run uplift trials where they see if AI helps them more than YouTube access, genuinely test what the worst thing someone could do with 200k and a garage is etc), evaluate it for themselves and make that call with each new capability level. The second is why has Dario/us talked about risk so much? Fundamentally it’s because we’ve wanted to be honest with people. Employment risk is the classic here. I actually disagree with Dario on the pace - I think it’s most likely that compute shortages, diffusion complexity, policy and unmet demand for services mean that even for years after we have models which could automate 95% of computer facing jobs (models will get there in 28), people will work at them (well into 2030s) - but I do think we as a society should take the possibility far more seriously than we are now, and prepare contingency policies for what to do at various levels of unemployment (e.g. you could imagine not letting profitable companies lay off more than 5% per year), as well as METR style evals to measure progress on different job families so we have a clear picture. Our opinion has always been that we need to be straight up and honest with people. Completely agree that as both a company and an industry we have utterly failed to present a positive picture of a future people want to fight for, and that this is actively decreasing our chance of getting to that future. So much we need to do better there. One final half baked one - I don’t think the Mark essay engages with the actually hard part of the problem, which is that inequality of compute will matter far more than open models in having personal superintelligence. Will talk more about this later, but TLDR think the essay presents open models as a kind of panacea without getting into the heart of the issues I actually expect in the future