Anthropic co-founder: AGI predictions, leaving OpenAI, what keeps him up at night | Ben Mann
Ben Mann, co-founder of Anthropic and tech lead for product engineering, highlights the significant impact AI is already having on software engineering. He notes that at Anthropic, their Claude Code team sees approximately 95% of the code written by Claude, which allows a much smaller team to be significantly more impactful, potentially writing 10X or 20X more code. This advancement suggests a shift in how work is done, where AI assists human efforts to dramatically increase output. The concept of "Reinforcement Learning from AI Feedback (RLAIF)" is also mentioned, where models write code and other models critique it for aspects like maintainability and correctness, demonstrating AI's ability to self-improve in code generation.
The broader impact extends beyond just coding departments; for instance, Anthropic's legal and finance teams also leverage Claude Code for tasks such as redlining documents and analyzing customer and revenue metrics using BigQuery. This indicates that AI coding tools can provide value across various functions, not just traditional software development.
Ben also emphasizes the importance of using these AI tools effectively and adapting to their unique capabilities. He stresses that the focus should be on building for the future, anticipating that current partial functionalities will become fully reliable. He sees AI agents capable of computer use (e.g., handling credentials on a computer) as a huge opportunity, contingent on solving safety and alignment challenges.
Key Takeaways in regards to using AI for coding:
AI dramatically enhances productivity: AI tools like Claude Code allow for a massive increase in output, enabling smaller teams to achieve significantly more.
Example: "Our Claude Code team, like 95% of the code is written by Claude. But I think a different way to phrase that is that we write 10X more code or 20X more code, and so a much, much smaller team can just be much, much more impactful".
Be ambitious and adaptive when using AI tools: Users should be willing to learn new ways of interacting with AI and push the boundaries of what they ask the AI to do, rather than applying old methods to new tools.
Example: "The difference between people who use Claude Code very effectively and people who use it not so effectively is like are they asking for the ambitious change?".
Persistence and iteration are key: AI models can be stochastic, meaning they may not get it right the first time. Trying multiple times, even with the same prompt, or refining the prompt based on previous failures, can lead to success.
Example: "If it doesn't work the first time, asking three more times because our success rate when you just completely start over and try again is much, much higher than if you just try once and then just keep banging on the same thing that didn't work". Or, "You can just literally ask the exact same question. These things are stochastic and sometimes they'll figure it out and sometimes they won't".
AI's impact on job roles is about augmentation, not just replacement (in the near term): While there may be displacement in some lower-skill jobs, AI currently enables human workers to focus on more complex tasks and expand the scope of their work, leading to an "expansion of the pie" in terms of labor.
Example: "In the immediate term, there will be a massive expansion of the pie and the amount of labor that people can do". The sources also note that Anthropic is not slowing down on hiring because "the next couple of years are really critical to get right and we're not at the point where we're doing complete replacement".
The benefit of AI in coding extends beyond traditional software development: Tools designed for code generation can also be highly valuable for non-technical teams, facilitating tasks that involve data analysis or document processing.
Example: "We have seen internally that our legal team and our finance team are getting a ton of value out of using Claude Code itself... using it to redline documents and use it to run BigQuery analyses of our customers and our revenue metrics".
Anticipate future capabilities and build accordingly: Given the exponential progress of AI, it's crucial to "skate to where the puck is going" and design tools with the expectation that current limitations will be overcome, allowing for more advanced applications like AI agents using computers.
Example: The "Labs" (now "Frontiers") team built Claude Code on the premise that "people are not going to be locked to their IDEs forever. People are not going to be auto completing. People will be doing everything that a software engineer needs to do and a terminal is a great place to do that". This approach requires thinking "six months from now, build for a year from now".