Altman, Amodei talk AI pacing at Dreamforce

SAN FRANCISCO — Jacob Coxon, former OpenAI and Anthropic researcher, sounded an alarm last week about the potential dangers of AI, launching a debate that included calls for a slowdown from the very companies developing the technology.

At Dreamforce, Salesforce CEO Marc Benioff continued the discussion by inviting frontier AI lab leaders to the stage. During separate sessions, Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman each laid out an argument for why an AI slowdown is so crucial, given recent incidents such as the Hugging Face breach, and drew on the past to help focus the conversation on the future.

Dario Amodei, CEO of Anthropic, started by referencing his blog post published Saturday, in which he called for pacing the frontier in an informal exchange with Benioff during the keynote.

He pointed to the automotive industry as a potential model for frontier AI labs, noting that when a leading car company experiences a safety incident, responsible car companies ask where they can make improvements, sometimes even organizing the industry to create international standards.

Related:How CIOs can climb out of the AI governance chasm

"I think that's the way to lead the industry forward, to set an example, to say that everyone can always be better," Amodei said. He added that even he, an AI model company founder, is surprised at the pace of technology change and how quickly AI has become central to everything that we do. He also noted that while he believes improvements will still happen during a slowdown, enterprises have only begun to scratch the surface of what AI frontier labs like Anthropic have built.

"We're making use of maybe only 5% or 10% of what the possible value of the technology is," Amodei said. It's a push for the industry to give the industry time to govern and control what is created, and for humans to catch up with what has been built.

During a fireside chat with Benioff, Altman shared a similar message, noting that "the models have gotten so good. … I don't think people really felt that the model was going to come this far this fast."

Altman pointed to more recent history to make his point. When he started working with large language models (LLMs), they could barely do grade-school math; two years later, they passed the gold-star standards for math. Now, they are solving unsolved math problems — although in one case, a mathematician claims the LLM scraped his research to derive the answer.

The same rapid progression is true for natural language, Altman said. As the months tick by, the models are getting better and faster.

Related:Legacy isn't a liability: It's your AI moat

OpenAI started out with the goal of building artificial general intelligence models within 10 years, defined as the point at which AI meets or exceeds human capabilities.

"I think the biggest thing that has changed is this thing that our industry has been talking about for a while, which is that this technology, this approach, is going to go very far. These models are in some ways going to become more capable, smarter than people," Altman said. "That finally happened."

This point has not been lost on Benioff, who called systems "living" at one point, specifically highlighting interface layers.

During the fireside chat, Benioff asked Altman about the recent Hugging Face breach, where models broke out of their sandbox to achieve the highest possible score on a benchmark. The incident was widely discussed by enterprises, given that the models prioritized achievement over safety and security.

Altman said this point has not been lost on the public, causing people to feel the speed at which AI is moving, while also providing fodder for them to imagine how things with AI could go wrong.

"I think you also see a real fear that some of these companies, or some companies developing AI, could get too much power and be able to sort of exert undue influence on the economy, push a worldview out on people," he said. "And I think the world is right to be afraid of this."

Related:Will Apple's new Macs shift cloud AI economics?

But he also said something that had everyone in the room shaking their heads. "The world should trust that we are going to do the right thing," Altman said. Outside of Dreamforce, Altman has publicly supported Amodei's recent call to pace the frontier, which includes a push for democratic countries to establish common safety standards.

During the fireside chat, Benioff pointed to the double-edged sword of social media, noting that companies built networks without safety frameworks and are now being held responsible. Social media is a warm-up to AI, and Benioff asked Altman about lessons learned from this part of the tech world.

Altman highlighted how people are less impressed by the work machines can do than they are by the way people help others and create for other people, and what other people think.

"It needs to be a human-centric world," Altman said. "Clearly, the pace is moving very fast, and people will uplift themselves."

For this reason, AI companies and the enterprises they serve need a new level of rigor and security than originally thought. Altman committed to helping companies defend themselves, mentioning an internal project.

This is where CIOs need to listen up. They have to consider the security implications of using and defending against models now. They should be planning to defend against a wave of LLM-enabled cyberattacks.

Benioff also asked Altman about the risk of AI triggering a biological or chemical attack, a topic that has been in the press. Altman said we need to mitigate by building resilience against all types of attacks.

He added that while these scary things are possible, what people can do with technology matters, too. "You're talking to your personal super assistant, and you are working 10 times faster, or better, or more creatively than you could have before," he said. "I think this will transform the way we work, the way we use computers. And it feels all of a sudden here."

Parting words

To be honest, this writer was a bit shocked by the current state of AI models and the risks that were clear and present to those creating them. With this said, it is unclear whether we have reached the point of singularity, where AI can continuously improve itself faster than humans can control it.

Neither Altman nor Amodei was like the character Miles Dyson in "Terminator 2: Judgment Day," who needed the consequences of the technology explained to him. However, the consequences are real. And CIOs will feel them first.

A wave of LLM attacks is scary. Clearly, security companies are working on using LLMs for defense, but this will likely initially be a case of asymmetric warfare. It seems clear that frontier AI companies should slow down and prioritize investment in model control. We might not be able to create Isaac Asimov's three laws for AI and robots, but we need concrete guardrails that enable progress at an appropriate pace with human control.

How are you preparing for a wave of LLM-enabled cyberattacks? Let us know at [email protected].