We need to start wrestling with the ethics of AI agents

Prabhakar was a key player in passing the president’s executive order on AI in 2023, which sets rules for tech companies to make AI safer and more transparent (though it relies on voluntary participation). Before serving in President Biden’s cabinet, she held a number of government roles, from rallying for domestic production of semiconductors to heading up DARPA, the Pentagon’s famed research department.

I had a chance to sit down with Prabhakar earlier this month. We discussed AI risks, immigration policies, the CHIPS Act, the public’s faith in science, and how it all may change under Trump.

The change of administrations comes at a chaotic time for AI. Trump’s team has not presented a clear thesis on how it will handle artificial intelligence, but plenty of people in it want to see that executive order dismantled. Trump said as much in July, endorsing the Republican platform that says the executive order “hinders AI innovation and imposes Radical Leftwing ideas on the development of this technology.” Powerful industry players, like venture capitalist Marc Andreessen, have said they support that move. However, complicating that narrative will be Elon Musk, who for years has expressed fears about doomsday AI scenarios and has been supportive of some regulations aiming to promote AI safety. No one really knows exactly what’s coming next, but Prabhakar has plenty of thoughts about what’s happened so far.

For her insights about the most important AI developments of the last administration, and what might happen in the next one, read my conversation with Arati Prabhakar.

Deeper Learning

These AI Minecraft characters did weirdly human stuff all on their own

The video game Minecraft is increasingly popular as a testing ground for AI models and agents. That’s a trend startup Altera recently embraced. It unleashed up to 1,000 software agents at a time, powered by large language models (LLMs), to interact with one another. Given just a nudge through text prompting, they developed a remarkable range of personality traits, preferences, and specialist roles, with no further inputs from their human creators. Remarkably, they spontaneously made friends, invented jobs, and even spread religion.

Why this matters: AI agents can execute tasks and exhibit autonomy, taking initiative in digital environments. This is another example of how the behaviors of such agents, with minimal prompting from humans, can be both impressive and downright bizarre. The people working to bring agents into the world have bold ambitions for them. Altera’s founder, Robert Yang sees the Minecraft experiments as an early step towards large-scale “AI civilizations” with agents that can coexist and work alongside us in digital spaces. “The true power of AI will be unlocked when we have truly autonomous agents that can collaborate at scale,” says Yang. Read more from Niall Firth.