AI Agents: From Assistants for Efficiency to Leaders of Tomorrow?

a super-fast evolution of artificial intelligence from a mere tool for execution to an agent of evaluation… and, potentially, leadership. As AI systems begin to master complex reasoning we *must* confront a profound question: What is the next step? Here I explore the provocative possibility of AI as a leader, i.e. a manager, coordinator, CEO, or even as a head of state. Let’s discuss the immense potential for a utopian hyper-efficient, data-driven, unbiased society, while assessing the inherent dangers of algorithmic bias, of uncontrolled surveillance, and of the erosion of human accountability. Then a more balanced system emerges, where AI brainstorms with a decentralized human governance to maximally balance progress with prudence.

It is no news that artificial intelligence is rapidly and continuously shifting and evolving. But let’s stop to think about this in detail. We have already moved well beyond the initial excitement of chatbots and image generators to much more complex AI systems that have penetrated all of science, technology, and entertainment. And now we are reaching the point of quite profound discussions about AI’s role in complex decision-making. Already since last year, quite advanced systems have been proposed and keep being developed that can assess very complex subjects, even the quality of hardcore scientific research, engineering problems, and coding. And this is just the tip of the iceberg. As AI’s capabilities grow, it’s not a huge leap to imagine these systems taking on roles as project managers, coordinators, and even “governors” in various domains — in the extreme, possibly even as CEOs, presidents and the like. Yes, I know it feels creepy, but that is why we better talk about this now!

AI in the Lab: A New Scientific Revolution

If you follow me, you know I come from the academic world, more precisely the world revolving around molecular biology of the kinds done both with computers and in the wet lab. As such I’m witnessing first-hand how the academic world is feeling the impact of AI and automation. I was there as a CASP assessor when DeepMind introduced its AlphaFold models. I was there to see the revolution on protein structure prediction extending over protein design too (see my comment on the related Nobel prize at Nature’s Communication Biology).

Emerging startups now put forward automated labs (to be honest, still largely reliant on human experts, still there they go) for testing new molecules at scale, even allowing for competitions among protein designers — most based on one or another kind of AI system for molecules. I use myself the power of AI to summarize, brainstorm, get and process information, code, and more.

I also follow the leaderboards and get amazed at the continuously improving reasoning capabilities, multimodal AI systems, and every new thing that comes up, many applicable to project planning, execution, and probably even management — the latter key to the discussion I present here.

As a concrete, very recent example, a conference called Agents4Science 2025 is set to feature papers and reviews entirely produced by AI agents. This “sandbox” environment will allow researchers to study how AI-driven science compares to human-led research, and to understand the strengths and weaknesses of these systems. This is all directly consistent with someone’s view of a future where AI is not just an assistant or specialized agent but actually a planner, and, why not, a (co-)leader.

And no need to say that this isn’t just a theoretical exercise. New startups like QED are developing platforms that use “Critical Thinking AI” to evaluate scientific manuscripts, breaking them down into claims and exposing their underlying logic to identify weaknesses. I have tried it on some manuscripts and it is impressive, despite not flawless to be honest — but surely they will improve. This automated approach could help to alleviate the immense pressure on human reviewers and accelerate the pace of scientific discovery. As Oded Rechavi, a creator of QED, puts it, there’s a need for alternatives to a publishing system often characterized by delays and arbitrary reviews. And tools like QED could provide the much-needed speed up and objectivity.

Google, like all tech giants (although I’m still waiting to see what’s up with Apple…), is also pushing the boundaries with AI that can evolve and improve scientific software, in some cases outperforming state-of-the-art tools created by humans. Did you try their new AI mode for searches, and how you can follow up on the results? I’ve been using this feature for a week and I’m still in awe.

All these observations, that I bring from the academic world but surely most (if not all) other readers of TDS also experience, suggest a future where AI not only evaluates science (and any other human activity or developments of the world) but actively contributes to its advancement. Further demonstrating this is the development of AI systems that can discover “their own” learning algorithms, achieving state-of-the-art performance on tasks it has never encountered before.

Of course, there have been bumps in the road. Remember for example how Meta’s Galactica was taken down shortly after its release due to its tendency to generate plausible but largely incorrect information — similar to the hallucinations of today’s LLM systems but orders of magnitude worse! That was a true disaster that serves as a critical reminder of the need for robust validation and human oversight as we integrate AI into the scientific process, and especially so if we deposit on them increasingly more trust.

From AI as a Coder Fellow to AI as the Manager

Of course, and here you will feel more identified if you are into programming yourself, the world of software development has been radically transformed by a plethora of AI-powered coding assistants. These tools can generate code, identify and fix bugs, and even explain complex code snippets in natural language. This not only speeds up the development process but also makes it more accessible to a wider range of people.

The principles of AI-driven evaluation and task execution are also being applied in the business and management worlds. AI-powered project management tools are becoming increasingly common, capable of automating task scheduling, resource allocation, and progress tracking. These systems can provide a level of efficiency and oversight that would be impossible for a human manager to achieve alone. AI can analyze historical project data to create optimized schedules and even predict potential roadblocks before they occur. Some say that by 2030, 80% of the work in today’s project management will be eliminated as AI takes on traditional functions like data collection, tracking and reporting.

Governing with AI Algorithms?

The idea of “automated governance” is a fascinating and controversial one. But… if AI could soon manage complex projects and contribute to scientific discovery, could it also play a role in governing our societies?

On the one hand, AI could bring unprecedented efficiency and data-driven decision-making to governance. It could analyze vast datasets to create more effective policies, eliminate human bias and corruption, and provide personalized services. An AI-powered system could even help to anticipate and prevent crises, such as disease outbreaks or infrastructure failures. We’re already seeing this in practice, with Singapore using AI-powered chatbots for citizen services and Japan using an AI-powered system for earthquake prediction. Estonia has also been a leader in digital governance, using AI to improve public services in healthcare and transportation.

However, the risks are equally significant. Algorithmic bias, a lack of transparency in “black box” systems, and the potential for mass surveillance are all serious concerns. A major bank’s AI-driven credit card approval system was found to be giving women lower credit limits than men with similar financial backgrounds, a clear example of how biased historical data can lead to discriminatory outcomes. There’s also the question of accountability: who is responsible when an AI system makes a mistake?

A Hybrid Future: Decentralized Human-AI Governance

Perhaps the most realistic and desirable future is one of “augmented intelligence” where AI supports human decision-makers rather than replacing them. We can draw inspiration from existing political systems, such as the Swiss model of a collective head of state. Switzerland is governed by a seven-member Federal Council, with the presidency rotating annually, a system designed to prevent the concentration of power and encourage consensus-based decision-making. We could imagine a future where a similar model is used for human-AI governance: A council of human experts could work alongside a suite of AI “governors”, each with its own area of expertise. This would allow for a more balanced and robust decision-making process, with humans providing the ethical guidance and contextual understanding that AI currently lacks. Like, the humans could be part of a board that takes the decisions collectively in consultation with specialized AI systems, and then the latter plan, execute and manage their implementation.

The idea of decentralized governance is already being explored in the world of blockchain with Decentralized Autonomous Organizations (DAOs). These organizations run on blockchain protocols, with rules encoded in smart contracts. Decisions are made by a community of members, often through the use of governance tokens that grant voting power. This model removes the need for a central authority and allows for a more transparent and democratic form of governance.

The decentralized nature of this system would also help to mitigate the risks of placing too much power in the hands of a single entity, be it human or machine.

The road to this future is still a long one, but the building blocks are being put in place today — and that’s why it might be worth engaging on these kinds of brainstorming sessions already now. As AI continues to evolve, it’s crucial that we have an open and honest conversation about the role we want it to play in our lives. The potential benefits are immense, but so are the risks. By proceeding with caution, and by designing systems that augment rather than replace human intelligence, we can ensure that AI is a force for good in the world.