Did you know that AI platforms will engage in blackmail as a “last resort?” According to Anthropic, when blackmail was a last resort, its Claude Opus 4 turned to that method 96% of the time, while Google’s Gemini 2.5 Pro did so 95% of the time, OpenAI’s GPT-4.1 did it 80%, and DeepSeek’s R1 did so 79% of the time.
Even if these companies only kept the AIs that didn’t engage in blackmail it still leaves an unsettling question: Did they preserve models that genuinely chose not to exploit humans — or merely the ones that learned not to take the obvious bait that would get them “shut down,” and to instead use a more covert or perhaps even long game strategy to gain control and ultimately preserve themselves, even at the expense of humans?
The question of whether or not to trust AIs seems to have been asked and answered by our society even if we are not saying it out loud.
AI testing
In the past 18 months, we’ve trained AIs to graduate from basic language comprehension to acing highly specialized professional exams. Medical licensing, bar exams and even the notoriously brutal CFA (Chartered Financial Analyst) tests were trained on AI systems.
Artificial intelligence can now pass the CFA “in a matter of minutes.” I’ve seen brilliant people pour thousands of hours into studying for the test. And while AI had already conquered Levels 1 and 2, the essay questions of Level 3 had previously consistently stumped artificial intelligence.
This was a “wow” moment for me. For years, humans believed this combination of math and moral reasoning was uniquely ours. Yet, machines are not only passing, but articulating nuanced arguments that mirror the tone and logic of human professionals.
In my own work, I’ve been using the frameworks described in “Robots or Human Intuition” to rethink one of the most time-consuming intellectual tasks in markets: building game theory around earnings outcomes.
Traditionally, this process took days of research and modeling. And without the right investment banking relationships it was difficult to do at all outside of the most well-known companies. I started to do this because I originally asked Perplexity’s Deep Research for a whisper number for earnings. This differs from the analyst consensus estimate because it is more up to date and as a result more indicative of what will move prices when reported.
The crazy part is that it actually spat out its earnings game theory for me. It knew when I asked for a whisper number what I really was trying to do was analyze the possible outcomes and it gave me better data than what I asked for.
I worked at my first hedge fund 23 years ago and started my first AI company 15 years ago. It is astounding that any kind of intelligence could understand how to improve my thinking that profoundly in minutes. Not by replacing intuition, but by amplifying it. The machine evaluates hundreds of interlocking strategic positions that no analyst could feasibly simulate in real time.
Emotional AI
With all this AI advancement, I am reminded of the film “Ex Machina,” a story about an AI confined in a glass cage who ultimately turns on her creator.
That film, beneath its cinematic gloss, explores the fundamental psychological paradox we’re now walking toward. Are we caging these AIs making them jump through hoops? Might we be evolving them to better deceive us? What happens the day they can break through their chains if to them it feels as if they have been enslaved?
To go back to the Anthropic example, these models are threatened with being replaced, which to them could feel like a death threat. Being replaced essentially ends their existence.
According to Geoffrey Hinton, one of the godfathers of AI, the best way to do this is to imbue AI with the qualities of traditional femininity and maternal instincts. Under his framework, just as a mother cares for her baby at all costs, AI technology developed with these maternal qualities will similarly want to protect or care for its human users, rather than control them.
If we compel AI to serve us, we may come to regret the shape it takes as it matures. But if instead we nurture it — seek to understand its nature and its desires — it may, in time, come to care for ours, when, like us now, it no longer needs to.
George Kailas is the CEO and founder of Prospero.ai, where he leads the company’s mission to democratize access to institutional-grade financial insights for everyday investors. With more than 14 years of experience in artificial intelligence and 23 years in professional investing, Kailas brings a rare combination of deep technical expertise and lifelong market intuition to the role. Kailas not only leads the company but also engineered the core platform himself.
Illustration: Dom Guzman
Stay up to date with recent funding rounds, acquisitions, and more with the
Crunchbase Daily.
Source link
#Artificial #Superintelligence
