Import AI 477：我告诉 IMF，奇点已经开始

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.

Subscribe now

Here’s a speech I gave on Saturday the 17th of October at the IMF meeting in Bangkok. This speech is inspired by the megatrend readers have seen me cover this year of AI beginning to operate at the frontier of human science, sometimes autonomously. While we all may quibble over the details, I think it’s evident we’re in the early stages of the singularity, and I wanted to use my privileged position of giving the Per Jacobsson lecture to try and tell the assembled audience of economists, bankers, and policymakers, that they should expect significant, rapid change in the future. In this speech, I focused primarily on how humanity can share in the benefits of AI if we exercise proper foresight and restraint, and I’m going to be expanding on the stakes of the singularity and the risks that we’ll all face in coming issues.

As I said a few times in the speech - some of which was ad libbed and not captured here - I take my own responsibility here extremely seriously. I find myself both a narrator of what is happening in AI, a principle agent behind what is happening in AI, and a translator that goes between many communities. In all of this, I am seeking to be truthful and to be clear about my own culpability. This is a forever work-in-progress.

Thank you all for reading. Speeches like this draw heavily from the newsletter, and the newsletter is my attempt to provide week-by-week information to all of you. I appreciate your readership.

***
Inside the Event Horizon: Why Everyone Should Take AI Progress Seriously

Hello, thank you for having me. It is a great privilege to be here.

To start, I should list my relevant credentials: I am a human being. There is one of me. It has taken me almost 40 years to build the experience to give this speech. I will come back to those credentials later.

My whole career has been defined by finding information that seemed important to the future of the world, then telling the world about that information.Two decades ago, I started my career as a journalist, focusing on data centers. Through that work, I became so obsessed with AI that in 2016 I left journalism to work at OpenAI, then co-founded Anthropic a few years later. Ever since, I’ve been trying to convince the world that one day, AI systems might become so powerful that they could alter national security, change the economic trajectory of the planet, and perhaps eventually gain the ability to build themselves.

Well, the moment has arrived. AI is now changing national security, it is altering the economic trajectory of the world - as we’ve heard in so many sessions here - and is contributing to its own development.

Of course, many of you know this, and you are having to track AI as one of a multitude of destabilizing forces that is shaping your world and your work.

What I am here to explain today is that AI is on a course to bring about even more change than what we are expecting and we should prepare accordingly. And I’m going to explain why.

For many years, researchers and science fiction authors have forecast a future in which AI systems drastically increase the pace of overall scientific and technical progress, making things move so quickly that it is hard for humans to keep up.

They call this the singularity. I believe that this year we have gained convincing evidence that it has now begun.

The writer Vernor Vinge forecast this in 1993. “When greater-than-human intelligence drives progress, that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities -- on a still-shorter time scale,” he wrote, describing the singularity as “a point where our models must be discarded and a new reality rules.”

The singularity is here. The implications reach across every aspect of life. But the most important implication is a simple one: we should expect that in the coming years, the whole pace of progress of not just AI, but science as a whole, will speed up. This will feel confusing and disorienting, and it will bring with it great wealth and potentially great peril.

Today, I’m going to concentrate on the first point: the potential benefits of rapid AI progress, and what we need to do to share in these benefits together.

So many of the ways that humanity will benefit from AI come from science. I believe that AI-driven scientific invention will change the course of the global economy, alter the ways countries plan and build their own infrastructure, make us all healthier and able to spend more time with our loved ones, and –– at the same time –– cause us all to ask deep philosophical questions about choices we should make about its continued development.

I’ll develop my claim: For the last half century, most countries have dealt with falling research productivity while researcher headcount grows rapidly - ideas and inventions appear to be getting harder to find.

But something important has changed: we’ve now built AI systems which have been able to augment, extend, and automate aspects of human labor. This has most obviously happened in software development, where AI systems are now more capable and efficient than the vast majority of the world’s software engineers.

As a consequence, the field of software development is changing - A single person is now able to write software for anything they can imagine, creating things that previously would have taken entire teams of humans.

We’ve seen concrete evidence of this inside Anthropic, where our researchers now produce almost 10 times the amount of code than they did for the first four years of the company’s life.

But new AI capabilities bring new challenges - the coding improvements have meant AI systems have got much better at cybersecurity and cyberattacks –– leading the world to race to deploy these powerful capabilities to create better cyberdefenses, while bad actors race to use these systems to carry out hacks.

The important thing to notice here is that these things are bound up, inseparably - the advance of AI systems in a general area like coding has also created both positive and negative impacts in domains of international relevance, like cybersecurity.

This is a taste of things to come. Of progress yielding new capabilities, and new challenges.

These capabilities will not stop at coding; the same thing will happen to science and robotics. It has already begun to happen. And it means some important properties of the global economy may change - like the rate of innovation, or the frequency with which we invent entirely new general purpose technologies that create new industries around themselves, like the internet, phones, and artificial intelligence.

So, what causes me to believe this? In 2026, we have got the first signs that AI systems may be capable of doing science at the frontier, and we have some evocative examples of what the future could look like. For instance:

In mathematics, AI systems have been used to propose solutions to longstanding mathematical problems, like the Navier Stokes Millenium Prize, the Jacobian Conjecture, and several open Erdos problems. In many cases, AI systems were responsible for core insights that made the solution possible.

In biology, researchers have used AI to design viruses from scratch, to identify new drug candidates for testing, and there are already drugs in trials today which have been in large part designed by AI.

In robotics, AI systems are now increasingly able to autonomously operate laboratory equipment remotely, making it possible for us to further automate experimentation in domains from biology to materials science, making progress faster in areas like batteries and fuel-cell development.

Taken individually, each of these examples is exciting. But the profound fact is that they all relate to the same underlying technology - and for most of these use-cases, the same single underlying frontier model. We now have singular entities - AI systems like Claude or ChatGPT or Gemini - which can carry out work at the frontier of multiple scientific disciplines.

Quantitative metrics bear this out.
One such metric is the score of AI systems at a test called Humanity’s Last Exam - I didn’t choose the name!

This ominously named test was developed in response to the profound advances seen in prior years by AI systems at a variety of difficult and varied scientific challenges, covering topics as varied as advanced chemistry, computer science, mathematics, classical literature, and ecology.

When the exam launched in January 2025, top scoring AI systems obtained a score of 8%. As of this week, the highest scoring system is Claude Opus 5.5 with 67.7%.

Another example is FrontierMath, a test released in November 2024 that sought to assess how well AI systems could solve hundreds of challenging math problems, curated and evaluated by professional mathematicians. When it launched, the best scoring systems got 2%. Now, they get 94%.

More broadly, it looks like this everywhere. Pick your favorite part of science and go looking for a test and you’ll find this pattern - sudden and steep increases in capability, everywhere you look.

TheoPhysEval
I’ll close my evidence with a story. The latest example comes to us from a paper published just last week, called TheoPhysEval. This is a paper that tests out how well AI systems can solve unpublished problems in theoretical physics. The 50 questions in the paper were compiled by physicists around the world.

Unsurprisingly, AI systems got high scores. But what is surprising is how they got the high scores. Here, I am going to quote from Indian physicist Suvrat Raju, who wrote a post commenting on the results.

“I don’t know of any individual who can solve any significant fraction of our final collection of problems without AI assistance,” he said. “Our initial evaluation suggested that GPT-6-Astra’s score was in the late eighties followed closely by Opus 5.5. But, as we started to investigate the mistakes that the models might have made, the models, especially Astra, insisted that they were right and that it was the expert-supplied answers that were erroneous. When we went back to the experts it turned out, in multiple instances, that the experts found some subtle errors buried deep inside their calculations. As they revised their answers, Astra’s score started to inch up and it finally ended up with 98% (49/50).”

Think about this for a moment. Some of the world’s scientific experts built a test out of problems an AI could never have possibly read the answer to. And when they tested the AI, not only did it score high, but it got a higher score by finding errors in the answers that the humans had worked on!

There is a scene in films, usually two thirds of the way through, when the young protagonist finally proves to their mentor that they have what it takes to go out into the world and succeed independently. It is a narrative trope - when the teacher becomes the student. That is what is happening here.

So, why am I telling you this? Why does it matter that AI systems are getting better at science?

I believe it is because doing true frontier science - being capable of making breakthroughs that extend the frontier of knowledge - implicitly tests for one of the last frontiers of AI capability: creativity. My view is that if AI systems can creatively problem solve to succeed at science, we should expect this to be a sign of their ability to generalize into many domains, including certainly AI research itself, which could have an explosive effect on innovation.

Therefore, if AI systems get better at science, this will strongly drive the growth and shape of the future economy, which means that you can make decisions today to set up your nations and people to benefit from this trend in the future.

Science and growth
Let’s talk about growth. One of the main sources of growth is in research yielding new ideas. One of the things that drives growth is our stock of human capital - the people all over the world who ask questions and conduct scientific research and thus help the world move forward.

But we also know that, as some economists have said, “good ideas are getting harder to find.” The cost of breakthroughs is going up, and we’ve had to pour an ever-rising amount of resources into research to keep going at the same pace.

What if this changed? With this technology, we can fundamentally change both the number of researchers on the planet, and how long it takes to train them.

As I said at the beginning, when I listed my credentials, there is one of me and it took me about 40 years to get here. What would it mean to be able to run hundreds or thousands of copies of me simultaneously? What would it mean if new versions of me could be created instantly, rather than waiting 40 years? How would my work change if I could make copies of myself and order them to do work under my supervision? That is what the next few years will be like for large swathes of the economy - we will be able to augment human labor in frontier disciplines, like science, in a way we’ve never been able to do before.

This will likely change the rate at which we find good ideas, because we’ll be able to add 100s or 1000s of times as many researchers to any problem, force multiplying our human scientists.

We can also do this very quickly. If you wanted to train a new generation of researchers in a certain area, it might take you 10 or 15 years of targeted education, and millions of dollars, to even get a small cohort. On the other hand, these AI systems can be created immediately, in arbitrary numbers, as long as there is enough computing capacity - and they can be used across domains to benefit from new scientific insights near instantaneously, and they can combine existing knowledge in new and surprising ways to make discoveries. This means that we can rapidly augment humans in domains that show promise by generating large amounts of virtual scientists to work alongside them.

This means that in the coming years, every human scientist on this planet will become more important and more powerful, as they gain the ability to supervise hundreds or thousands of like-minded experts on any problem they find interesting.

We have done some of our own modeling here, where we have tried to simulate different versions of the economy according to different trajectories of the technology. If you assume that models will be able to autonomously complete a broad range of tasks and will diffuse quickly in the economy, then under these singularity-esque conditions, we could see an unprecedented economic transformation, with US GDP reaching 44 trillion by 2030. This sounds outlandish, but it’s implicated by the technology continuing to get better. If GDP were to grow this much, then economic doctrine suggests that new industries and inventions would need to have been created to power such major economic expansion - and that is exactly what is implied by this notion of having AI systems be capable of autonomous scientific inquiry.

So, what do we do about this? How do we set up the world broadly to benefit from these advancing capabilities? How do we balance this opportunity for progress with appropriate caution?

We need to both plan for success so that humanity can take advantage of this technology and use it to create abundance, and be prepared to make wise choices about how to use the technology itself - just because AI will be a capable scientist, that does not mean we should use it for absolutely everything.

On planning for success, I’ll start with a story from my own experience.

For the last ten years, I have gone to governments around the world and tried to convey a very simple but important fact: that AI systems are getting far more powerful and useful, and that these systems require large amounts of computers. Therefore, I said, the single best thing a government could do to position itself to take advantage of the AI revolution was to build enough electricity capacity and data center space to be able to host the kinds of powerful AI systems discussed here, and to invest in the AI supply chain.

Not many policymakers listened. Now, we deal with the consequences of that. In today’s world, data center capacity is concentrated in the United States, China, and increasingly the Middle East, and the supply chains are concentrated and are under such contention that prices are going up.

That buildout remains important, but it was just the first phase of this economic and scientific revolution implied by AIs continued advance. Now, there is more to be built, and I believe more countries can participate in this - if they choose to.

The next phase of the AI revolution is about the expansion into the physical world, specifically by having AI systems increasingly operate robots, and to perform scientific experimentation, sometimes together. Therefore, just as we did with compute and power, we will want to use much more scientific equipment and robotic systems or we will run very quickly out of capacity. We must build this capacity now to let us take maximum advantage of converting this boom in science into benefits for us all.

According to estimates by UNESCO, there are approximately ~10 million scientists in the world. Even with only 10 million scientists, we have too few human scientists to oversee the work, too few laboratories to do materials science analysis, too few biological labs to develop and test new drugs, and too few large pieces of scientific equipment in space or on the earth for the experiments we want to run.

What happens when instead of ~10 million scientists, we have a billion virtual scientists directed by our own ten million? Or ten billion? That is the world that is coming right for us. We have to get ready.

If we do get ready, we are going to be in a better position to translate the intelligence of AI systems into gains for us all: longer, healthier lives, more energy, more food, more wealth. And just as with the data centers, the nations which build and develop this supply chain, will be the ones that will benefit the most.

This is an opportunity for everyone: if you are a developed country with a rich scientific ecosystem, you have an opportunity to make all of your scientists more productive. If you are a developing country that supplies the basic ingredients into the AI and scientific enterprise, like raw materials, then you have an opportunity to leapfrog: you can both build scientific testing facilities that, paired with human scientists and digital AI scientists, will let you grow a scientific community, and continue to develop your contribution to the supply chain, which will become more valuable in the future.

But infrastructure alone will not be enough. If discovery becomes abundant, we need to avoid bottlenecking ourselves downstream - an avalanche of new proposed drugs is of little use if only a handful can ever be tested. The policymakers and their colleagues in this room will need to help us find ways to accelerate our ability to move discoveries out of the lab and into the clinic, the factory, and the field.

The better we prepare for this world, the better able we will be to translate the gains of AI to gains for the world.

We also need to prepare ourselves as a global community to make choices about the future. Because as I have said, there is risk here. Being able to use AI for science, opens us up to the misuses of science as well - biological and cyber weapons, and things we can not yet imagine.

We have many choices to make, and it’s only by recognizing this technology for what it is - for how powerful it is - that we will see the urgency behind making these choices.

The first choice is about where we should direct this technology. Just because we have an AI system that can do science, that doesn’t mean that it should be left to do science unattended. One such use case is biological research, where we want the strictest human oversight and transparency to maintain safety.

Another is AI research itself, where labs like Anthropic will need to make decisions about how much oversight we apply to AI-driven research. This could let us make more progress on the technology but also introduces risks. We will need to move extremely carefully here, because otherwise we could lose control of the technology. We will also likely need rules to govern this.

I was in Tokyo earlier this week, and a student at a university there said they worked in the physics department, and they said AI was already beginning to be able to do many experiments. What experiments should they spend time on as a person, and what experiments should they let the AI system do itself? This is a subtle question to which I don’t know the answer, but it is a question each discipline of science will need to answer.

The second choice is about how much we need to understand about what these AI systems do. As we have seen this year with the rise of AI in mathematics, we are already dealing with a world where the advances made by AI systems are sufficiently creative that we as humans are now spending time trying to decode them - trying to understand what they mean and how to apply these insights for ourselves.

How much should we understand? Again, the answer here is going to be subtle, and it relates to fundamental questions of meaning in the age of the singularity - for what parts of scientific progress do we want to have true understanding? We are only a few short years from AI systems not only doing science, but inventing their own new scientific equipment built for the ways AI systems work, rather than human minds. Do we need to understand that equipment? Should we? This is another choice we will need to make.

Most importantly, taking AI’s progress seriously means we should give ourselves license to imagine a fundamentally different world

When I was a teenager, I read a lot about the world, and I was afraid. I believed that the 21st century was going to be difficult - that population growth, climate change, resource extraction, and other crises would add up to cause a malthusian collapse of the human species. That after rising to invent computers and the internet, we might consume ourselves into the ground.

And then over the subsequent years, I saw time and time again how technology could allow us to fundamentally alter these odds: how solar panels came along and became much cheaper, much faster than people expected. How our growing understanding of the human body allowed us to invent cures for things that had killed many people - including some of my own family.

How we could do things I had never even imagined but find utterly amazing, like how we are now using AI systems to decode the speech of animals, or how companies have built rockets that take off and then land back on this planet.

I work on AI because it can help us do more of that. I work on AI because whether it does that is about choices we make - choices people here in this room will make - about how we direct it and how we govern it. Choices I need to make. I work on AI because it is the most important technology our species has built.

It is a technology that, if we are able to use correctly, will let us deal with problems that might otherwise doom us as a species. But it is also a technology that has danger woven into it. For us to get through this - to chart a course through the singularity – we are going to need to work together to create unusual and serious rules for this technology and the companies building it, and to exercise the hardest and I think most noble quality of people - restraint.

We will need to build the infrastructure to take advantage of this technology, but we will also need to go slowly and deliberately on developing the technology itself, to pace or otherwise control its development. We need to constantly differentiate between the things we can do and the things we should do. And we will need to invest today in the scientific capabilities to make that progress possible.

I am reminded in all of this by the great Carl Sagan, who said in The Pale Blue Dot, that the humans who may one day reach other stars, will be “a species very like us, but with more of our strengths and fewer of our weaknesses… more confident, farseeing, capable, and prudent.”

That is what AI demands from us. And where Sagan thought we had hundreds of thousands of years to achieve this - to become a better and wiser people - we instead have a handful of years. I believe we can do it. But it is only by treating AI with the seriousness it deserves that we will succeed. Thank you.

Watch the lecture here: