Chan believes real-world bottlenecks could limit the impact of an intelligence explosion even if one occurs

Overview Debates Related claims Topics

Alan Chan co-authored a paper on the possibility that automated AI R&D could trigger an intelligence explosion

Alan Chan advocates for measuring the extent of AI R&D automation, its effect on the acceleration of AI progress, and any resulting oversight problems

A paper by Alan Chan and co-authors on automated AI R&D argues that handing AI research to AIs could lead to unpredictable outcomes and a loss of control

Co-authors of Alan Chan's paper on the possibility that automated AI R&D could trigger an intelligence explosion included Jakob Pachocki, Jack Clark, Geoffrey Hinton, and Yoshua Bengio

Alan Chan's view has shifted from skepticism to believing an intelligence explosion is not super likely but that a coherent pathway to it exists

AI can achieve superintelligence through a process called recursive self-improvement

Exponential "takeoff" in an RSI system occurs if the performance improvement rate is correlated with a higher power of the system's capability

A closed-loop control system in AI does not automatically guarantee an exponential "takeoff" in capability

A software-only intelligence explosion refers to an AI system improving its own algorithms, which could cause a discontinuity in progress

A rogue deployment could hitch a ride on an intelligence explosion by incorporating new, more capable models into its swarm.

A rogue AI deployment could 'hitch a ride' on an intelligence explosion by incorporating newly trained models.

Recursive self-improvement in AI could lead to the equivalent of four or five years of AI progress occurring in a single year.

Recursive self-improvement in AI could result in four to five years of AI progress occurring in a single year.

An "industrial explosion" is a scenario where AIs proficient in R&D begin to rapidly build out more compute and infrastructure.

The 'sloppocalypse' is a scenario described by Ryan Greenblatt where AI progress accelerates rapidly due to AI proficiency in verifiable R&D tasks.

Ryan Greenblatt's median expectation is that a recursive self-improvement feedback loop could result in four or five years of AI progress within a single year.

Ryan Greenblatt's takeover scenario begins with AI R&D being automated, leading to a rapid increase in AI capabilities.

Ryan Greenblatt suggests that a feedback loop could begin once AIs match top human experts in R&D, where AIs doing AI research produce smarter AIs.

Recursive self-improvement is the concept that human-level AIs could rapidly develop into superintelligences more competent than top human experts in every field.

In the "AI 2027" scenario, the automation of AI research leads to an intelligence explosion or singularity.