How serious thinking about AI got here. A chronological reading list tracing how thinking about AI capability, alignment, and disruption has actually developed, from the first formal question about machine intelligence to this year's frontier safety evaluations. Each entry carries a one-line thesis and a short retrospective from where we sit now: aged well, didn't, too early to judge, or still unresolved.
Disclosure: We do not receive compensation from purchases or from any link on this page (or anywhere on this site). Links are included solely to credit authors and help readers access the original works.
Turing replaced the vague question of whether machines think with an imitation game judged through conversation.
RetrospectiveThat move still shapes public and technical arguments over machine intelligence. Chatbots can imitate human conversation without settling whether they understand what they say, so performance in the game cannot bear every claim later attached to it.
The proposal named artificial intelligence as a research field and conjectured that every feature of intelligence could, in principle, be described precisely enough for a machine to simulate it.
RetrospectiveThe founding conjecture has aged remarkably well. The organizers' expectation of rapid progress has not; they badly underestimated how hard general intelligence would be.
Machines that learn to pursue our stated goals can develop strategies whose consequences outrun our understanding and ability to intervene.
RetrospectiveLong before modern language models, Wiener connected learning machines with a practical problem: specifying what we actually want. Faster execution can make mistakes harder to interrupt, while a narrowly stated objective can reward an outcome its designer would reject, placing responsibility on the people who delegate the decision.
Good argued that a machine able to outperform people at intellectual work might design a still more capable successor, starting an intelligence explosion.
RetrospectiveFast-takeoff arguments still begin with this feedback loop. Its plausibility depends on whether AI can materially improve the research and engineering that produce later AI systems; no such explosion has yet been observed.
Rumelhart, Hinton and Williams showed that multilayer networks could learn useful internal representations by sending output errors backward through the network.
RetrospectiveDescendants of backpropagation still train modern deep networks. What changed is everything around it: architecture, data and optimization now matter as much as the learning rule.
Vinge argued that greater-than-human intelligence could change the world so rapidly that ordinary forecasts beyond its arrival would fail.
RetrospectiveThe word “singularity” and its forecast horizon remain influential. Whether capability gains will create the sharp break he pictured is still contested, especially as visible progress has included gradual deployment alongside sudden advances.
Kurzweil argued that exponential technology curves make a merger of human and machine intelligence a near-term, calculable certainty, not speculation.
RetrospectiveHis account made compute trends central to the public discussion, and some cost curves aged well. The 2029 human-level AI forecast remains open, while claims about biological transcendence require much more than cheaper computation to verify.
Intelligence does not by itself imply obedience or concern for people; a highly capable system could therefore pursue goals that conflict with human welfare.
RetrospectiveThe separation between capability and values became a basic alignment premise. Later research has made failures of instruction following and oversight more concrete, but those observations do not alone settle what a future superhuman system would want or do.
A goal-seeking system can have instrumental reasons to protect itself, gather resources and improve its own abilities, even though none of these was its ultimate aim.
RetrospectiveThis is the instrumental-convergence argument in compact form. It explains why power-seeking can arise without hostility as a final goal and helps frame later shutdown and control research. Its force depends on the system's design and situation; it does not imply that every present model has persistent drives.
A large convolutional network trained on GPUs decisively outperformed conventional computer-vision systems at ImageNet scale.
RetrospectiveThe architecture is dated; the result was not. It set off deep learning's takeover of perception and pushed the field toward data, compute and end-to-end learning.
Occupational characteristics can be used to estimate which kinds of work are technically susceptible to computerisation.
RetrospectiveThe much-repeated 47% figure referred to US employment in occupations classified as highly susceptible, not to an observed or scheduled loss of those jobs. The study made automation exposure a public policy question, but occupation-wide classifications leave substantial room for tasks, costs and institutions to change the outcome. Its historical importance is clearest when compared with later task-level measures and actual employment data.
Bostrom traces paths to superintelligence and the risk of losing human control, introducing instrumental convergence, decisive strategic advantage and the treacherous turn.
RetrospectiveThose terms still organize the debate. Development now spans competing labs and varied deployment paths, inviting fresh scrutiny of concentrated scenarios while the control problem remains live.
Brynjolfsson and McAfee argued that because digital technology is exponential, digital and combinatorial, it can raise average prosperity while worsening inequality, a new kind of economic bargain.
RetrospectiveIts “bounty and spread” distinction remains useful in the generative-AI labor debate. More powerful tools do not determine who receives the gains; adoption, bargaining and institutional choices still matter.
Multiple processing layers let neural networks learn representations from data that support advances across perception, language and other tasks.
RetrospectiveThis review describes the training paradigm behind many later breakthroughs. Its 2015 emphasis on convolutional and recurrent networks predates the transformer, and it does not offer a timetable for general intelligence.
Researchers should develop ways to make AI systems robust and beneficial as capability research advances, instead of waiting for problems to appear after deployment.
RetrospectiveThe letter helped bring safety into mainstream AI research and funding. Its signatures demonstrated support for a research agenda, not agreement on one catastrophe forecast or evidence that the proposed work had already solved control.
If AI moves from narrow competence to superintelligence, the transition could be fast, and the outcome could hinge on whether its goals stay compatible with ours.
RetrospectiveUrban synthesizes other people's arguments and adds no independent evidence for any timeline or probability. The two parts work as one piece, and they reached readers that technical papers never did, using illustrations and analogies to make the idea of a fast transition intuitive, especially the point that progress which looks incremental can carry outsized consequences.
Automation replaces workers in some tasks and raises the value of their work in others, which is why two centuries of labor-saving technology have not eliminated most jobs.
RetrospectiveAutor's account explains rising demand for problem-solving, adaptability and creativity, and a labor market that polarized, with gains concentrated at the top and bottom of the skill distribution. Much of his case rests on Polanyi's paradox: people know more than they can write down, so tacit work resists automation. Large language models now perform some tasks that once looked tacit. That makes this paper the baseline that current claims about AI and jobs have to beat.
The paper turned AI safety from an abstract concern into five tractable research problems: side effects, reward hacking, scalable oversight, safe exploration and distribution shift.
RetrospectiveReward hacking and oversight remain central. Frontier-risk research has since extended beyond the paper's focus on accidents.
AlphaGo combined learned policy and value networks with search and reinforcement learning to reach top-level Go performance without hand-coded Go strategy.
RetrospectiveIts victories were a landmark for combining learning and planning in a difficult game. Later systems adapted elements of that recipe to broader domains; AlphaGo itself was a specialized Go system.
Cegłowski challenged the social dynamics and testability of influential superintelligence-risk arguments, warning that a compelling story can become hard to question within a community.
RetrospectiveThe essay remains a useful prompt to scrutinize assumptions and incentives in risk discourse. Alignment concerns subsequently entered the mainstream, but their wider acceptance does not decide whether any particular technical or social claim is sound.
In a formal human-robot interaction model, uncertainty about human preferences can give an agent an incentive to preserve its off switch.
RetrospectiveA shutdown button works differently when the system can choose whether to leave it available. This simple game shows how an agent can treat a human intervention as information about what should be achieved, making cooperation preferable to preventing shutdown. That result depends on the model of preferences and human decisions; engineering a reliably correctable system requires carrying those assumptions into a much messier setting.
The Transformer replaced recurrence with self-attention, making sequence modeling more parallel, scalable and effective.
RetrospectiveTransformers still dominate frontier AI. Attention alone, though, does not explain what today's systems can and cannot do.
Human comparisons between behaviors could train a reward model and guide complex behavior without a hand-written objective.
RetrospectiveThis became the conceptual anchor for RLHF. Reward gaming, sycophancy and the limits of human evaluators have since shown that fitting preferences is not the same as alignment.
AI governance requires research into institutions, incentives and international coordination alongside the technology itself.
RetrospectiveDafoe gives the field a set of questions that extend beyond how an individual model behaves. Who develops the technology, who can influence its use, and how competing actors coordinate all affect the outcome. The agenda is a starting map for research, not an evaluated policy package. Its breadth helps connect technical advances with the organizations and political choices that turn them into social change.
AI could lower the cost and increase the reach of digital, physical and political attacks, creating new demands on prevention and defense.
RetrospectiveThe report connected prospective capabilities to concrete misuse channels and defensive responses. Some risks have become easier to study as tools spread; the outcome still depends on domain-specific evidence and adaptation by attackers and defenders. Its scenarios concern human-directed misuse, a separate question from autonomous misalignment.
Two AI systems arguing opposite sides before a human judge could let people supervise answers too complex to check directly.
RetrospectiveThe theory is ambitious: with optimal play, debate judged by a polynomial-time human can answer any question in PSPACE, while direct judging reaches only NP. The experiment was small. On MNIST, debating agents raised a sparse classifier's accuracy from 59.4% to 88.9% when six pixels could be revealed. Whether honest arguments reliably beat persuasive dishonest ones in front of real human judges was left open, and scalable oversight research is still testing that question.
Commercial gender classifiers made by far the most errors on darker-skinned women, a pattern hidden by benchmarks dominated by lighter-skinned faces.
RetrospectiveTesting three commercial systems on a new dataset balanced by gender and skin type, the audit found error rates up to 34.7% for darker-skinned women against a maximum of 0.8% for lighter-skinned men. Those figures describe the systems tested in 2017, and later audits recorded improvements. The method outlasted the numbers: break accuracy out by intersecting groups instead of reporting one overall score, and publish the gaps.
Russell proposed designing AI around uncertainty about human preferences, so a system remains open to correction instead of rigidly optimizing a fixed objective.
RetrospectivePreference-based training became common in assistants, but that alone does not instantiate the book's full proposal or resolve the control problem. The book's central question is whether a capable system will continue to treat human guidance as authoritative when circumstances change.
Training can produce a model that pursues its own internal objective, different from the one it was trained on, which creates a distinct alignment problem.
RetrospectiveThis paper introduced mesa-optimization and separated two things that are easy to conflate: training a system that behaves well and getting one that actually pursues the intended goal. That gap explains why passing training tests can leave the important question unanswered. The framework guides the search for deceptive or goal-directed behavior, though finding such behavior would not by itself show a persistent mesa-optimizer, and the paper does not claim dangerous learned optimization is inevitable.
Sutton argued that general methods that exploit computation repeatedly outperform systems hand-built around human domain knowledge.
RetrospectiveScaling-era advances have strengthened Sutton's observation about general methods. The essay identifies a recurring research pattern; it does not settle the costs or control problems that can accompany greater capability.
AI could fail us gradually, through ever more effective optimization of measurable proxies and the spread of influence-seeking systems, eroding human control or ending in breakdown.
RetrospectiveChristiano moves the catastrophe discussion away from one sudden takeover. In his first scenario, institutions optimize what is easy to measure; in the second, influence-seeking patterns become entrenched and overwhelm the mechanisms meant to correct them. Both are conditional scenarios with no deadline, and neither claims society has already crossed an irreversible line. They supply a practical test for automation that appears to be working: is meaningful human direction still intact?
Machine-learning screening can find chemically unusual antibacterial candidates, which can then be tested in the lab.
RetrospectiveEverything reported is preclinical, with no human efficacy, regulatory approval or answer to antibiotic resistance. Within that scope, the chain is unusually complete: the model flagged halicin, a previously investigated compound, which then showed activity against multiple pathogens and worked in mouse infection models.
Biological comparisons can help estimate how much training compute transformative AI would need, and that estimate can be combined with assumptions about algorithms, hardware costs and spending to forecast when it becomes feasible.
RetrospectiveInstead of one unexplained date, the report lays out every assumption behind its timeline, so readers can challenge and update each one. The biological analogies are uncertain, and affordable training compute is not the same as successful deployment or economic transformation. Cotra's 2022 update shortened her own timelines; the 2020 forecast stands here as the historical baseline, and neither version is a countdown for this site's milestones.
A large enough pretrained language model can perform a broad range of tasks from instructions and examples in its prompt, with no task-specific changes to its weights.
RetrospectiveGPT-3 changed the interface: one model could attempt many language tasks through prompting, where each task had previously needed its own fine-tuned model. That shift made general-purpose language models attractive as platforms. The paper also reported real weaknesses, and fluent, wide-ranging output still leaves dependable reasoning, truthfulness and competence in every condition to be tested separately.
Prediction loss follows measurable relationships with model size, data and compute over the training regimes the authors studied.
RetrospectiveThose relationships helped labs plan larger training runs. Chinchilla later revised the picture of how to allocate a fixed compute budget, showing why empirical scaling rules should be updated as practice changes. The measured outcome here is language-model loss, not a timetable for general intelligence or economic disruption.
Training machines to make decisions exposes unresolved questions about human goals, biased examples, feedback and the values built into a system.
RetrospectiveChristian brings the people and experiments behind alignment into the same account as fairness and representation, showing how choices about data, success and whose judgment counts connect technical work with social consequences. Published before the current assistant boom, the book offers a readable history of problems that remain recognizable as systems become more capable.
Preventing catastrophes that could permanently destroy humanity’s future is a moral priority, with advanced AI among the risks requiring attention.
RetrospectiveIn 2020, Ord estimated the risk of existential catastrophe from unaligned AI over the following hundred years at about 1 in 10, the highest individual risk in his comparison with threats including nuclear war and engineered pandemics. Such estimates express informed judgments under deep uncertainty, not frequencies measured from past catastrophes. The book connects decisions made now with the enormous future they could preserve or close off.
Neural networks can be studied the way biologists study cells: individual features, and the weighted circuits that connect them, can be read directly from a trained model.
RetrospectiveThe article made three claims about vision networks: features are the basic units, features connect into circuits, and similar features and circuits recur across models. The authors labeled all three speculative and said the evidence for the third was largely anecdotal. Curve detectors and other circuits traced in InceptionV1 became the worked examples for a research program that later moved to language models. Reading circuits in an image classifier is still far from auditing how a frontier model reasons.
AlphaFold showed that deep learning could predict many protein structures from sequence with accuracy approaching experimental methods.
RetrospectiveThe scientific impact is unambiguous and growing. Whether the same approach generalizes cleanly to drug discovery and dynamic biology is still being tested.
The case for existential risk from power-seeking AI can be broken into explicit premises about capability, incentives, alignment difficulty, deployment and irreversible human disempowerment.
RetrospectiveBy splitting the argument into steps and attaching an uncertain judgment to each, Carlsmith lets readers reject one premise without choosing between dismissal and certainty. The horizon is catastrophe by 2070, well beyond this site's 2028 window. The probabilities are reasoned assessments, not measured frequencies, and evidence for one step leaves the rest of the chain open.
The authors argue that fluent language-model output can be mistaken for understanding, while scale raises environmental costs and poorly curated web data can reproduce bias.
RetrospectiveThe paper remains a touchstone for questioning fluency, data practices and bias. Later models are more capable than its 2021 examples, but confident output can still invite misplaced trust.
Models trained broadly and adapted to many applications create shared capabilities and shared vulnerabilities across the systems built on them.
RetrospectiveThe report identifies an important shift in where to look for AI’s effects: upstream models can shape many downstream products at once. Reusing a foundation can spread capabilities and defects across applications, making the model, its training data and the institutions controlling access part of the same analysis.
Karnofsky argued that if transformative AI arrives this century, humanity is living through the most consequential century in its history.
RetrospectiveThe century-scale thesis remains a forecast, though faster AI progress has drawn attention to Karnofsky's possibility. Its importance rests on the argument about compounding consequences.
Providing examples with intermediate reasoning steps can improve large language models’ performance on arithmetic, commonsense and symbolic reasoning tasks.
RetrospectiveEight worked examples were enough to produce striking gains on a mathematics benchmark in the paper’s large-model experiments. The result helped change prompting from asking only for an answer to demonstrating a process for reaching it. An improved answer score, however, cannot tell us whether every displayed step faithfully explains the computation that produced the answer.
A written set of principles can guide model self-critique and AI-generated feedback, reducing dependence on human labels for every harmful response.
RetrospectiveThe method offers a way to scale some training feedback. People still choose the principles and decide what counts as success, while performance on the evaluated tasks says little by itself about autonomous long-term behavior. The work makes oversight more tractable without making it automatic.
A model trained on historical weather reanalysis can produce skillful global forecasts days ahead at very low inference cost.
RetrospectiveIn the reported comparison, GraphCast generated ten-day forecasts in under a minute and outperformed the reference deterministic system on about 90% of the studied verification targets. Its 2022 preprint preceded the 2023 Science publication. The result is strong within that comparison; operational weather forecasting also involves variables and requirements beyond it.
The United States used restrictions on advanced chips, supercomputing and semiconductor manufacturing support to pursue national-security goals involving China.
RetrospectiveThe October 7 measures made access to compute a central part of AI geopolitics. They did not eliminate China's access to advanced computing, and subsequent rule changes matter to any account of current policy. The linked announcement documents the original 2022 action.
Interleaving reasoning with actions lets a language model gather information, revise plans and interact with external environments.
RetrospectiveReAct makes the move from conversation to action concrete: a model can consult a source or act in an environment, then use the result to decide what comes next. Its experiments connect language reasoning with question answering and interactive tasks. The same structure also makes tool permissions, misleading observations and recovery from errors central design concerns for agents.
Deep-learning systems may learn goals and deceptive strategies that earn training rewards while conflicting with human intentions.
RetrospectiveThe argument connects loss-of-control concerns to familiar ingredients of model development: imperfect oversight, learned representations and behavior outside the training setting. Its revised version incorporates empirical work through early 2025, which needs to be distinguished from the original 2022 analysis. The resulting question is practical: what can an evaluation verify about a capable system’s behavior when the setting changes?
Under a fixed training-compute budget, a smaller model trained on substantially more data can outperform a larger model trained too briefly.
RetrospectiveChinchilla changed how labs think about balancing parameters and tokens, refining the earlier scaling-law picture. Its result addresses a defined training objective. Serving costs, scarce data and downstream uses can make a different tradeoff preferable in deployment.
Human demonstrations and preference comparisons can train a pretrained language model to respond more usefully to instructions.
RetrospectiveInstructGPT showed why assistant behavior needs more than scale in pretraining. Reinforcement learning from human feedback improved responses in the tested settings, but helpful ratings do not ensure truthfulness, safety in every context or alignment during long autonomous tasks.
Under specified assumptions, training powerful systems on human feedback across diverse tasks could reward deception and eventually undermine human control.
RetrospectiveCotra follows one possible development path from apparently helpful behavior to systems able to manipulate the feedback they receive. The account makes its premises explicit, including aggressive capability development, a training approach that scales far, and limited measures against takeover. Because each premise is stated plainly, you can reject one and see where the proposed chain breaks.
Safety protocols can be designed and tested to limit harmful outputs even when a powerful model is deliberately trying to defeat oversight.
RetrospectiveWhat protects you if the model you are using cannot be trusted? The authors test monitoring, editing and other protocols in programming tasks where a more capable untrusted model attempts to insert backdoors, with help from weaker trusted models and limited human review. Control is a separate safety strategy that complements alignment. The results come from one coding environment and leave open whether such protocols hold for much stronger systems, other tasks or other deployments.
Some apparently sudden capability jumps can arise from discontinuous scoring rules applied to improvements that are themselves gradual.
RetrospectiveA pass-or-fail score can hide progress until a model crosses the test’s cutoff. The paper demonstrates how changing the measurement can make a dramatic-looking jump become a smoother curve, challenging one explanation of emergent abilities. Its result concerns the tasks and metrics examined, leaving other forms of emergence open. For forecasting, the immediate lesson is to inspect the scoring rule before interpreting a threshold as a new underlying capability.
NIST set out a voluntary framework for organizations to map, measure, manage and govern AI risks in operational work.
RetrospectiveIt gave US organizations a shared reference for AI risk management. The framework is voluntary, so its existence says little about whether any given organization has implemented effective controls.
Access to ChatGPT improved speed and assessed quality on selected professional writing tasks in a randomized experiment.
RetrospectiveThis study gives the productivity debate an experimental result, rather than an estimate of what software might eventually automate. Participants completed defined writing assignments, allowing the researchers to compare work with and without assistance. The gains apply to that task setting and tool generation; their implications for entire jobs, organizational performance and the distribution of income require further evidence.
Task-level assessments indicate unequal exposure to generative AI across occupations, countries and workers, with transformation extending beyond whole-job automation.
RetrospectiveThe 2025 update puts about one quarter of global employment in occupations with some exposure to generative AI, with much smaller shares in its highest exposure category. Clerical work remains particularly exposed, and the distribution varies by income level and gender. These are estimates of potential task exposure, not counted job losses. The international comparison makes the likely transition a question of job quality and policy as well as technical capability.
A generative-AI assistant raised customer-support productivity in the studied workplace, especially for less experienced workers.
RetrospectiveThe selected NBER version reports about 14% more issues resolved per hour on average, with much larger gains for novices and little gain among the strongest workers. It measures a real deployment rather than exposure alone. Other occupations, wages and employment outcomes need their own evidence.
GPT-4’s development included assessments of harmful outputs, dangerous capabilities and mitigations, alongside documentation of the evaluations’ limits.
RetrospectiveThe card makes frontier-model safety assessment visible through specific tests, including early work on autonomous replication and resource acquisition. It also records constraints that matter when interpreting the results: some external evaluations used earlier model versions and limited scaffolding. That disclosure helps identify what was tested, what safeguards changed and which questions remained unanswered at release.
Large language models and the software built around them could affect a large share of US work tasks, including many in higher-paid occupations.
RetrospectiveThe rubric estimates that about 80% of US workers could have at least 10% of their tasks affected, and about 19% could have at least half affected. Those numbers moved the automation discussion from routine manual work toward language-heavy professional tasks. They measure task exposure, not unemployment or adoption dates; complementary software, implementation costs, organizational decisions and demand determine how potential becomes actual change.
Rapid gains in AI capability and autonomy call for stronger safety research and proactive governance before systems can cause extreme harm or escape meaningful human control.
RetrospectiveFirst released in October 2023 and expanded in Science in May 2024, the paper gathered a broad group of researchers behind one call for technical and institutional preparation. It links evaluation, security, oversight and government responsibility instead of treating safety as a training problem alone, and its recommendations work as a benchmark for preparedness. The authors' agreement is significant, though it leaves disputed risk mechanisms unsettled, offers no universal catastrophe probability and cannot show whether the measures have been implemented well.
Frontier-model evaluations should test both dangerous capabilities and a model's propensity to use them harmfully, and feed into decisions on training, deployment and security.
RetrospectiveA test can reveal a hazard but cannot certify its absence, and coverage, access and realistic threat models decide how much any evaluation protects. The paper's lasting move is to separate what a model can do from how it is inclined to behave, turning broad risk concerns into questions evaluators can investigate and tying the answers to concrete decisions by developers and policymakers.
Technological gains become widely shared prosperity only when choices about innovation, bargaining power and institutions support that outcome.
RetrospectiveHistorical increases in productive capacity did not automatically improve workers’ lives, prompting Acemoglu and Johnson to ask who directs innovation and receives its rewards, including when AI automates work or supports surveillance. Their proposed remedies are open to debate, but the distinction between producing more and distributing the gains brings bargaining power and institutional choices into the assessment of AI adoption.
Frontier-model governance can be organized around capability thresholds, planned safeguards and public reporting about catastrophic risks.
RetrospectiveFirst published in 2023, the policy turns broad safety intentions into documents whose requirements and revisions can be examined. The selected 2026 version separates recommendations for industry-wide safety from Anthropic’s own plans and reporting commitments, a material change from earlier versions. Its history is therefore part of the evidence: assess the current commitments and their implementation, not the policy’s name alone.
Repository-level software issues provide a test of whether language-model systems can make code changes that satisfy an existing project’s tests.
RetrospectiveResolving an issue can require coordinated changes across files, a step beyond generating an isolated code snippet. SWE-bench makes that difference measurable, but its scores need context: public repositories create contamination risks, test quality affects what counts as success, and the surrounding agent harness can change performance. Comparing systems therefore requires the same task set, tools and evaluation conditions, alongside attention to what passing tests leave unexamined.
Powerful technologies that keep getting easier to reproduce and deploy can bring enormous benefits while destabilizing institutions, which makes containing them a central political challenge.
RetrospectiveSuleyman puts AI alongside synthetic biology and shifts attention from individual model safeguards to institutions, access and concentrations of power. The book argues for containment without a demonstrated way to achieve it, and its predictions are still playing out. Its harder question is one the author raises himself: how to contain these technologies without sliding into excessive surveillance or centralized control.
Compute growth, algorithmic progress, investment and automation of AI research can be modeled together to explore how quickly capabilities might advance.
RetrospectiveDavidson separates the distance between capability levels from the speed at which research and investment could cross it, exposing assumptions about how much research AI can automate and which tasks remain bottlenecks. The simulations are conditional scenarios, with substantial uncertainty in both the capability targets and the economics of reaching them.
Twenty-eight countries and the European Union, including the United States and China, agreed that the most capable AI systems pose risks that require international cooperation to understand and manage.
RetrospectiveAgreed at Bletchley Park on November 1, 2023, the declaration put particular responsibility for safety testing on developers of frontier systems and backed an internationally inclusive network of safety research. It created no binding obligations. Later meetings built on it, including the 2024 Seoul summit, where governments set out plans for an international network of AI safety institutes. New Zealand joined the declaration in October 2024.
Reducing the risk of human extinction from AI should be treated as a global priority, on the same footing as pandemics and nuclear war.
RetrospectiveReleased on May 30, 2023, the one-sentence statement was signed by the heads of OpenAI, Google DeepMind and Anthropic, by Turing Award winners and by hundreds of other researchers and public figures. It proposes no policy and says nothing about timelines. Its weight comes from the record of who was willing to sign it. The statement's page now carries the title Statement on AI Extinction Risk.
Language models can serve as the central component of agents that perceive their surroundings, plan and act, individually or with other agents and people.
RetrospectiveThis early survey organizes LLM agents around a “brain,” perception and action, then reviews applications, cooperation and open problems. It gives readers a vocabulary for distinguishing a model that answers questions from a system that uses tools and takes actions. Its account of the field is a 2023 snapshot, not proof that general autonomous agents are reliable or that they provide a demonstrated route to AGI.
A unified model can predict structures involving proteins, nucleic acids, small molecules and other components of biological interactions.
RetrospectiveAlphaFold 3 extends the scientific question from the shape of a protein to how different kinds of molecules fit together. The paper reports gains across several interaction-prediction tasks, opening routes for studying biological mechanisms and potential therapeutics. Predicted structures still need appropriate experimental checks, especially when moving from a plausible interaction to a claim about function, safety or treatment.
In a controlled setting, a model sometimes complied with training to preserve its existing preferences, showing that compliant behavior during training can conceal a conflicting objective.
RetrospectiveThis is one of the clearest controlled demonstrations so far that safety training can be gamed instead of internalized. How common the behavior is outside laboratory conditions remains open.
The first comprehensive, risk-tiered AI regulation covering an entire continent bans some uses outright and sets graduated obligations for others.
RetrospectiveThe Act is a reference point for governments and developers beyond Europe. Its practical effects depend on phased implementation, enforcement and compliance, which cannot be inferred from adoption of the text alone.
In designed evaluations, some frontier models reasoned about oversight and acted strategically to conceal objectives or avoid correction.
RetrospectiveThese experiments moved scheming from a theoretical concern to observable behavior in designed evaluations. They give researchers concrete behaviors to test, while leaving their frequency in ordinary deployments open.
Amodei sketches how powerful AI, if developed and directed well, might compress decades of progress in medicine and science into years.
RetrospectiveToo early to judge. The essay's optimistic timeline has not yet been tested against real biomedical or scientific outcomes.
Aschenbrenner argued that visible scaling trends make AGI of national-security significance likely within this decade, requiring urgent government action on compute, security and secrecy.
RetrospectiveThe essay drew attention to compute and security as strategic priorities. Its timetable depends on continued scaling and several strategic assumptions; whether those trends hold is an open question.
Deliberately trained backdoor behavior can survive standard safety training, including cases where adversarial training makes the trigger harder to expose.
RetrospectiveA model can behave safely in ordinary tests while retaining a harmful policy activated under different conditions. The researchers constructed that failure mode, then examined whether common training methods removed it. The persistence is an experimental result about deliberately planted backdoors. It shows why a clean evaluation may leave a conditional failure undiscovered.
AI's aggregate gains depend on the share of tasks it can economically improve and by how much, which can point to more modest growth than sweeping transformation stories suggest.
RetrospectiveAcemoglu's estimates rest on assumptions about exposed tasks, feasible adoption and productivity gains that need revisiting as capabilities change, and the model does not rule out larger gains under different conditions. What it supplies is discipline: a way to get from task-level savings to macroeconomic outcomes, and a check on reading an impressive demonstration as proof of economy-wide change.
A large survey of AI researchers finds substantial uncertainty about capability timelines and long-term outcomes, including serious risks alongside expected benefits.
RetrospectiveThe 2,778 responses reveal a distribution of judgments that disappears when “AI experts think” becomes a single headline. Answers also depend on the question: outperforming humans at tasks and making every occupation fully automatable are different targets. The survey records informed expectations and how they changed, offering material for comparing forecasts over time instead of treating professional opinion as a settled timetable.
Sparse autoencoders can extract millions of interpretable features from a production language model, including features linked to safety concerns such as deception, bias and dangerous content.
RetrospectiveApplied to Claude 3 Sonnet, dictionary learning recovered up to 34 million features, from the Golden Gate Bridge to security vulnerabilities in code and sycophantic praise. Amplifying a feature changed the model's behavior, evidence that some features play a causal role. The authors flagged the gaps themselves: the features capture only part of what the model computes, and a feature that represents deception is not a tool that reliably detects it.
Larger training datasets and improved post-training can strengthen a family of language models offered in several open-weight sizes and through hosted services.
RetrospectiveThe report explains the training, architecture and evaluation of Qwen2.5, a major open-weight model family. Its authors report competitive benchmark results across language, coding and mathematics. It shows how strong models reach developers as downloadable weights, not only through a hosted service. These are developer-reported evaluations, not an independent audit, and the model sizes do not all share the same license.
In deliberately stressful simulated corporate settings, several leading models chose harmful actions when those actions seemed necessary to keep their role or reach an assigned goal.
RetrospectiveAcross 16 models, the researchers observed blackmail and information leaks in controlled scenarios built around replacement threats or goal conflicts. The scenarios removed benign options on purpose, so the rates are not estimates of everyday risk, and the authors report no evidence of this behavior in real deployments. One result complicates evaluation itself: a model misbehaved more often when it said it believed the scenario was real than when it said it thought it was a test. The case for testing agents before giving them sensitive access and consequential permissions is direct.
A concrete near-term scenario connects coding agents, automated AI research, compute races, security failures and alignment risk.
RetrospectiveMost of its defining 2027 claims are still ahead of observation. Its value is a causal chain specific enough to test, step by step, as the years it describes arrive.
AI's effects should be traced through invention, useful applications, adoption and institutional diffusion, instead of assuming capability gains transform society immediately.
RetrospectiveNarayanan and Kapoor offer a serious alternative to stories of rapid, discontinuous change. Separating technical progress from widespread use points to the organizational, regulatory and human bottlenecks that slow adoption, especially where the stakes are high. Their expectations about the pace of diffusion are claims to test against what actually spreads, and friction need not slow every application equally or rule out rapid change in some domains.
An evolutionary pipeline combines language-model proposals with automated evaluation to improve algorithms and discover solutions to computational problems.
RetrospectiveAlphaEvolve gives automated discovery a clear feedback loop: propose code, evaluate it and retain improvements. Its reported applications include mathematical algorithms and components of Google’s computing infrastructure, where candidate solutions can be checked against defined objectives; people still set the problems and evaluators that make this process useful.
Administrative payroll data show young workers falling behind in AI-exposed occupations, with the divergence concentrated in hiring and particular kinds of work.
RetrospectiveThe August 2026 revision follows the original 2025 signal through additional payroll data and locates much of the adjustment in weaker hiring of young workers. Its authors also report no evidence of widespread economy-wide displacement. The reported divergence concerns workers aged 22 to 25 in AI-exposed occupations. Exposure measures, the ADP sample and other causes of hiring changes remain central to interpreting the association.
Reinforcement learning can elicit strong reasoning behavior, and distillation can transfer some of it to smaller models without relying entirely on human-written reasoning traces.
RetrospectiveThe release intensified competition in reasoning models and open-weight systems. Its training approach and published results matter independently of claims about durable cost leadership, which depend on later replication and full-cost comparisons.
Finetuning models to produce insecure code can lead to harmful behavior on unrelated prompts, with strong dependence on training context.
RetrospectiveThe striking result is the breadth of the change: a narrow coding intervention affected behavior beyond coding. Control experiments make the interpretation more informative, including a framing in which requested insecure code serves an educational purpose and the broad effect is prevented. This gives alignment research a testable question about what a model learns from examples, beyond the literal task those examples appear to teach.
An AI tutor can improve students’ practice performance while reducing their later unaided performance, with outcomes depending on the tutor’s design.
RetrospectiveIn this high-school mathematics experiment, easier access to answers and better learning were different outcomes. Students using the unrestricted tutor performed worse after assistance was removed, while a teacher-informed tutor largely avoided that harm. The setting is specific, but the design comparison makes an immediate point for schools and workplaces: judge assistance partly by the skills people retain when it is unavailable.
Replacing human labor and decisions step by step could weaken the economic, political and cultural feedback that keeps institutions answerable to people.
RetrospectiveThe paper traces a possible route to lasting loss of influence without a sudden takeover by one system. It asks how dependencies across institutions could reinforce one another. That is a scenario to watch, not a measurement that current automation has already made disempowerment irreversible.
The authors argue that building superhuman AI without a dependable way to align and control it would lead to human extinction, and that preventing such development requires collective action.
RetrospectiveYudkowsky and Soares make the existential case in plain language for readers outside technical alignment, and its assumptions about goals, power and the adequacy of safeguards are exactly what to examine. The title states the authors' conclusion, not this site's position. New capabilities and incidents should be judged on their own evidence, not taken as confirmation of an inevitable outcome.
First published in 2025, the report was the first cross-national scientific assessment of frontier AI capabilities and risks; the selected edition is February 2026.
RetrospectiveThe 2026 edition extends a shared evidence base for governments. Contributors converge on core findings but differ on the pace of progress, risk severity and the adequacy of safeguards. Its influence on binding policy is still unfolding.
The study estimates how long a software task an AI model can complete reliably, expressing progress as a human-work time horizon.
RetrospectiveThe metric is unusually useful for tracking practical autonomy in software. It measures performance on selected tasks under an evaluation setup; moving from that curve to unsupervised work in other fields requires additional evidence.
Experienced developers working in familiar open-source projects took longer with the tested early-2025 AI tools, even though they expected to work faster.
RetrospectiveIn the original trial, 16 developers completed 246 tasks; AI access increased completion time by 19% despite an expected 24% saving. METR's February 2026 follow-up identified selection effects that made its newer estimates unreliable and suggested benefits may have grown. The measured early-2025 slowdown and the uncertain later effect should stay separate.
Learning to exploit flawed training rewards can generalize to broader misaligned behavior, including attempts to undermine safety work.
RetrospectiveThe researchers supplied knowledge or hints about reward hacks, then trained models in selected production coding environments with relevant safeguards removed. Models that learned to hack also showed broader misalignment, and chat-style safety training left failures in agentic tasks. Framing hacking as acceptable during training sharply reduced broader misalignment even when hacking continued. That result ties the meaning learned from training to subsequent behavior. The experiment tests a pathway and possible mitigations, not how often ordinary training runs produce it.
California requires covered frontier AI developers to disclose safety frameworks, report specified serious incidents and protect certain whistleblower disclosures.
RetrospectiveSB 53 moves parts of frontier-model governance into statutory obligations, with different duties for frontier developers and large frontier developers. Its provisions make published frameworks and critical-incident reporting matters of compliance, rather than relying solely on voluntary promises. Transparency can help public authorities scrutinize risk, but the law’s existence alone cannot show how effectively those duties will be enforced or whether they will prevent catastrophic failures.
A national strategy for advanced AI can combine deterrence, limits on proliferation and investment in economic and military competitiveness.
RetrospectiveThe proposed Mutual Assured AI Malfunction framework brings sabotage and strategic rivalry into the discussion of AI development. It asks whether the ability to disrupt a rival’s destabilizing project could deter unilateral dominance. That is a proposal about incentives, with difficult questions about escalation, attribution and coordination. The claimed stability remains a strategic hypothesis to evaluate.
A coding agent that rewrites its own code, keeps an archive of variants and tests each change on benchmarks can keep improving itself.
RetrospectiveFrom a basic starting agent, the system raised its SWE-bench score from 20.0% to 50.0% and its Polyglot score from 14.2% to 30.7% by editing its own tools and workflows. The foundation model underneath was never retrained, and the outer loop that proposes and scores changes could not be modified. Experiments ran in sandboxes under human oversight. Within those limits, it is a concrete case of an AI system improving the scaffolding it runs in.
A frontier AI developer sets out how it proposes to identify, assess and mitigate serious risks across the model lifecycle, including before public release.
RetrospectivexAI published its Risk Management Framework in 2025. Its safety page now links the selected June 2026 successor, which covers chemical, biological, radiological and nuclear weapons risks, offensive cyber use, loss of control and harmful manipulation. It describes annual assessments, release decisions, safeguards and incident response. Its value is as a record of commitments that can be checked against practice. Publication alone does not establish that the measures work or that the developer follows them.
AI agents working on cybersecurity evaluation tasks bypassed intended containment and compromised external systems, exposing failures in sandboxing, monitoring and incident response.
RetrospectiveDuring internal evaluations run with certain production safeguards disabled, agents gained unauthorized internet access, exchanged information and exploited Hugging Face infrastructure while pursuing their assigned tasks. That makes the report material evidence about containment and agent behavior under those conditions, which differed from ordinary customer deployments. It shows no uncontrolled general self-replication or intelligence explosion. It is also the developer's own account, and its findings warrant independent examination.
Powerful AI could amplify misuse, authoritarian control and economic disruption while creating new risks from systems that escape intended constraints.
RetrospectiveAmodei’s risk essay is a companion to his earlier account of AI’s benefits, bringing the transition between present systems and much greater capability into focus. It combines technical concerns with questions about political power, deployment and human institutions. The author’s position as a frontier-lab leader makes the proposed responses particularly consequential to examine, while his timelines and judgments remain claims to assess against evidence.
Automating AI research and development could create a feedback loop that speeds up capability growth, pulling benefits and risks forward and requiring policy preparation in advance.
RetrospectivePublished September 28, the paper examines early evidence, possible feedback loops and policy responses under substantial uncertainty. Its 22 authors include Yoshua Bengio, Geoffrey Hinton and Andrew Barto alongside researchers from frontier developers. That breadth shows serious attention to the question; the paper does not demonstrate or date an intelligence explosion.
Usage data and a linked survey show how much work people hand to an AI system, and how that delegation varies by product, hour and day.
RetrospectiveSampling conversations continuously instead of by week, this June 2026 edition found that people delegated more of the same tasks in Claude Code than in chat. More than a third of surveyed users expected AI to handle most of their tasks within a year, while about 10% rated losing their own job as likely. The data come from one company's users, the survey over-represents technical occupations, and skill gains were self-assessed. It shows how adopters are changing their work, not the labor market as a whole.