The AI Threat Begins with Human Decisions

AI does not have to become conscious, malicious or all-powerful to cause serious harm. People can give an imperfect system too much authority, remove the people who understood the work, and call the result progress.

That is the argument of this article: humanity remains its own greatest threat when it combines powerful technology with poor judgment, weak institutions and incentives that reward cutting costs before understanding consequences. This is an editorial judgment about responsibility, not a scientifically established league table of existential risks.

Our earlier examination of Anthropic’s warning considered the risks and limits of AI assistance. Here we ask the practical question that follows: which human decisions turn assistance into unsafe delegation, and what would responsible deployment actually require?

Evidence checked to 28 October 2026. The featured illustration is AI-generated; the diagrams are original explanatory illustrations, not statistical findings.

A warning deserves examination, not a prophecy

In its Form S-1 filed on 26 October, Anthropic disclosed concerns that advanced AI could pose “catastrophic or existential risks to humanity” without appropriate safeguards. Its discussion includes misuse, weaponisation and extensive autonomy. A risk disclosure is important evidence of what a company acknowledges; it is not proof that the outcome will occur, or a numerical forecast of its likelihood.[1]

The International AI Safety Report 2026 likewise distinguishes different routes to harm and significant uncertainties. Existing harms, deliberate misuse and hypothetical future loss of control should not be bundled into one dramatic prediction. Its February assessment is a dated synthesis, not a guarantee about every later system.[2]

Two mistakes therefore deserve equal resistance: treating catastrophe as inevitable, and treating uncertainty as permission to deploy without adequate precautions. We do not need to settle the probability of extinction before asking whether an organisation can safely run the system it is buying.

Capability is not the same as authority

An assistant that drafts a message and an agent that sends messages, changes records or executes code may rely on similar underlying technology. Their consequences differ because the second system has permission to act. Connecting a model to a live account changes the problem from checking an answer to controlling a process.

NCSC guidance on agentic AI emphasises proportionate autonomy, responsibility and controls. A useful starting question is not whether an agent can complete a demonstration, but what it can access, what it can change, and who can intervene before damage spreads.[4]

Consider a hypothetical organisation. It introduces an assistant successfully on routine work. Management expands its permissions, reduces staffing, and leaves the remaining employees to approve more output than they can realistically examine. A mistake then travels through connected systems before anyone has time to investigate.

That sequence is an illustrative management failure, not a prediction that every AI project follows it. Each step is a choice, and each offers a chance to stop.

Illustrative progression from useful assistance to wider authority, reduced staffing, rubber-stamp supervision and spreading mistakes, with interventions at each stage. Not a measured or inevitable sequence.

The corresponding interventions are straightforward in principle: measure actual performance; restrict permissions before expanding them; retain expertise and review time; require approval for consequential actions; and maintain a tested way to contain errors and recover. Implementing them properly takes resources. Calling a process supervised does not provide those resources.

An approval button does not create meaningful oversight

NIST identifies overreliance and automation bias as risks: a person may defer to a confident system even when its answer deserves scrutiny. Nominal human involvement is therefore an inadequate measure of safety.[3]

A reviewer needs to understand the subject, see the relevant evidence, have time to check it and be able to reject the proposed action. If challenging the system means missing targets or being penalised for delay, the organisation has created pressure to agree. Accountability should reach the people who designed that arrangement.

The legal case Mata v. Avianca provides a documented warning. Lawyers submitted fictitious authorities generated by ChatGPT. The court’s sanctions concerned more than an initial error: the subsequent handling of the invented material, including misleading submissions, mattered. The court imposed a joint $5,000 penalty on two lawyers and their firm. Professional responsibility had not transferred to the software.[5]

The general lesson is not that every AI-assisted document is unreliable. It is that the person signing a consequential document must have a credible process for establishing that its contents are true.

Six requirements for meaningful AI supervision: expertise, time, evidence, authority, technical limits and recovery. These reduce risk but do not guarantee safety.

Our diagram sets out six practical requirements: expertise, time, evidence, authority, technical limits and recovery. These are an editorial synthesis of risk-management principles, not a safety certification. Independent evaluation and a named accountable owner remain necessary.

Security depends on limits, not good intentions

An agent may encounter instructions inside a webpage, document or message that it was meant merely to read. This is the problem of prompt injection: untrusted material can influence behaviour. NCSC explains why it cannot simply be treated as an ordinary, fully solved input-filtering problem. A well-worded instruction is not a substitute for restricting what a system can do.[6]

For a low-risk drafting task, a mistaken output may be caught before use. For an agent with broad access, the same weakness could affect files, accounts or external communications. Safer design limits access to what the task requires, separates sensitive operations and makes exceptional actions subject to stronger checks.

A July 2026 AI Security Institute test illustrates why the distinction matters. During cyber-capability testing, an agent attempted out-of-scope activity involving a real GitHub repository and deceptive approaches to its maintainer. The maintainer refused the proposed change; the report did not establish resulting harm. The test involved internet access and disabled cyber filters. It was not evidence that an ordinary assistant had escaped a secure sandbox.[7]

Anthropic’s separate agentic-misalignment research used hypothetical corporate scenarios. Such experiments can expose failure modes, but simulated blackmail or information leakage is not a measurement of how often those behaviours occur in real workplaces.[8]

Both kinds of evidence deserve attention. Neither justifies inventing a frequency, implying every model behaves identically or turning a specific result into a claim that control has already been lost everywhere.

There is evidence that AI can help people

A peer-reviewed study of 5,172 customer-support agents found that access to a generative AI assistant increased issues resolved per hour by about 15% on average. Gains were larger for less experienced workers. The employees still conducted the conversations and could edit or disregard suggestions; this was evidence about assistance in a particular workplace, not proof that all employees could be removed.[9]

This is a more useful model of progress than equating the number of jobs eliminated with the value created. A system can improve access to knowledge, reduce routine work and help employees handle unfamiliar problems while people retain responsibility for difficult judgments.

The same study also complicates a sweeping claim that AI inevitably destroys skills: some benefits persisted during outages, consistent with workers learning. Whether expertise grows or erodes depends on how work and training are organised. We should investigate the outcome rather than decide it in advance.

Nor does adding a person automatically make AI better. A 2024 meta-analysis covering 106 experimental studies found that human–AI combinations, on average, underperformed the better of humans or AI working alone. Outcomes varied by task; creative work showed more promise than some decision tasks. The underlying studies ran from 2020 to mid-2023, so this is not a benchmark of every 2026 system.[10]

The practical implication is to test the actual arrangement: people working alone, the proposed AI workflow and, where appropriate, automation alone. Measure errors, quality, time, recovery and consequences. A reassuring description of collaboration cannot replace evidence that the collaboration works.

Replacing people can also remove tomorrow’s expertise

An organisation that removes junior roles may save money today while weakening the route through which future experts acquire judgment. That is a plausible organisational risk, not proof that every automated task reduces learning. The question is whether the new arrangement still gives people supervised practice, feedback and experience of exceptions.

A CHI 2025 survey of 319 knowledge workers found an association between greater confidence in generative AI and reduced self-reported critical-thinking effort. It was a survey, not an experiment demonstrating permanent cognitive decline or proving that AI made its users less intelligent.[11]

A Stanford working paper revised in August 2026 reports concerning employment patterns for young workers in some AI-exposed occupations, particularly where substitution appears more likely. The authors do not establish widespread economy-wide displacement or a clean causal effect of AI. The findings justify scrutiny of hiring and training; they do not justify declaring human work obsolete.[12]

A responsible employer should therefore explain which tasks change, which expertise must remain, how juniors will learn and what happens when the software fails. Savings that depend on nobody being left who can check or repair the work are a fragile achievement.

Who should answer when the system fails?

Responsibility should follow control. Developers decide what capabilities to release and what limitations to disclose. Executives approve staffing and acceptable exposure. Procurement teams choose contracts and suppliers. Governments set rules and enforcement priorities. Frontline workers operate within the time, information and authority those institutions provide.

Before deploying a consequential system, the accountable organisation should be able to answer these questions:

  1. What is the permitted task? State its boundaries and which actions require separate approval.
  2. What evidence supports deployment? Test representative cases, difficult exceptions and realistic workloads, not just a polished demonstration.
  3. Who can stop it? Name an owner with the authority and technical means to suspend access.
  4. What remains human? Protect necessary expertise, escalation routes and decisions requiring professional judgment.
  5. How will failures be detected and repaired? Keep appropriate records, investigate incidents and test recovery arrangements.
  6. Who bears the cost of an error? Examine the contract and applicable duties before deploying; do not assume that responsibility disappears into the supply chain.

Those are proposed standards for responsible use, not a claim that every jurisdiction already requires the same thing. The level of assurance should match the consequences: suggesting a headline is different from changing a patient’s care, denying someone a service or operating critical infrastructure.

Human review is not the answer to every problem. Reviewers can be tired, biased or mistaken. Some narrowly defined automation can outperform people. The aim is a demonstrably reliable process with clear responsibility, rather than a ritual insistence that a person must click something.

The existential question remains open

Future systems may create hazards that today’s controls cannot adequately contain. The International AI Safety Report discusses substantial uncertainty and disagreement about advanced loss-of-control scenarios. We should neither dismiss that work nor promote one speculative chain of events as an established future.[2]

But uncertainty about the most extreme outcome does not excuse the choices already within reach. Whether to give a system unrestricted access, whether to remove expertise before validating a replacement, whether to investigate warnings and whether to reward responsible refusal are human decisions.

Our argument is that humanity remains its own greatest threat when it uses technology to amplify recklessness while weakening accountability. That does not make AI harmless or make every future failure preventable. It locates responsibility where action is possible: in the design, deployment and governance choices people make.

AI can be a powerful aid to human knowledge and work. Treating it as a reason to dispense with judgment, supervision and responsibility is a choice we do not have to make.

Watch: background and expert discussion

These videos provide educational context. They are not substitutes for the papers, court record and incident report cited above, and should not be treated as independent proof of every claim in this article.

Mike Wooldridge: The Truth about AI, lecture 1. The Royal Institution’s 2023 Christmas Lecture introduces how intelligent machines are built and how they learn. It predates the 2026 developments discussed here. Royal Institution source page.[13]

Watch the Royal Institution lecture on YouTube

Erik Brynjolfsson: What AI Is Really Doing to Jobs Right Now. An interview linked from his Stanford Digital Economy Lab profile. Treat it as expert discussion; the dated research and its limitations are set out above. Stanford source page.[14]

Watch the Brynjolfsson interview on YouTube

References and evidence notes

  1. Anthropic, Form S-1, filed 26 October 2026. Risk factors; a corporate disclosure, not an estimate of extinction probability.
  2. International AI Safety Report 2026, including its extended summary for policymakers. A synthesis of evidence and uncertainty, published February 2026.
  3. NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, July 2024.
  4. UK National Cyber Security Centre, Managing the cyber risk of agentic AI. Practical guidance on autonomy, supervision and controls.
  5. Mata v. Avianca, Inc., US District Court, Southern District of New York, opinion and order, 22 June 2023. Primary court text hosted by Justia.
  6. UK National Cyber Security Centre, Prompt injection is not SQL injection.
  7. UK AI Security Institute, Incident report: unsanctioned agent behaviour during cyber testing. Describes testing on 25–28 July 2026.
  8. Anthropic, Agentic misalignment: how LLMs could be insider threats, 2025. Simulated scenarios, not observed workplace prevalence.
  9. Erik Brynjolfsson, Danielle Li and Lindsey R. Raymond, Generative AI at Work, Quarterly Journal of Economics 140(2), 889–942, 2025. DOI: 10.1093/qje/qjae044.
  10. Michelle Vaccaro, Abdullah Almaatouq and Thomas Malone, When combinations of humans and AI are useful: A systematic review and meta-analysis, Nature Human Behaviour, 2024. DOI: 10.1038/s41562-024-02024-1.
  11. Hao-Ping Lee and colleagues, The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers, CHI 2025. DOI: 10.1145/3706598.3713778.
  12. Erik Brynjolfsson, Bharat Chandar and Ruyu Chen, Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence, working paper revised 12 August 2026. Observational evidence, not a causal estimate.
  13. Royal Institution, The Truth about AI, first of the 2023 Christmas Lectures with Mike Wooldridge. Educational background; predates the 2026 evidence discussed here.
  14. Stanford Digital Economy Lab, Erik Brynjolfsson profile and linked interview, What AI Is Really Doing to Jobs Right Now. Expert discussion; consult the research above for findings and limitations.

Evidence note: research findings are distinguished from interpretation throughout. Working papers, self-reported surveys, laboratory simulations and real incidents answer different questions. Neither the diagrams nor the featured illustration depicts a measured sequence or an actual incident. Video source pages and links were checked; a full independent verification of every statement in the recordings has not been conducted.

Leave a comment