Home Office used 'AI hallucinated' information to refuse asylum claim, judge 英国内政部使用“人工智能幻觉”信息拒绝庇护申请，法官指出

TL;DR

• A UK Home Office asylum refusal relied on a non-existent Country Policy Information Note (CPIN), which the tribunal judge suggested was an "AI hallucination."
• The missing document, cited as confirming Morocco's safety for return, could not be verified by the Home Office or found in public records.
• The judge ruled that relying on this fabricated evidence constituted a procedural irregularity akin to using bogus evidence.
• An internal Country Information Note (CIN) was later produced but contradicted the CPIN's claims regarding safety and legal protections for women in Morocco.

Why It Matters

This case highlights critical risks associated with integrating generative AI into high-stakes government decision-making processes, particularly where automated systems may generate plausible-sounding but entirely false citations ("hallucinations"). It underscores the urgent need for rigorous human oversight, verification protocols, and transparency when AI tools assist in legal and administrative rulings affecting fundamental rights like asylum.

Technical Details

• The Home Office used a purportedly authoritative Country Policy Information Note (CPIN) dated July 2021 to justify refusing a Moroccan woman’s asylum claim based on safety assurances from Morocco.
• No such CPIN exists in public archives or within the Home Office’s own records; the document was never disclosed during initial proceedings.
• A senior Upper Tribunal judge noted stylistic hallmarks consistent with AI-generated text in the refusal letter, suggesting possible use of large language models without proper validation.
• Subsequently, an unpublished internal Country Information Note (CIN) was provided, which did not support the safety conclusion and instead highlighted risks including child marriage exemptions and lack of criminalization for marital rape.
• The discrepancy between claimed external documentation and internal guidance reveals potential misalignment between automated outputs and verified policy sources.

Industry Insight

Organizations deploying AI in sensitive domains must implement mandatory cross-checking mechanisms against primary source databases before any output influences official decisions. Additionally, audit trails should log whether AI-assisted content was reviewed by humans and what sources were consulted to prevent reliance on fabricated or misleading references. This incident serves as a cautionary tale about automating bureaucratic functions without sufficient accountability frameworks.

TL;DR

• 英国上议院法官指控内政部在拒绝摩洛哥妇女的庇护申请时，可能依赖了由AI生成的虚假文件（“AI幻觉”）。
• 该案件涉及一份不存在的“国家政策信息备忘录”（CPIN），其引用被指为程序性违规且等同于使用伪造证据。
• 实际存在的内部指导文件（CIN）并未支持摩洛哥是安全国家的结论，反而揭示了该国针对女性的法律风险。
• 此案凸显了公共部门在使用生成式AI进行决策时的严重合规与透明度风险。

为什么值得看

此案例是政府机构滥用AI技术导致司法不公的标志性事件，对AI从业者而言，它警示了自动化决策系统在缺乏人类审核和可解释性机制下的巨大伦理与法律风险，强调了在关键公共服务中建立严格AI治理框架的紧迫性。

技术解析

• 核心问题：内政部律师信中的引用格式和内容特征符合大型语言模型生成文本的典型模式，法官推测其引用的2021年7月摩洛哥CPIN文档实为AI虚构。
• 文档差异：官方公开的CPIN（国家政策信息注记）自2017年后未更新，而内政部实际使用的是未公开的内部CIN（国家信息注记），两者性质不同，前者具权威性后者仅为内部参考。
• 数据矛盾：内部CIN明确指出摩洛哥存在童婚豁免、缺乏强奸罪刑事化及对报案女性可能面临非法性交易指控等风险，与内政部声称的“安全”结论相悖。
• 流程漏洞：AI生成内容未经过事实核查即作为法律依据提交法庭，暴露了行政流程中对数字证据验证机制的缺失。

行业启示

• 政务AI监管亟需强化：政府部门引入AI辅助决策必须建立强制性的“人机回环”验证机制，确保所有输出信息可追溯、可核实，禁止将AI生成内容直接作为法定证据。
• 透明性与问责制是关键：任何基于AI的公共决策系统都应保留完整的审计日志，明确标注哪些部分由AI生成，并设立独立第三方评估机制以防止系统性偏见或幻觉导致的权利侵害。
• 推动可解释AI发展：未来应优先研发具备高可解释性和事实锚定能力的AI模型，特别是在司法、移民等高敏感领域，避免黑箱操作损害公民基本权益。