Manifold Grace

INDEPENDENT EVIDENCE · AI SECURITY · DATA GOVERNANCE

AI Tool Security and Trust: 20 Documented Case Studies

Comparative evidence for assessing external AI models, research platforms, hosting providers, public code, automation services and third-party connections. Updated 10 November 2026. Cases include AI-specific systems and wider technology dependencies that an AI system may use.

Interpretation: An incident demonstrates a type of failure, not that every employee or service in the named organisation acts maliciously. Distinguish confirmed incidents, research findings, allegations and regulatory conclusions. Historic security failures do not establish present insecurity; an absence of known incidents is not proof of safety. Nationality or jurisdiction alone is not evidence of intent.

Risk categories for any AI integration

Data and intellectual property

Retention, provider training, contractor access, deletion, licences, copyright, transcripts and metadata.

Cybersecurity

Access controls, API keys, dependencies, update integrity, vulnerabilities, prompt injection, model execution and incident response.

Governance and geopolitical exposure

Applicable law, processing location, lawful disclosure requirements, ownership changes, service continuity and documented external dependencies.

Human and public-interest risks

Unfair or misleading outputs, surveillance, personal-data misuse, concentration of infrastructure and inadequate avenues to challenge decisions.

Case-study register

20 case studies displayed

No.YearCase studyEvidence statusWhat occurred or was establishedApplication to safer AI developmentPrimary/authoritative source
12023OpenAI ChatGPT history and payment-data exposureConfirmed software defectAn open-source library bug could reveal other users’ chat titles and limited payment-related details during a defined period.Isolation tests; minimise confidential prompts; verify provider incident reporting.Evidence and report
22025OpenAI–Mixpanel analytics incidentConfirmed third-party incidentOpenAI reported that Mixpanel systems were accessed and limited identifying analytics data exported; OpenAI said prompts, chat content and API keys were not compromised.Audit third-party analytics, identifiers and subprocessors separately from core AI infrastructure.Evidence and report
32024Hugging Face Spaces secrets incidentConfirmed platform incidentHugging Face detected unauthorised Spaces access and suspected some stored secrets had been accessed; it rotated tokens and introduced stronger controls.Use narrowly scoped, disposable tokens; minimise hosted secrets; log key rotations.Evidence and report
42023Microsoft AI repository storage exposureResearcher-verified exposureSecurity researchers found an over-permissive Azure share token exposing access to approximately 38 TB of additional private storage; the token was revoked.Never publish broad cloud access tokens with models or data; segregate public and private storage.Evidence and report
52025DeepSeek publicly accessible databaseResearcher-disclosed exposureSecurity researchers identified a database accessible without authentication containing logs, chat histories and sensitive fields; exposure does not establish that unknown parties downloaded them.Require network isolation, authenticated database access, and documented retention.Evidence and report
62023Samsung confidential material submitted to external AIReported organisational data handling incidentNews reports described employees placing sensitive code into an external AI service and the organisation responding with usage restrictions.Classify information before submitting to hosted AI; use approved local or contracted environments.Evidence and report
72022PyTorch nightly package compromiseConfirmed dependency compromiseA malicious torchtriton package affected specified PyTorch nightly installations on Linux during December 2022; stable packages were not affected.Pin dependencies, verify release provenance and minimise build-machine permissions.Evidence and report
82024XZ Utils maintenance-chain backdoorConfirmed malicious software supply chainAn intentionally inserted backdoor appeared in affected xz releases; the operational impact depends on distribution and environment.Review maintainership, dependency changes and signed build artifacts, not only licences.Evidence and report
92025tj-actions/changed-files workflow compromiseConfirmed CI/CD supply-chain compromiseCompromised GitHub Actions versions exposed the risk of leaking workflow secrets to logs; NHS England published an advisory.Pin reviewed action commits, restrict secrets in public CI and monitor workflow changes.Evidence and report
102023CircleCI employee-session compromise and token exposureConfirmed service provider intrusionAn attacker compromised an engineer’s session and extracted some customer environment variables, keys and tokens, prompting key rotations.Treat CI providers as privileged processors; limit secrets and rotate after credible incidents.Evidence and report
112020SolarWinds Orion update compromiseConfirmed supply-chain intrusionA compromised software update was used in an extensive intrusion campaign affecting government and commercial environments; subsequent targeting affected a subset of users.Verify software update chains, environment segregation and privileged-service access.Evidence and report
1220233CX cascading supply-chain incidentConfirmed vendor intrusion and distributionMandiant traced compromise of a communications application through a previously compromised software dependency, with malicious desktop-app distribution.Check dependency provenance, vendor incident response and endpoint isolation.Evidence and report
132024Snowflake customer account compromise campaignConfirmed customer-account attacksMandiant reported attackers using stolen customer credentials against multiple Snowflake customer environments, with approximately 165 potentially exposed organisations notified.Enforce MFA, least-privilege queries, access monitoring and short-lived service credentials.Evidence and report
142023Okta support-system file and session exposureConfirmed support-system intrusionAn attacker accessed support files for 134 customers and could hijack a small number of sessions; follow-up findings identified wider support-user contact-data exposure.Scrub session data from support bundles; isolate vendor-support access and require session revocation.Evidence and report
152022LastPass backup-data breachConfirmed provider intrusion and regulatory actionAttackers accessed some archived customer information; UK regulators later found inadequate protections for sensitive personal information. Encryption limited but did not eliminate the risks.Minimise concentrated sensitive data, independently review encryption and recovery processes.Evidence and report
162023MOVEit Transfer exploited vulnerabilityConfirmed mass third-party vulnerabilityAn exploited file-transfer vulnerability was associated with theft of customer and employee data from multiple organisations and their supply chains.Inventory managed-transfer dependencies and vendor exposure; patch and independently validate.Evidence and report
17202323andMe account compromise and related-person exposureRegulator-confirmed breachCredential-stuffing attacks exposed account and related-person data; a 2025 UK–Canada investigation found security safeguards inadequate.Protect sensitive relational datasets; MFA, anomaly detection and limits on downstream sharing.Evidence and report
182015–2017Royal Free NHS and DeepMind health-data sharing reviewRegulator-confirmed governance deficiency, not a breach findingUK oversight examined whether the data-sharing arrangement, transparency and purpose limitation adequately protected patient information.Confirm lawful purpose, necessity, minimisation, disclosure and independent oversight before AI partnerships.Evidence and report
192024Change Healthcare ransomware and services disruptionConfirmed incident and regulator investigationA ransomware event disrupted healthcare and billing operations and triggered an investigation of privacy/security obligations.Require exit and continuity planning for concentrated service providers and critical infrastructure.Evidence and report
202023Microsoft Storm-0558 cloud email intrusionIndependent federal review of a security incidentA government review described forged authentication tokens used to access enterprise email accounts and gaps in the provider’s explanation of key exposure.Evaluate independent incident investigations, authentication boundaries and national-security exposure.Evidence and report

Case-study inclusion reflects transferable technical or governance lessons, not a ranking of current provider risk. Do not use these incidents to infer motives of present staff, national groups or unrelated services.

Evidence required before connecting any tool

Data custody and disclosure

Record processing and backup locations, retention windows, deletion controls, permitted model-training uses, subprocessors, support access, lawful requests and the rights of people whose data is included. Obtain a documented legal basis where personal data is processed.

Security architecture and supply chain

Record the software and model licence, repository source and maintainer, pinned release, software bill of materials where applicable, isolation of secrets, authentication method, audit events, vulnerability reporting and incident notification terms. Local open-source software still requires review.

Geopolitical and corporate dependencies

Record public corporate ownership, hosting and processing jurisdictions, relevant legal obligations, known material service restrictions and available exit paths. Assess actual policy and technical evidence, not assumptions about ethnicity, citizenship or personal political beliefs.

AI output safety and intellectual provenance

Test source accuracy, hallucination, independent corroboration, model-generated misinformation, prompt injection, improper retrieval, idea attribution and the reliability of recorded corrections. Keep human approval for consequential communications or publishing.

Decision record and reassessment

For each trial record who approved it, what data classes may be sent, what was tested, what failed, total expected cost, revocation steps and the next review date. A free tier is a pricing decision, not a security certification.

Return to the searchable AI tool inventory · Public-interest AI programme · Productivity log