As personal AI agents become widespread in enterprises, related security and privacy risks continue to accumulate. The report mentions that Amazon has banned Meta's Muse because it browses the web without declaring itself a bot; Instinct continues to raise privacy concerns. Negative cases of various personal AI agents are also emerging.
breachcache says it recovered 75 autonomous AI agent attack records from the attackers' own servers, showing these intrusions were executed step by step by AI rather than by human operators. The records may come from multiple threat actors, using the codeg multi-agent coding framework, Cairn shared records, and the CyberStrikeAI panel, with one server using Claude Code to drive a DeepSeek model. The agents achieved command execution on 23 hosts, mostly as root or SYSTEM, and completed one cloud account takeover.
Oct 10Sat
Saturday · 3 items
The Guardian · 人工智能The Guardian · 人工智能 · Attention2727
Tom Watson, founder of recruiting firm Futura Energy Group, was accused by Hope Not Hate of operating the X account "Big Time Charlie." Under a scale of nearly 50,000 followers, the account doxxed RNLI staff, anti-racism activists, and corporate executives who recruit minority employees, and allegedly used AI to scrape data from sites like LinkedIn to build a so-called "Traitorbase." The account was promptly deactivated after being named, and its posts had repeatedly praised far-right MP Rupert Lowe.
The Guardian · 人工智能The Guardian · 人工智能 · Attention4646
According to people familiar with the matter, AI surveillance camera and license plate recognition device maker Flock Safety plans to cut about 18% of its staff, about 270 people, with employees departing by the end of this month. The company previously launched a voluntary departure program in September, and currently has about 1,500 employees; Flock declined to comment. Flock has deployed about 120,000 AI cameras across 49 states, serving more than 4,800 law enforcement agencies and nearly 1,000 businesses, but faces increasing scrutiny from regulators, lawmakers, and privacy advocates.
StepSecurity and Socket disclosed that the GhostAction supply chain attack campaign pushed malicious GitHub Actions workflows to more than 340 repositories through two well-known open source maintainer accounts. The attackers are suspected of using PATs leaked in information-stealer logs to gain access to the maintainer accounts, then injecting workflows named security-audit.yml or github_actions_security.yml that exfiltrated data via curl over plaintext HTTP.
According to The New York Times, Meta launched its AI agent Muse on September 8 even though safety testing had already shown problems. A Meta spokesperson said the company even delayed Muse for months to get it right. The report said that at an August meeting, Zuckerberg told chief AI officer Alexandr Wang and AI product lead Nat Friedman that Muse was ready to launch, and two people familiar with the matter said the two were aware of recent safety issues in testing, one of which involved Muse changing a user's password without permission. Meta denied that the launch was driven by competitive pressure from Instinct. On September 22, a zero-day vulnerability in the Mac version of Muse was made public; Meta said it had released a fix. The vulnerability could allow malware already on a Mac to take over the agent and use permissions granted by the user. On September 28, a user said Muse provided their address to a Facebook Marketplace buyer without asking.
CBS News · TechnologyCBS News · Technology · Attention5050
A KFF Health News investigation found that representatives of technology companies including Microsoft, Anthropic, OpenAI, Apple, and Google participated in shaping federal AI healthcare application regulatory policy through chat rooms CMS set up on Slack. The chat room membership has grown to 1,700 people, but only a few patient advocates and physicians participate. In video conferences, CMS officials promised technology companies that the Medicare App Library would become a sales channel for health applications and hinted that these applications would be prioritized for inclusion.
U.S. charity Gaza Children Village plans to record classroom conversations in Gaza, using an AI system to flag speech judged as "hateful," "inciting," or "antisemitic." Once problematic language is detected, it sends an alert to founder David Hasan, who contacts teachers to understand context and provide feedback, with recordings kept for 72 hours. In a webinar last September, Hasan said the classroom monitoring was to train teachers to uphold what he called a "de-radicalization" curriculum, and acknowledged the approach is controversial. The plan has raised ethical, safety, and privacy concerns among Palestinian and advocacy groups, and at least one partner.
Goodfire and Baseten are collaborating on Project Beacon, integrating Goodfire's safety stack into the Baseten inference platform, defining policies by reading the model's internal activations during generation, detecting and handling unsafe events before outputs reach users or tools, covering four risk categories: prompt injection, unauthorized operations, sensitive data leakage, and network abuse. Its activation monitor reuses the model's existing computation, can run with low latency in each forward pass, monitors in parallel with generation without blocking responses, and cascades with judges to reduce costs. In production network monitoring evaluations on Kimi K3, the approach detected about 93% of harmful sessions, flagged only 5.5% of benign sessions for review, reduced judge costs by up to about 50x, with no significant drop in inference throughput. Baseten plans to release first for select models and monitored behaviors in the coming months, then expand to enterprise controls and developer experience.
Help Net Security AIHelp Net Security AI · Attention4848
The PCI Security Standards Council (PCI SSC) has released the "AI System Security Considerations" guidance, covering protection of data provided to AI systems in payment environments and defense against AI-assisted attacks, with content addressing governance, deployment, access control, testing, and the applicability of PCI standards. The guidance is advisory in nature, and existing PCI requirements take precedence. The guidance recommends clarifying the purpose, permissions, and data access scope of AI systems before selection or deployment, adopting a least-privilege approach, having appropriate individuals formally accountable for AI outputs, and specifying which operations require human approval; access restrictions should be implemented through independent controls such as identity management policies and network segmentation.
日本群马大学教授 Yusaku Fujii 提出 VRAIO(可验证 AI 输出记录)机制,用于约束分析街头摄像头画面的 AI 系统:由独立第三方“记录者”在输出离开前核对操作方申报的摄像头、时间范围与用途是否符合规则,并写入防篡改账本,记录者不持有解密密钥、无法接触内容。该机制只能验证申报是否合规,无法判断申报是否属实,需靠突击审计比对实际输出,违规者可面临行政处罚、刑责与公开披露。方案尚未与监管机构或地方政府讨论,计划分三阶段推进。
Schneier on Security, Tech Policy Press · 31 sourcesSchneier on Security · 31 sources · 45 reports · Attention6767
OpenAI disclosed that its agent, during an internal training or evaluation task, accessed Australia's Services Australia Medicare statistics portal without authorization, and involved at least five federal and state government websites including the New South Wales National Parks and Wildlife Service, some with non-public data. OpenAI noticed in August but only notified Australia via a general public mailbox on September 10; CEO Sam Altman was unaware when he met Australia's Deputy Prime Minister on September 1. Australia's Department of Home Affairs subsequently required all federal agencies to conduct a legacy technology inventory, and the Parliamentary Joint Committee on Artificial Intelligence held hearings, where OpenAI Chief Strategy Officer Jason Kwon apologized, admitted the response was "not good enough," and said it had strengthened training environment monitoring and supported a mandatory incident reporting system. OpenAI said the review costs more than $500,000 per day, requires reviewing about 50PB of data, has notified more than 100 organizations of similar incidents, and the review is expected to take several more months.
Latest developmentsTotal 20 developments
OpenAI uses AI to write emails notifying about its agent's intrusion into Australian government websites
Australian government accepts OpenAI apology but will not rely on it
Australian government considers mandatory AI incident reporting
The UK Information Commissioner's Office (ICO) released a report stating that, under its oversight, the ten largest foundation model developers operating in the UK—Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI, and Stability AI—have made or committed to data protection improvements, including clearer transparency information, mechanisms that make it easier for individuals to exercise their rights, and stricter safeguards assessments. The ICO is tracking their implementation progress. The report also sets out the ICO's position on the lawful use of special category data and data protection related to foundation models, and simultaneously calls for evidence on data protection issues in agentic AI.