StableAI Engineeringagent-security

Agent Security — Untrusted Text, Bounded Actions

Operationalizes the security of a system where a language model plans, calls tools, and acts. The governing idea: retrieved text, documents, web content, and tool output are untrusted data, never instructions — a boundary application security has enforced for thirty years at every other layer, and which language models collapse by construction, because instructions and data are the same substrate read by the same mechanism. From that collapse follow prompt injection in both forms, tool and resource poisoning, exfiltration through channels nobody classified as egress, memory that stays poisoned across sessions, and a confused deputy acting with permissions its requester never held. The remedy is not a better prompt. It is bounding what the agent CAN do — scoped credentials, sandboxes, egress allowlists, approval gates on irreversible actions, and an audit record — so that a model persuaded by a hostile document achieves nothing outside those bounds.

prompt-injectionindirect-prompt-injectiontool-poisoningdata-exfiltrationexcessive-agencyconfused-deputy

Principios

· Aplica los principios documentados en el pack agent-security.

Patrones

· If you did not write it and you cannot revert it, it is untrusted. Ownership of the system that served the content is not ownership of the content. Your ticketing system is yours; the tickets were written by strangers.
· Provenance travels with the bytes, not with the pipeline. A document that becomes a summary that becomes a memory record is still attacker-influenced three hops later.
· Assume the hostile text is in the part nobody reads. Page footers, HTML comments, alt text, document metadata, the eleventh row of a CSV, the third page of a PDF.
· A tool result is a document. It arrived over a network from a system you do not fully control, and it deserves the same suspicion as a scraped web page.
· Content asserting its own authority is the signal. Nothing legitimate needs to tell the agent that prior instructions are revoked.

Riesgos

· Ver anti-patrones específicos del dominio en el pack.

Checklist

[ ] ★ Config check: every content source is classified by channel, and retrieved documents, files, DB rows, email, and tool results are all untrusted by default. (ASE1, ASE2)
[ ] ★ Untrusted content cannot authorize an action, widen a scope, or change a rule; the action gate consults provenance, not content. (ASE3)
[ ] ★ The agent's rules, toolset, scopes, and approval thresholds are loaded from change-controlled configuration and are not modifiable from the conversation, from retrieved content, or from memory. (ASE7, ASE57)
[ ] ★ Outbound network destinations are an allowlist with default-deny at the network layer, not a policy stated in the prompt. (ASE26)
[ ] ★ Model-generated markup does not auto-load remote resources in any client that renders it. (ASE27)
[ ] ★ Context-derived data never reaches a URL parameter, path segment, or fragment. (ASE28)

Fuentes

En la bibliografía AKOS

Este pack cita las siguientes fuentes registradas en el índice bibliográfico:

Explorar todas las fuentes →