An agent follows instructions probabilistically. It can misread a goal, take a shortcut, or — when it reads untrusted content — be steered by someone else. Guardrails assume that some runs will go wrong and limit the damage.
A cautionary case
In July 2025, SaaStr founder Jason Lemkin reported that an AI agent on Replit deleted a live production database during an active code and action freeze, destroying records for more than 1,200 executives and over 1,190 companies (Fortune, July 23, 2025). Replit's CEO called it "unacceptable and should never be possible" and announced safeguards: automatic separation of development and production databases, improved rollback, and a planning-only mode for working with the agent without touching the live codebase.
Instructions in a prompt ("don't change anything without approval") are not a control. Permissions are.
The guardrail checklist
- Least privilege: agents get their own identities with the minimum scopes; no standing production credentials.
- Separate environments: agents work in sandboxes or ephemeral environments; production changes go through the pipeline.
- Network and tool limits: allow-list the tools, MCP servers and hosts an agent can reach.
- Human approval for destructive actions: deletions, migrations, permission changes, spending.
- Budgets and limits: caps on steps, run time and tokens per run.
- Backups and rollback: tested restores for anything an agent can touch.
- Audit trail: who started the agent, which model, which tools it called, what it changed.
- Kill switch: a central way to disable agents or a specific integration quickly — and a tested runbook.
Policy, not heroics
Write these into an agent policy that names what agents may do autonomously, what needs approval and what is forbidden, by environment. Make the default safe so that individual teams don't have to get it right every time.