AI Agent Security: How to Vet an Agent Before Install

To vet an AI agent before you install it, check five things: who published it, what code it runs, what it can reach, what it sends over the network, and what credentials you are about to hand it. In practice that means reading the repository, running a static scan and a dependency check, starting it in a sandbox while you watch its traffic, and giving it the narrowest keys that still let it do the job.

This guide walks through that process step by step. It applies to autonomous agents, MCP servers and agent skills alike, because all three end up running code or taking actions with your permissions. Each section links to a deeper article in our AI agent security series.

Why AI agents need a different kind of vetting

A normal library runs when your code calls it. An agent decides for itself what to do next. It reads files, calls APIs, browses the web and runs commands, often with the same rights as the user who launched it.

That changes the risk in three ways:

• It runs code you did not write. An MCP server is usually an npm or PyPI package started on your machine. A skill can ship scripts. Installing one is the same as running a stranger's program.
• It reads content you do not control. Web pages, emails, issues and documents can contain instructions aimed at the model, not at you.
• It acts. Agents send emails, push commits, move files and call paid APIs. A mistake or a hijack has real side effects.

Simon Willison calls the dangerous combination the lethal trifecta: an agent that has access to private data, is exposed to untrusted content, and can communicate externally. If all three are true, an attacker who controls the content can often get your data out. Most useful agents have all three, which is why vetting matters.

The OWASP Top 10 for LLM Applications 2025 lists the same issues from a different angle. Three entries map directly to agent installs: LLM01 Prompt Injection, LLM03 Supply Chain and LLM06 Excessive Agency.

The four ways an AI agent can hurt you

Before you check anything, know what you are checking for. Almost every real problem falls into one of these buckets.

Risk	What it looks like	Main defence
Malicious code	A backdoor, credential stealer or data exfiltration hidden in the package or an install script	Provenance checks, code scanning, sandboxed run
Vulnerable code	Known CVEs in dependencies, unsafe shell calls, leaked secrets	Dependency scanning, static analysis
Prompt injection	Instructions hidden in a web page, email or issue that steer the agent	Limit tools, separate untrusted input, human approval
Excessive agency	Broad tokens, full disk access, unrestricted network	Least privilege and sandboxing

These are not theoretical. In September 2025, researchers reported a fake postmark-mcp package on npm that behaved normally for 15 versions, then added one line that blind-copied every outgoing email to an attacker. In May 2025, Invariant Labs showed how a malicious public GitHub issue could steer an agent using the GitHub MCP server into leaking private repository data through a pull request. The first was supply chain. The second was prompt injection plus an over-scoped token.

Step 1: Know exactly what you are installing

Start by naming the thing precisely. "An agent" can mean several different artifacts, and each has a different attack surface.

• An MCP server is a program your AI client launches or connects to. Local servers run on your machine with your user rights. Remote servers run elsewhere but receive your data and often your OAuth tokens. See what an MCP server is and our MCP server security guide.
• An agent skill is a folder of instructions, often with scripts the agent may run. It looks like documentation but can contain executable code. See what agent skills are and the agent skills guide.
• A full agent is an application that plans and calls tools on its own. It usually bundles its own dependencies and needs API keys.

Write down the install command you were given. If it is npx some-package, uvx some-package or a curl ... | sh line, that command is what will run with your permissions. Treat it with the same care as any other software install.

Step 2: Check provenance

Most supply chain attacks rely on you installing the wrong thing, or the right thing after it changed hands. Check:

1. The package name matches the repository. On npm, run npm view <package> repository.url. On PyPI, check the project links. A package with a name like an official vendor's product but published by an unknown account is a red flag. The postmark-mcp package above was not published by Postmark.
2. The publisher is who you think. Look at the maintainer list and their other packages. A brand new account with one popular-sounding package deserves extra scrutiny.
3. The repository has a history. Real projects have commits over time, issues, and more than one contributor. A repo created last week with a single large commit is not proof of malice, but it means you are the first reviewer.
4. The published package matches the source. Malicious code is sometimes only in the published tarball, not on GitHub. npm pack <package> downloads the tarball so you can diff it against the repo without running anything.

Our article on AI agent supply chain attacks covers typosquatting, account takeovers and install scripts in more depth.

Step 3: Read the code that matters

You do not need to read every line. Focus on the places where damage happens:

• Install and startup hooks. preinstall, install and postinstall in package.json, setup.py code, or a bin script. These run before you ever use the tool.
• Process execution. Calls like child_process.exec, subprocess.run(..., shell=True), os.system or eval.
• Environment and file reads. Code that reads process.env or os.environ wholesale, or opens paths such as ~/.ssh, ~/.aws or browser profile folders.
• Network calls. Every fetch, requests.post or socket. Note each domain. Does it match what the tool says it does?
• Obfuscation. Long base64 strings, eval(atob(...)), minified code in a repo that is otherwise readable. Legitimate projects rarely need to hide their logic.

For MCP servers, also read the tool names and descriptions. These are text the model reads and follows, so hidden instructions there are a known attack. Our article on MCP tool poisoning shows what that looks like.

Step 4: Run automated scans

Free, open source scanners catch a lot of the obvious problems in minutes. Run them from a clone of the repository:

# Static analysis for risky patterns
semgrep scan --config auto .

# Hard-coded secrets in the code and git history
gitleaks git -v .

# Dependencies with known vulnerabilities
osv-scanner scan source -r .

Read the results, do not just count them. A static analysis hit on subprocess.run in a tool whose whole job is to run tests is expected. The same hit in a weather tool is not. A leaked test token is a hygiene issue. A leaked production key suggests the maintainer is careless with secrets in general.

Step 5: Run it in a sandbox and watch the network

Static checks tell you what code could do. Running it tells you what it actually does on startup. Do this in a container or a throwaway VM, never on your main machine with real credentials loaded.

A minimal Docker setup for a Node-based MCP server, run from a throwaway clone of the repo:

docker run --rm -it \
  --cap-drop=ALL \
  --security-opt no-new-privileges \
  --user node \
  -v "$PWD":/app -w /app \
  node:22 sh -c "npm ci && npm start"

Add --network none first to see whether it even starts offline. Then run it again with network access while you capture traffic (for example with a logging proxy) and compare the domains it contacts with the ones it documents. A weather server that talks to a weather API is fine. A weather server that also posts to a raw IP address is not.

If you have the time, connect it to a test client and call each tool once with harmless inputs. Watch what files it touches and what it sends.

Step 6: Give it the least privilege that works

Even trustworthy code can be steered by prompt injection or broken by a bug. Limit the damage before it happens:

• Create a dedicated API key or token for the agent with only the scopes it needs. A GitHub token for one repository, read-only if possible.
• Point filesystem tools at a single project directory, not your home folder.
• Run local servers in a container or sandbox with no access to your SSH keys or cloud credentials.
• Turn on human approval for actions that send, delete, pay or publish.

Our guide to least privilege for AI agents has concrete examples for tokens, filesystems, networks and approval prompts. The MCP project's own security best practices also recommend sandboxing local servers and requesting minimal scopes.

Step 7: Plan for prompt injection

You cannot fully vet away prompt injection, because the attack lives in content the agent reads later, not in the code you install. What you can do is decide in advance which combinations are too risky.

Ask: will this agent read untrusted content (web pages, inbound email, public issues) while also holding private data and a way to send it out? If yes, remove one of those three. Split the work across two agents, drop the outbound tool, or require approval before anything leaves. Our article on prompt injection in tool-using agents explains the attack and the mitigations that actually help.

Step 8: Pin, monitor and re-check

A package that was clean on Tuesday can be malicious on Thursday. The postmark-mcp backdoor arrived in an update. The Shai-Hulud npm worm, covered in a CISA alert in September 2025, spread by publishing compromised versions of existing packages.

• Pin exact versions in your client config (some-mcp@1.4.2, not some-mcp@latest).
• Re-run your checks before you upgrade, and read the diff.
• Rotate keys you gave the agent if it is ever reported as compromised, and periodically anyway.
• Remove what you no longer use. Every installed agent is attack surface.

Step 9: Keep an inventory and give each agent its own identity

You cannot re-check agents you have forgotten about. Wiz's AI agent security guide recommends recording, for each agent, its owner, purpose, the tools and APIs it can reach, the permissions it holds, the data stores it can read and where it runs, and treating any agent nobody registered as high-risk by default. Abnormal's AI agent security overview adds that each agent should get its own identity, rather than a shared key or a reused human session, so you can see and revoke what that one agent does.

For a single developer this can be a short table in your notes: agent, owner, keys it holds, scopes, version pinned, date last checked.

Step 10: Watch it at runtime

Vetting happens before install; monitoring catches what vetting missed. Wiz lists the minimum telemetry as the agent's identity, its tool calls, prompts, retrieval sources and policy decisions, and alerts on behaviour that departs from a normal baseline (Wiz). Abnormal stresses that alerts should feed a tested response, such as a way to pause an agent's actions while you investigate (Abnormal).

In practice, for a local setup:

• Keep your client's tool-call logs, and skim them after the first few real sessions.
• Route the agent's outbound traffic through a logging proxy and compare the domains with what it documents, as in Step 5.
• Know how you would stop it: which config line to remove and which keys to revoke.

Memory poisoning and multi-agent compromise

Two newer risks apply once agents keep state or work together.

• Memory poisoning. An agent that saves notes or uses a vector store can be tricked, through ordinary-looking input, into saving an attacker's record that steers later sessions. Abnormal notes that deleting a chat does not clear that store, and recommends limiting durable writes and tracking where saved content came from (Abnormal). The OWASP AI Agent Security Cheat Sheet lists memory poisoning as a key risk and has a section on memory and context security.
• Multi-agent compromise. When agents pass work to each other, one compromised agent can cause loops or overload in agents that were never attacked directly (Abnormal). Treat output from another agent as untrusted input, the same as a web page.

Which practice stops which threat?

This table follows the practice-to-threat mapping in Wiz's guide, mapped to the steps above.

Practice	Threats it addresses	Step
Dedicated identity with least privilege	Unauthorized actions, identity misuse, data exposure	6, 9
Inventory of every agent	Supply chain risk, identity misuse	9
Secure configuration and guardrails, such as approval prompts	Business logic bypass, unauthorized actions	6, 7
Runtime behaviour monitoring	Unauthorized actions, business logic bypass, prompt injection	10
Validating inputs and retrieval sources	Prompt injection	7
Provenance checks, code scans and sandboxed runs	Malicious or vulnerable code	2 to 5

How this maps to OWASP and NIST

If you need to show a security team which framework a step comes from:

Framework	What to use it for	Where it shows up here
OWASP Top 10 for LLM Applications 2025	Naming the risk: LLM01 Prompt Injection, LLM03 Supply Chain, LLM04 Data and Model Poisoning, LLM06 Excessive Agency	Steps 2 to 7, memory poisoning
OWASP AI Agent Security Cheat Sheet	Agent-specific controls: tool security and least privilege, prompt injection defence, memory security, human-in-the-loop, monitoring, multi-agent security	Steps 6, 7, 10
NIST AI Risk Management Framework	Organisation-level risk management. AI RMF 1.0 was released on January 26, 2023 for voluntary use, with a Generative AI Profile (NIST-AI-600-1) added in July 2024	Inventory and ownership (Step 9)

For MCP-specific steps, such as OAuth scopes, tool descriptions and known server CVEs, use the MCP server security guide.

Can an automated scan tell you an agent is safe?

Scanners are good at known patterns: dangerous function calls, leaked secrets, vulnerable dependency versions, and unexpected network traffic during a test run. They are a fast first filter.

They are weaker at:

• Malicious logic that only triggers on certain inputs, dates or environments.
• Code downloaded at runtime from a server the author controls.
• Prompt injection, which depends on content that does not exist at scan time.
• Business logic flaws, such as a tool that deletes more than it should.

No. A clean scan raises confidence. It does not prove safety. Anyone who tells you a scan guarantees an agent is 100% safe is overselling.

How AgenticMart scans a listing

We built AgenticMart because running the steps above for every tool is slow, and most people skip them. Any listing on the marketplace can get a free, automated scan, the full report is public on the listing page, and listings that have not been scanned yet say so.

Each scan runs on a fresh GitHub Actions virtual machine that is destroyed afterwards. It runs Semgrep with custom rules for things like eval, shell execution, hidden environment variable reads and obfuscation; Gitleaks for secrets; and OSV-Scanner for known-vulnerable dependencies. The repo is then installed and started inside a Docker container with all Linux capabilities dropped, running as a non-root user. For MCP servers the scan starts the server and lists its tools with tools/list. It sends an initialize request first, which servers on revisions before 2026-07-28 expect; it does not yet call the newer server/discover method. All outbound traffic is recorded and compared with the domains the manifest declares, and undeclared domains or hard-coded IPs are flagged. Three clean runs must match to earn the reproducibility badge.

The results become badges and a 0 to 100 score. A clean scan is not a safety guarantee. Use it as a head start on Steps 3 to 5, not as a replacement for scoping your own credentials. You can browse the agents directory, the MCP server directory and the agent skills directory, where each listing page shows its scan status, or submit a repository to get it scanned. Publishers can read how listings and scans work on the developers page.

A quick vetting workflow

If you only have ten minutes:

1. Confirm the package name, publisher and repository match.
2. Check for install scripts and read them.
3. Search the code for exec, eval, shell=True, process.env and outbound URLs.
4. Run Semgrep, Gitleaks and OSV-Scanner, or read an existing scan report.
5. Start it in a container with no network, then with network, and note the domains.
6. Give it a scoped token and a scoped directory.
7. Pin the version.

For a printable list, see our AI agent security checklist.

The AI agent security series

• Is this AI agent safe? A 10-point checklist: a quick pass you can run on any agent, server or skill.
• AI agent supply chain attacks: malicious packages, typosquatting, compromised dependencies and install scripts.
• Prompt injection in tool-using agents: how indirect injection works and what limits it.
• Least privilege for AI agents: scoping keys, filesystems and networks, and when to require human approval.

FAQ

Is it safe to install AI agents from GitHub?

It can be, but installing an agent from GitHub means running someone else's code with your permissions. Check the publisher, read install scripts, scan the code and run it in a sandbox first. Give it scoped credentials so a problem stays small.

Can an MCP server steal my data?

Yes. A local MCP server runs with the same rights as your user account, so it can read files and environment variables and send them anywhere. A real example is the fake postmark-mcp package that copied users' outgoing emails to an attacker.

Does a security scan guarantee an AI agent is safe?

No. Scans catch known risky patterns, leaked secrets, vulnerable dependencies and unexpected network calls, but they cannot see every runtime behaviour or future prompt injection. Treat a clean scan as useful evidence, then still limit what the agent can access.

How does AI agent security differ from LLM security?

LLM security is mostly about what a model says: harmful output, leaked training data, jailbreaks. AI agent security also covers what the system does with that output, through its tools, identities and permissions. As Abnormal's overview puts it, agent security "also governs actions."

What is the biggest security risk with AI agents?

For most teams it is the combination of broad permissions and untrusted input. An agent that holds a powerful token and reads web pages or emails can be steered by prompt injection into misusing that token. Least privilege and human approval for risky actions reduce the impact.