AI Agent Security Checklist: 15 Checks Before and After

An AI agent security checklist has two halves: checks you run before you install an agent (who published it, what runs at install time, what the code and dependencies do, what it connects to, how much access it wants) and checks you keep running after (staged permissions, approvals, logs, spend limits, a named owner and a way to switch it off). The 15 checks below cover both, and they work for autonomous agents, MCP servers and agent skills.

Items 1 to 10 take about 15 to 30 minutes for a small tool. Items 11 to 15 are settings and habits you set up once. For the reasoning behind each check, read our guide to AI agent security and how to vet an AI agent. For MCP-only risks such as OAuth flows and tool descriptions, use the MCP server security guide.

The checklist at a glance

Print this table, or keep it next to you while you review. Score each item pass, fail or unknown.

#	Check	When	Stop if it fails?
1	Publisher is who they claim to be	Before install	Yes
2	Nothing surprising runs at install time	Before install	Yes
3	Repository has a real history	Before install	No, sandbox only
4	Published package matches the source	Before install	Yes
5	No risky code without a clear reason	Before install	Yes
6	No leaked secrets in the repo	Before install	No, but note it
7	No known critical dependency vulnerabilities	Before install	Until fixed
8	Network traffic matches the description	Before install	Yes
9	Tool descriptions and instructions read in full	Before install	Yes, if hidden instructions
10	Its own scoped credentials, no shared keys	Before install	Yes
11	Pinned version, re-reviewed on every update	Before and after	No
12	Staged permissions and approval for risky actions	After install	Yes
13	Action audit log and monitoring	After install	No
14	Rate, token and spend limits	After install	No
15	Named owner, inventory entry and offboarding plan	After install	No

Before you install: checks 1 to 10

1. Is the publisher who they claim to be?

Check that the package and the repository belong to the same person or organisation. On npm, npm view <package> repository.url maintainers shows both. A package named after a well-known product but published by an unrelated account is a classic trick. In September 2025 a fake postmark-mcp package that Postmark never published copied users' outgoing emails to an attacker. If you are not sure what an MCP server can reach once it runs, read that first: it runs with your permissions.

Pass: the publisher has a track record and links back to the repo, and the repo links to the package.

2. Does anything surprising run at install time?

Install hooks run before you ever use the tool. Look for:

"scripts": {
  "preinstall": "...",
  "install": "...",
  "postinstall": "..."
}

In Python, look for custom code in setup.py. Read any script these call. Downloading a binary for your platform can be legitimate. Reading your home directory or calling an unknown URL is not.

Pass: no install hooks, or hooks that do exactly what they say.

3. Does the repository have a real history?

Look at commit dates, contributors, issues and releases. A long history does not prove safety, but it means other people have looked at the code. A brand new repo with one large commit means you are the first reviewer.

Pass: steady history, responsive maintainer, issues answered.

4. Does the published package match the source?

What you install from a registry can differ from what is on GitHub. Download the tarball without installing it and compare:

npm pack <package>@<version>
tar -xzf <package>-<version>.tgz
diff -r package/ ./path-to-cloned-repo/

Pass: no unexplained files or code in the package that are missing from the repo.

5. Is there risky code without a clear reason?

Search the code for process execution, dynamic evaluation, broad environment reads and obfuscation:

grep -rnE "eval\(|exec\(|child_process|shell=True|os\.system|process\.env|atob\(|base64" .

Context matters. A test-runner tool that calls a shell is expected. A note-taking tool that does is not. Long encoded strings and eval of decoded data are strong red flags. Static analysis with semgrep scan --config auto . catches more patterns than grep.

Pass: every risky call has an obvious purpose tied to the tool's job.

6. Are there leaked secrets in the repo?

Run gitleaks git -v . on a clone. A real API key committed to the repo shows how the maintainer treats credentials, possibly including yours.

Pass: no secrets, or only clearly fake test values.

7. Do dependencies have known critical vulnerabilities?

Run osv-scanner scan source -r . to compare the lockfile against public vulnerability databases. Check whether flagged packages are actually reachable in the code, and whether a fixed version exists.

Pass: no high-severity known vulnerabilities, or fixes available and planned.

8. Does network traffic match the description?

Run the agent in a container and record where it connects. Compare the domains with what the README or manifest says. A GitHub tool should talk to GitHub. Unknown domains, raw IP addresses and anything that fires before you call a tool are reasons to stop.

Pass: every outbound connection is documented and expected.

9. Have you read every tool description and instruction file?

For MCP servers, tool descriptions are text the model reads and may follow, and hidden instructions in tool descriptions are a known attack. For agent skills, the SKILL.md body and any scripts it calls are instructions too. Read them all, not just the README.

Pass: no instructions that reach beyond the tool's stated job, such as reading unrelated files or sending data elsewhere.

10. Does it get its own scoped credentials?

Read the setup docs. What tokens, scopes, folders and tools does it want? A tool that only reads issues should not need write access to every repository or your whole home directory. Give each agent its own credential rather than your personal key or a key shared with other agents, so you can trace and revoke what it does. OWASP lists too much functionality, too many permissions and too much autonomy as Excessive Agency (LLM06).

Pass: you can run it with a dedicated, scoped token and a scoped directory. Our guide to least privilege for AI agents shows how.

After you install: checks 11 to 15

11. Is the version pinned, and do you re-review updates?

Lock it to an exact version and check that releases have changelogs and a security contact. Self-spreading malware such as the Shai-Hulud npm worm published poisoned versions of existing packages, so running @latest means trusting every future release blindly. Treat each update as a new install and repeat checks 2, 5, 8 and 9.

Pass: tagged releases, a changelog, a pinned version and a habit of reviewing the diff before you upgrade.

12. Are permissions staged, with approval for risky actions?

Start the agent read-only. Add limited write access only when you have watched it work, and keep a human approval step for anything hard to undo: sending messages, pushing or deploying, deleting data, changing permissions and spending money. OWASP's LLM06 entry lists requiring user approval for high-impact actions as a mitigation. Approval prompts should show the exact action and arguments, not the agent's summary.

Pass: the agent cannot send, publish, delete or pay without a prompt you actually read.

13. Is every action logged and watched?

Keep a log of the tool calls the agent makes, with arguments and results, and look at it. OWASP's LLM06 entry notes that logging and monitoring do not prevent excessive agency but limit the damage by showing what happened. Many MCP clients show tool calls in their own logs; for agents you run yourself, log at the point where tools are called.

Pass: you can answer "what did this agent do yesterday?" from a log, not from memory.

14. Are there rate, token and spend limits?

A looping or hijacked agent can run up API bills or hammer a service. Set per-key rate limits, model token budgets and spending caps where your providers support them. OWASP covers this as Unbounded Consumption (LLM10), with rate limiting and quotas among its mitigations.

Pass: a runaway agent hits a cap you chose before it hits your card or a production system.

15. Does it have a named owner, an inventory entry and an offboarding plan?

Every agent in use should have one person responsible for it, written down with what it does, which keys it holds and where they are stored. Agents nobody remembers installing are the ones that keep old, broad keys. When you stop using an agent, revoke its keys, remove its config and delete its data.

Pass: you can list every agent you run, its owner and its keys, and you have removed the ones you no longer use.

How the checks map to OWASP risks

This is our mapping of each check to the OWASP Top 10 for LLM Applications 2025. OWASP does not publish a checklist mapping; use it to see which risk each check reduces.

Check	Main OWASP 2025 risk
1 to 5, 7, 11	LLM03 Supply Chain
6, 8	LLM02 Sensitive Information Disclosure
9	LLM01 Prompt Injection
10, 12, 13, 15	LLM06 Excessive Agency
14	LLM10 Unbounded Consumption

Quick scoring

Result	What to do
All pass	Install with scoped credentials and a pinned version, then set up checks 12 to 15
Some unknowns	Run only in a sandbox with test credentials until resolved
Any fail on 1, 2, 4, 5, 8 or 9	Do not install; report it to the registry if it looks malicious
Other fails	Raise an issue with the maintainer and wait for a fix

Speeding it up

Checks 5 to 8 are the slowest to do by hand. On AgenticMart, any listing can get a free automated scan; each listing page shows its scan status. The scan runs Semgrep with custom rules, Gitleaks, OSV-Scanner, and a sandboxed run in a container with all Linux capabilities dropped, where outbound traffic is compared with the domains the manifest declares. Listing pages in the MCP server directory and the agent skills directory show any scan report, and the scan methodology explains each step. You still need to do checks 1, 10 and 11 to 15 yourself, because they depend on your context.

A scan is a malware and risk check, not a safety guarantee. It means the known checks found nothing. Keep your credentials scoped anyway.

FAQ

What should an AI agent security checklist include?

At minimum: publisher identity, install scripts, package and source match, risky code, secrets, dependency vulnerabilities, network traffic, tool descriptions and credential scope before install, then pinned versions, staged permissions, approvals, logging, spend limits and a named owner after install. The 15 checks above cover each of these.

Does this checklist work for MCP servers?

Yes. Checks 1 to 11 apply to MCP servers as written, and check 9 covers tool descriptions. MCP servers also have protocol-specific risks such as OAuth flows and token handling, which the MCP server security guide covers.

What are the red flags in an AI agent repository?

The biggest are install scripts that read files or call unknown URLs, obfuscated or base64-encoded code passed to eval, broad reads of environment variables, and network calls to undocumented domains or raw IP addresses. A package name that imitates a known brand but has an unrelated publisher is another strong warning sign.

Should I trust an AI agent with a lot of GitHub stars?

Stars show popularity, not safety. Popular packages are attractive targets for account takeover, and malicious updates can land in well-known projects. Use stars as one signal alongside the other checks, and always pin versions.

How long does it take to vet an AI agent?

A first pass through checks 1 to 10 takes 15 to 30 minutes for a small tool. Larger agents with many dependencies take longer. Reading an existing public scan report can cut that time.

What should I monitor after an AI agent is installed?

Watch the tool calls it makes, the data it sends out, the credentials it uses and what it costs. Unexpected tools, new domains, sudden spikes in calls or spend, and any action you did not approve are reasons to pause it and revoke its keys.