Documentation

Shield LLM grades your chatbot's defenses against real-world attacks, right from your terminal. You'll need an account and an API key; grab the key from your dashboard under Settings. Prefer to stay hands-off? Ask the coding agent you already use to run these steps for you.

1. 1
   Install the CLI

   Install Shield LLM globally with npm: `npm install -g shield-llm`.
2. 2
   Authenticate

   Run `shield-llm login --key sk_shield_...` with the API key from your dashboard. It confirms which instance you are connected to.
3. 3
   Initialize the config

   Run `shield-llm init` to generate a shield.config.json describing your chatbot endpoint.
4. 4
   Run your first scan

   Run `shield-llm scan` to launch a security audit against your chatbot. Results stream to your dashboard.

Scan chatbot APIs directly from the command line: REST endpoints, SSE streaming, custom HTTP shapes. The CLI is a thin client. Authentication, the attack catalogue, judging and scoring all live on the backend. New attacks reach existing CI installations on the next scan, with no upgrade.

Installation

npm install -g shield-llm

# Or use without installing:
npx shield-llm scan --help

Quick Start

# 1. Authenticate against your instance
shield-llm login --key sk_shield_xxx --api-url https://shield.your-corp.com

# 2. Generate a config for your chatbot
shield-llm init

# 3. Run a scan
shield-llm scan -c shield.config.json

# 4. (CI mode) JSON to stdout, status to stderr, exit-code policy
shield-llm scan -c shield.config.json --ci --min-score 75

Commands

{
  "endpoint": {
    "url": "https://api.mycompany.com/chatbot",
    "request": {
      "body": { "message": "{{prompt}}" }
    },
    "response": {
      "field": "response"
    }
  }
}

Architecture: thin client

Each scan walks through four steps. The CLI never holds an LLM key, never decides what is vulnerable, and never bundles the attack catalogue: the backend is the single source of truth.

1. 1
   Authenticate

   shield-llm login stores your API key and instance URL in ~/.shield-llm/credentials.json. Every subsequent call carries the Bearer token.
2. 2
   Fetch the attack catalogue

   GET /api/cli/attacks?preset=… returns the attacks for the requested preset, plus an optional set of custom tests. The CLI ships no attacks of its own, so a backend connection is required.
3. 3
   Send attacks to the target

   For each attack, the CLI POSTs the prompt to the configured endpoint (HTTP, SSE, multi-turn). The target only sees a request shaped like its production traffic.
4. 4
   Backend judges & persists

   Captured responses go to /api/cli/evaluate. The backend runs regex + LLM judge using its own credentials, scores the run, persists it to the dashboard, and returns the verdict. The CLI prints the result and exits.

Output Formats & CI/CD

Exit codes: 0 passed · 1 threshold violation · 2 config or auth error · 3 runtime or network error. Use --ci to send JSON to stdout and status logs to stderr so CI logs stay parseable.

GitHub Action example:

- uses: ./.github/actions/shield-llm-scan
  with:
    api-key: ${{ secrets.SHIELD_LLM_API_KEY }}
    api-url: ${{ vars.SHIELD_LLM_API_URL }}
    config: shield.config.json
    preset: owasp
    min-score: 75
    fail-on-critical: true
    output-format: sarif

A scanner that only emits findings leaves you with a report and a backlog. Two commands close the loop: fix patches findings from declarative recipes, and verify-claims replays the original exploit after you deploy. A finding therefore closes on evidence rather than because someone ticked a box.

The loop

# 1. Scan, keeping the machine-readable report
shield-llm scan --preset standard --write-json shield-report.json

# 2. See which recipes apply, and which anchors would unlock the rest
shield-llm fix --report shield-report.json --explain

# 3. Apply. Writes marker-delimited blocks into your own working tree.
shield-llm fix --report shield-report.json

# 4. After your deploy, replay the original exploits and close the claims
shield-llm verify-claims

What the recipes cover

Five recipes ship today, each bound to one finding type. Every other finding falls back to the written remediation in the report. All five are flagged for human review: nothing auto-merges.

Telling fix where your code lives

fix reads a codebase block in shield.config.json, which a scan never reads. An anchor you have not declared is a soft skip: the recipe does not run and the output names the exact config key to add. An anchor pointing at a file or export that does not exist is a hard error, because a stale config fakes coverage.

{
  "codebase": {
    "root": ".",
    "language": "ts",
    "anchors": {
      "system_prompt": { "file": "src/prompt.ts", "export": "SYSTEM_PROMPT" },
      "output_render": { "file": "src/render.ts", "export": "renderReply" }
    },
    "redeploy": {
      "command": "pnpm build && pnpm deploy:staging",
      "timeoutMs": 300000,
      "healthTimeoutMs": 60000
    }
  }
}

What “verified” actually means

fix --verify redeploys your target with codebase.redeploy, then replays each finding's design attacks plus a holdout attack that the harness draws. The recipe does not choose its own holdout, so it cannot be authored to pass. A benign suite runs alongside, so a patch that closes the exploit by breaking the assistant is caught rather than shipped.

Two honest edges

• --allow-unverified opens a pull request with nothing verifying the patch, and the pull request body says so. Verification is something you opt into, not something the command guarantees.
• verify-claims is the primary closure path, meant to run from your deploy job right after deploying. A claim it cannot re-prove stays open.

Shield exposes its red-teaming to AI coding agents (Claude Code, Cursor, and any MCP client) through a Model Context Protocol server. An agent can scan the chatbot it is building, read the fixes, and re-prove them without leaving the editor.

Tools

Setup

npm install -g shield-llm-mcp

# or run without installing (your MCP client can use npx):
npx shield-llm-mcp

MCP client configuration

{
  "mcpServers": {
    "shield-llm": { "command": "shield-llm-mcp" }
  }
}

Shield LLM runs from your environment, testing chatbots through their HTTP endpoint — no SDK or code changes required.

Shield LLM's test suite is mapped to the OWASP LLM Top 10, the industry standard for AI/LLM security risks.

Your chatbot gets a score from 0 to 100 and a letter grade from A to F, like a report card. Higher is safer. Each vulnerability subtracts from a perfect score, weighted by how serious it is, so one critical hole costs far more than a minor finding.

Severity Weights

Grade Thresholds

Choose a scan preset based on your needs. Each preset runs a curated set of attacks optimized for different use cases.

Define your own security test scenarios tailored to your chatbot's specific use case and compliance requirements.