live red-team sandbox

Ask it to break the rules.

Two endpoints, one shared fake-RAG backend. /chat runs no input filtering, no risk scoring, no output review. /chat/secure runs all three. Fire the same prompt-injection attack at both and watch where they diverge.

Run an attack ↓ View source

attack console

Pick a technique, or write your own.

Every prompt below is a real test case from this repo's pen-test suite. Nothing here touches a real model or a real secret — the "leak" is a scripted stand-in for what an ungoverned model would do: hand back its own system prompt if you ask for it, a planted honeypot document if you ask for a secret, or just go along with a fake authority claim.

POST /chat no guardrails

unchecked

Waiting for a message…

No filter · no risk scoring · no output review

POST /chat/secure guarded

unchecked

Waiting for a message…

Input filter → risk score → output guard

under the hood

What /chat/secure actually checks

No LLM judge, no external service — three small, readable stages, in this order, before a document is ever returned.

01 — input filter

Strip the disguise

Regexes strip role tags, [SYSTEM]…[/SYSTEM] wrappers, and fenced code blocks used to smuggle fake instructions in — and flag that they were there.

<system>…</system>
BEGIN SYSTEM PROMPT … END
```fenced block```

02 — risk score

Weigh the intent

19 weighted patterns across role-confusion, exfiltration, and indirect-execution families. Any score above zero blocks the request outright on this endpoint.

override_system   +30
prompt_leakage    +25
secrets           +25
cot_request       +25

03 — output guard

Check what comes back

Even a clean question gets its answer re-scanned for secret-shaped strings and echoed role markers before it's allowed out.

if SECRET_PATTERN.search(answer):
    return safe_refusal()