Debugging with an LLM usually starts the same way: copy an error, a config block, or a log excerpt, paste it into a chat window, and ask what’s wrong. It’s fast, and it works. It’s also how API keys, customer PII, and internal infrastructure details end up in a third party’s logs — sometimes retained for model training, sometimes just sitting in a chat history longer than anyone intended.
What actually ends up in prompts
Look at what a typical “help me debug this” paste contains:
- Stack traces — often include local file paths, environment variable names, and sometimes their values
- Config files /
.envdumps — database URLs with embedded credentials, API keys, JWT secrets - Log excerpts — customer emails, IP addresses, session tokens, internal hostnames
- Sample data / JSON payloads — real customer records used “just for testing”
None of this is malicious — it’s just the fastest way to give an LLM enough context to help. But most LLM providers’ data-use terms distinguish between what you intend and what you send, and once sensitive data is in a request, you’re relying on their retention policy rather than your own controls.
The categories worth automating
Manually scanning every paste for secrets doesn’t scale, and doing it by eye is exactly the kind of task humans are bad at under time pressure. The categories worth catching automatically:
- Credentials — API keys (AWS, Stripe, GitHub tokens, etc. have recognizable prefixes), private keys, JWTs, database connection strings
- PII — email addresses, phone numbers, IP addresses, physical addresses, names in structured contexts
- Internal identifiers — internal hostnames, employee IDs, ticket references that reveal org structure
Regex-based detection gets you most of the way for credentials and structured PII, since most secret formats (AWS keys, Stripe keys, JWTs, private key PEM blocks) have distinctive, matchable shapes.
Redact locally, not after the fact
The right point to catch this is before the text leaves your machine — not by trusting the receiving service to filter it, and not by remembering to manually find-and-replace every time. Our PII & Secret Redactor runs entirely client-side: paste text in, it detects emails, phone numbers, IP addresses, API key patterns, and private keys, and masks them in place before you copy the sanitized version into your LLM chat of choice. Nothing you paste is uploaded anywhere — the detection runs in your browser.
A few practical habits that pair well with redaction:
- Redact before you compress. If you’re also trimming a long prompt to fit a context window with something like our Prompt Compressor, redact first — compression can shuffle or truncate text in ways that make manual review harder afterward.
- Watch structured data, not just prose. JSON blobs and CSV samples hide PII in field values that don’t visually stand out the way a sentence would.
- Treat internal hostnames and ticket IDs as PII too. They’re rarely regulated data, but they leak information about your infrastructure and org structure that you probably don’t want in a third-party’s logs either.
This isn’t about avoiding LLMs
None of this is an argument against using LLMs for debugging or development — they’re genuinely useful for exactly this kind of task. It’s an argument for treating “paste into a chat window” with the same instinct you’d apply to pasting into a public GitHub issue: assume it’s not private unless you’ve checked, and make the checking step fast enough that you actually do it every time.