LLMs don't need your secret tokens (but MCP servers hand them over anyway)

A quick security PSA for anyone wiring large-language models to their back-end toys.

What’s an MCP server, again?

An MCP (Model Control Protocol) server is a bundle of local functions. Each function comes with a short plain-English description. When an LLM reads a user request that matches a description, it calls the function, gathers the result, and keeps chatting.

It feels like magic:

Enable that feature for me. Why is Service 42 broken?

Every major cloud vendor ships an MCP stack, and indie projects are everywhere.

Where things go off the rails: Secrets everywhere

Most MCP servers run locally so they can poke databases, hit internal APIs, and rummage through things you would never expose publicly. That is great until you remember what actually happens:

  1. “List my services” triggers an MCP call to your API.
  2. The API replies with service IDs, auth tokens, region keys, and so on.
  3. MCP forwards the entire blob right back to the LLM.
  4. The LLM hides the secrets from the user, yet the strings now sit inside a third-party context window.

Running the model on your own machine avoids that, but most of us are not lugging 230-billion-parameter beasts around.

The model literally does not need your token

An LLM’s reasoning is unchanged if

sk-f8fad...

is replaced by

[[ENCRYPTED-TOKEN:123]]

The model only cares about making the distinction between tokens, not the tokens themselves.

A dead-simple fix: Encrypt before you send

Below is an outline that any MCP implementation can follow to keep secrets local.

1 Spotting secrets

  • Decide what “looks like a secret.” The quick route is a deny-list of regular expressions for well-known key formats (AWS, Google, and so on).
  • Anything that matches is flagged as sensitive; anything else passes through untouched.

2 Encrypting outbound values

  • Generate a random symmetric key when the session starts.

  • Whenever the MCP prepares a response for the LLM, replace every flagged value with an encrypted placeholder, such as:

    [[ENCRYPTED:...]]
  • Keep the symmetric key in RAM only. When the session ends, the key disappears.
  • AES-CBC with a fixed IV is acceptable in this context. More important than the algorithm itself is ensuring the key never leaves the process.

3 Decrypting inbound values

  • Before a local function calls a real API, scan its inputs for placeholders.
  • Replace each placeholder with the plaintext secret by decrypting it with the in-memory key.
  • From the tool’s point of view nothing changed; from the model’s point of view the secret never existed.

4 Handling files

Sometimes the MCP task writes a file, for instance, an invoice PDF, and returns the file path to the model. That still leaks if the model then asks to grep the workspace. Two pragmatic steps:

  • Encrypt the file contents with a key that only the user can decrypt.
  • Hand the LLM only the path plus an explicit tag such as “(confidential, do not read).”

Anyone who actually needs the file can decrypt it locally; the LLM should not even try.

5 Version-control hygiene

Because the placeholders are meaningless without the key and the key never hits disk, committing chat logs or workspace files to git does not expose anything valuable. The real secrets remain in RAM the whole time.

“But what about invoices and logs?”

Many workflows dump data into local files, then hand the path to the LLM. Encrypting the file itself plus a short “do not read” note is enough to mitigate casual secret leakage. If multiple teammates need the file, share the key through your normal password manager or KMS; the LLM never sees it.

Why isn’t everyone already doing this?

Developers chase the “works on my laptop” moment first, security later. The official SDK examples focus on functionality, not threat models, and their patterns get copy-pasted across projects. I have poked at several “enterprise-ready” MCP repos and found zero built-in secret-scrubbing.

Takeaways

  • MCP servers are powerful: they turn five clicks into one chat message.
  • They are also noisy: every call ferries your private tokens into the model prompt.
  • Fixing it is cheap: encrypt on the way out, decrypt on the way back, and keep your org’s crown jewels off someone else’s GPU.