Llm Data Leakage in Flask with Mongodb

When a Flask application uses MongoDB as a data store and exposes endpoints that interact with LLMs—such as sending user input to an unauthenticated LLM endpoint or returning LLM-generated text—there is a risk of LLM data leakage. This occurs when model responses, system prompts, or intermediate data inadvertently expose sensitive information stored in MongoDB documents.

In this stack, a common pattern is for Flask to retrieve user records from MongoDB and include them in prompts sent to an LLM for summarization, classification, or assistance. If the prompt is constructed by directly interpolating MongoDB document fields (e.g., email, ssn, or internal identifiers) into the system or user messages, and the LLM response is returned to the client without filtering, sensitive data can leak through the LLM output.

For example, consider a Flask route that fetches a profile from MongoDB and sends it to an LLM to generate a support summary:

from flask import Flask, jsonify
from pymongo import MongoClient

app = Flask(__name__)
client = MongoClient('mongodb://localhost:27017/')
db = client['mydb']

@app.route('/summary/')
def user_summary(user_id):
    user = db.users.find_one({'_id': user_id})
    if not user:
        return jsonify({'error': 'not found'}), 404
    # Risky: sensitive fields included in prompt
    prompt = f"Summarize this user: {user['email']}, {user.get('ssn')}"
    llm_response = call_llm(prompt)  # hypothetical LLM call
    return jsonify({'summary': llm_response})

If the LLM echoes back the provided data or includes it in verbose responses, the email or ssn can be leaked to the caller or logged by the LLM provider. This is a form of LLM data leakage because sensitive MongoDB fields appear in the LLM context and may be reflected in outputs accessible to users or captured in logs.

Additionally, if the Flask app calls an unauthenticated or improperly scoped LLM endpoint, the system prompt itself might be exposed. The LLM/AI Security checks in middleBrick specifically test for system prompt leakage using regex patterns tailored to ChatML, Llama 2, Mistral, and Alpaca formats. Even if the prompt is built safely, a verbose LLM response can reveal internal instructions if the model is tricked through prompt injection or jailbreak techniques.

The combination of Flask routing, MongoDB document access, and LLM interaction amplifies risks when data flows from persistent storage into generative model inputs. Without strict input sanitization, output scanning, and prompt validation, sensitive MongoDB fields can travel through the LLM pipeline and become exposed in responses, logs, or error messages.

To prevent LLM data leakage in Flask applications using MongoDB, design data flows so that sensitive fields never enter LLM prompts and responses are filtered before being returned to clients. Apply the following concrete fixes and patterns.

1. Exclude sensitive fields from prompts

Never interpolate raw MongoDB documents into prompts. Instead, explicitly select only safe, non-sensitive fields and validate their content. For example:

def build_safe_prompt(user):
    # Only use non-sensitive fields
    return f"Summarize preferences for user with plan: {user.get('plan', 'unknown')} and last login: {user.get('last_login')}"

@app.route('/summary-safe/')
def user_summary_safe(user_id):
    user = db.users.find_one({'_id': user_id}, {'email': 0, 'ssn': 0})  # exclude sensitive fields
    if not user:
        return jsonify({'error': 'not found'}), 404
    prompt = build_safe_prompt(user)
    llm_response = call_llm(prompt)
    return jsonify({'summary': llm_response})

By projecting out email and ssn in the find_one query and using a dedicated prompt builder, sensitive data stays in MongoDB and does not enter the LLM context.

2. Sanitize and scan LLM responses

LLM outputs should be scanned for accidental inclusion of PII, API keys, or MongoDB identifiers before being returned. If your deployment includes output scanning (as in middleBrick’s LLM/AI Security checks), integrate a validation layer:

import re

PII_REGEXES = [
    re.compile(r'\b\d{3}-\d{2}-\d{4}\b'),  # SSN-like
    re.compile(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b'),  # email
]

def contains_pii(text):
    for pat in PII_REGEXES:
        if pat.search(text):
            return True
    return False

@app.route('/summary-guarded/')
def user_summary_guarded(user_id):
    user = db.users.find_one({'_id': user_id}, {'email': 0, 'ssn': 0})
    if not user:
        return jsonify({'error': 'not found'}), 404
    prompt = build_safe_prompt(user)
    llm_response = call_llm(prompt)
    if contains_pii(llm_response):
        return jsonify({'error': 'response blocked due to potential data leakage'}), 400
    return jsonify({'summary': llm_response})

This guards against reflections of MongoDB data and matches patterns commonly found in LLM leakage scenarios, such as SSNs and emails that might originate from stored documents.

3. Use authenticated LLM endpoints and strict system prompts

Ensure LLM endpoints are authenticated and avoid embedding secrets in system prompts that could be extracted via prompt injection probes. Validate and escape any dynamic content used in prompts, and prefer structured instructions over free-form concatenation.

4. Continuous monitoring and testing

Use tools like middleBrick to run LLM/AI Security checks against your endpoints. Its Active Prompt Injection testing (5 sequential probes) and System Prompt Leakage detection help identify weaknesses specific to Flask routes that feed MongoDB data into LLMs. Findings include guidance aligned with OWASP API Top 10 and compliance frameworks.