Hallucination Attacks in Flask with Firestore

A hallucination attack in the context of a Flask application using Google Cloud Firestore occurs when an LLM or AI-assisted component generates plausible but false data that is then accepted as authoritative and used to influence Firestore reads or writes. This can happen when application logic relies on LLM output to construct queries, interpret user intent, or fill document fields without strict validation against the canonical Firestore data model.

Flask, as a lightweight Python framework, does not enforce strict schema validation on incoming or outgoing data. When developers integrate LLMs to enrich request handling—such as summarizing a document, suggesting field values, or generating Firestore query filters—the LLM may invent document IDs, field names, or values that do not exist in Firestore. If the application trusts the LLM output and directly uses it in Firestore operations (e.g., fetching a document by an LLM-suggested ID or updating fields based on LLM-derived suggestions), the mismatch between hallucinated content and actual Firestore state can lead to inconsistent behavior, data integrity issues, or information leakage.

For example, an LLM might hallucinate a Firestore document path like users/chat_12345/preferences based on incomplete user input, prompting the Flask app to attempt to read or write that path. If the document does not exist, the app might create it with fabricated data, effectively injecting false information into the database. Conversely, if the LLM suggests a field name that is not part of the expected schema, Firestore will still accept the write (since it is schemaless), but downstream consumers may misinterpret the data, leading to logic errors or compliance violations.

Another vector involves query construction. An LLM might generate a Firestore query with a filter condition based on imagined attributes—such as where('status', '==', 'active') when the correct field is account_status. If the Flask code executes this query without validating the field names against the known Firestore schema, it may return empty results while the application believes it has retrieved valid data. This creates a scenario where the application logic proceeds on the basis of hallucinated query outcomes, potentially bypassing intended access controls or business rules.

Because Firestore is schemaless and tolerant of arbitrary fields, it does not inherently prevent an application from storing LLM-generated content. The risk therefore shifts to the Flask application layer, where insufficient validation, over-reliance on LLM suggestions, and improper handling of Firestore references can turn hallucinations into persistent, impactful data anomalies. Secure integration requires strict schema enforcement, explicit allowlisting of field names and paths, and treating LLM output as unverified suggestions rather than authoritative instructions.

To mitigate hallucination attacks in Flask when working with Firestore, enforce strict schema validation and treat LLM output as advisory. The following patterns demonstrate secure practices for referencing documents, constructing queries, and validating data before writing to Firestore.

1. Validate Document Paths and IDs

Do not use LLM-generated document IDs directly. Instead, map them against an allowlist or verify their existence before use.

from google.cloud import firestore
from flask import Flask, request, jsonify

app = Flask(__name__)
db = firestore.Client()

ALLOWED_USER_IDS = {'user_001', 'user_002', 'user_003'}  # Example allowlist

@app.route('/user-profile', methods=['GET'])
def get_user_profile():
    user_id = request.args.get('user_id')
    if user_id not in ALLOWED_USER_IDS:
        return jsonify({'error': 'invalid user_id'}), 400
    doc_ref = db.collection('users').document(user_id)
    doc = doc_ref.get()
    if not doc.exists:
        return jsonify({'error': 'not found'}), 404
    return jsonify(doc.to_dict())

2. Use Schema-Defined Field Names in Queries

Avoid constructing queries with field names derived from LLM output. Instead, use hardcoded or configuration-driven field names that match Firestore document schemas.

import firebase_admin
from firebase_admin import firestore
from flask import Flask, jsonify

app = Flask(__name__)
db = firestore.client()

# Define allowed fields explicitly
ALLOWED_FILTER_FIELDS = {'status', 'created_at', 'category'}

@app.route('/search', methods=['GET'])
def search_items():
    field = request.args.get('field')
    value = request.args.get('value')
    if field not in ALLOWED_FILTER_FIELDS:
        return jsonify({'error': 'invalid filter field'}), 400
    # Safe: field is validated against allowlist
    docs = db.collection('items').where(field, '==', value).stream()
    results = [doc.to_dict() for doc in docs]
    return jsonify(results)

3. Sanitize LLM Suggestions Before Write

If using LLM output to populate document fields, validate and sanitize each field against a known schema before writing to Firestore.

from google.cloud import firestore
from flask import Flask, request, jsonify
import re

app = Flask(__name__)
db = firestore.Client()

def is_valid_status(value):
    return value in {'draft', 'published', 'archived'}

@app.route('/update-settings', methods=['POST'])
def update_settings():
    data = request.get_json()
    status = data.get('status')
    # Reject hallucinated or unexpected values
    if not is_valid_status(status):
        return jsonify({'error': 'invalid status value'}), 400
    # Use server timestamp for controlled updates
    doc_ref = db.collection('settings').document('app_config')
    doc_ref.update({
        'status': status,
        'updated_at': firestore.SERVER_TIMESTAMP
    })
    return jsonify({'success': True})

4. Prefer Server-Side Field Mapping

Map LLM suggestions to canonical field names on the server instead of trusting raw output.

from google.cloud import firestore
from flask import Flask, request, jsonify

app = Flask(__name__)
db = firestore.Client()

FIELD_MAP = {
    'stat': 'status',
    'created': 'created_at',
    'cat': 'category'
}

@app.route('/record', methods=['POST'])
def record_event():
    data = request.get_json()
    mapped = {}
    for key, value in data.items():
        canonical = FIELD_MAP.get(key, key)
        # Only allow known canonical fields
        if canonical in {'status', 'created_at', 'category', 'priority'}:
            mapped[canonical] = value
    doc_ref = db.collection('events').document()
    doc_ref.set(mapped)
    return jsonify({'id': doc_ref.id}), 201

5. Use Firestore Transactions for Consistency

When reading and writing based on dynamic inputs, use transactions to ensure that the data you read is the data you write, reducing the impact of hallucinated references.

from google.cloud import firestore
from flask import Flask, jsonify

app = Flask(__name__)
db = firestore.Client()

@app.route('/increment-counter', methods=['POST'])
def increment_counter():
    doc_ref = db.collection('counters').document('main')
    def update_transaction(transaction):
        snapshot = transaction.get(doc_ref)
        if not snapshot.exists:
            transaction.set(doc_ref, {'value': 0})
        current = snapshot.get('value')
        transaction.set(doc_ref, {'value': current + 1})
        return current + 1
    new_value = db.run_transaction(update_transaction)
    return jsonify({'new_value': new_value})