Unicode Normalization in Aspnet with Cockroachdb

Unicode normalization inconsistencies between ASP.NET and CockroachDB can lead to authentication bypass, data integrity issues, and security-sensitive logic flaws. When an ASP.NET application accepts user input (e.g., usernames, identifiers, or API keys) and normalizes it using one Unicode normalization form (e.g., NFC or NFD) before comparison or storage, but CockroachDB stores and compares the value in a different form, attackers can exploit the mismatch.

For example, consider the character "é", which can be represented as a single code point U+00E9 (NFC) or as the sequence "e" + combining acute accent U+0301 (NFD). If ASP.NET normalizes incoming strings to NFC using string.Normalize(NormalizationForm.FormC) before using the value in business logic or queries, but the data in CockroachDB is stored in NFD (or vice versa), two seemingly identical credentials or identifiers may not compare as equal. This can cause authentication bypass where a normalized token fails to match the stored value, or it may allow two logically equivalent identifiers to be treated as distinct, violating uniqueness constraints.

In the context of middleBrick’s security checks, this normalization mismatch appears as an input validation and property authorization finding. An attacker could supply carefully crafted Unicode input to exploit inconsistent normalization across layers, potentially bypassing access controls or escalating privileges when identifiers are compared in an unnormalized or differently normalized state. Because the scan tests the unauthenticated attack surface, it can detect endpoints where user-controlled strings flow into database operations without canonical normalization, placing the application at risk.

When integrated with OpenAPI/Swagger spec analysis, middleBrick cross-references endpoint definitions with runtime behavior, highlighting paths where string parameters interact with CockroachDB without explicit normalization guidance. This is especially relevant for endpoints that accept identifiers, tokens, or search filters that may include non-ASCII characters. Without consistent normalization, even basic operations like lookups, updates, or membership checks can behave unpredictably, leading to security-sensitive anomalies that align with OWASP API Top 10 categories such as Broken Object Level Authorization and Injection.

To resolve Unicode normalization issues between ASP.NET and CockroachDB, enforce a single normalization form at the boundary where data enters the application and before it is persisted or compared. The following practices and code examples illustrate a robust approach.

1. Normalize input in ASP.NET before database interaction

Always normalize incoming strings to a canonical form (preferably NFC) in your controller or service layer. Use System.Globalization.StringInfo or string.Normalize consistently.

using System.Globalization;
using System.Text;

public static class UnicodeHelper
{
    public static string NormalizeUnicode(string input)
    {
        if (string.IsNullOrEmpty(input))
            return input;
        // Use FormC (NFC) as the canonical form
        return input.Normalize(NormalizationForm.FormC);
    }
}

Apply this helper to identifiers, usernames, tokens, and any string that participates in database queries or comparisons.

2. Ensure CockroachDB stores normalized values

When inserting or updating records, ensure the value sent to CockroachDB is already normalized. Use parameterized queries via Npgsql to avoid injection and guarantee consistent encoding.

using Npgsql;
using System;

public class UserRepository
{
    private readonly string _connectionString = "Host=your-cockroachdb-host;Database=yourdb;Username=youruser;SslMode=Require;TrustServerCertificate=true";

    public void CreateUser(string userId, string email)
    {
        var normalizedUserId = UnicodeHelper.NormalizeUnicode(userId);
        var normalizedEmail = UnicodeHelper.NormalizeUnicode(email);

        using var conn = new NpgsqlConnection(_connectionString);
        conn.Open();
        using var cmd = new NpgsqlCommand(
            "INSERT INTO users (id, email) VALUES (@id, @email)", conn);
        cmd.Parameters.AddWithValue("id", normalizedUserId);
        cmd.Parameters.AddWithValue("email", normalizedEmail);
        cmd.ExecuteNonQuery();
    }

    public bool ValidateUser(string userId, string email)
    {
        var normalizedUserId = UnicodeHelper.NormalizeUnicode(userId);
        var normalizedEmail = UnicodeHelper.NormalizeUnicode(email);

        using var conn = new NpgsqlConnection(_connectionString);
        conn.Open();
        using var cmd = new NpgsqlCommand(
            "SELECT COUNT(*) FROM users WHERE id = @id AND email = @email", conn);
        cmd.Parameters.AddWithValue("id", normalizedUserId);
        cmd.Parameters.AddWithValue("email", normalizedEmail);
        var count = Convert.ToInt64(cmd.ExecuteScalar());
        return count > 0;
    }
}

3. Normalize on retrieval and comparison

If you cannot guarantee that existing data is normalized, normalize both sides during comparison or use database functions to normalize stored values at query time. With CockroachDB, you can apply Unicode normalization in SQL using extension-aware processing or handle it in ASP.NET before constructing queries.

public bool ValidateUserNormalizedInQuery(string userId, string email)
{
    var normalizedUserId = UnicodeHelper.NormalizeUnicode(userId);
    var normalizedEmail = UnicodeHelper.NormalizeUnicode(email);

    using var conn = new NpgsqlConnection(_connectionString);
    conn.Open();
    // Normalize stored values at query time if needed
    using var cmd = new NpgsqlCommand(
        "SELECT COUNT(*) FROM users " +
        "WHERE id = UPPER(@id) COLLATE ""und-u-ks-level2"" " +
        "AND email = LOWER(@email) COLLATE ""und-u-ks-level2""", conn);
    // Note: For full Unicode normalization in SQL, consider storing normalized values
    // and using the helper consistently; CockroachDB does not provide built-in NFC/NFD functions.
    cmd.Parameters.AddWithValue("id", normalizedUserId);
    cmd.Parameters.AddWithValue("email", normalizedEmail);
    var count = Convert.ToInt64(cmd.ExecuteScalar());
    return count > 0;
}

4. Middleware for consistent normalization

In ASP.NET, add normalization early in the request pipeline so that all incoming strings are canonicalized before model binding or authentication logic executes.

using Microsoft.AspNetCore.Http;
using System.Threading.Tasks;

public class UnicodeNormalizationMiddleware
{
    private readonly RequestDelegate _next;

    public UnicodeNormalizationMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext context)
    {
        if (context.Request.Query.TryGetValue("username", out var usernameValues))
        {
            var normalized = UnicodeHelper.NormalizeUnicode(usernameValues.ToString());
            // Optionally replace or store normalized value for downstream use
            context.Items["NormalizedUsername"] = normalized;
        }
        await _next(context);
    }
}

// In Startup.cs or Program.cs:
// app.UseMiddleware<UnicodeNormalizationMiddleware>();

By normalizing at the edge and persisting normalized forms in CockroachDB, you eliminate cross-layer mismatches. middleBrick’s scans help identify endpoints where inconsistent normalization may expose authentication or authorization weaknesses, allowing you to remediate before attackers can exploit them.