Time Of Check Time Of Use in Cockroachdb

A Time‑Of‑Check Time‑Of‑Use (TOCTOU) race condition occurs when a program first evaluates a condition (the “check”) and later acts on the result (the “use”) without guaranteeing that the state has not changed in the intervening interval. In CockroachDB‑backed APIs this pattern often appears in code that first queries for the existence of a record and then decides to insert, update, or delete based on that query’s result.

Consider a typical Go handler that creates a user‑scoped API key only if the user does not already have one:

func createAPIKey(w http.ResponseWriter, r *http.Request) {
    userID := r.Context().Value("userID").(string)
    var exists bool
    err := db.QueryRow(context.Background(), "SELECT EXISTS(SELECT 1 FROM api_keys WHERE user_id = $1)", userID).Scan(&exists)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    if exists {
        http.Error(w, "API key already exists", http.StatusConflict)
        return
    }
    // TOCTOU window: another request could insert a key between the SELECT and the INSERT
    _, err = db.Exec(context.Background(), "INSERT INTO api_keys (user_id, key) VALUES ($1, $2)", userID, generateKey())
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    w.WriteHeader(http.StatusCreated)
}

If two concurrent requests arrive for the same user, both may see exists == false and both proceed to insert, causing a duplicate key violation or, worse, two distinct keys being issued. In security‑sensitive contexts this can lead to privilege escalation (e.g., issuing an extra admin token) or resource exhaustion.

CockroachDB provides strong serializable isolation, but the check‑then‑act pattern bypasses that guarantee because the decision is made outside the transaction. The race window exists between the SELECT and the INSERT, and CockroachDB will serialize the two transactions only after they have both read the same snapshot, allowing both to commit unless a conflict is detected at commit time — which, for an INSERT without a uniqueness constraint, may not happen.

Real‑world analogues include CVE‑2021‑21315 (a TOCTOU in a container runtime) and numerous API‑key generation bugs reported in public bug‑bounty programs. The OWASP API Security Top 10 (2023) lists “Broken Object Property Authorization” (API1:2023) and “Unrestricted Access to Sensitive Business Flows” (API3:2023) as categories where race conditions can be exploited to bypass intended checks.

Detecting TOCTOU in a black‑box setting requires observing inconsistent outcomes when the same request is sent repeatedly or concurrently. middleBrick’s unauthenticated surface scan can be leveraged for this purpose:

• It issues a baseline request to the target endpoint (e.g., POST /api/keys) and records the HTTP status and response body.
• It then sends a burst of parallel requests (default concurrency of 5) to the same endpoint with identical payloads.
• If the responses diverge — for example, some return 201 Created while others return 409 Conflict or 400 Bad Request — middleBrick flags this as a potential race condition under its Input Validation check, noting that the endpoint’s behavior is not idempotent under concurrent load.
• The scanner also records any duplicate resource identifiers returned in the bodies (e.g., two different API key values) which strongly suggests a TOCTOU flaw.

Because middleBrick does not require agents or credentials, the test can be run against any publicly reachable API, including staging or production endpoints that expose key‑generation, invitation‑code issuance, or role‑assignment flows.

In practice, a finding might look like:

Finding: Potential TOCTOU race condition
Endpoint: POST /v1/api-keys
Severity: Medium
Evidence:
- Baseline request: 201 Created, key="abc123"
- Concurrent request #2: 201 Created, key="def456"
- Concurrent request #3: 409 Conflict, detail="API key already exists"
Remediation guidance: Make the check‑and‑use operation atomic using CockroachDB’s UPSERT or SELECT … FOR UPDATE within a serializable transaction.

Thus, middleBrick provides an automated, reproducible way to surface the symptom of a TOCTOU flaw without needing source code or internal instrumentation.

The correct fix is to eliminate the separate check step and make the insertion atomic, relying on CockroachDB’s built‑in conflict detection. Two idiomatic approaches are:

1. UPSERT (INSERT … ON CONFLICT) – Attempt to insert; if a unique constraint already exists, the operation either does nothing or returns the existing row.
2. Select‑for‑update within a transaction** – Lock the relevant row (or a placeholder) before deciding, ensuring no other transaction can modify the same key concurrently.

func createAPIKey(w http.ResponseWriter, r *http.Request) {
    userID := r.Context().Value("userID").(string)
    key := generateKey()
    // The INSERT will succeed only if no row with this user_id exists;
    // otherwise it does nothing and we can fetch the existing key.
    _, err := db.Exec(context.Background(),
        `INSERT INTO api_keys (user_id, key) VALUES ($1, $2) ON CONFLICT (user_id) DO NOTHING`,
        userID, key)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    // Determine whether we inserted or a row already existed
    var existingKey string
    err = db.QueryRow(context.Background(),
        `SELECT key FROM api_keys WHERE user_id = $1`, userID).Scan(&existingKey)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    if existingKey == "" {
        // We inserted the new key
        w.WriteHeader(http.StatusCreated)
        json.NewEncoder(w).Encode(map[string]string{"key": key})
    } else {
        // A key already existed; return it to avoid leaking that a race occurred
        w.WriteHeader(http.StatusOK)
        json.NewEncoder(w).Encode(map[string]string{"key": existingKey})
    }
}

cmd := db.Prepare(context.Background(), "",
    `INSERT INTO api_keys (user_id, key) VALUES ($1, $2) ON CONFLICT (user_id) DO UPDATE SET key = EXCLUDED.key RETURNING (xmax = 0) AS inserted`)
var inserted bool
err = cmd.QueryRow(context.Background(), userID, key).Scan(&inserted)
if err != nil { /* handle */ }
if !inserted {
    http.Error(w, "API key already exists", http.StatusConflict)
    return
}
// proceed with success response

tx, err := db.BeginTx(context.Background(), pgx.TxOptions{IsoLevel: pgx.Serializable})
if err != nil { /* handle */ }
var exists bool
err = tx.QueryRow(context.Background(), "SELECT EXISTS(SELECT 1 FROM api_keys WHERE user_id = $1 FOR UPDATE)", userID).Scan(&exists)
if err != nil { tx.Rollback(context.Background()); /* handle */ }
if exists {
    tx.Rollback(context.Background())
    http.Error(w, "API key already exists", http.StatusConflict)
    return
}
_, err = tx.Exec(context.Background(), "INSERT INTO api_keys (user_id, key) VALUES ($1, $2)", userID, key)
if err != nil { tx.Rollback(context.Background()); /* handle */ }
err = tx.Commit(context.Background())
if err != nil { /* handle */ }
w.WriteHeader(http.StatusCreated)