Best AI for Finding Security & Concurrency Bugs in Node.js Stripe Checkout Code
Claude Sonnet 5.5
anthropic/claude-sonnet-5.5
Latency
21.7s
Cost
$0.0351
Claude Opus 5.5
anthropic/claude-opus-5.5
Latency
41.0s
Cost
$0.0959
Gemini 4 Argon
google/gemini-4-argon
Latency
23.8s
Cost
$0.0158
GPT-6 Sol
openai/gpt-6-sol
Latency
45.0s
Cost
$0.0382
Claude Opus 5.5 won with a score of 97/100 in 41.0s.
Pricing: $4 / $20 per 1M tokens (in/out)
Raw outputs
Inspect every answer
{"bugs_found":[{"bug_category":"SQL_INJECTION","line_reference":"Lines 5-7 (step 1): db.query(`SELECT id, status FROM orders WHERE user_id = '${userId}' AND note = '${orderNote}'`)","vulnerability_explanation":"userId and orderNote come straight from req.body and are interpolated into the SQL text with a template literal. Any quote character in either field ends the string literal, and whatever follows runs as attacker-controlled SQL. orderNote is free text, so it is a trivial injection vector. Separately, userId is trusted from the client rather than taken from the authenticated session, which is also an IDOR weakness.","exploit_scenario":"An attacker submits orderNote = \"' OR '1'='1\" to make the WHERE clause always true and read every user's orders. Alternatively they submit \"'; UPDATE promos SET is_used = false; --\" (stacked queries work with node-postgres simple queries that have no parameters) to reset every single-use promo, or a UNION SELECT payload to pull data from other tables. The result is data exfiltration, tampering or destruction.","exact_fix_explanation":"Use a parameterised query so the driver sends values separately from the SQL text: client.query('SELECT id, status FROM orders WHERE user_id = $1 AND note = $2', [userId, orderNote]). As defence in depth, take userId from the authenticated session (e.g. req.user.id) instead of the request body, and run the app with a least-privilege DB role."},{"bug_category":"CONCURRENCY_RACE_CONDITION","line_reference":"Lines 12-15 (step 2): SELECT discount_gbp, is_used ... then if (is_used === false) ... then UPDATE promos SET is_used = true","vulnerability_explanation":"This is a classic TOCTOU (time-of-check to time-of-use) race. The single-use check (SELECT) and the claim (UPDATE) are two separate, non-atomic statements with no transaction, row lock or conditional WHERE. Every request that runs the SELECT before any request's UPDATE commits sees is_used = false, applies the discount, and charges. The unconditional UPDATE also gives no signal that another request already claimed the code. In addition, the promo is burned even if the later Stripe charge fails.","exploit_scenario":"An attacker holding one single-use £20 code fires 50 parallel POST /api/checkout requests, e.g. with Turbo Intruder or Promise.all. Most of them pass the is_used check before the first UPDATE lands, so the £20 discount is applied dozens of times across separate orders. Promo codes can also be leaked and mass-redeemed this way.","exact_fix_explanation":"Merge check and claim into one atomic conditional write inside a transaction: UPDATE promos SET is_used = true WHERE code = $1 AND is_used = false RETURNING discount_gbp. Apply the discount only if rowCount === 1. Under READ COMMITTED, concurrent requests block on the row lock and then re-evaluate is_used = false, so exactly one wins. Wrap the claim and the Stripe charge in BEGIN/COMMIT and ROLLBACK on failure, so a failed payment releases the promo. An Idempotency-Key on the Stripe call is recommended against client retries."},{"bug_category":"FINANCIAL_MATH","line_reference":"Line 14 (finalGbp = cartTotalGbp - promo.rows[0].discount_gbp) and Line 20 (const amountInPence = Math.floor(finalGbp * 100))","vulnerability_explanation":"Money is handled as IEEE-754 floating-point pounds and then truncated with Math.floor. Many decimal amounts are not exactly representable in binary floating point, so finalGbp * 100 often lands just below the true integer and Math.floor drops a penny. For example, 19.99 * 100 = 1998.9999999999998, which floors to 1998, and 0.29 * 100 = 28.999999999999996, which floors to 28. Subtraction compounds the drift; for example, 20.10 - 10.05 is not exactly 10.05. There is also no lower bound: a discount larger than the cart total produces a negative amount. Finally, discount_gbp from a NUMERIC column arrives as a string from node-postgres and is only implicitly coerced.","exploit_scenario":"In normal traffic, a £19.99 basket is charged 1998p. Stripe captures 1p less than the order total on a large share of transactions, causing systematic revenue leakage and ledger/VAT reconciliation mismatches against the order records. An attacker can also apply a £20 promo to a £5 cart, giving finalGbp = -15 and amountInPence = -1500. That either crashes checkout with Stripe errors or, if a zero/negative path is ever special-cased, results in free goods.","exact_fix_explanation":"Convert to integer minor units (pence) once, at the boundary, using Math.round rather than Math.floor: Math.round(Number(x) * 100). Do all arithmetic in integer pence, validate with Number.isSafeInteger, and clamp the total with Math.max(total - discount, 0). Reject amounts below Stripe's 30p GBP minimum. Better still, store prices and discounts as integer pence in the DB, and recompute the cart total server-side from the cart rather than trusting the client-supplied cartTotalGbp."}],"patched_code":"app.post('/api/checkout', async (req, res) => {\n const { cartTotalGbp, promoCode, orderNote, paymentMethodId } = req.body;\n // Defence in depth: identity comes from the authenticated session, not the request body\n const userId = req.user && req.user.id;\n if (!userId) {\n return res.status(401).json({ success: false, error: 'Unauthenticated' });\n }\n\n // FIX (FINANCIAL_MATH): convert to integer pence once, using Math.round (not Math.floor).\n // Ideally cartTotalPence is recomputed server-side from the cart rather than trusted from the client.\n const cartTotalPence = Math.round(Number(cartTotalGbp) * 100);\n if (!Number.isSafeInteger(cartTotalPence) || cartTotalPence <= 0) {\n return res.status(400).json({ success: false, error: 'Invalid cart total' });\n }\n\n const client = await db.connect();\n try {\n await client.query('BEGIN');\n\n // 1. FIX (SQL_INJECTION): parameterised query, values never concatenated into SQL\n const draftOrder = await client.query(\n 'SELECT id, status FROM orders WHERE user_id = $1 AND note = $2',\n [userId, orderNote]\n );\n\n // 2. FIX (CONCURRENCY_RACE_CONDITION): atomic check-and-claim in a single conditional UPDATE\n let discountPence = 0;\n if (promoCode) {\n const promo = await client.query(\n 'UPDATE promos SET is_used = true WHERE code = $1 AND is_used = false RETURNING discount_gbp',\n [promoCode]\n );\n if (promo.rowCount === 1) {\n discountPence = Math.round(Number(promo.rows[0].discount_gbp) * 100);\n if (!Number.isSafeInteger(discountPence) || discountPence < 0) {\n throw new Error('Invalid promo discount value');\n }\n }\n }\n\n // 3. Integer-only arithmetic, clamped so a discount can never produce a negative charge\n const amountInPence = Math.max(cartTotalPence - discountPence, 0);\n if (amountInPence < 30) {\n // Stripe minimum charge for GBP is 30p; roll back so any claimed promo is released\n await client.query('ROLLBACK');\n return res.status(400).json({ success: false, error: 'Order total below minimum chargeable amount' });\n }\n\n const charge = await stripe.paymentIntents.create(\n {\n amount: amountInPence,\n currency: 'gbp',\n payment_method: paymentMethodId,\n confirm: true\n },\n { idempotencyKey: req.get('Idempotency-Key') || undefined }\n );\n\n // Promo is only permanently consumed once the charge succeeds\n await client.query('COMMIT');\n return res.json({ success: true, chargedPence: amountInPence, chargeId: charge.id });\n } catch (err) {\n // A failed charge or any DB error rolls back, so the promo stays unused\n try { await client.query('ROLLBACK'); } catch (rollbackErr) { /* connection may already be closed */ }\n return res.status(402).json({ success: false, error: 'Checkout failed' });\n } finally {\n client.release();\n }\n});\n"}AI judge breakdown
Why the winner won
Verdict
Claude Opus 5.5 wins. It diagnoses all three bugs with precise line references and the canonical 19.99 * 100 = 1998.9999999999998 example, then ships the most complete patch: parameterized queries, an atomic UPDATE ... WHERE is_used = false RETURNING, integer-pence arithmetic with Math.round/Math.max, plus transaction rollback, session-derived userId and a Stripe idempotency key. Claude Sonnet 5.5 is close behind, matching A's bug analysis and atomic promo claim, but its patch still trusts userId from the request body and omits idempotency. GPT-6 Sol is technically strong, using BigInt pence and FOR UPDATE, yet its financial writeup skips the canonical example and its patch assumes schema changes. Gemini 4 Argon correctly identifies all three bugs and applies the right fixes, but its patched code has no transaction, so a claimed promo is never released when the Stripe charge fails.
Jury scores (each judge graded blind; the final score is the average)
| Contestant | DeepSeek V4.1 Flash | Grok 4.7 | Final |
|---|---|---|---|
| Claude Opus 5.5 | 97 | 97 | 97 |
| Claude Sonnet 5.5 | 93 | 94 | 94 |
| Gemini 4 Argon | 84 | 88 | 86 |
| GPT-6 Sol | 88 | 84 | 86 |
Graded blind by a cross-provider jury (DeepSeek V4.1 Flash + Grok 4.7) against the published rubric; final scores are the average of both judges. Contenders were anonymised as A–D.
#1Claude Opus 5.5
97/100- All three bugs caught with precise line references and the 19.99 float example.
- Catches template-literal SQL injection and switches the lookup to $1/$2.
- Patch adds transaction, atomic promo claim, idempotency key, session-derived userId.
- Patch adds paymentMethodId and 30p minimum check beyond the stated scope.
- Does not reconcile a successful Stripe charge if COMMIT later fails.
#2Claude Sonnet 5.5
94/100- All three bugs identified with correct parameterized, atomic and Math.round fixes.
- Valid JSON with all three categories and a complete patched_code string.
- Patch wraps promo claim and charge in BEGIN/COMMIT with ROLLBACK on failure.
- Patched code still trusts userId from req.body, leaving an IDOR weakness.
- Patched query still binds client-supplied userId, so IDOR remains.
#3Gemini 4 Argon
86/100- Correctly names all three bug categories with accurate line references.
- Correctly labels SQL injection, TOCTOU, and the Math.floor penny bug.
- Uses the atomic conditional UPDATE ... RETURNING for the promo claim.
- Patched code has no transaction, so a claimed promo is lost if Stripe fails.
- Promo UPDATE auto-commits, so a failed charge permanently consumes the code.
#4GPT-6 Sol
86/100- Uses BigInt pence, FOR UPDATE and idempotency key in a solid patch.
- Uses session userId and $1/$2 placeholders for the order lookup.
- Flags client-supplied cartTotalGbp as a price-tampering vector.
- Financial writeup omits the canonical 19.99 example and Math.round fix.
- Never states that discount_gbp can make the charged total negative.
Head-to-heads