Payments related fraud is rising worldwide, and a recent Coinbase (NASDAQ: COIN) evaluation has shown that simply moving to newer large language models does not automatically improve specialized fraud detection.
In its “Owning Intelligence” engineering series, published in early October 2026, crypto exchange Coinbase detailed a controlled historical replay of its Onramp service.
The test covered 16,140 transactions involving 7,293 users and 813 confirmed fraudulent payments.
These transactions came from nine weeks of production data collected before the company introduced its own risk agent, ensuring that later blocking decisions did not skew the observable outcomes.
Each model was assessed under identical instructions, the same contextual-review process, and a fixed risk-to-decision policy.
The sole variable was the model version.
Across three major families, the newer releases caught a smaller share of fraudulent transactions and a smaller portion of the associated value than the versions they replaced.
Recall (the share of fraud cases identified) and dollar-weighted recall (the share of fraudulent value identified) both declined, as did the combined F1 score.
Sonnet 5 underperformed Sonnet 4.6 most sharply, with recall falling 22.2 percentage points and dollar-weighted recall dropping 22.9 points; precision also declined.
Opus 5 showed more modest slippage versus Opus 4.5, including a 0.8-point drop in recall, an 8.7-point drop in precision, a 5.5-point drop in F1, and a 1.9-point drop in dollar-weighted recall.
GPT-5.6 (sol) improved precision by 11.5 points over GPT-5.4, consistent with more conservative classification, yet recall fell 20.7 points, dollar-weighted recall fell 21.8 points, and F1 declined 6.1 points.
Coinbase engineers summarized the pattern by noting that newer LLMs are not necessarily better at domain-specific tasks.
The replay identified a regression without establishing its cause and did not measure live customer losses.
These findings arrive against a backdrop of increasing payment fraud (globally).
UK Finance’s Annual Fraud Report 2026 reported that criminals stole £1.28 billion through payment fraud in 2025, a 4 percent rise from the prior year. Confirmed cases reached a record nearly 4.1 million, up 11 percent.
Authorised push payment (APP) fraud losses rose 19 percent to £576.4 million, while unauthorised losses stood at £703.4 million (down 5 percent).
Banks prevented a further £1.68 billion in unauthorised fraud, equivalent to 70p in every pound attempted.
Lloyds Banking Group has highlighted sharp increases in certain purchase scams reported by its customers.
Driving-lesson and driving-test scams rose more than 240 percent year-over-year in the period ending November 2025, while vehicle, ticket, clothing, and personal-item scams remained among the most frequently reported.
NatWest Group, along with other major UK banks, has continued to invest in real-time analytics, behavioural monitoring, and AI-assisted tools such as fraud-triage capabilities within its customer assistant, reflecting the same industry-wide pressure captured in UK Finance data.
In the United States, Federal Reserve Financial Services’ 2026 Risk Officer Report, based on a late-2025 survey of more than 400 institutions, documented rising fraud attempts and losses across major rails.
Debit card fraud was the most widespread, reported by 75 percent of respondents and estimated to account for roughly 40 percent of total payments-fraud losses.
Check fraud remained elevated, account-takeover activity increased, and institutions noted growth in account-holder scams, business-email compromise, and related threats on wires and ACH.
Broader indicators, including FBI Internet Crime Complaint Center data and industry analyses, point to continued growth in cyber-enabled and social-engineering-driven payment losses.
Coinbase’s own live testing of a purpose-built LLM risk agent had previously shown meaningful reductions in fraudulent Onramp transactions and value relative to its prior stack.
The newer-model benchmark underscores that general capability gains do not automatically translate into better performance on narrow, high-stakes tasks such as payment-fraud screening. As fraud volumes climb, institutions are under pressure to evaluate upgrades against their specific risk policies and data rather than assuming later releases will improve coverage.