Every time a customer pays you through a payment gateway, an e-wallet, or a bank transfer with a virtual account, that transaction is scored by a machine learning model before the money moves. AI fraud detection in payments is one of the oldest production uses of machine learning, running quietly for years at the banks, card networks, and Indonesian payment gateways you already use. It is worth understanding, both because it protects you and because its limits define the fraud you still have to handle yourself.

The scale explains why this had to be automated. A mid-size payment gateway processes millions of transactions a day. No human team can review that volume, and rule books alone, "block transactions above Rp X from new accounts," are too rigid: fraudsters learn the rules within weeks, while legitimate customers get blocked by them daily.

Here is how the machine side works, in plain terms, and then the part that matters more to most readers: the fraud that lands on merchants directly, below the radar of any payment-rail AI.

How Anomaly Scoring Actually Works

Strip away the vendor marketing and AI fraud detection in payments does one thing: it estimates, in a few milliseconds, how unusual this transaction is compared to what normal looks like.

"Normal" is learned from history along many dimensions at once:

  • Behavioral: Does this cardholder or account usually transact at this hour, at this merchant category, at this amount? A card that buys groceries in Tangerang every week suddenly buying electronics at 3 am is a spike on several dimensions simultaneously.
  • Velocity: How many transactions from this account, device, or IP address in the last minutes and hours? Stolen credentials tend to be used fast, before the theft is noticed, so bursts are a classic signal.
  • Network: Does this device, address, or bank account appear across many unrelated accounts? Fraud operations reuse infrastructure, and graph-style analysis catches accounts that look independent but share fingerprints.
  • Population: Even with no history for this specific customer, does the transaction resemble known fraud patterns learned from millions of labeled past cases?

Each transaction gets a risk score. Low scores pass instantly. High scores get blocked. The interesting zone is the middle, where systems respond with friction instead of refusal: an OTP challenge, a delay, a manual review queue. That design acknowledges the real trade-off in fraud detection: every false positive is a legitimate customer you just embarrassed and possibly lost. The models are tuned as much to avoid blocking good customers as to catch bad ones.

Two honest limitations. First, models trained on history are weakest against genuinely new fraud patterns, which is why the systems are retrained constantly and paired with human fraud analysts. Second, a score is a probability, not a verdict, and treating model output as certain truth is a mistake in fraud just as it is everywhere else in applied machine learning. The same discipline applies when you deploy AI in your own business, which I wrote about in AI Hallucinations: How to Deploy AI Without Embarrassment.

The Fraud That Reaches Merchants Anyway

Here is the part that surprises business owners: the payment rails being well defended does not mean you are well defended. The AI protects the transaction. It does not protect your business model. The fraud that hurts Indonesian SMEs mostly happens around the payment, not inside it.

Fake COD orders. Cash on delivery involves no payment rail until delivery, so no payment AI ever sees the order. Fraudulent or prank COD orders that get refused on arrival cost you double shipping plus handling plus tied-up stock. For sellers doing high COD volume, refusal rates of 5 to 15 percent are common, and each refused package can burn Rp 25,000 to 60,000.

Refund and return abuse. The buyer claims the package was empty or the item damaged, files for a refund with recycled photos, and does it across many stores. Individually each case looks like a customer-service issue. In aggregate it is an organized pattern.

Fake payment confirmations. A "customer" sends a doctored transfer screenshot and pressures your admin to ship before the money clears. During busy periods, especially campaign season, tired admins approve these. The screenshot is free to fake; your goods are not.

Promo and voucher abuse. New-customer vouchers harvested by one person with twenty accounts. A miscoded discount stacked into nearly free orders and shared in bargain-hunter groups within the hour.

Low-Tech Defenses That Actually Work

You do not need machine learning to cut most of this. You need the same principle the ML uses, "flag what deviates from normal," applied at merchant scale with rules and habits.

  1. Kill the screenshot problem structurally. Never ship on a transfer screenshot. Use virtual accounts or a payment gateway so confirmation comes from the system, not the customer. This single policy eliminates an entire fraud category.
  2. Score your own COD risk with three rules. New customer, plus high order value, plus an address or phone pattern you have seen refused before, equals a verification WhatsApp call before shipping. Sellers who add a simple confirmation step routinely cut COD refusals by a third or more.
  3. Keep a refusal and abuse log. Phone numbers, addresses, and account names from refused COD orders and dubious refund claims, in one shared spreadsheet. Fraudsters repeat. Most SMEs never notice because the records live in five different heads. This is exactly the kind of unread evidence I described in Dark Data: The Answers Already Buried in Your Business.
  4. Cap and monitor every promo. Per-account limits, per-device where the platform allows it, hard quantity caps on deep discounts, and someone actually watching redemption in the first hour of a campaign.
  5. Give staff a pressure rule. Any buyer who is in a hurry, insists on skipping standard steps, or overexplains is a mandatory escalation to the owner or a senior admin. Social pressure on junior staff is the most reliable fraud tool there is, and a standing rule removes the decision from the moment.

Where AI Fits for a Small Merchant

Should an SME build its own AI fraud detection? Almost never. The economics only work at payment-processor scale, where millions of labeled transactions feed the models. Your leverage is choosing rails that already run this machinery, using a licensed payment gateway rather than manual transfers, and turning on the fraud controls your marketplace or gateway already offers, which many sellers never open.

Where your own data does become useful is the log from point three above. A few hundred rows of refusal history is not machine learning, but it is the same idea at human scale: patterns beat instincts.

The Practical Takeaway

Trust the payment rails, they are genuinely well defended, and put your energy where the AI cannot see: COD verification for risky orders, system-confirmed payments only, one shared abuse log, capped promos, and an escalation rule for pressure tactics. An afternoon of policy-setting covers all five.

AI fraud detection in payments is a good reminder of what machine learning does well: pattern recognition at inhuman speed and scale. The merchant lesson is not to buy AI. It is to steal its principle, define normal, watch for deviation, and never let one tired admin be the last line of defense.