Machine learning for fraud detection:
essentials, use cases, and guidelines

Machine learning for fraud detection: essentials, use cases, and guidelines

September 26, 2023

Aleksandr Ahramovich

by Aleksandr Ahramovich,

Head of AI/ML Center of Excellence

Machine learning-based fraud detection systems rely on ML algorithms that can be trained with historical data on past fraudulent or legitimate activities to autonomously identify the characteristic patterns of these events and recognize them once they recur.

Explore the nature, payoffs, and applications of this technology in common fraud scenarios, along with some guidelines to streamline its adoption. Finally, find out how machine learning development experts can help you protect your company and customers from ever-evolving fraud threats.

ML-based vs rule-based fraud detection

ML-enabled anomaly detection applications differ from traditional software in terms of detection technique:

UserTransactionsML modelOngoing model fine-tuningPatterns anomaliesDetectionMachine LearningFraudsterFraudRulesHigher frequencyAddress mismatchOver limitDetectionRule Based

ML-based fraud detection

  • ML solutions autonomously identify and use more complex and variable rules than traditional systems. To do so, ML algorithms process data on past fraud cases, discover patterns and relationships between data points, and build models trained to identify those patterns once they recur in future datasets.
  • ML systems can predict imminent criminal actions by identifying anomalies, namely subtle and unconventional behavioral patterns that humans would probably overlook but that still deviate from the norm, which could be clues to upcoming fraud.
  • ML-powered solutions improve with experience, refining their models over time as they process new data, including unmapped data points. So, if they encounter new fraud scenarios, machine learning-based anomaly detection systems will quickly adapt to such threats, automatically integrating and updating the existing rules without human intervention.
Example
The fraud detection system of an ecommerce platform runs into a suspicious credit card transaction that doesn't fit its user’s behavioral patterns based on multiple subtle parameters, such as the product pages browsed before placing an order.

Rule-based fraud detection

  • Traditional rule-based solutions follow an "if/then" logic and trigger a response when a predefined condition is violated.
  • These tools must be manually instructed to detect fraud cases by compiling libraries with hundreds of rules.
Example
The fraud detection system of a bank flags supposed customers who use credit cards in unusual locations, or increase their payment frequency.

Consult our experts to implement machine learning the right way

Get in touch

Technical overview of ML for fraud detection

ML approaches

Do machines need human intervention to learn, or can they just observe the surrounding reality? The main approaches to training machine learning algorithms are supervised, unsupervised, and reinforcement learning, depending on the degree of human involvement and control over the ML training process.

Supervised learning

ML-based fraud detection systems are trained with large amounts of labeled data, previously annotated with certain labels describing its key features. This can be data from legitimate and fraudulent transactions described with "fraud" or "non-fraud" labels, respectively. These labeled datasets, which require rather time-consuming manual tagging, provide the system with both the input (transaction data) and the desired output (groups of classified examples), allowing algorithms to identify which patterns and relationships connect them and apply such findings to classify future cases.

Unsupervised learning

These algorithms are fueled with unlabeled transaction data and have to autonomously group these transactions into different clusters based on their similarities (shared behavioral patterns) and differences (typical vs unusual patterns which can correspond to fraudulent activity). This approach, typically associated with deep learning, is computationally demanding but can be the only choice when facing fraud attempts that have never been met before and therefore unlabelled.

Reinforcement learning

This trial-and-error approach involves multiple training iterations in which the algorithm performs a fraud detection task in different ways several times until it can accurately identify fraudulent and non-fraudulent attempts. Since it does not require labeled inputs, reinforcement learning can be applied without prior knowledge of the current fraud scenario. However, it requires considerable computing power.

Input Raw DataAlgorithmModel TrainingLabeled dataProcessingClassificationOutputSupervised learningInput Raw DataAlgorithmInterpretationUnannotated training data setUnknown outcomeProcessingClustersOutputUnsupervised learning

Scheme title: Supervised vs unsupervised learning

Data source: medium.com — An Executive’s View: Introduction to Machine Learning

Machine learning vs deep learning

Experts generally distinguish between machine learning as a whole and its more advanced branch, known as deep learning.

Machine learning

A subset of AI focusing on systems that can learn and improve with experience. Identifying patterns and anomalies is one of their most common tasks and a key enabler for fraud detection. “Traditional” ML solutions can be trained with a relatively small amount of data and don’t need substantial computational power, at least compared to deep learning-based software.

Deep learning

A machine learning method typically associated with complex sets of algorithms known as deep neural networks. Such structures, comprising connected layers of artificial neurons that mimic the human brain, can process data flowing from one layer to the next and spot the most subtle patterns and features hidden within an immense amount of data points. This particular process makes neural networks very effective in detecting fraud, but it also requires enormous computational power and large data sets for proper training.

ML algorithms

Identifying the best algorithm or ensemble of algorithms to perform fraud-related data analysis can be a challenging task, as their performance can depend on the scenario in which we operate. Here are some options to consider:

Logistic regression

A supervised learning algorithm which calculates the probability of one event out of two alternatives, such as "fraud" and "non-fraud" based on a set of relevant parameters.

Decision tree

Another algorithm of the supervised learning subset is a tree-like decision-making model in which every bifurcation represents the analysis of a certain metric or condition (spending threshold, location, etc.) to determine whether an operation is fraudulent.

Random forest

A combination of several decision trees to further expand the amount of data types and conditions examined and identify non-linear relations among multiple variables.

Support vector machine

Several systems rely on this supervised learning algorithm for credit card fraud detection because of its excellent performance with large datasets despite it being computationally demanding.

K-nearest neighbor

This supervised learning algorithm, which proves quite accurate but difficult to interpret when making a certain decision, can frame the nature of an event (be it fraud or non-fraud) by comparing it with similar occurrences recorded in the past.

Neural networks

Complex, multi-layered architecture and superior big data analysis capabilities make neural networks the go-to algorithms when it comes to spotting non-linear relations and dealing with unprecedented fraud scenarios through supervised, unsupervised and reinforcement learning.

Machine learning in top fraud scenarios

Machine learning can be deployed to keep fraudsters and cybercriminals at bay in a variety of scenarios. Let’s take a look at some popular use cases of this technology.

Machine learning in top fraud scenarios

Financial institutions have come to understand the synergistic potential between the stock market and machine learning and the benefits of predictive analytics in finance, considering the large sums involved and the required compliance with increasingly strict regulations.

ML-driven systems can help prevent financial fraud, such as churning, spoofing, and wash trading, by spotting anomalies in stock traders' activity and cross-checking transactions and brokers' data to detect inconsistencies in the information provided.

Applying machine learning in banking has proved useful to track anomalous transactions that could be a sign of criminal activity, such as large sums of money exchanged among a group of newly-established companies registered in tax havens.

Machine learning models can be trained with data relating to three possible scenarios: lawful transactions, money transfers flagged as suspicious by bank alert systems, and potential money laundering cases reported to the authorities. For each case, machine learning systems analyze the senders’ and receivers' background or their previous transaction histories. This way, they can spot the same patterns once they recur in future scenarios and therefore distinguish between legitimate activities and criminal actions.

Electronic payment fraud represents a widespread crime taking many forms and encompassing almost any industry, from banking to ecommerce. However, machine learning systems in retail have a good chance of containing this escalation by detecting abnormal account behavior that could be a sign of fraud. Some of the factors to consider are a growing transaction frequency (especially to purchase premium goods), payment on a card carried out significantly before the due date, associations with different high-risk accounts, and multiple payment methods added in a short time.

Machine learning-driven systems for payment fraud detection can also update card users' behavioral profiles after each transaction, making future predictions more precise and avoiding false positives.

Another type of crime strictly connected to payment card fraud and remarkably widespread in many scenarios, including fraudulent loan applications and ecommerce scams, is identity theft. This involves a scammer exploiting the personal information of another individual, including their name and credit card number, without their consent to commit a crime. To deal with this kind of fraud, we can count once again on machine learning-driven analysis of users' habits and transaction data.

Furthermore, we can leverage machine learning-powered computer vision to analyze identity documents or add additional verification mechanisms such as face recognition and biometrics.

Machine learning solutions can also strengthen fraud detection in the insurance sector, especially in health insurance. Algorithms can identify false and duplicate claims, pointing out a customer who reported an incorrect diagnosis or exaggerated medical coverage costs.

One of the most valuable machine learning techniques in healthcare is natural language processing, which ensures an in-depth analysis of unstructured data such as medical reports. These solutions leverage machine learning to scan documents written by doctors, insurers, or clients, searching for suspicious inconsistencies.

Public authorities can use machine learning capabilities in identifying unusual patterns to enhance audit and tax compliance. For example, ML can examine the general ledger for anomalous entries which could be the signs of attempted fraud.

Algorithms can spot a wide range of clues easier and faster than human auditors, taking into account various parameters. These include monthly variations in companies' gross sales, relations among different taxpayers, inconsistencies among purchases, or the itemized deductions in an income peer group.

Get these ML benefits with Itransition’s guidance

Let’s talk

Real-world examples of ML for fraud detection

Capgemini reported that customers adopting its ML-enabled CPP Fraud Analytics software for fraud detection and prevention have benefited from an increase in detection rate between 50% and 90% and a reduction in investigation time for each fraud case up to 70%. The solution relies on an ensemble of technologies and approaches, including both rule-based fraud detection and neural network-powered pattern analysis, to perform tasks ranging from KYC operations to anti-money laundering and credit card fraud.
LendingBankingCardsOther relationshipsCustomer view internal dataUnstructured data – social media, call centers etc.Credit bureau, fraud vendors, watch lists, Mkt & reference dataCustomer view external dataInternetATMPOSPhone bankingOther channels/tradesCustomer view transaction dataBig dataArtificial neural networks/pattern recognitionAccount opening decisioningAdaptive analytics, rule based modelsCustomer level fraud manage-mentOffline entity link analysisBiometricsTransaction fraud monitoring3d secureCounterfeit measuresDevice trackingTokenizationFraud monitoring & detectionDecisionsAlert managementDataNotificationsCase managementVisualization/reports/dashboardsAnalyticsFraud management actionsAdvanced analytics methodologies

Scheme title: The scope of Capgemini’s fraud detection solution

Data source: capgemini.com — Next generation fraud management solutions

Danske Bank complemented its legacy fraud detection system with a deep learning-based solution by Teradata, leading to a 60% reduction in false positives and a 50% increase in true positives. The new software automates most audit processes for credit card transaction monitoring and anti-money laundering, with human experts supervising specific cases flagged by the system as anomalies.
Yes/MaybeYes!Mobile payeBankingBusiness onlineGlobal payment interfaceCentral fraud engineFraud?NoProcess payment to beneficiaryCustomer product dataHistoric transactionsBasic customer dataAggregated customer dataAdvanced analytics platform for fraudWeblog dataFraud?Manual evaluationCreate verificationReturn payment to customerFraud paymentUpdate fraud dataNo

Scheme title: Danske Bank’s fraud detection system operation

Data source: assets.teradata.com — Danske Bank Fights Fraud with Machine Learning and AI

The leading South African banking and finance group implemented an ML-based solution built on top of Amazon Fraud Detector to automate low-risk insurance claim approval and better assess high-risk claims. This resulted in two times more fraudulent cases being identified and the turnaround time reduced from 48 hours to 6 hours, mitigating the business' exposure to risk and offering a superior customer experience.
Amazon S3Amazon Fraud Detector APILoad dataInspect dataEnrich dataIdentify featuresSelect algorithmsTrain & optimize modelsValidate performanceHost modelsAmazon Fraud DetectorBuild a fraud detection machine learning model that is customized to your data in a few clicks using a fully automated processAmazon Fraud Detector Detection LogicCombine your model with decision rules to turn model scores into actionable outcomes (e.g., review, pass)Amazon Fraud Detector Prediction APIFor real-time fraud detection, call the Amazon Fraud Detector API with online event data (e.g., new account creation) to receive fraud predictions

Scheme title: Amazon Fraud Detector’s architecture

Data source: aws.amazon.com — Amazon Fraud Detector

Nasdaq adopted a deep learning-based solution to monitor over 17.5 million trades per day, recognize fraudulent equity orders, report them to the authorities, and thus ensure transparent markets. By probing unfamiliar trading patterns, it can even shed light on suspicious events which can end up being previously unknown types of fraud. This system operates in combination with human analysts and the preexisting system based on rules and statistical techniques.

It [points] out what we call an ‘interesting event’. It’s not necessarily a prohibited activity, but it’s what the model has deemed to be interesting because it’s not normal market behavior.

Mike O'Rourke

Mike O'Rourke

Senior Vice President, Head of Artificial Intelligence and Investment Intelligence Technology, Nasdaq

How to set up an ML system for fraud detection

1

Business analysis

Identify fraud prevention-related needs and challenges via discovery workshops and process observations

Evaluate your current tech ecosystem

Understand if it makes sense to opt for ML-based fraud detection software over conventional solutions

Define the solution’s functional and non-functional requirements

Set up a project roadmap, including scope, deliverables, and timeframes

2

Initial data analysis

Perform an exploratory analysis to map available data sources (corporate databases and connected devices such as ATMs or POSs)

Identify external data sources (public records, law enforcement, or government watch lists)

3

Product design

Draw up a specification detailing the solution’s architecture, modules, core features, UI/UX, and integrations with other software

Identify a suitable tech stack to build your software

Optionally, deliver a proof of concept to ensure the project’s feasibility and financial viability while pointing out potential adoption challenges

4

Building the CV solution

Perform data pre-processing, including data cleansing, annotation, and transformation

Extract the most relevant features (customer's IP, preferred payment methods, number of failed transactions, average order value, fraud rate of the issuing bank)

Outline the fraud detection system’s evaluation criteria

Process your datasets with ML algorithms to train the model to recognize patterns and anomalies, or build multiple models until you achieve the desired output

5

Model integration and deployment

Integrate the ML model into the solution to power its fraud detection capabilities with the model’s output

Deploy the system to the target environment (on-premise or cloud-based)

Configure all necessary API- or ESB-based integrations with other corporate systems and data sources

6

Support

Closely monitor the system’s operation and perform ongoing maintenance with the help of experienced ML engineers 

Provide your staff with user training and support 

Following MLOps' best practices, retrain the ML model with new datasets across multiple iterations to fine-tune its output and address model drift issues

The benefits of ML in fraud detection

The distinct mechanisms driving ML-based fraud detection and fraud prevention solutions make them superior to more traditional, rule-based systems in several respects.

1 Higher flexibility and reactivity

The rule-based approach is not flexible enough to deal with rapidly evolving fraud patterns. After all, the rule sets are manually coded and built on previous fraud scenarios, so they should be continuously adjusted according to new types of events. Thanks to self-learning abilities and sheer processing speed, ML-driven systems can deal with this process far quicker than rule-based tools and their human reviewers by adjusting machine learning models on their own based on new kinds of threats.

2 Wider data pool for analysis

Statistical and rule-based fraud detection methods can process structured data, such as transaction figures, but can struggle when handling unstructured data, for example, written reports, insurance claims, and pictures from IDs and other documents. In the meantime, machine learning and its related technologies, such as natural language processing and computer vision, can deal well with any kind of data, enabling companies to analyze more criteria and variables across data points.

3 Lower rate of incorrect outcomes

The traditional approach tends to follow a "black or white" mindset. This results in a massive amount of false positives that don’t represent a real threat but still need to be double-checked through an expensive and time-consuming manual review. Machine learning-based systems, instead, are far more accurate and cost-efficient even in complex scenarios, as they don’t rely on strict rules but search for patterns and dependencies among a wider amount of variables. This helps them to grasp some nuances, understand the logic behind a suspicious case, and avoid false alarms.

4 Superior compliance

The need for constant human intervention and interaction with data of rule-based systems can clash with the increasingly strict trading, fiscal, and data management regulations. With less reliance on manual operations, the adoption of machine learning systems can minimize human errors and mitigate compliance-related business risk.

Challenges of ML for fraud detection

Despite their proven efficiency, ML-based fraud detection systems can be complex to develop and adopt. Here are a few tips to streamline their implementation while overcoming potential drawbacks of this technology.

ML model interpretability

Issue
A common dilemma when building an ML-based fraud detection system is to select suitable algorithms since the best-performing ones typically suffer from the "black box" issue. For instance, random forests and neural networks can easily identify non-linear relationships and therefore build very accurate models portraying complex fraud events. However, their sprawling architectures make it difficult to interpret how they generate outputs from certain inputs, negatively affecting the model’s explainability.
Recommendation
The black-box nature of ML still represents an unsolved enigma for professionals in this field. That said, ML engineers should strive to identify the right metrics to track algorithms' performance during training and thereby shed some light on their operation, along with the reasons behind potential bias and inaccuracies.

False positives/negatives tradeoff

Issue
Finding a reasonable compromise between security and a hassle-free user experience can be challenging. Indeed, the first goal requires fraud detection software to be particularly strict and block even vaguely suspicious transactions, which may end up being completely legitimate (false positives). On the other hand, great user experience implies a wider tolerance range when assessing potential anomalies, which can result in successful fraud (false negatives). Machine learning's low rate of false positives certainly mitigates this issue, but algorithms are still far from being perfect.
Recommendation
When creating a fraud detection model, it's important to train it on large validation datasets and identify a suitable threshold, or minimum requirements that will trigger a response. The right balance depends on the level of risk your business can afford. For example, a fraud detection system can be more tolerant when scanning large volumes of low-value transactions and more rigorous when probing premium product purchases.

Data privacy concerns

Issue
ML systems' hunger for data and compliance with personal data protection standards and applicable legislation (PCI DSS, GDPR, IFRS, etc.) can be conflicting elements, especially in highly regulated fields like finance and accounting.
Recommendation
Using data masking techniques to obfuscate your data sets before they’re processed by ML algorithms for training or analysis will make sure that your solution complies with such regulations.

Development complexities

Issue
Businesses interested in fully personalized fraud detection software should opt for a bespoke solution built from the ground up. However, this can be demanding in terms of data availability, ML model training times, and computational resources.
Recommendation
Any company looking to reduce upfront costs, implementation timeframe, and maintenance efforts needs to consider adopting one of the off-the-shelf, machine learning-powered systems available on the market. These include Amazon Fraud Detector, IBM Security Trusteer, Sift's Digital Trust & Safety, and Signifyd’s Authorization Rate Optimization. Alternatively, you can rely on the ML tools and services from major cloud providers (such as Azure Machine Learning or Amazon SageMaker), which offer built-in algorithms, pre-trained models, and scalable computing resources to help you create your own solution.

Fighting crime with algorithms

The more our society moves towards full digitalization, the more cybersecurity attacks grow in impact, frequency, and complexity, with scammers and cybercriminals leveraging the same technologies that legitimate institutions deploy to counter them. As a result, AI and ML crack passwords, power adaptive bots, and manipulate datasets on the one hand, and power highly effective fraud detection systems on the other. Fraud detection solutions have proved to live up to the hype, thanks to their adaptability to new threats, smart context-based data analysis, and real-time identification capabilities. At the same time, ML’s limited interpretability, appetite for sensitive data, and substantial training and computing requirements should be approached with proper expertise and compliance in mind. To smooth out the challenges of ML adoption, rely on an experienced partner like Itransition.

Machine learning in ecommerce:
10 use cases, examples, and best practices

Insights

Machine learning in ecommerce: 10 use cases, examples, and best practices

Discover the key trends, use cases, payoffs, and examples of ML in ecommerce, along with some best practices to streamline its adoption.

Machine learning consulting
services & solutions we deliver

Service

Machine learning consulting services & solutions we deliver

Explore our range of machine learning consulting services, along with related technologies, use cases, implementation roadmap, and payoffs.

Machine learning in real estate: 
use cases, examples, and adoption guidelines

Insights

Machine learning in real estate: use cases, examples, and adoption guidelines

Explore the top trends, use cases, examples, and pros of ML in real estate, along with key adoption challenges and best practices for addressing them.

AI use cases: 3 rules to find the right one for your business

Insights

AI use cases: 3 rules to find the right one for your business

One of the main barriers to AI adoption is identifying the right AI use cases. Find out the best practices to make it less challenging.