Continuous Embedding Spaces for Bank Transaction Data

CONTINUOUS EMBEDDING SPACES
for BANK TRANSACTION DATA
Ali Batuhan Dayıoğlugil 1
and Yusuf Sinan Akgül2
1
Yeditepe University, Cybersoft R&D Center, İstanbul, Turkey
2
Gebze Technical University, Kocaeli, Turkey
ISMIS 2017, WARSAW

28/07/2017Dayioglugil A. B., Akgul Y. S. Page 2 of 18
OUTLINE
● Introduction & Motivations
● Modern NLP Techniques
● Purpose of the Work
● An NLP Approach for Bank Data
● Experiments
● Future Works

Introduction
● Growing data: todays treasure
● Health, e-commerce, banking, assurance, games...
● Biggest challenge is extracting valuable
information
● Behavior recognition
● Forecasting
● Detecting abnormal activities

Introduction
In case of Banks:
● Millions of transactions everyday
● Processing of data:
● rule based systems
● domain experts
● Missing complicated relations and patterns between same
and different customer transactions (limited capacity)
● As a result
● Less successful product offers
● Economical loss due to undetected frauds
● Dissatisfaction of customers

Motivations
● New era in banking: Smart systems
– Learning models
– Automatically discovering hidden patterns
– Faster than human decisions
– Less prone to human errors
● Essential tasks by Smart Systems:
– Fraud Detection
– New product offers
– Customer behavior analysis

Modern NLP Techniques
● Representation of words with their hidden features:
● Continuous embedding workspaces
– Fast in process, highly promising, domain independent, fully unsupervised...
● Application Fields:
– Language modeling, language translations

Nature of Bank Data
● Specifications:
– Structured and ordered (Row-Column)
– Large variety of attributes;
● Demographic, Transactional, Logs, Engineered Features
– Various customer behavior
– Fast, Cumulative, High Volume
– Lack of labeled data
● Role of Domain Experts
– Segmentation
– Campaign Planning
– Customer Propensity & New product design
– Outlier Identification

Nature of Bank Data
● Popular techniques for financial data extraction:
– Rule based approaches
● Difficult to keep up to date
● Hard to add/remove rules regularly
– Supervised machine learning methods
● Incapable of detecting deep relations in data
● Require labeled data
– Domain Experts
● Subjective decisions
● Prone to human errors
● Not enough resource to process all data

Purpose of the Work
“Predicting customer behavior using transaction and
demographic data without any manual labeling”
● Explaining deep relations in data
● Objective approach
● Up-to-date models
● Decisions with less domain expert dependent

NLP Design over Bank Data
● Similarities:
– Time series transactions ~ Sentences in a context
– Attributes of transactions ~ Words in a sentence
● Successful results of continuous embedding
spaces with libraries: Word2Vec, GloVe and
FastText
● Need of numerical inputs for machine learning
models

NLP Design over Bank Data
● Transaction Data:
– TRX= {Ti
}, i=1..ni
● Transaction Elements
– Ti = {ti,j}, j=1..m
● Mostly categorical attributes
● Numerical elements are clustered and irrelevant elements are ignored
● Transaction elements
are converted to vectors
and compared by cosine
similarity

Experiment: Space Transformation
● Data:
– 1.8 M transaction data (4 weeks) of a medium-sized Turkish
Bank
– Enriched with customer demographic data
– 8 categorical and 2 numerical attributes (clustered)
– Dictionary size is 137
● Vector Transformation
– Word2Vec library
● Skip-gram, 20-length vectors (for each element value)
● Comparison of vectors in 2D (PCA)

● Nearest neighbours of three transaction element values
(Age, business segment and profession respectively) with
respect to cosine similarities

● PCA of business segment element value embedding vectors

●
Embedding vectors of same element values with artificially divided
‘High-income’ value

Experiment: Classification
● Predicting business segment vectors Bi (20) using Transaction vectors i
(180)
(without segment vector)
● Aim: model produce vectors positions close to business segment element value
vectors
● ANN parameters:
– Cosine similarity (loss func.)
– Stochastic Gradient
Descent as optimizer
– tanh as activation func.
– Single hidden layer, 60 nodes
– Learning rate: 0,018
– Batch size: 100

Experiment: Classification
● 4 cross-fold validation with designed ANN model
● Comparison of predicted business segment vectors i
with real vectors Bi and selecting nearest (top) 5
● Proposed model accuracies for business segment
attribute:

Future Work
● Extracting deep and hidden patterns in customer
transactions may be used in:
– Creating relevant products
– Detecting fraud attempts by examining customer
behaviours
– Defining abnormal customer activities

Thank you for listening.
Any question?

Continuous Embedding Spaces for Bank Transaction Data

Recommended

Recommended

More Related Content

Similar to Continuous Embedding Spaces for Bank Transaction Data

Similar to Continuous Embedding Spaces for Bank Transaction Data (20)

Recently uploaded

Recently uploaded (20)

Continuous Embedding Spaces for Bank Transaction Data