Recomendaciones, predicciones y detección de fraude usando servicios de inteligencia artificial

1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | 1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | 1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | 1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Javier Ramirez
AWS Developer Advocate
Recomendaciones, predicciones y
detección de fraude usando servicios de
inteligencia artificial
@supercoco9

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The case for forecasting
Forecasting is the science of predicting the future
Product demand
Actual demand vs. forecasted demand ($ Millions)
Actual Demand Forecast Demand
Over-forecasting leads
to wasted resources
Under-forecasting leads
to lost opportunity

The case for Forecasting
Inventory
planning
Workforce
planning
Capacity
planning
Excess inventory Unutilized labor
Uncapitalized
infrastructure
Lost sale Overtime costs Unmet demand
Over-forecasting
Under-forecasting
Financial
planning
Depleted cash
reserves
Undercutting
Impact of under and over forecasting

Product
availability
Fast
delivery
Lower
price
Demand forecasting for
over 400 million
products every day
Inventory and fulfillment
cost reduction to provide
low prices to customers
12 shipping options
with free same-day
delivery
Forecasting at Amazon.com
Accurate forecasting is critical for delivering on customer promises

Forecasting process
Three standard steps involved in forecasting
• Looking backward
• We always begin with historical data; ideally, that contains a timestamp, an item, a value
• Larger datasets contain potentially millions of rows over thousands of items, but the construct is the
same
• This provides the baseline
• Identifying trends
• Using statistical, deep learning, or other approaches, you look over the historical data to determine
trends within your data—trends that hopefully continue into the future
• Projecting forward
• Given the trends identified, take each item and predict in increments the expected future values

An overview
Forecast
Amazon

Amazon Forecast
Fully managed
service
Highly accurate Easy to use Your data,
your models
Automatically sets
up data pipeline,
training, and
prediction
50% improvement
in accuracy over
traditional
methods
No deep learning
experience
required
Encrypted with
customer keys
through Amazon Key
Management Service
(AWS KMS)
Automated machine learning service for accurate forecasting

Use of historical data
to predict future values
Target time-series
dataset
The primary variable to predict with its
historical values
(demand, sales)
Datasets used for forecasting
Use of related
attributes and
categorical data
Item metadata
(non-time-varying)
Categorical data that provide more context
about items
(color, city, channel)
Use of known time-
varying data specific to
your business
Related time-
series
dataset
Time-varying related features that may impact
the target value
(price, promotion, weather)
Amazon Forecast

Private
forecasting API
Amazon Forecast
The technology that powers the world’s largest ecommerce business
Get started with the console or API
Point Amazon Forecast to your data
stored in Amazon Simple Storage Service
(Amazon S3)
Automatically train your custom ML
model
Let Amazon Forecast auto select the best one for
your data through AutoML
Generate accurate forecasts
Retrieve forecasts through the console or
private API
Historical data
Related data
Sales, call volume, inventory,
resource demand
Price, promotions, weather
data, custom events
Item metadata
Color, city, country, category,
author, album name
Built-in dataset
(holiday, weekends)
Amazon Forecast

Customized
forecasting API
Inspect
data
Identify
features
Select most
accurate model
from multiple
algorithms
Select
Hyper-
parameters
Host
models
Load
data
Train
models using
multiple
algorithms
Optimize
models
Amazon Forecast
Behind the scenes
Fully managed by Amazon Forecast
Historical data
Related data
sales, call volume, inventory,
resource demand.
Price, promotions, weather
data, custom events
Item metadata
Color, city, country, category,
author, album name
Amazon Forecast

Handles tricky forecasting scenarios
Missing values
Cold start
(new product introduction)
Irregular seasonality
Product discontinuation
Highly spiky data
Sensitivity analysis
(future price change)
Amazon Forecast

Amazon Forecast Quick Demo

Amazon Forecast in production

Amazon Forecast vs. legacy system TCO
Other
(legacy system)
Amazon
Forecast
Installation time 2–8 months 6–8 weeks
Pricing model License fee Pay-as-you-go
model
Maintenance costs (3 years) Up to a $1,000,000 $6,000*
Professional services Up to $150K As needed
*Includes training, model maintenance for monthly forecast for 35,000 SKUs, and
two quantiles

Amazon Forecast in production
Manages a set of EC2 instances
Maintains cache pools for EC2 instances
Needs to forecast cache pool size
Demand
publisher
Cluster
scaling
EC2 Cache Pool
Amazon
Redshift
Demand
Forecast
table
ETL Lambda
Amazon
Forecast
Demand
Forecasts
Demand
history
1
2
34
1. EC2 Cache pool demand changes are
published to S3 bucket
2. New data is ingested into Amazon Forecast
and new forecast predictions are stored in
S3 bucket
3. A Lambda function copies new forecasts to
an Amazon DynamoDB table
4. The cluster scaling logic reads forecasts
and adjust the cache pool size based on
projected demand
Amazon Redshift Cluster Management

Why personalization?

Personalization offers material business benefits
Everybody’s tastes, preferences, and needs are different
Improve customer engagement with your website, app, and content
Help customers discover products and services they need
Increase revenue for your business
Increase conversion – purchases, subscriptions, app downloads,
movie & music streams

History of personalization at Amazon
First personalized experience on Amazon.com
Early features such as “Customers who bought this also bought …” loved by all
Early investment in personalization to make shopping easy
Over two decades of research in personalization techniques
Feature launched & patent filed in 1998

Our learnings on personalization
Rule-based strategies are
not effective and are
effort-intensive to
maintain
Real-time
recommendations
and handling
Coldstart scenarios
(new user/new item)
One single ML algorithm is not a
good fit for all personalization
use cases
Avoid a bias for
recommending
popular items
Machine learning (ML)-based techniques perform well but need to solve multiple hard
problems to do so
Building accurate and effective
personalization models requires
ML experts

Personalize
Amazon
Personalization using the same ML
technology as at Amazon.com; no ML
experience needed

Customized
personalization
API
Amazon Personalize
Inspect
data
Identify
features
Select
hyper-
parameters
Train
models
Optimize
models
Host
models
Real-time
feature
store
Amazon Personalize
Behind the scenes
Fully managed by Amazon Personalize
Item metadata
(details of articles,
products, videos, etc).
User metadata
(age, location, etc.)
User events/
interactions
(views, signups,
conversions, etc.)

Different types of personalization

User personalization Personalized rankingSimilar items
Amazon Personalize solves foundational
personalization use cases

User personalization
When to use:
• Landing pages
• Cart recommendations
• Email promotions (often
preprocessed)
• Detail pages in addition to
similar items
How to use:

User personalization with sequence models
The evolution of historical interest and disinterest is a good indicator of future preferences
ü Assigns more attention to recent events
ü Real-time interactive responses
Other
information
Learned user
representation
(Order and timing matter)
(recurrent neural network)

Cold item personalization
• Use for recommending relevant newly added items to users; special
case of user personalization
• Personalized recommendations for new items
• Use via HRNN-Coldstart recipe

Similar items
When to use:
• Item detail page – multiple
variants possible with different
events (users who purchased X
also purchased Ys; users who
clicked X also clicked on Ys)
• Up- sell recommendations (SIMS
+ business rules)
How to use:

Similar items recommendations using item-item
collaborative filtering
ü An item is similar to another if they were interacted by same users
ü Scalable; only needs interaction data to calculate similarity
ü Customers can choose the popularity vs. correlation trade-off
SIMILAR

Personalized reranking
When to use:
• Category-based
recommendations
• Recommendations under
business constraints (e.g.,
recommend only from free-to-
watch movies)
How to use:
Sequence
algorithm

Amazon Personalize Tour

How Coursera used Amazon
Personalize for recommending
courses

8M
LATAM
13M
North America
9M
Europe
2M
Africa
12M
Asia4.8M
India
Source: Coursera data, September 2019
Coursera reaches more than 44M global learners

What changed in 2019?
Rapid Machine Learning Development
2018: Senior Data Scientist - 3 months
2019: Intern - 1 week (using Amazon)

Business/systems challenges
1. Recs for 40 Million members
2. Recs for 4 Million Unique Visitors/Day
3. Leverage 100 M+ events
4. Personalized Recs in Real-time
5. Scalable / HA Solution

Goal: Better course recommendations

Amazon Personalize SIMS recommendations

Architecture
Amazon Redshift
Amazon
S3 bucket
Amazon
Personalize
USER

Model retraining script

Results of Amazon Personalize (SIMS)
Rapid Development & Deployment
• Train: 20x faster
• Total: 3 days development
Automatically Scales
Revenue increased (A/B tested)

Next: HRNN-Metadata recipe
User
Language ru
Country RU
Region MOW
Device iPhone
Item
Language Russian
Domain data-science
Sub-domain data-analysis
Interactions
User, item, timestamp
2996376, gCCEeiZdg6RDGBSdg,1556241493

Recommendations: Russian browser

Fraud Prevention at Amazon and
AWS

Cloud Computing offers easy access to compute services.
We protect our customers and services from bad actors.
Payment Fraud
• Compromised Payment Instruments (e.g., stolen cards)
• Intentional Non-Payment (e.g., pre-paid cards)
Account Takeover/Compromise
• Username/Password
• API Key
Abuse
• Free Tier Misuse
• Premium Phone Number
Fraud comes in all shapes and forms

Business Rules vs Machine Learning
Business Rules look for specific conditions or behaviors
• Business Rules are easily explained and validated
• Sample New Account Registration rule:
ML Models learn more general patterns by looking at lots of examples
• When fraudsters make small tweaks, the model still recognizes them as
suspicious since it’s unlike anything it has seen from legitimate
customers
• ML models are not just good at finding the risky patterns, they’re much
less brittle than rules
If IP_ADDRESS_LOCATION == [’Japan’] and CUST_ADDRESS_COUNTRY == [‘JAPAN’] and
CUSTOMER_PHONE_LOC == [‘Spain’] THEN Investigate
Prevention Detection

Supervised Machine Learning
• Data
• Inputs: “large” number of samples
• Predictors/independent variables/features
• Types: qualitative (categorical), quantitative (numerical)
• Labels
• Each sample needs a tag/class/value you want to
predict
• Types: numerical => regression, categorical =>
classification
• Algorithm
• Wide variety from simple (Linear Regression) to
complex (Deep Neural Networks)
MODEL = DATA + LABELS + ALGORITHM

Preparing Data for Machine Learning

Closing the loop

Fraud detection is difficult
$$$ billions lost
each year
Bad actors change
tactics often
Costly human
reviews
Embedded
detection logic
Fraud detection with ML is also difficult
ML experts costly,
hard to find
Generic models
underperform
Needle in a
haystack
Time-consuming
data transforms

Introducing Amazon Fraud Detector
A fraud detection service that makes it easy
for businesses to use machine learning to
detect online fraud in real-time, at scale.

Benefits of Amazon Fraud Detector
• Build high quality fraud detection ML models faster
• Stop bad actors at the door
• Built-in online fraud expertise
• Give fraud teams more control

Detect common types of online fraud
Designed to help companies detect common types of online fraud
Examples:
• New account fraud
• Online payment fraud (coming soon)
• Guest checkout fraud
• ‘Try Before You Buy’ + post-paid online service abuse

How it works

Generating Fraud Predictions
Guest Checkout: Purchase
IP: 1.23.123.123
email: joe@example.com
Payment: Bank123
…
Fraud Detector returns:
Outcome: Approved
ML Score: 160
Purchase Approved
Call service with:
IP: 1.23.123.123
email: joe@example.com
Payment: Bank123
…
AWS Cloud

Key features
Pre-built
fraud
detection
model
templates
Automatic
creation of
custom fraud
detection
models
Models learn
from past
attempts to
defraud
Amazon
Amazon
SageMaker
integration
Interface to
review past
events and
detection
logic

Automated model building
1 2 4 5
Training data in
Amazon S3
63

AutoML: Does one size fit all?
General AutoML
• Domain agnostic
• Works ok/well with any dataset
• Transformation techniques are general
• Example service: SageMaker AutoPilot
Specialized AutoML
• Domain specific (fraud, forecasting, recommendations)
• Works very well with specific dataset
• Transformation techniques are customized (domain expertise applied)
• Example service: Amazon Fraud Detector
Performance on example fraud dataset
• SageMaker AutoPilot = 0.62 AUC
• Amazon Fraud Detector = 0.90 AUC
order_amt ip_address email_address card_bin phone_number user_agent event_timestamp billing_address shipping_address is_fraud
351 20.194.8.124 fake_ramirezsally@gmail.com 482389 001-560-879-8102x87564 Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_4 like Mac OS X) AppleWebKit/536.1 (KHTML, like Gecko) FxiOS/11.9i8455.0 Mobile/92Y979 Safa11/18/2018 17:16 32695 Murphy 7271 Michael 0
413 192.10.132.144 fake_trevorleon@gmail.com 408848 (716)960-5930x917 Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_8_1; rv:1.9.4.20) Gecko/2013-01-14 11:19:58 Firefox/14.02/10/2019 8:08 750 Anthony 242 Wise Station 1
517 192.190.200.13 fake_sarah59@gmail.com 351330 150-179-7560x4176 Mozilla/5.0 (Android 4.0.2; Mobile; rv:55.0) Gecko/55.0 Firefox/55.0 12/2/2018 5:51
931 Martinez
Drives Suite
3414 Johnathan
Landing 0
482 203.0.97.150 fake_fwebb@hotmail.com 389643 620.853.0627x27001 Mozilla/5.0 (X11; Linux i686; rv:1.9.5.20) Gecko/2018-06-13 16:04:51 Firefox/6.08/24/2019 0:24
29910 Brooks
Shore Suite 343
6751 Villarreal
Port 0

Data Enrichment
Amazon Fraud Detector enriches the customer’s raw dataset using 3rd party data sources
as well as models trained on Amazon’s fraud data.
Raw IP
Address
1.2.3.4
2.3.4.5
3.4.5.6
…
15+ IP Enrichments
Country ISP City
US CenturyLink Seattle
JP Asahi Net Tokyo
FR Orange Paris
… … …
AMZN IP risk scores
AWS Amazon …
85 45 …
26 74 …
35 41 …
… … …

Demo: Amazon Fraud Detector

What you need to get started
• Sign up for the Preview: https://aws.amazon.com/frauddetector/
• Gather your historical data to use with a model template
• Provide at least 10K historical online events (the more, the better)
• Include examples of both fraud and legit events
• Save data in CSV format

65© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Gracias
Síguenos en twitter: https://twitter.com/awscloud_es
Webinars y eventos: https://aws.amazon.com/es/about-aws/events/eventos-es/
Contacto: https://aws.amazon.com/es/contact-us/
Noticias y novedades: https://aws.amazon.com/es/new
No olvides rellenar la encuesta
para ayudarnos a mejorar
Javier Ramirez
AWS Developer Advocate
@supercoco9

Recomendaciones, predicciones y detección de fraude usando servicios de inteligencia artificial

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Recomendaciones, predicciones y detección de fraude usando servicios de inteligencia artificial

Similar to Recomendaciones, predicciones y detección de fraude usando servicios de inteligencia artificial (20)

More from javier ramirez

More from javier ramirez (20)

Recently uploaded

Recently uploaded (20)

Recomendaciones, predicciones y detección de fraude usando servicios de inteligencia artificial