SlideShare a Scribd company logo
1 of 51
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Brad Bebee, AWS, Principal Product Manager, Amazon Neptune
January 22, 2018
Deep Dive on Amazon Neptune
Fast, reliable graph database built for the cloud
B r a d B e b e e , A W S , P r i n c i p a l P M , A m a z o n N e p t u n e
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
• Building Applications on Highly Connected Data
• Types of Graphs and How to Query Them
• Property Graph and Apache TinkerPop Friend Recommendation
Example
• RDF Knowledge Graph Example
• Behind Amazon Neptune’s Fully managed, Enterprise-ready
Features
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
HIGHLY CONNECTED DATA
Retail Fraud DetectionRestaurant RecommendationsSocial Networks
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
U SE C ASES FOR H IGH LY C ON N EC TED D ATA
Social Networking
Life
Sciences
Network & IT OperationsFraud Detection
Recommendations Knowledge Graphs
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RECOMMENDATIONS BASED ON RELATIONSHIPS
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
KNOWLEDGE GRAPH APPLICATIONS
What museums should Alice
visit while in Paris?
Who painted the Mona Lisa?
What artists have paintings
in The Louvre?
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
N AVIGATE A W EB OF GLOBAL TAX POLIC IES
“Our customers are increasingly required to navigate a complex web of global tax policies and
regulations. We need an approach to model the sophisticated corporate structures of our
largest clients and deliver an end-to-end tax solution. We use a microservices architecture
approach for our platforms and are beginning to leverage Amazon Neptune as a graph-based
system to quickly create links within the data.”
said Tim Vanderham, chief technology officer, Thomson Reuters Tax & Accounting
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Challenges Building Apps with Highly Connected DataRELATIONAL DATABASE CHALLENGES
BUILDING APPS WITH HIGHLY CONNECTED
DATA
Unnatural for
querying graph
Inefficient
graph processing
Rigid schema inflexible
for changing data
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
DIFFERENT APPROACHES FOR HIGHLY
CONNECTED DATA
Purpose-built for a business process
Purpose-built to answer questions about
relationships
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A GR APH D ATABASE IS OPTIMIZED FOR EFFIC IEN T
STOR AGE AN D R ETR IEVAL OF H IGH LY C ON N EC TED
D ATA.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open Source Apache TinkerPop™
Gremlin Traversal Language
W3C Standard
SPARQL Query Language
R E S O U R C E
D E S C R I P T I O N
F R A M E W O R K ( R D F )
P R O P E RT Y
G R A P H
LEADING GRAPH MODELS AND
FRAMEWORKS
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CHALLENGES OF EXISTING GRAPH
DATABASES
Difficult to maintain
high availability
Difficult to scale
Limited support for
open standards
Too expensive
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON NEPTUNE
F u l l y m a n a g e d g r a p h d a t a b a s e
FAST RELIABLE OPEN
Query billions of
relationships with
millisecond latency
6 replicas of your data
across 3 AZs with full
backup and restore
Build powerful queries
easily with Gremlin and
SPARQL
Supports Apache
TinkerPop & W3C RDF
graph models
EASY
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON NEPTUNE HIGH LEVEL
ARCHITECTURE
Bulk load
from S3
Database
Mgmt.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Types of Graphs and How to
Query Them
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
PROPERTY GRAPH
A property graph is a set of vertices and edges with
respective properties (i.e. key/value pairs)
• Vertex represents entities/domains
• Edge represents directional relationship
between vertices.
• Each edge has a label that denotes the
type of relationship
• Each vertex & edge has a unique identifier
• Vertex and edges can have properties
• Properties express non-relational information about the
vertices and edges
FRIENDname:
Bill
name:
Sarah
UserUser
Since 11/29/16
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CREATING A TINKERPOP GRAPH
//Connect to Neptune and receive a remote graph, g.
user1 = g.addVertex (id, 1, label, "User", "name", "Bill");
user2 = g.addVertex (id, 2, label, "User", "name", "Sarah");
...
user1.addEdge("FRIEND", user2, id, 21);
Gremlin (Apache TinkerPop 3.3)
FRIEND
name:
Bill
name:
Sarah
User
User
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RDF GRAPHS
• RDF Graphs are described as a collection of triples: subject, predicate, and object.
• Internationalized Resource Identifiers (IRIs) uniquely identify subjects.
• The Object can be an IRI or Literal.
• A Literal in RDF is like a property and RDF supports the XML data types.
• When the Object is an IRI, it forms an “Edge” in the graph.
<http://www.socialnetwork.com/person#1>
rdf:type contacts:User;
contact:name: ”Bill” .
subject
predicate
Object (literal)
name:
Bill
User
<http://www.socialnetwork.com/person#1>IRI
<http://www.socialnetwork.com/person#1>
contacts:friend
<http://www.socialnetwork.com/person#2> .
subject
predicate
Object (IRI)
FRIEND
#1 2#2
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“THERE’S NO TROUBLE WITH TRIPLES”:
RDF EXAMPLE
@prefix contacts: <http://www.socialnetwork.com/people#>.
<http://www.socialnetwork.com/person#1>
rdf:type contacts:User;
contact:name: ”Bill” .
<http://www.socialnetwork.com/person#1>
contacts:friend <http://www.socialnetwork.com/person#2> .
<http://www.socialnetwork.com/person#2>
rdf:type contacts:User;
contact:name: ”Sarah” .
RDF
(Turtle Serialization)
FRIEND
name:
Bill
name:
Sarah
User
User
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GRAPH VS. RELATIONAL DATABASE
MODELING.
* Source : http://www.playnexacro.com/index.html#show:article
Relational model Graph model subset
CompanyName:
Acme
…
Customers
OrderDate:
8/1/2017
…
Order
PURCHASED
HAS_DETAILS
UnitPrice:
$179.99
…
Order
DetailsProductName:
“Echo”
…
Product
HAS_PRODUCT
CompanyName:
“Amazon”
…
SupplierSUPPLIES
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SQL RELATIONAL DATABASE QUERY
SELECT distinct c.CompanyName
FROM customers AS c
JOIN orders AS o ON /* Join the customer from the order */
(c.CustomerID = o.CustomerID)
JOIN order_details AS od /* Join the order details from the order
*/
ON (o.OrderID = od.OrderID)
JOIN products as p /* Join the products from the order details
*/
ON (od.ProductID = p.ProductID)
WHERE p.ProductName = ’Echo'; /* Find the product named ‘Echo’ */
Find the name of companies that purchased the ‘Echo’.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SPARQL DECLARATIVE GRAPH QUERY
PREFIX sales_db: <http://sales.widget.com/>
SELECT distinct ?comp_name WHERE {
?customer sales_db:HAS_ORDER ?order ; #customer graph pattern
sales_db:CompanyName ?comp_name .
?order sales_db:HAS_DETAILS ?order_d . #orders graph pattern
?order_d sales_db:HAS_PRODUCT ?product . #order details graph
pattern
?product sales_db:ProductName “Echo” . #products graph pattern
}
* Source : http://www.playnexacro.com/index.html#show:article
Find the name of companies that purchased the ‘Echo’.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GREMLIN IMPERATIVE GRAPH TRAVERSAL
/* All products named ”Echo” */
g.V().hasLabel(‘Product’).has('name',’Echo')
.in(’HAS_PRODUCT') /* Traverse to order details */
.in(‘HAS_DETAILS’) /* Traverse to order */
.in(’HAS_ORDER’) /* Traverse to Customer */
.values(’CompanyName’).dedup() /* Unique Company Name */
Find the name of companies that purchased the ‘Echo’.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Property Graph and Apache
TinkerPop Social Network Friend
Recommendation Example
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TRIADIC CLOSURE – CLOSING TRIANGLES
FRIEND
FRIEND
Terry
Bill
Sarah
FRIEND
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RECOMMENDING NEW CONNECTIONS
Terry
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IMMEDIATE FRIENDSHIPS
FRIEND
Terry
Bill
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
MEANS AND MOTIVE
FRIEND
FRIEND
Terry
Bill
Sarah
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RECOMMENDATION
FRIEND
FRIEND
Terry
Bill
Sarah
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recommend New Connections
g = graph.traversal()
g.V().has('name','Terry').as('user').
both('FRIEND').aggregate('friends').
both('FRIEND').
where(neq('user')).where(neq('friends')).
groupCount().by('name').
order(local).by(values, decr)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FIND TERRY
g = graph.traversal()
g.V().has('name','Terry').as('user').
both('FRIEND').aggregate('friends').
both('FRIEND').
where(neq('user')).where(neq('friends')).
groupCount().by('name').
order(local).by(values, decr)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FIND TERRY’S FRIENDS
g = graph.traversal()
g.V().has('name','Terry').as('user').
both('FRIEND').aggregate('friends').
both('FRIEND').
where(neq('user')).where(neq('friends')).
groupCount().by('name').
order(local).by(values, decr)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AND THE FRIENDS OF THOSE FRIENDS
g = graph.traversal()
g.V().has('name','Terry').as('user').
both('FRIEND').aggregate('friends').
both('FRIEND').
where(neq('user')).where(neq('friends')).
groupCount().by('name').
order(local).by(values, decr)
user
friend
fof
FRIEND
FRIEND
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
...WHO AREN’T TERRY AND AREN’T
FRIENDS WITH TERRYg = graph.traversal()
g.V().has('name','Terry').as('user').
both('FRIEND').aggregate('friends').
both('FRIEND').
where(neq('user')).where(neq('friends')).
groupCount().by('name').
order(local).by(values, decr)
user
friend
fof
X
FRIEND
FRIEND
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RDF Knowledge Graph
Example
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
URIs AS GLOBALLY UNIQUE
IDENTIFIERS
URIs to identify nodes and edge labels
<https://permid.org/1-4295902158>
=> identifies the company “Netflix Inc”
organization:isIncorporatedIn1
=> identifies the relationship “is incorporated in”
<http://sws.geonames.org/6252001/>
=> identifies country “USA”
1 This is a shortcut for
<http://permid.org/ontology/organization/isIncorporatedIn>.
RDF uses XML prefix notation, where the prefix organization is a
shortcut for <http://permid.org/ontology/organization/>.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RDF GRAPH: COLLECTION OF TRIPLES
(1)
# Every edge in the RDF graph is represented as
# a (subject, predicate, object) triple
<https://permid.org/1-4295902158>
organization:isIncorporatedIn
<http://sws.geonames.org/6252001/> .
subject
predicate
object
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RDF GRAPH: COLLECTION OF TRIPLES
(2)
# Every edge in the RDF graph is represented as
# a (subject, predicate, object) triple
<https://permid.org/1-4295902158>
organization:isIncorporatedIn
<http://sws.geonames.org/6252001/> .
<https://permid.org/1-4295902158>
vcard:organization-name
"Netflix Inc" .
subject
predicate
object
(URI)
subject
predicate
object
(literal)
Literals are “sinks” in the graph. They do not
have any outgoing edges.
RDF supports strings and other XML datatypes
(bool, integer, dates, floats, doubles, …)
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RDF GRAPH: COLLECTION OF TRIPLES
(3)
# Every edge in the RDF graph is represented as
# a (subject, predicate, object) triple
<https://permid.org/1-4295902158>
organization:isIncorporatedIn
<http://sws.geonames.org/6252001/> .
<https://permid.org/1-4295902158>
vcard:organization-name
"Netflix Inc" .
<https://permid.org/1-4295902158>
organization:hasRegisteredPhoneNumber
"13026587581" .
<http://sws.geonames.org/6252001/>
iso:countryCode
”US” .
Same URI used as both subject and object, depending on
whether we represent outgoing vs. incoming RDF edges.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THE BENEFIT OF URIs: LINKED DATA
Linking across datasets by referencing globally unique URIs
GeoNames
Wikidata
PermID
Example: PermID (re)uses <http://sws.geonames.org/6252001/>
as a global Identifier for the USA, which is an identifier rooted in GeoNames.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
QUERYING RDF USING SPARQL (1)
?name
Variables are
prefixed with “?”.
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
QUERYING RDF USING SPARQL (2)
?property
?property
?property
?node ?node
?node
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
QUERYING RDF USING SPARQL (3)
?org
?phone
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Behind Amazon Neptune’s Fully
Managed, Enterprise Ready
Features
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fully Managed Service
Easily configurable via the Console
Multi-AZ High Availability, ACID
Support for up to 15 read replicas
Supports Encryption at rest
Supports Encryption in transit (TLS)
Backup and Restore, Point-in-time
Recovery
B E N E F I T S
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Secure deployment in a VPC
• Increased availability through
deployment in two subnets in two
different Availability Zones (AZs)
• Cluster volume always spans three
AZ to provide durable storage
• See the Amazon Neptune
Documentation for VPC setup details
AMAZON NEPTUNE: VPC DEPLOYMENT
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON NEPTUNE READ REPLICAS
Availability
• Failing database nodes are
automatically detected and replaced
• Failing database processes are
automatically detected and recycled
• Replicas are automatically promoted to
primary if needed (failover)
• Customer specifiable fail-over order
AZ 1 AZ 3AZ 2
Primary
Node
Primary
Node
Primary
Master
Node
Primary
Node
Primary
Node
Read
Replica
Primary
Node
Primary
Node
Read
Replica
Cluster
and
Instance
Monitoring
Performance
• Customer applications can scale out read
traffic across read replicas
• Read balancing across read replicas
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON NEPTUNE FAILOVER TIMES ARE
TYPICALLY < 30 SECONDS
Replica-Aware App Running
Failure Detection DNS Propagation
Recovery
Database
Failure
1 5 - 2 0 s e c 3 - 1 0 s e c
App
Running
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Questions
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AMAZON NEPTUNE IS AVAILABLE FOR
PREVIEW
• Preview Sign-up
• https://pages.awscloud.com/NeptunePreview.html
• Amazon Neptune Web Page
• https://aws.amazon.com/neptune/
• Amazon Neptune Documentation
• https://docs.aws.amazon.com/neptune/latest/userguide
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!

More Related Content

What's hot

AWS Neptune - A Fast and reliable Graph Database Built for the Cloud
AWS Neptune - A Fast and reliable Graph Database Built for the CloudAWS Neptune - A Fast and reliable Graph Database Built for the Cloud
AWS Neptune - A Fast and reliable Graph Database Built for the CloudAmazon Web Services
 
Graph Gurus 15: Introducing TigerGraph 2.4
Graph Gurus 15: Introducing TigerGraph 2.4 Graph Gurus 15: Introducing TigerGraph 2.4
Graph Gurus 15: Introducing TigerGraph 2.4 TigerGraph
 
AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례
AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례
AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례Amazon Web Services Korea
 
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017Amazon Web Services
 
Build a Serverless Application using GraphQL & AWS AppSync
Build a Serverless Application using GraphQL & AWS AppSyncBuild a Serverless Application using GraphQL & AWS AppSync
Build a Serverless Application using GraphQL & AWS AppSyncAmazon Web Services
 
Introduction to GraphQL: Mobile Week SF
Introduction to GraphQL: Mobile Week SFIntroduction to GraphQL: Mobile Week SF
Introduction to GraphQL: Mobile Week SFAmazon Web Services
 
Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...
Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...
Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...Amazon Web Services
 
Your Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph StrategyYour Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph StrategyNeo4j
 
Modern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph TechnologyModern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph TechnologyNeo4j
 
Beyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFramesBeyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFramesDatabricks
 
GraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLGraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLSpark Summit
 
ACM DBPL Keynote: The Graph Traversal Machine and Language
ACM DBPL Keynote: The Graph Traversal Machine and LanguageACM DBPL Keynote: The Graph Traversal Machine and Language
ACM DBPL Keynote: The Graph Traversal Machine and LanguageMarko Rodriguez
 
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축Sungmin Kim
 
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...Amazon Web Services
 
Introducing AWS AppSync: serverless data driven apps with real-time and offli...
Introducing AWS AppSync: serverless data driven apps with real-time and offli...Introducing AWS AppSync: serverless data driven apps with real-time and offli...
Introducing AWS AppSync: serverless data driven apps with real-time and offli...Amazon Web Services
 
Resiliency-and-Availability-Design-Patterns-for-the-Cloud
Resiliency-and-Availability-Design-Patterns-for-the-CloudResiliency-and-Availability-Design-Patterns-for-the-Cloud
Resiliency-and-Availability-Design-Patterns-for-the-CloudAmazon Web Services
 
Accelerate Your Cloud Migration Journey.pdf
Accelerate Your Cloud Migration Journey.pdfAccelerate Your Cloud Migration Journey.pdf
Accelerate Your Cloud Migration Journey.pdfAmazon Web Services
 

What's hot (20)

AWS Neptune - A Fast and reliable Graph Database Built for the Cloud
AWS Neptune - A Fast and reliable Graph Database Built for the CloudAWS Neptune - A Fast and reliable Graph Database Built for the Cloud
AWS Neptune - A Fast and reliable Graph Database Built for the Cloud
 
Graph Gurus 15: Introducing TigerGraph 2.4
Graph Gurus 15: Introducing TigerGraph 2.4 Graph Gurus 15: Introducing TigerGraph 2.4
Graph Gurus 15: Introducing TigerGraph 2.4
 
MariaDB Temporal Tables
MariaDB Temporal TablesMariaDB Temporal Tables
MariaDB Temporal Tables
 
AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례
AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례
AWS Summit Seoul 2023 | 스타트업의 서버리스 기반 SaaS 데이터 처리 및 데이터웨어하우스 구축 사례
 
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017How to build a data lake with aws glue data catalog (ABD213-R)  re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
 
Build a Serverless Application using GraphQL & AWS AppSync
Build a Serverless Application using GraphQL & AWS AppSyncBuild a Serverless Application using GraphQL & AWS AppSync
Build a Serverless Application using GraphQL & AWS AppSync
 
Introduction to GraphQL: Mobile Week SF
Introduction to GraphQL: Mobile Week SFIntroduction to GraphQL: Mobile Week SF
Introduction to GraphQL: Mobile Week SF
 
Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...
Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...
Work Backwards to Your Graph Data Model & Queries with Amazon Neptune (DAT330...
 
Dynamodb ppt
Dynamodb pptDynamodb ppt
Dynamodb ppt
 
Your Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph StrategyYour Roadmap for An Enterprise Graph Strategy
Your Roadmap for An Enterprise Graph Strategy
 
Modern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph TechnologyModern Data Challenges require Modern Graph Technology
Modern Data Challenges require Modern Graph Technology
 
Beyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFramesBeyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFrames
 
GraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLGraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQL
 
ACM DBPL Keynote: The Graph Traversal Machine and Language
ACM DBPL Keynote: The Graph Traversal Machine and LanguageACM DBPL Keynote: The Graph Traversal Machine and Language
ACM DBPL Keynote: The Graph Traversal Machine and Language
 
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
 
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
 
Introducing AWS AppSync: serverless data driven apps with real-time and offli...
Introducing AWS AppSync: serverless data driven apps with real-time and offli...Introducing AWS AppSync: serverless data driven apps with real-time and offli...
Introducing AWS AppSync: serverless data driven apps with real-time and offli...
 
ElastiCache & Redis
ElastiCache & RedisElastiCache & Redis
ElastiCache & Redis
 
Resiliency-and-Availability-Design-Patterns-for-the-Cloud
Resiliency-and-Availability-Design-Patterns-for-the-CloudResiliency-and-Availability-Design-Patterns-for-the-Cloud
Resiliency-and-Availability-Design-Patterns-for-the-Cloud
 
Accelerate Your Cloud Migration Journey.pdf
Accelerate Your Cloud Migration Journey.pdfAccelerate Your Cloud Migration Journey.pdf
Accelerate Your Cloud Migration Journey.pdf
 

Similar to Deep Dive on Amazon Neptune - AWS Online Tech Talks

Graph and Amazon Neptune - Bill Baldwin
Graph and Amazon Neptune - Bill BaldwinGraph and Amazon Neptune - Bill Baldwin
Graph and Amazon Neptune - Bill BaldwinAmazon Web Services
 
Graph & Neptune: Database Week San Francisco
Graph & Neptune: Database Week San FranciscoGraph & Neptune: Database Week San Francisco
Graph & Neptune: Database Week San FranciscoAmazon Web Services
 
Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018
Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018
Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018Amazon Web Services
 
Going Graph With Amazon Neptune - AWS Summit Sydney 2018
Going Graph With Amazon Neptune - AWS Summit Sydney 2018Going Graph With Amazon Neptune - AWS Summit Sydney 2018
Going Graph With Amazon Neptune - AWS Summit Sydney 2018Amazon Web Services
 
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...Amazon Web Services
 
Understanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web SummitUnderstanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web SummitAmazon Web Services
 
Non-Relational Revolution: Database Week SF
Non-Relational Revolution: Database Week SFNon-Relational Revolution: Database Week SF
Non-Relational Revolution: Database Week SFAmazon Web Services
 
Let me graph that for you - Amazon Neptune
Let me graph that for you - Amazon NeptuneLet me graph that for you - Amazon Neptune
Let me graph that for you - Amazon NeptuneAmazon Web Services
 
What’s the big deal with Graph Databases?
What’s the big deal with Graph Databases?What’s the big deal with Graph Databases?
What’s the big deal with Graph Databases?Daniel Zivkovic
 
NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017
NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017
NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017Amazon Web Services
 
Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...
Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...
Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...Amazon Web Services
 
Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018
Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018
Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018Amazon Web Services
 
Build and Innovate with a Modern Data Architecture
Build and Innovate with a Modern Data ArchitectureBuild and Innovate with a Modern Data Architecture
Build and Innovate with a Modern Data ArchitectureAmazon Web Services
 

Similar to Deep Dive on Amazon Neptune - AWS Online Tech Talks (20)

Graph & Neptune
Graph & NeptuneGraph & Neptune
Graph & Neptune
 
Graph and Amazon Neptune - Bill Baldwin
Graph and Amazon Neptune - Bill BaldwinGraph and Amazon Neptune - Bill Baldwin
Graph and Amazon Neptune - Bill Baldwin
 
Graph and Amazon Neptune
Graph and Amazon NeptuneGraph and Amazon Neptune
Graph and Amazon Neptune
 
Graph and Neptune
Graph and NeptuneGraph and Neptune
Graph and Neptune
 
Graph and Amazon Neptune
Graph and Amazon NeptuneGraph and Amazon Neptune
Graph and Amazon Neptune
 
Graph & Neptune: Database Week San Francisco
Graph & Neptune: Database Week San FranciscoGraph & Neptune: Database Week San Francisco
Graph & Neptune: Database Week San Francisco
 
Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018
Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018
Connecting the Unconnected using GraphDB - Tel Aviv Summit 2018
 
Going Graph With Amazon Neptune - AWS Summit Sydney 2018
Going Graph With Amazon Neptune - AWS Summit Sydney 2018Going Graph With Amazon Neptune - AWS Summit Sydney 2018
Going Graph With Amazon Neptune - AWS Summit Sydney 2018
 
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
Connecting the dots - How Amazon Neptune and Graph Databases can transform yo...
 
Understanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web SummitUnderstanding Graph Databases: AWS Developer Workshop at Web Summit
Understanding Graph Databases: AWS Developer Workshop at Web Summit
 
Non-Relational Revolution
Non-Relational RevolutionNon-Relational Revolution
Non-Relational Revolution
 
Non-Relational Revolution: Database Week SF
Non-Relational Revolution: Database Week SFNon-Relational Revolution: Database Week SF
Non-Relational Revolution: Database Week SF
 
Let me graph that for you - Amazon Neptune
Let me graph that for you - Amazon NeptuneLet me graph that for you - Amazon Neptune
Let me graph that for you - Amazon Neptune
 
What’s the big deal with Graph Databases?
What’s the big deal with Graph Databases?What’s the big deal with Graph Databases?
What’s the big deal with Graph Databases?
 
NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017
NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017
NEW LAUNCH! Deep dive on Amazon Neptune - DAT318 - re:Invent 2017
 
Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...
Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...
Building a Data Lake for Your Enterprise, ft. Sysco (STG309) - AWS re:Invent ...
 
Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018
Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018
Amazon Cloud Directory Deep Dive (DAT364) - AWS re:Invent 2018
 
Neptune webinar AWS
Neptune webinar AWS Neptune webinar AWS
Neptune webinar AWS
 
Build and Innovate with a Modern Data Architecture
Build and Innovate with a Modern Data ArchitectureBuild and Innovate with a Modern Data Architecture
Build and Innovate with a Modern Data Architecture
 
BI & Analytics
BI & AnalyticsBI & Analytics
BI & Analytics
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Deep Dive on Amazon Neptune - AWS Online Tech Talks

  • 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Brad Bebee, AWS, Principal Product Manager, Amazon Neptune January 22, 2018 Deep Dive on Amazon Neptune Fast, reliable graph database built for the cloud B r a d B e b e e , A W S , P r i n c i p a l P M , A m a z o n N e p t u n e
  • 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • Building Applications on Highly Connected Data • Types of Graphs and How to Query Them • Property Graph and Apache TinkerPop Friend Recommendation Example • RDF Knowledge Graph Example • Behind Amazon Neptune’s Fully managed, Enterprise-ready Features
  • 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. HIGHLY CONNECTED DATA Retail Fraud DetectionRestaurant RecommendationsSocial Networks
  • 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. U SE C ASES FOR H IGH LY C ON N EC TED D ATA Social Networking Life Sciences Network & IT OperationsFraud Detection Recommendations Knowledge Graphs
  • 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RECOMMENDATIONS BASED ON RELATIONSHIPS
  • 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. KNOWLEDGE GRAPH APPLICATIONS What museums should Alice visit while in Paris? Who painted the Mona Lisa? What artists have paintings in The Louvre?
  • 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. N AVIGATE A W EB OF GLOBAL TAX POLIC IES “Our customers are increasingly required to navigate a complex web of global tax policies and regulations. We need an approach to model the sophisticated corporate structures of our largest clients and deliver an end-to-end tax solution. We use a microservices architecture approach for our platforms and are beginning to leverage Amazon Neptune as a graph-based system to quickly create links within the data.” said Tim Vanderham, chief technology officer, Thomson Reuters Tax & Accounting
  • 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Challenges Building Apps with Highly Connected DataRELATIONAL DATABASE CHALLENGES BUILDING APPS WITH HIGHLY CONNECTED DATA Unnatural for querying graph Inefficient graph processing Rigid schema inflexible for changing data
  • 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DIFFERENT APPROACHES FOR HIGHLY CONNECTED DATA Purpose-built for a business process Purpose-built to answer questions about relationships
  • 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A GR APH D ATABASE IS OPTIMIZED FOR EFFIC IEN T STOR AGE AN D R ETR IEVAL OF H IGH LY C ON N EC TED D ATA.
  • 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Open Source Apache TinkerPop™ Gremlin Traversal Language W3C Standard SPARQL Query Language R E S O U R C E D E S C R I P T I O N F R A M E W O R K ( R D F ) P R O P E RT Y G R A P H LEADING GRAPH MODELS AND FRAMEWORKS
  • 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CHALLENGES OF EXISTING GRAPH DATABASES Difficult to maintain high availability Difficult to scale Limited support for open standards Too expensive
  • 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AMAZON NEPTUNE F u l l y m a n a g e d g r a p h d a t a b a s e FAST RELIABLE OPEN Query billions of relationships with millisecond latency 6 replicas of your data across 3 AZs with full backup and restore Build powerful queries easily with Gremlin and SPARQL Supports Apache TinkerPop & W3C RDF graph models EASY
  • 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AMAZON NEPTUNE HIGH LEVEL ARCHITECTURE Bulk load from S3 Database Mgmt.
  • 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Types of Graphs and How to Query Them
  • 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. PROPERTY GRAPH A property graph is a set of vertices and edges with respective properties (i.e. key/value pairs) • Vertex represents entities/domains • Edge represents directional relationship between vertices. • Each edge has a label that denotes the type of relationship • Each vertex & edge has a unique identifier • Vertex and edges can have properties • Properties express non-relational information about the vertices and edges FRIENDname: Bill name: Sarah UserUser Since 11/29/16
  • 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CREATING A TINKERPOP GRAPH //Connect to Neptune and receive a remote graph, g. user1 = g.addVertex (id, 1, label, "User", "name", "Bill"); user2 = g.addVertex (id, 2, label, "User", "name", "Sarah"); ... user1.addEdge("FRIEND", user2, id, 21); Gremlin (Apache TinkerPop 3.3) FRIEND name: Bill name: Sarah User User
  • 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RDF GRAPHS • RDF Graphs are described as a collection of triples: subject, predicate, and object. • Internationalized Resource Identifiers (IRIs) uniquely identify subjects. • The Object can be an IRI or Literal. • A Literal in RDF is like a property and RDF supports the XML data types. • When the Object is an IRI, it forms an “Edge” in the graph. <http://www.socialnetwork.com/person#1> rdf:type contacts:User; contact:name: ”Bill” . subject predicate Object (literal) name: Bill User <http://www.socialnetwork.com/person#1>IRI <http://www.socialnetwork.com/person#1> contacts:friend <http://www.socialnetwork.com/person#2> . subject predicate Object (IRI) FRIEND #1 2#2
  • 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “THERE’S NO TROUBLE WITH TRIPLES”: RDF EXAMPLE @prefix contacts: <http://www.socialnetwork.com/people#>. <http://www.socialnetwork.com/person#1> rdf:type contacts:User; contact:name: ”Bill” . <http://www.socialnetwork.com/person#1> contacts:friend <http://www.socialnetwork.com/person#2> . <http://www.socialnetwork.com/person#2> rdf:type contacts:User; contact:name: ”Sarah” . RDF (Turtle Serialization) FRIEND name: Bill name: Sarah User User
  • 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GRAPH VS. RELATIONAL DATABASE MODELING. * Source : http://www.playnexacro.com/index.html#show:article Relational model Graph model subset CompanyName: Acme … Customers OrderDate: 8/1/2017 … Order PURCHASED HAS_DETAILS UnitPrice: $179.99 … Order DetailsProductName: “Echo” … Product HAS_PRODUCT CompanyName: “Amazon” … SupplierSUPPLIES
  • 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SQL RELATIONAL DATABASE QUERY SELECT distinct c.CompanyName FROM customers AS c JOIN orders AS o ON /* Join the customer from the order */ (c.CustomerID = o.CustomerID) JOIN order_details AS od /* Join the order details from the order */ ON (o.OrderID = od.OrderID) JOIN products as p /* Join the products from the order details */ ON (od.ProductID = p.ProductID) WHERE p.ProductName = ’Echo'; /* Find the product named ‘Echo’ */ Find the name of companies that purchased the ‘Echo’.
  • 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SPARQL DECLARATIVE GRAPH QUERY PREFIX sales_db: <http://sales.widget.com/> SELECT distinct ?comp_name WHERE { ?customer sales_db:HAS_ORDER ?order ; #customer graph pattern sales_db:CompanyName ?comp_name . ?order sales_db:HAS_DETAILS ?order_d . #orders graph pattern ?order_d sales_db:HAS_PRODUCT ?product . #order details graph pattern ?product sales_db:ProductName “Echo” . #products graph pattern } * Source : http://www.playnexacro.com/index.html#show:article Find the name of companies that purchased the ‘Echo’.
  • 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GREMLIN IMPERATIVE GRAPH TRAVERSAL /* All products named ”Echo” */ g.V().hasLabel(‘Product’).has('name',’Echo') .in(’HAS_PRODUCT') /* Traverse to order details */ .in(‘HAS_DETAILS’) /* Traverse to order */ .in(’HAS_ORDER’) /* Traverse to Customer */ .values(’CompanyName’).dedup() /* Unique Company Name */ Find the name of companies that purchased the ‘Echo’.
  • 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Property Graph and Apache TinkerPop Social Network Friend Recommendation Example
  • 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TRIADIC CLOSURE – CLOSING TRIANGLES FRIEND FRIEND Terry Bill Sarah FRIEND
  • 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RECOMMENDING NEW CONNECTIONS Terry
  • 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IMMEDIATE FRIENDSHIPS FRIEND Terry Bill
  • 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MEANS AND MOTIVE FRIEND FRIEND Terry Bill Sarah
  • 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RECOMMENDATION FRIEND FRIEND Terry Bill Sarah
  • 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Recommend New Connections g = graph.traversal() g.V().has('name','Terry').as('user'). both('FRIEND').aggregate('friends'). both('FRIEND'). where(neq('user')).where(neq('friends')). groupCount().by('name'). order(local).by(values, decr)
  • 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. FIND TERRY g = graph.traversal() g.V().has('name','Terry').as('user'). both('FRIEND').aggregate('friends'). both('FRIEND'). where(neq('user')).where(neq('friends')). groupCount().by('name'). order(local).by(values, decr)
  • 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. FIND TERRY’S FRIENDS g = graph.traversal() g.V().has('name','Terry').as('user'). both('FRIEND').aggregate('friends'). both('FRIEND'). where(neq('user')).where(neq('friends')). groupCount().by('name'). order(local).by(values, decr)
  • 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AND THE FRIENDS OF THOSE FRIENDS g = graph.traversal() g.V().has('name','Terry').as('user'). both('FRIEND').aggregate('friends'). both('FRIEND'). where(neq('user')).where(neq('friends')). groupCount().by('name'). order(local).by(values, decr) user friend fof FRIEND FRIEND
  • 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ...WHO AREN’T TERRY AND AREN’T FRIENDS WITH TERRYg = graph.traversal() g.V().has('name','Terry').as('user'). both('FRIEND').aggregate('friends'). both('FRIEND'). where(neq('user')).where(neq('friends')). groupCount().by('name'). order(local).by(values, decr) user friend fof X FRIEND FRIEND
  • 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RDF Knowledge Graph Example
  • 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. URIs AS GLOBALLY UNIQUE IDENTIFIERS URIs to identify nodes and edge labels <https://permid.org/1-4295902158> => identifies the company “Netflix Inc” organization:isIncorporatedIn1 => identifies the relationship “is incorporated in” <http://sws.geonames.org/6252001/> => identifies country “USA” 1 This is a shortcut for <http://permid.org/ontology/organization/isIncorporatedIn>. RDF uses XML prefix notation, where the prefix organization is a shortcut for <http://permid.org/ontology/organization/>.
  • 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RDF GRAPH: COLLECTION OF TRIPLES (1) # Every edge in the RDF graph is represented as # a (subject, predicate, object) triple <https://permid.org/1-4295902158> organization:isIncorporatedIn <http://sws.geonames.org/6252001/> . subject predicate object
  • 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RDF GRAPH: COLLECTION OF TRIPLES (2) # Every edge in the RDF graph is represented as # a (subject, predicate, object) triple <https://permid.org/1-4295902158> organization:isIncorporatedIn <http://sws.geonames.org/6252001/> . <https://permid.org/1-4295902158> vcard:organization-name "Netflix Inc" . subject predicate object (URI) subject predicate object (literal) Literals are “sinks” in the graph. They do not have any outgoing edges. RDF supports strings and other XML datatypes (bool, integer, dates, floats, doubles, …)
  • 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. RDF GRAPH: COLLECTION OF TRIPLES (3) # Every edge in the RDF graph is represented as # a (subject, predicate, object) triple <https://permid.org/1-4295902158> organization:isIncorporatedIn <http://sws.geonames.org/6252001/> . <https://permid.org/1-4295902158> vcard:organization-name "Netflix Inc" . <https://permid.org/1-4295902158> organization:hasRegisteredPhoneNumber "13026587581" . <http://sws.geonames.org/6252001/> iso:countryCode ”US” . Same URI used as both subject and object, depending on whether we represent outgoing vs. incoming RDF edges.
  • 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THE BENEFIT OF URIs: LINKED DATA Linking across datasets by referencing globally unique URIs GeoNames Wikidata PermID Example: PermID (re)uses <http://sws.geonames.org/6252001/> as a global Identifier for the USA, which is an identifier rooted in GeoNames.
  • 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. QUERYING RDF USING SPARQL (1) ?name Variables are prefixed with “?”.
  • 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. QUERYING RDF USING SPARQL (2) ?property ?property ?property ?node ?node ?node
  • 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. QUERYING RDF USING SPARQL (3) ?org ?phone
  • 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Behind Amazon Neptune’s Fully Managed, Enterprise Ready Features
  • 45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fully Managed Service Easily configurable via the Console Multi-AZ High Availability, ACID Support for up to 15 read replicas Supports Encryption at rest Supports Encryption in transit (TLS) Backup and Restore, Point-in-time Recovery B E N E F I T S
  • 46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Secure deployment in a VPC • Increased availability through deployment in two subnets in two different Availability Zones (AZs) • Cluster volume always spans three AZ to provide durable storage • See the Amazon Neptune Documentation for VPC setup details AMAZON NEPTUNE: VPC DEPLOYMENT
  • 47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AMAZON NEPTUNE READ REPLICAS Availability • Failing database nodes are automatically detected and replaced • Failing database processes are automatically detected and recycled • Replicas are automatically promoted to primary if needed (failover) • Customer specifiable fail-over order AZ 1 AZ 3AZ 2 Primary Node Primary Node Primary Master Node Primary Node Primary Node Read Replica Primary Node Primary Node Read Replica Cluster and Instance Monitoring Performance • Customer applications can scale out read traffic across read replicas • Read balancing across read replicas
  • 48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AMAZON NEPTUNE FAILOVER TIMES ARE TYPICALLY < 30 SECONDS Replica-Aware App Running Failure Detection DNS Propagation Recovery Database Failure 1 5 - 2 0 s e c 3 - 1 0 s e c App Running
  • 49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Questions
  • 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AMAZON NEPTUNE IS AVAILABLE FOR PREVIEW • Preview Sign-up • https://pages.awscloud.com/NeptunePreview.html • Amazon Neptune Web Page • https://aws.amazon.com/neptune/ • Amazon Neptune Documentation • https://docs.aws.amazon.com/neptune/latest/userguide
  • 51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!