• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
raph Databases with Neo4j – Emil Eifrem
 

raph Databases with Neo4j – Emil Eifrem

on

  • 879 views

Graph Databases with Neo4j

Graph Databases with Neo4j

Statistics

Views

Total Views
879
Views on SlideShare
742
Embed Views
137

Actions

Likes
0
Downloads
23
Comments
0

1 Embed 137

http://www.neotechnology.com 137

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    raph Databases with Neo4j – Emil Eifrem raph Databases with Neo4j – Emil Eifrem Presentation Transcript

    • 1Neo4j is Teh AwesomeGraph Database 101Graph Database 1011@emileifrem#neo4j
    • 2“So what’s a graph database?”2
    • 33“A traditional relational databasemay tell you the average age ofeveryone in this room...”
    • 44“... but a graph database will tellyou who is most likely tobuy you a beer!”
    • 5No. Srsly.๏ Nodes๏ Relationships๏ Properties5
    • 6How fast is it?6
    • 7How fast is it?๏ a sample social graph•with ~1,000 persons๏ average 50 friends per person๏ pathExists(a,b) limited to depth 4๏ caches warmed up to eliminate disk I/O# persons query timeRelational database 1,000 2000msNeo4j 1,000 2msNeo4j 1,000,000
    • 8How fast is it?๏ a sample social graph•with ~1,000 persons๏ average 50 friends per person๏ pathExists(a,b) limited to depth 4๏ caches warmed up to eliminate disk I/O# persons query timeRelational database 1,000 2000msNeo4j 1,000 2msNeo4j 1,000,000 2ms
    • 9So how do you query it?9
    • 1010Cypher(A) -[:LOVES]-> (B)LOVESAA BBGraph PatternsSTART A=node:person(name=“A”)MATCHRETURN B as loverASCII art
    • 11// step 1: find starting pointSTART andreas=node:persons(name = ‘Andreas’)// step 2: describe pattern and resultsSTART andreas=node:persons(name = ‘Andreas’)MATCH (andreas)-->()-->(foaf) RETURN foafExample: Finding Friends of Friends11(andreas)
    • 12Neo4j is a Graph Database๏ A Graph Database:•a Property Graph with Nodes, Relationships•and Properties on both•perfect for complex, highly connected data๏ A Graph Database:•reliable with real ACID Transactions•scalable: high availability clustering in Neo4j Enterprise•server with HTTP API, or embeddable on the JVM•high-performance with High-Availability (read scaling)12
    • 13Who’s using graphs today?13Accenture
    • 14So how do you get your hands on Neo4j?14๏ Option A: Download and install locally...•go to http://neo4j.org•click the shiny “Download Neo4j Now” button•expand the archive, read the readmes๏ Or B...
    • 15Graphs In The Cloud (BETA)// new to heroku? get help$ heroku help// create a new application$ heroku create intro-to-neo4j// add Neo4jheroku addons:add neo4j --app intro-to-neo4j// find out about the applicationheroku info --app intro-to-neo4j// find the Neo4j Webadminheroku config --app intro-to-neo4j// done trying it out? remove the applicationheroku destroy --app intro-to-neo4j15
    • 16I needs thy help๏ Cloud Devops Engineer (San Mateo, London or Malmö, SE)•Come help us build a world class graph database cloud platform!๏ Director of Community North America (San Mateo)•Head up our developer outreach and evangelism in NA๏ Developer Evangelist (San Mateo)•Preach graphs to silicon valley and the world16
    • 1717thank you!stay connected
    • Core IndustriesCore Industries& Use Cases:& Use Cases:Web / ISVWeb / ISVFinance &Finance &InsuranceInsuranceDatacom /Datacom /TelecomTelecomNetwork /Cloud MgmtNetwork /Cloud MgmtMDMMDMSocialSocialGeoGeoEarly Adopter Graph Database Segments
    • Core IndustriesCore Industries& Use Cases:& Use Cases:Web / ISVWeb / ISVFinance &Finance &InsuranceInsuranceDatacom /Datacom /TelecomTelecomNetwork /Cloud MgmtNetwork /Cloud MgmtMDMMDMSocialSocialGeoGeoCore IndustriesCore Industries& Use Cases:& Use Cases:Web / ISVWeb / ISVFinance &Finance &InsuranceInsuranceDatacom /Datacom /TelecomTelecomLogisticsLogisticsLifeLifeSciencesSciencesMedia &Media &PublishingPublishingEducation,Education,Not-for-Not-for-ProfitProfitGovernment,Government,Aerospace,Aerospace,Gaming, ...Gaming, ...Network /CloudNetwork /CloudMgmtMgmtMDMMDMSocialSocialGeoGeoResource Auth &Resource Auth &Access ControlAccess ControlContentContentManagementManagementRecommend-Recommend-ationsationsData CenterData CenterManagementManagementFraudFraudDetection, ...Detection, ...Early Adopter Graph Database SegmentsEarly Adopters Going Mainstream
    • 21TelenorResource authorization & Access Control in TelecommunicationsResource authorization & Access Control in Telecommunications21Background• 10th largest Telco provider in the world, leading inthe Nordics• Online self-serve system where large businesscustomers manage employee subscriptions and plans• 24/7 availability critical to customer satisfactionBusiness problem Solution & Benefits
    • 22TelenorResource authorization & Access Control in TelecommunicationsResource authorization & Access Control in Telecommunications22
    • 23TelenorResource authorization & Access Control in TelecommunicationsResource authorization & Access Control in Telecommunications23Background• 10th largest Telco provider in the world, leading inthe Nordics• Online self-serve system where large businesscustomers manage employee subscriptions and plans• 24/7 availability critical to customer satisfactionBusiness problem Solution & Benefits• Resource authorization and access control acrossmillions of plans, customers, administrators andgroups, all interconnected, becomes a challenge• Used Sybase RDBMS for pre-computing access rightsdaily• Pre-computation time projected to reach 9 hours in2014• Users cannot log in until their rights are computed• Resource graph easily modeled and queried in Neo4j• 1500 lines of stored procedures => 10s of lines ofNeo4j code• All requests computed in real time in milliseconds• Changes to customer resources reflectedimmediately• Customer retention risks mitigated
    • 24SFRNetwork Management in TelecommunicationsNetwork Management in Telecommunications24Background• Second largest Telco in France• Part of Vivendi Group, partnering with VodaphoneBusiness problem Solution & BenefitsRouterRouterServiceServiceDEPENDS_ONSwitchSwitch SwitchSwitchRouterRouterFiberLinkFiberLink FiberLinkFiberLinkFiberLinkFiberLinkOceanfloor CableOceanfloor CableDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONLINKEDLINKEDLINKEDDEPENDS_ON
    • 25SFRNetwork Management in TelecommunicationsNetwork Management in Telecommunications25RouterRouterServiceServiceDEPENDS_ONSwitchSwitch SwitchSwitchRouterRouterFiber LinkFiber LinkFiber LinkFiber LinkFiber LinkFiber LinkOceanfloorCableOceanfloorCableDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONLINKEDLINKEDLINKEDDEPENDS_ON
    • 26SFRNetwork Management in TelecommunicationsNetwork Management in Telecommunications26Background• Second largest Telco in France• Part of Vivendi Group, partnering with VodaphoneBusiness problem Solution & Benefits• Need for flexible network inventory management,aggregation, and troubleshooting• Impact analysis of planned and unplanned networkoutages, so that affected services can be notified orreceive increased redundancy• Highly volatile network structure changing daily, withbusiness requirements changing as well• Neo4j Enterprise with a highly available cluster• Dynamic system allowing for new applications to tieinto network structure data• Near 1:1 mapping of real world to graph, greatlyreducing modeling work• High adaptability to changing business requirementsRouterRouterServiceServiceDEPENDS_ONSwitchSwitch SwitchSwitchRouterRouterFiberLinkFiberLink FiberLinkFiberLinkFiberLinkFiberLinkOceanfloor CableOceanfloor CableDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONDEPENDS_ONLINKEDLINKEDLINKEDDEPENDS_ON
    • 27AdobeContent Management, Access Control & CollaborationContent Management, Access Control & Collaboration27Background• Creative Cloud, announced 2011, is a cloud-basedoffering for professional users of Adobe’s creativesuiteCollaborative Cloud is the social element of theCreative Cloud, connecting professional usersaround the worldBusiness problem Solution & Benefits
    • 2828Domain GraphDeployment ArchitectureAdobeContent Management, Access Control & CollaborationContent Management, Access Control & Collaboration
    • 29AdobeContent Management, Access Control & CollaborationContent Management, Access Control & Collaboration29Background• Creative Cloud, announced 2011, is a cloud-basedoffering for professional users of Adobe’s creativesuiteCollaborative Cloud is the social element of theCreative Cloud, connecting professional usersaround the worldBusiness problem Solution & Benefits• Identifies which collections a user has access toFindsthird-party assets that are like a user’s assetsInfersprofessional relations based on user-generatedcontent• Fit:• Graph model is a natural fit for social network• Collaborative user experience adds competitiveadvantage to Adobe offering• Flexibility: Data model can be easily evolved tosupport permissions and more sophisticatedrecommendation strategies• Performance: Sub-second results for large,densely-connected datasets
    • 30ViadeoRecommendations in SocialRecommendations in Social๏ Customer: a professional social network• 35 millions users, adding 30,000+ each day๏ Goal: up-to-date recommendations• Scalable solution with real-time end-user experience• Low maintenance and reliable architecture• 8-week implementation30๏ Problem:• Real-time recommendation imperative to attract new usersand maintain positive user retentionClustered MySQLsolution not scalable or fast enough to support real-timerequirementsUpgrade from running a batch job๏ initial hour-long batch job• but then success happened, and it became a day• then two days• With Neo4j, real time recommendations๏ With Neo4j, real time recommendations
    • 32First off: the name๏ WE ALL HATES IT, M’KAY?32
    • NOSQL is NOT...๏ NO to SQL๏ NEVER SQL
    • Not Only SQLNOSQL is simply
    • But why now?
    • 36Trends in BigData & NOSQL36๏ 1. increasing data size (big data)•“Every 2 days we create as much information as we did up to2003” - Eric Schmidt๏ 2. increasingly connected data (graph data)•for example, text documents to html๏ 3. semi-structured data•individualization of data, with common sub-set๏ 4. architecture - a facade over multiple services•from monolithic to modular, distributed applications
    • 374 Categories of NOSQL37
    • 38Key-Value Category๏ “Dynamo:Amazon’s Highly Available Key-Value Store” (2007)๏ Data model:•Global key-value mapping•Big scalable HashMap•Highly fault tolerant (typically)๏ Examples:•Riak, Redis,Voldemort38
    • 39Key-Value: Pros & Cons๏ Strengths•Simple data model•Great at scaling out horizontally•Scalable•Available๏ Weaknesses:•Simplistic data model•Poor for complex data39
    • 40Column-Family Category๏ Google’s “Bigtable: A Distributed Storage System for StructuredData” (2006)•Column-Family are essentially Big Table clones๏ Data model:•A big table, with column families•Map-reduce for querying/processing๏ Examples:•HBase, HyperTable, Cassandra40
    • 41Column-Family: Pros & Cons๏ Strengths•Data model supports semi-structured data•Naturally indexed (columns)•Good at scaling out horizontally๏ Weaknesses:•Unsuited for connected data41
    • 42Document Database Category๏ Data model•Collections of documents•A document is a key-value collection•Index-centric, lots of map-reduce๏ Examples•CouchDB, MongoDB42
    • 43Document Database: Pros & Cons๏ Strengths•Simple, powerful data model•Good scaling (especially if sharding supported)๏ Weaknesses:•Unsuited for connected data•Query model limited to keys (and indexes)43
    • 44Graph Database Category๏ Data model:•Nodes & Relationships•Hypergraph, sometimes (edges with multiple endpoints)๏ Examples:•Neo4j (of course), OrientDB, InfiniteGraph,AllegroGraph44
    • 45Graph Database: Pros & Cons๏ Strengths•Powerful data model, as general as RDBMS•Fast, for connected data•Easy to query๏ Weaknesses:•Requires conceptual shift‣though graph-like thinking becomes addictive45
    • 46Scaling to SizeScaling to ComplexityKey/Value storesColumnFamily storesDocument databasesGraph databasesMy subjective view: > 90% of use cases100+ billion of nodesand relationshipsThe NOSQL Space