• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Solving Problems with Graphs
 

Solving Problems with Graphs

on

  • 8,985 views

Who am I and why do I feel that the world is not infinitely perfect? Which technologies should I use to rectify this situation? Enter the graph and the graph traversal.

Who am I and why do I feel that the world is not infinitely perfect? Which technologies should I use to rectify this situation? Enter the graph and the graph traversal.

Statistics

Views

Total Views
8,985
Views on SlideShare
8,766
Embed Views
219

Actions

Likes
52
Downloads
404
Comments
0

14 Embeds 219

http://www.scoop.it 96
https://twitter.com 68
http://www.sonnia.comyr.com 18
http://www.darmgezondheid.nl 7
http://tages.shanti.virginia.edu 6
http://localhost 5
http://sonnia.comyr.com 4
http://tweetedtimes.com 4
http://www.facebook.com 3
http://www.twylah.com 3
http://twitter.com 2
http://ams.activemailservice.com 1
https://si0.twimg.com 1
https://web.tweetdeck.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Solving Problems with Graphs Solving Problems with Graphs Presentation Transcript

    • SOLVING PROBLEMS WITH GRAPHS MARKO A. RODRIGUEZ http://THINKAURELIUS.COM
    • MARKO A. RODRIGUEZ
    • MARKO A. RODRIGUEZ
    • MARKO A. RODRIGUEZ
    • YOU WILL NOT KNOW ME BY LOOKING WITHIN
    • marko
    • marko TO KNOW ME IS TO KNOW MY WORLD
    • gremlin marko AND IN MY WORLD THERE ARE THINGSrussell
    • gremlin a ted cre marko s ow knrussell
    • gremlin a ted cre marko AND TO KNOW THESE THINGS s ow kn IS TO KNOW THEIR WORLD ... AD INFINITUMrussell
    • pavel ted crea gremlin de pe nd s a ted cre marko blueprints s ow knrussell hortonworks works
    • pavel ted created crea cql gremlin works de pe nd s twitter a ted cre marko s de d pe en nd d ep blueprints s titan s ow kn faunusrussell use use hadoop s s hbase hortonworks works
    • pavel ted created crea cql gremlin works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen dependsrussell uses use use hadoop s s hbase hortonworks works
    • I POSIT THAT THE SOLUTIONS TO PROBLEMS EXIST IN THE WORLD AND THAT RESOLUTION IS SOUGHT IN LINKAGE. pavel ted created crea cql gremlin works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen dependsrussell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavel ted created crea cql gremlin works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavelSOLUTION #1 crea ted created cql• Hortonworks and Cloudera gremlin use Big Data technology. works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavelSOLUTION #1 crea ted created cql• Hortonworks and Cloudera gremlin use Big Data technology. works I know Russell. de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavelSOLUTION #1 crea ted created cql• Hortonworks and Cloudera gremlin use Big Data technology. works I know Russell. de pe depe Partnership? nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works ner part r partne
    • pavel ted created crea cql gremlin works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen dependsrussell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavel ted created crea cql gremlin works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavel ted createdSOLUTION #2 crea cql gremlin• Pavel and I created Gremlin. works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavel ted createdSOLUTION #2 crea cql gremlin• Pavel and I created Gremlin. works He is a Cassandra engineer de at Twitter. pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • PROBLEM• I want organizations with Big Graph Data to use the Aurelius Graph Cluster. pavel ted createdSOLUTION #2 crea cql gremlin• Pavel and I created Gremlin. e ts me works He is a Cassandra engineer de at Twitter. pe depe nd Meeting? s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen depends russell uses use use hadoop s s hbase hortonworks works
    • pavel ted created crea cql gremlin works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen dependsrussell uses use use hadoop s s hbase hortonworks works
    • pavel ted created crea cql gremlin SOFTWARE works de pe depe nd s twitter nds a ted cre depends marko uses s de nd pe pe blueprints nd de s titan cassandra s s created epend ow d kn s depends nd faunus pe cloudera de ds de pen dependsrussell uses use use hadoop s s hbase hortonworks works
    • marko SOFTWARE
    • marko SOFTWARE
    • marko SOFTWARE
    • marko
    • marko pet pet mamapuppy
    • marko pet pet sniffs mamapuppy
    • marko chula pet pet i ffs sn sn iffs sniffs mama sniffs puppy curley biscuitmia iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs sniffs mama sniffs puppy curley biscuitmia males iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs sniffs mama 1 sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet 2 pet i ffs sn sn iffs sniffs mama 2 1 sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet 2 pet i ffs sn sn iffs sniffs mama 2 sniffs sniffs puppy curley biscuit sn iffs 3 sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs 4 sniffs mama sniffs sniffs puppy curley biscuit sn iffs 3 sniffsmia males sni 4 sni ffs s iff ffs sn iffs sn sn scout iffs 4 4 4 sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs 5 4 sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni 4 sni 5 ffs s iff ffs sn iffs sn sn scout iffs 4 4 4 sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs 5 sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni 5 ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • PROBLEM • What is the best way to save this dog community from disease? marko chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • PROBLEM • What is the best way to save this dog community from disease? marko chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • PROBLEM • What is the best way to save this dog community from disease? SOLUTION marko • Inoculate the most central dog. chula pet pet i ffs sn sn e ulat iffsinoc sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffs mia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • PROBLEM • What is the best way to save this dog community from disease? SOLUTION marko • Inoculate the most central dog. chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • marko chula DOGS pet pet i ffs sn sn iffs sniffs mama sniffs sniffs puppy curley biscuit sn iffs sniffsmia males sni sni ffs s iff ffs sn iffs sn sn scout iffs sniffs jackson missy whettle
    • markoDOGS
    • marko SOFTWAREDOGS
    • marko SOFTWAREDOGS
    • marko SOFTWAREDOGS
    • marko SOFTWAREDOGS
    • marko SOFTWAREDOGS
    • marko
    • avalanchemarko captain
    • blackhawks wild panthers avalanchemarko captain
    • game #1 played blackhawks #2 wild #3 panthers #4 avalanchemarko #5 captain ... #n
    • game #1 played blackhawks #2 period #1 wild p1 #3 #2 p2 p3 panthers #4 #3 avalanchemarko #5 captain ... #n
    • game #1 played blackhawks #2 period #1 wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 #1 avalanche penaltiesmarko #5 #2 #2 captain ... #3 #n
    • game #1 steve jason craig patrick ... played blackhawks #2 period #1 wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 #1 avalanche penaltiesmarko #5 #2 #2 captain ... #3 #n
    • game #1 steve jason craig patrick ... played blackhawks #2 period #1 received scored assisted scored wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 assisted #1 avalanche penaltiesmarko #5 #2 #2 captain ... #3 #n
    • PROBLEM game #1 steve jason craig patrick• Who do I draft for the 2012/2013 season? ... played blackhawks #2 period #1 received scored assisted scored wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 assisted #1 avalanche penalties marko #5 #2 #2 captain ... #3 #n
    • PROBLEM game #1 steve jason craig patrick• Who do I draft for the 2012/2013 season? ... played blackhawks #2 period #1 received scored assisted scored wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 assisted #1 avalanche penalties marko #5 #2 #2 captain ... #3SOLUTION #n• Craig/Patrick play well together.
    • PROBLEM game #1 steve jason craig patrick• Who do I draft for the 2012/2013 season? ... played blackhawks #2 period #1 received scored assisted scored wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 assisted #1 avalanche penalties draft marko #5 #2 #2 captain ... #3 draftSOLUTION #n• Craig/Patrick play well together. Draft them as a pair?
    • game #1 steve jason craig patrick ... played blackhawks #2 period #1 received scored assisted scored wild p1 #3 #2 p2 p3 panthers #4 #3 ls goal goa penalty #1 assisted #1 avalanche penaltiesmarko #5 #2 #2 captain ... #3 SPORTS #n
    • SPORTSmarko
    • SPORTSmarko SOFTWARE DOGS
    • THIS IS JUSTMY SUBSET OF THE WORLD ANDMY PERSONAL PROBLEMS.
    • OUR PERSONAL WORLDS ...
    • ... ARE EMBEDDED WITHIN A LARGER WORLD ...
    • ... OF OTHER PEOPLEAND ARTIFACTS ...
    • ... AND TOGETHER WE NAVIGATE THAT WORLD ...
    • ... TRYING TO BETTER OUR LIVES ...
    • ... BY SOLVING OUR PROBLEMS ...
    • ... BY OPTIMALLY LINKING OURSELVES WITHIN THE WORLD.
    • WHY DO WE ENCODE OURSELVES?
    • These are my friends. I think about these ideas. I like these books.This where I work. I am. I visit these webpages. I work on these projects. I wrote these articles. WHY ARE WE CREATING A UNIVERSAL MODEL?
    • Who should I befriend? What ideas will inspire me? Where is the best place for me to live?What movie should I watch tonight with the friends Im meeting up with? What career path should I choose given my interests and expertise? Who should I fall in love with and live my life with? BECAUSE WE HAVE FAITH IN THE ALGORITHM.
    • WE HAVE FAITH THAT TOGETHER, WITH COMPUTERS,WE WILL DETERMINE THE OPTIMAL EMBEDDING.
    • WE WILL GENERATE ENTHRALLING CONNECTIONS ...
    • THAT RESONATES US ...
    • TO CREATE AND EXPERIENCE ...
    • EVEN GREATER THINGS.
    • HOW DO WE STORE AND PROCESS A WORLD MODEL?BILLIONS OF VERTICES TRILLIONS OF EDGES SOLVING MILLIONS OF PROBLEMS/SECOND
    • AURELIUSGRAPH CLUSTER
    • GRAPH COMPUTING
    • GRAPH COMPUTINGDATA STRUCTURE ALGORITHM + GRAPH TRAVERSAL
    • GRAPHname:marko name:Fountain Head age:32 published:1943 likes star:5 time:2002
    • GRAPH KEYPROPERTIES VALUE EDGE LABEL name:marko name:Fountain Head age:32 published:1943 likes star:5 time:2002 VERTEX
    • TRAVERSAL 3 2 31 3 2 3 A traversal is an algorithmic walk over a (sub)graph in order to make explicit information that is implicit within its structure. 1.) graph derivation ("my fathers father is my grandfather.") 2.) graph statistic ("many paths lead to Rome.")
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel PROBLEM • What should I buy for any upcoming friends birthday?
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2}
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes likes x knows knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes likes x knows knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x).in(likes)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes likes x knows knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x).in(likes).out(like)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes likes x knows knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x).in(likes).out(like) .except(x)
    • TRAVERSAL lives lives name:jen birthmonth:2 name:santa fe likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x).in(likes).out(like) .except(x)
    • TRAVERSAL lives lives name:jen birthmonth:2 C:2 name:santa fe D:1 likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x).in(likes).out(like) .except(x).groupCount()
    • TRAVERSAL lives lives name:jen birthmonth:2 C:2 name:santa fe D:1 likes likes A name:russellname:marko knows C likes likes knows likes knows likes likes likes likes D name:josh B birthmonth:12 name:pavel g.V(name,marko).out(knows) .filter{(it.birthmonth + currentMonth) % 12 < 2} .out(likes).aggregate(x).in(likes).out(like) .except(x).groupCount()
    • GRAPH COMPUTING SYSTEMS
    • AURELIUS GRAPH CLUSTER TITAN FAUNUS FULGORADistributed Graph Database Graph Analytics Engine Fast Graph Processor Apache 2 Licensed
    • TITANDISTRIBUTED GRAPH DATABASE
    • TITAN DISTRIBUTED GRAPH DATABASERepresents the world as a single, atomic graph structure.
    • TITAN DISTRIBUTED GRAPH DATABASERepresents the world as a single, atomic graph structure. Distributed over a multi-machine cluster using existing distributed data systems such as Cassandra and HBase.
    • TITAN DISTRIBUTED GRAPH DATABASERepresents the world as a single, atomic graph structure. Distributed over a multi-machine cluster using existing distributed data systems such as Cassandra and HBase. Supporting numerous short-lived, topologically local, real-time traversals.
    • FAUNUSGRAPH ANALYTICS ENGINE
    • FAUNUS GRAPH ANALYTICS ENGINEExtracts an ephemeral snapshot of the master graph.
    • FAUNUS GRAPH ANALYTICS ENGINEExtracts an ephemeral snapshot of the master graph.Leverages Hadoop as its distributed computing engine.
    • FAUNUS GRAPH ANALYTICS ENGINEExtracts an ephemeral snapshot of the master graph.Leverages Hadoop as its distributed computing engine.Executing long-lived, topologically global analyses of the graph.
    • FULGORAFAST GRAPH PROCESSOR
    • FULGORA FAST GRAPH PROCESSORStores a compressed subset of the master graph.
    • FULGORA FAST GRAPH PROCESSORStores a compressed subset of the master graph. Contained within the confines of a single high memory/CPU machine.
    • FULGORA FAST GRAPH PROCESSORStores a compressed subset of the master graph. Contained within the confines of a single high memory/CPU machine.Evaluating heavily threaded, memory efficient graph and machine learning algorithms. }
    • AURELIUS GRAPH CLUSTER AN INTEGRATED BIG GRAPH DATA SOLUTION TITAN FAUNUS FULGORADistributed Graph Database Graph Analytics Engine Fast Graph Processor Output Graph Output Statistics Map/Reduce Output Statistics Update Graph Other Data Analysis Tools Update Graph Apache 2 Licensed
    • CREDITS PRESENTER MARKO A. RODRIGUEZ CONTRIBUTORSMATTHIAS BROCHELER STEPHEN MALLETTE DAN LAROCQUE VADAS GINTAUTAS MANY THANKS TO AURELIUS COMMUNITYTINKERPOP COMMUNITY KETRINA YIM
    • RELATED MATERIAL FAITH IN THE ALGORITHM 2:COMPUTATIONAL EUDAEMONICS http://arxiv.org/abs/0904.0027 A COLLECTIVELY GENERATED MODEL OF THE WORLD http://markorodriguez.files.wordpress.com/2011/01/collective-model.pdf STRUCTURAL ABSTRACTIONS IN THE BRAIN http://thinkaurelius.com/2012/05/08/structural-abstractions-in-brains-and-graphs/GRAPHS, BRAINS, AND GREMLIN http://markorodriguez.com/2011/07/14/graphs-brains-and-gremlin/