AURELIUSTHINKAURELIUS.COMTITANGraph Computing with CassandraMatthias Broecheler, CTO@mbroechelerJune XI, MMXIII#CASSANDRA13
Thank YouJOFF L?KO?MNM@?;NOL? MOAA?MNCIHM<OA L?JILNM=IGGOHCNS MOJJILN
June 14th2012September2012December2012March2013May2013AlphaReleaseTitan0.1.0Titan0.2.0Titan0.3.0Titan0.3.1%RJ?LCG?HN;F L?F...
June 14th2012September2012December2012March2013May2013AlphaReleaseTitan0.1.0Titan0.2.0Titan0.3.0Titan0.3.1%RJ?LCG?HN;F L?F...
TitanGraph Database>CMNLC<ON?>L?;F NCG?IJ?HMIOL=?
name: Herculestype: demigodname: Cerberustype: monsterbattledtime:126?LN?R%>A? ,;<?F%>A?0LIJ?LNS
Value in Relationshipslow highKey-Value7B?H MBIOF> SIO OM? ; L;JB $;N;<;M?gK VBigTableK V V V VDocumentRelationalGraph"
Educating the Planet
Educating the Planet
PersonPersonStudent TeacherCourseInstitutionConceptDiscussionCommentShareenrolledInteachesrelatesTohasCoursebelongsTofollo...
PersonPersonStudent TeacherCourseInstitutionConceptDiscussionCommentShareenrolledInteachesrelatesTohasCoursebelongsTofollo...
TitanIntegrative Data ModelCH ; JIFSAFINMNIL;A? QILF>
StudentPersonTeacherCourseInstitutionConceptDiscussionCommentShareenrolledInteachesrelatesTohasCoursebelongsTofollowsautho...
TitanAnalyze RelationshipsCH L?;F NCG?
Scaling TitanHOG<?L I@NL;HM;=NCIHMMCT? I@ NB? AL;JB
121 Billion Edges6.2 Billion VerticesU -CFFCIH 5HCP?LMCNC?M
0F;=?G?HN LIOJBCU .4RF
1.1 million edges / secOMCHA <;N=B GI>?Data Ingestion
^ GU .G?>COG
x = [] as Set; m = [:]!m = user.out(follows).aggregate(x)[0..(num*2)]!!.out(follows).except(x)[0..limit]!!.groupCount(m);!...
GenericGraph APIDataflowProcessingTraversalLanguageObject-GraphMapperGraphAlgorithmsGraphServer?R=CNCHA QILE=IGCHA2%34 h *3...
10,200 transactions / secUZ L;H>IGFS =BIM?H =IGJF?RNL;P?LM;F N?GJF;N?MThroughput
Transaction Description Avg (ms) Stdev (ms)Student retrieves all content for asingle course in their course list279.32 81....
Scaling TitanN?=BHC=;F J?LMJ?=NCP?
Vertex Representationtime: 1584927motherbattledbattledbattledfoughttime: 4time: 7 CH>O=?>IL>?Lname:Herculestype:demigod5Pr...
label id +directionprimary key edge idΔvertex idsignaturepropertiesotherpropertiesEdge RepresentationColumn Value=IGJL?MM?...
Token RingGraph Partitioning;MMCAHM C>M NI G;JP?LNC=?M CHNI “IJNCG;F”NIE?H L;HA?,INM I@ CHN?L?MNCHA KO?MNCIHM @IL@ONOL? QI...
Aurelius Graph ClusterStores a massive-scaleproperty graph allowing real-time traversals and updatesBatch processing of la...
Special ThanksSteve Hill (@kindageeky)Director Architecture & Innovationat Pearson Education
AURELIUSTHINKAURELIUS.COMWe are Hiring
Upcoming SlideShare
Loading in...5
×

Titan - Graph Computing with Cassandra

2,770

Published on

This presentation introduces Titan, Faunus, and scalable graph computing in general. We present a case study of how Pearson builds an education social network on top of Titan, Faunus, and Cassandra to support learning in the 21st century.

Titan is an open source distributed graph database build on top of Cassandra that can power real-time applications with thousands of concurrent users over graphs with billions of edges. Faunus is an open source global graph processing engine build on top of Hadoop and compatible with Cassandra that can analyze graphs, compute graph statistics, and execute global traversals. Titan and Faunus are components of the Aurelius Graph Cluster which enables scalable graph computation and powers applications in social networking, recommendation engines, advertisement optimization, knowledge representation, health care, education, and security.

0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,770
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
75
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Titan - Graph Computing with Cassandra

  1. 1. AURELIUSTHINKAURELIUS.COMTITANGraph Computing with CassandraMatthias Broecheler, CTO@mbroechelerJune XI, MMXIII#CASSANDRA13
  2. 2. Thank YouJOFF L?KO?MNM@?;NOL? MOAA?MNCIHM<OA L?JILNM=IGGOHCNS MOJJILN
  3. 3. June 14th2012September2012December2012March2013May2013AlphaReleaseTitan0.1.0Titan0.2.0Titan0.3.0Titan0.3.1%RJ?LCG?HN;F L?F?;M? I@ ;>CMNLC<ON?>m IJ?H rMIOL=?AL;JB >;N;<;M?&CLMN MN;<F? L?F?;M?2?QLCN? I@ =IL?)H>?RCHA h %F;MNC=3?;L=B0?L@ILG;H=? "OA@CRCHA
  4. 4. June 14th2012September2012December2012March2013May2013AlphaReleaseTitan0.1.0Titan0.2.0Titan0.3.0Titan0.3.1%RJ?LCG?HN;F L?F?;M? I@ ;>CMNLC<ON?>m IJ?H rMIOL=?AL;JB >;N;<;M?&CLMN MN;<F? L?F?;M?2?QLCN? I@ =IL?)H>?RCHA h %F;MNC=3?;L=B0?L@ILG;H=? "OA@CRCHAFaunus Release
  5. 5. TitanGraph Database>CMNLC<ON?>L?;F NCG?IJ?HMIOL=?
  6. 6. name: Herculestype: demigodname: Cerberustype: monsterbattledtime:126?LN?R%>A? ,;<?F%>A?0LIJ?LNS
  7. 7. Value in Relationshipslow highKey-Value7B?H MBIOF> SIO OM? ; L;JB $;N;<;M?gK VBigTableK V V V VDocumentRelationalGraph"
  8. 8. Educating the Planet
  9. 9. Educating the Planet
  10. 10. PersonPersonStudent TeacherCourseInstitutionConceptDiscussionCommentShareenrolledInteachesrelatesTohasCoursebelongsTofollowsauthorreferenceshasComment relatesToauthorpartOfrelatesTo
  11. 11. PersonPersonStudent TeacherCourseInstitutionConceptDiscussionCommentShareenrolledInteachesrelatesTohasCoursebelongsTofollowsauthorreferenceshasComment relatesToauthorpartOfrelatesTo
  12. 12. TitanIntegrative Data ModelCH ; JIFSAFINMNIL;A? QILF>
  13. 13. StudentPersonTeacherCourseInstitutionConceptDiscussionCommentShareenrolledInteachesrelatesTohasCoursebelongsTofollowsauthorreferenceshasComment relatesToauthorpartOfDiscussionRankrelatesTo
  14. 14. TitanAnalyze RelationshipsCH L?;F NCG?
  15. 15. Scaling TitanHOG<?L I@NL;HM;=NCIHMMCT? I@ NB? AL;JB
  16. 16. 121 Billion Edges6.2 Billion VerticesU -CFFCIH 5HCP?LMCNC?M
  17. 17. 0F;=?G?HN LIOJBCU .4RF
  18. 18. 1.1 million edges / secOMCHA <;N=B GI>?Data Ingestion
  19. 19. ^ GU .G?>COG
  20. 20. x = [] as Set; m = [:]!m = user.out(follows).aggregate(x)[0..(num*2)]!!.out(follows).except(x)[0..limit]!!.groupCount(m);!m.sort{-it.value}[0..num]._()!!.transform{ [userid: it.key.id, !! ! ! ! ! ! points: it.value]};!&IFFIQ 2?=IGG?H>;NCIH
  21. 21. GenericGraph APIDataflowProcessingTraversalLanguageObject-GraphMapperGraphAlgorithmsGraphServer?R=CNCHA QILE=IGCHA2%34 h *3/.4CN;H’M%=IMSMN?GKO?LSF;HAO;A?http://tinkerpop.com
  22. 22. 10,200 transactions / secUZ L;H>IGFS =BIM?H =IGJF?RNL;P?LM;F N?GJF;N?MThroughput
  23. 23. Transaction Description Avg (ms) Stdev (ms)Student retrieves all content for asingle course in their course list279.32 81.83Student follows another student 193.72 22.77Student is recommended peopleto follow241.33 256.48Student reads their stream andshares an item with followers284.07 68.20Student retrieves their profile 53.740 22.61Student reads the most recentcomments for their courses211.07 45.56
  24. 24. Scaling TitanN?=BHC=;F J?LMJ?=NCP?
  25. 25. Vertex Representationtime: 1584927motherbattledbattledbattledfoughttime: 4time: 7 CH>O=?>IL>?Lname:Herculestype:demigod5PropertyPropertyEdgeEdgeEdgeEdgeEdgeLIQ CH>C=?M@IL @;MNP?LN?R =?HNLC=KO?LC?M
  26. 26. label id +directionprimary key edge idΔvertex idsignaturepropertiesotherpropertiesEdge RepresentationColumn Value=IGJL?MM?> M?LC;FCT?> I<D?=NMP;LC;<F? FIHA ?H=I>CHA
  27. 27. Token RingGraph Partitioning;MMCAHM C>M NI G;JP?LNC=?M CHNI “IJNCG;F”NIE?H L;HA?,INM I@ CHN?L?MNCHA KO?MNCIHM @IL@ONOL? QILEOM?M "/0
  28. 28. Aurelius Graph ClusterStores a massive-scaleproperty graph allowing real-time traversals and updatesBatch processing of largegraphs with HadoopRuns global graph algorithmson large, compressed,in-memory graphsMap/Reduce Load & CompressAnalysis resultsback into TitanBulk LoadTITAN FAUNUS FULGORAApache 2aureliusgraphs@googlegroups.comtitan.thinkaurelius.com faunus.thinkaurelius.com
  29. 29. Special ThanksSteve Hill (@kindageeky)Director Architecture & Innovationat Pearson Education
  30. 30. AURELIUSTHINKAURELIUS.COMWe are Hiring
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×