• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Abstracting with pacer
 

Abstracting with pacer

on

  • 492 views

 

Statistics

Views

Total Views
492
Views on SlideShare
206
Embed Views
286

Actions

Likes
0
Downloads
5
Comments
0

3 Embeds 286

http://xnlogic.com 276
https://darrick-wiebe.squarespace.com 9
http://darrick-wiebe.squarespace.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Abstracting with pacer Abstracting with pacer Presentation Transcript

    • © All Rights Reserved 2013 | xnlogic.com Abstracting
    •   your
    •   Graph with
    •   Pacer
    • © All Rights Reserved 2013 | xnlogic.com Querying
    •   Twitter
    • © All Rights Reserved 2013 | xnlogic.com How would you query Twitter? Find conversations that are relevant to me, happened recently and were popular.
    • © All Rights Reserved 2013 | xnlogic.com 
    •    person.recent_popular_relevant_conversations!
    • © All Rights Reserved 2013 | xnlogic.com Layer
    •   1:
    •   Graph
    •   Foundations
    • © All Rights Reserved 2013 | xnlogic.com It’s all about moving through the graph and filtering out anything you are not interested in
    • © All Rights Reserved 2013 | xnlogic.com Moving
    •   Through
    •   The
    •   Graph Follow an edge pointing out of a vertex out_e.in_v or one pointing in to my vertex in_e.out_v
    • © All Rights Reserved 2013 | xnlogic.com These operations are fast
    • © All Rights Reserved 2013 | xnlogic.com Filtering
    •   for
    •   Understanding
    • © All Rights Reserved 2013 | xnlogic.com Most
    •   Fundamental:
    •   By
    •   Edge
    •   Label Who I follow out_e(:follow).in_v Who follows me in_e(:follow).out_v follow
    • © All Rights Reserved 2013 | xnlogic.com By
    •   Property out_e.in_v(name: 'elonmusk') filter(name: 'elonmusk') name: 'elonmusk'
    • © All Rights Reserved 2013 | xnlogic.com Layer
    •   2:
    •   Building
    •   Meaning
    • © All Rights Reserved 2013 | xnlogic.com Vertex
    •   Types
    •   in
    •   Your
    •   Domain g.v type: "person" Pacer just uses Ruby’s built-in modules as its types module Person def self.lookup { type: "person" } end end g.v(Person)
    • © All Rights Reserved 2013 | xnlogic.com Name
    •   Basic
    •   Movements
    •   From
    •   a
    •   Person module Person::Route def following out_e(:follow).in_v(Person) end def followed_by in_e(:follow).out_v(Person) end end Person follow Person
    • © All Rights Reserved 2013 | xnlogic.com Moving
    •   From
    •   One
    •   Type
    •   To
    •   Another module Person::Route def tweets out_e(:wrote).in_v(Tweet) end end Person Tweet wrote
    • © All Rights Reserved 2013 | xnlogic.com Aside:
    •   Using
    •   Our
    •   New
    •   Definitions graph = Pacer.neo4j(‘path/to/graph’) #=> <PacerGraph> graph.v(Person).tweets # <V[123] Tweet> <V[321] Tweet> # <V[456] Tweet> #=> <Route Index -> out(:tweet)> It’s really just that simple.
    • © All Rights Reserved 2013 | xnlogic.com Basic
    •   Domain
    •   Filters def female_only filter(gender: ‘female’) end def recent_only where(‘time > :recent’, recent: hours_ago(2)) end def max_length_only filter do |tweet| tweet[:text].length == 140 end end
    • © All Rights Reserved 2013 | xnlogic.com These methods are extremely simple. Q: Why even bother with them? A: Names, names, names
    • © All Rights Reserved 2013 | xnlogic.com Layer
    •   3:
    •   Chains
    •   of
    •   Thought
    • © All Rights Reserved 2013 | xnlogic.com Meaningful
    •   Movements def retweeted_by retweets.authors end def current_timeline following.tweets.recent_only end
    • © All Rights Reserved 2013 | xnlogic.com More
    •   Powerful
    •   Filters:
    •   Lookaheads Person followed by at least 1000 people def popular_only lookahead(min: 1000) { |person| person.followed_by } end Tweet retweeted at least 100 times def impactful_only lookahead(min: 100) { |tweet| tweet.retweets } end
    • © All Rights Reserved 2013 | xnlogic.com It’s
    •   Even
    •   Better:
    •   Filtered
    •   Lookaheads Have these people tweeted recently? def active_only lookahead { |person| person.tweets.recent_only } end Reversing a filter (lookahead Inception!) def inactive_only neg_lookahead { |person| person.active_only } end
    • © All Rights Reserved 2013 | xnlogic.com Layer
    •   4:
    •   Composition
    • © All Rights Reserved 2013 | xnlogic.com Extracting
    •   Deeper
    •   Meaning A shy twitterer retweets and stars stuff, follows people but never tweets: def shy_people_only has_retweeted. has_starred. is_following. neg_lookahead(&has_tweeted) end
    • © All Rights Reserved 2013 | xnlogic.com Reality
    •   Folds
    •   Back
    •   On
    •   Itself We are marking an element and circling back to it def relevant_conversation_tweets as(:me).following.tweets.lookahead do |tweet| tweet.responded_to.authors.followed_by.is(:me) end end This pattern is incredibly powerful!
    • © All Rights Reserved 2013 | xnlogic.com Exercise:
    •   How
    •   Many
    •   Lines
    •   Of
    •   Code? How did we build such a sophisticated query out of such simple pieces? def recent_popular_relevant_conversations! recent_only.popular_only.relevant_conversation_tweets end
    • © All Rights Reserved 2013 | xnlogic.com These methods are extremely simple. Q: Why even bother with them? A: Names, names, names
    • © All Rights Reserved 2013 | xnlogic.com You’ve just isolated your logic from the schema. Most of your business logic is at Layer 4+. Even major schema changes rarely impact beyond Layer 3.
    • © All Rights Reserved 2013 | xnlogic.com Layer
    •   5:
    •   Repeating
    •   Patterns
    • © All Rights Reserved 2013 | xnlogic.com Some
    •   Patterns
    •   are
    •   Variable
    •   Length The tweet this responded to, then the tweet that it responded to, etc, etc... def conversations all { |tweet| tweet.responded_to } end Any pattern can be repeated, no matter how complex def popular_conversations all { |tweet| tweet.responded_to.popular_only } end
    • © All Rights Reserved 2013 | xnlogic.com Ignoring
    •   Intermediate
    •   Vertices The tweet that started it all def conversation_starters deepest { |tweet| tweet.responded_to } end
    • © All Rights Reserved 2013 | xnlogic.com Combining
    •   Combination
    •   Combinations A lookahead with a loop with a lookahead, emerging with a loop. def conversations_with_a_recent_popular_exchange lookahead do |tweet| tweet.popular_conversations.recent_only end.conversations end
    • © All Rights Reserved 2013 | xnlogic.com Specialized
    •   Loop
    •   Logic Make common operations simple, uncommon operations possible... def rapid_conversations loop do |tweet| tweet.responded_to end.while do |tweet, depth, path| if depth >= 1 prev_tweet = path[-2] minutes = tweet.minutes_after(prev_tweet) if minutes < 2 :loop_and_emit if minutes < 5 :loop end else :loop end end end
    • © All Rights Reserved 2013 | xnlogic.com Sometimes the data is like this. Sometimes the data is like that... In the real world, data isn’t so clean. We’ve got you covered.
    • © All Rights Reserved 2013 | xnlogic.com Branching
    •   and
    •   Merging You’ve got the power to encode alternate routes def good_tweets branch do |person| person.starred_tweets end.branch do |person| person.tweets.retweeted end end That’s often all it takes to handle messy data from multiple sources.
    • © All Rights Reserved 2013 | xnlogic.com What
    •   Else?
    • © All Rights Reserved 2013 | xnlogic.com What
    •   else? • Transactional CRUD is easy • Indices are automatic • Summarization tools • Data export (for visualization, etc) • Process more than just graphs