Florian Hopf
http://blog.florian-hopf.de
Download
  Page
              Link
              downloaded
              already?

Parse Links
 & Content




  Index
 Content
Sequential
 Example
1. Scale Up
Thread A   Page    Thread B
           Store
Synchronize
  Thread A   Page    Thread B



 and Suffer
             Store
Actor-Framework
Message Passing
 Concurrency
Thread A   Thread B




 Actor A    Actor B
Master

              Page
    Fetch     Store     Index
    Page                Page
           Page
                  Indexed
 Page     Content
                            Indexing
Parsing
                              Actor
 Actor
Simple Example
Actors are accessed
   by references
Reference




              Actor
            Instance
Reference




              Actor
                Actor
                   n
            Instance
              Instance
                 Actor
               Instances
Fetching Pages
  in parallel
2. Scale Out
References can point
to remote machines
akka {
        actor {
           deployment {
              /serviceA/aggregation {
                 router = "round­robin"
                 nr­of­instances = 10
                 target {
                    nodes = 
                       ["akka://app@10.0.0.2:2552", 
                       "akka://app@10.0.0.3:2552"]
                 }
              }
           }
        }
    }
3. Fault Tolerance
Actors are supervised
   by their parent
Master

          Page
          Store



 Page
                   Indexing
Parsing
                     Actor
 Actor
Reference




              Actor
            Instance
Master

          Page
          Store



 Page
                   Indexing
Parsing
                     Actor
 Actor
Reference




              Actor
            Instance
Actions:
Escalate
Resume
Restart
  Stop
Network fail on download
Master

          Page
          Store



 Page
                   Indexing
Parsing
                     Actor
 Actor
Resume
PageParsingActor,
 resend message
Indexing Failure, I/O Error
Master

          Page
          Store



 Page
                   Indexing
Parsing
                     Actor
 Actor
Restart Master
Scaling Up
  Scaling Out
Fault Tolerance
Software Transactional
       Memory

  Durable Mailboxes
http://akka.io

Akka Presentation Schule@synyx