Your SlideShare is downloading. ×
0
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems

1,103

Published on

A talk held by Dr.-Ing. Kalman Graffi at 9. April 2008 at the QuaP2P Kolloquium in Darmstadt / Germany …

A talk held by Dr.-Ing. Kalman Graffi at 9. April 2008 at the QuaP2P Kolloquium in Darmstadt / Germany

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,103
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Roter Faden: Evtl. Unklare Details:
  • Roter Faden: Was ist P2P? Welche QoS Anforderungen bisher gestellt? Evtl. Unklare Details:
  • Roter Faden: Viele Qualitätsaspekte wichtig für P2P Systeme Evtl. Unklare Details:
  • Roter Faden: Zukünftige, “seriöse” P2P Applikationen bestehend aus effizienten Modulen die QoS bieten. Anwendungsbesipiel: Katastrophenszenario -> Überleitung zu Emergency Call Handling Evtl. Unklare Details: ??ja: rst?? FreePastry: Freie modulare Implementierung von Pastry, PAST: Storage+Replikationslogik modular aufbauen zu FreePastry, Scribe: Group Communication für FreePastry, POST: co-operative messageing, SplitStream: high-bandwidth content distribution Gegensatz zu früher: P2P Programme werden nicht mehr from the scratch entwickelt, sondern modular zusammengesteckt
  • | | November 19, 2007
  • | | November 19, 2007
  • Roter Faden: Zukünftige, “seriöse” P2P Applikationen bestehend aus effizienten Modulen die QoS bieten. Anwendungsbesipiel: Katastrophenszenario -> Überleitung zu Emergency Call Handling Evtl. Unklare Details: ??ja: rst?? FreePastry: Freie modulare Implementierung von Pastry, PAST: Storage+Replikationslogik modular aufbauen zu FreePastry, Scribe: Group Communication für FreePastry, POST: co-operative messageing, SplitStream: high-bandwidth content distribution Gegensatz zu früher: P2P Programme werden nicht mehr from the scratch entwickelt, sondern modular zusammengesteckt
  • Roter Faden: ECH: ungelöst, Anforderungen: Location-based search, QoS-mechanisms Evtl. Unklare Details: Grafiken rechts zeigen Bilder die für die Simualtion dann verwendet wurden: Populationsdichte, Notrufzuständigkeitsgebiete, Simulationssnapshot
  • Roter Faden: Globase.KOM löst das Problem der Location-based Search, Details Evtl. Unklare Details: Special Issue of the Proceedings of IEEE on Advances in Distributed Multimedia Communications
  • Roter Faden: Unsere Lösung: Eingeschobener Network Wrapper zum einbetten von QoS Mechanismen für Overlay Nachrichten. HiPNOS behandelt Nachrichten gemäß ihrer Delay und Loss Prioritäten. Results: Lösung funktioniert sehr gut. Evtl. Unklare Details: Sollen Referenzen zu den Lösungen hier eingefügt werden? Oder zu Globase?
  • Roter Faden: Unsere Lösung: Eingeschobener Network Wrapper zum einbetten von QoS Mechanismen für Overlay Nachrichten. HiPNOS behandelt Nachrichten gemäß ihrer Delay und Loss Prioritäten. Results: Lösung funktioniert sehr gut. Evtl. Unklare Details: Sollen Referenzen zu den Lösungen hier eingefügt werden? Oder zu Globase?
  • Roter Faden: Gute Lösung verschiebt den Fokus von den QoS Mechanismen zu der Frage: Wie kommen wir an die Informationen, die notwendig sind um gute QoS Entscheidungen zu treffen. Diese Folie soll das Thema Emergency Call Handling und HiPNOS / Overlay Bandwidth Management abschließen und zum zweiten Teil des Talks überleiten. Evtl. Unklare Details:
  • Roter Faden: Zeigen dass jetzige P2P Applications schon jeweils für sich Informationen zur Effizienzsteigerung einsammeln -> Lässt sich durch eine deddizierte Schicht besser erledigen. Evtl. Unklare Details:
  • Roter Faden: Zeigen dass jetzige P2P Applications schon jeweils für sich Informationen zur Effizienzsteigerung einsammeln -> Lässt sich durch eine deddizierte Schicht besser erledigen. Evtl. Unklare Details:
  • Roter Faden: Effizienz Management Lifecycle: Info einsammlen, aufbereiten/auswerten, Wissen wieder in das System (in Form von besseren Entscheidungen) einfließen lassen Evtl. Unklare Details:
  • Roter Faden: Nun Übergang zu unserer Lösung: Eine zusätzliche Schicht auf strukturierte P2P Overlays (die durch die Common API einheitlich angesprochen werden können). Die Query Form die unsere Architektur bietet, wird vorgestellt Evtl. Unklare Details: Common API, Paper von “F. Dabek and B. Zhao and P. Druschel and I. Stoica“ zum Vereinheitlichen der Services von DHTs, wichtig hier: Route(ID, Msg) – mittels der eine Nachricht (Msg) zu einem Peer geroutet werden kann der für eine ID (ID) zuständig ist.
  • Roter Faden: Nun Übergang zu unserer Lösung: Eine zusätzliche Schicht auf strukturierte P2P Overlays (die durch die Common API einheitlich angesprochen werden können). Die Query Form die unsere Architektur bietet, wird vorgestellt Evtl. Unklare Details: Common API, Paper von “F. Dabek and B. Zhao and P. Druschel and I. Stoica“ zum Vereinheitlichen der Services von DHTs, wichtig hier: Route(ID, Msg) – mittels der eine Nachricht (Msg) zu einem Peer geroutet werden kann der für eine ID (ID) zuständig ist.
  • Roter Faden: Nun Übergang zu unserer Lösung: Eine zusätzliche Schicht auf strukturierte P2P Overlays (die durch die Common API einheitlich angesprochen werden können). Die Query Form die unsere Architektur bietet, wird vorgestellt Evtl. Unklare Details: Common API, Paper von “F. Dabek and B. Zhao and P. Druschel and I. Stoica“ zum Vereinheitlichen der Services von DHTs, wichtig hier: Route(ID, Msg) – mittels der eine Nachricht (Msg) zu einem Peer geroutet werden kann der für eine ID (ID) zuständig ist.
  • Roter Faden: wo ist die EM Architektur (=Information Einsammel Service) einzuordnen: Über der common API (die über den Strukt. Overlays ist), Update-Flüsse: in einem virtuellen Baum hoch, Peers kennen ihre Vater-Knoten (woher: nächste Folie) Evtl. Unklare Details:
  • Roter Faden: Vater-Knoten ist der Peer zuständig für die mittlere ID in einer Domain (=ID Space Intervallabschnitt), Rekursive Domains Supporting peers zum Load Balancing (da sonst schwache Knoten an wichtigen Stellen sitzen könnten) Evtl. Unklare Details: Peer responsible for a specific ID (e.g. middle) is responsible for ID domain: Determinstische Zuweisfunktion, dadurch kann jeder Peer ermitteln, an welchen Peer er ein Update schicken muss (und zwar an den zuständigen für eine bestimmte ObjectID) Durch diese deterministische Funktion, spart man sich die Maintenance Kosten Storyline zur Animation: Zuerst sehen wir einen ID Space auf den die Overlay IDs abgebildet werden. Die Peers sind in diesem ID Space verteilt. Wir betrachten nun die Protokollschritte aus der Sicht eines einzelnen Peers (roter Pfeil), dieser bestimmt zuerst an wen er seine Updates schicken muss. Dazu halbiert er den ID Space jeweils soweit bis er seinen Coordinator identifiziert (die nächste Halbierung würde den Peer sich selbst als Coordinator zuweisen). An den Coordinator wird nun das Update geschickt. Auch der Coordinator hat einen Coordinator eine Ebene höher, an den die Updates weiter propagiert werden. Mit der Zeit baut sich der Baum von unten her auf und wächst zusammen. Die Coordinatoren können sich Supporting Peers zur Ünterstützung auswählen, diese werden anhand ihrer Kapazitäten ausgewählt.
  • Roter Faden: Soll zeigen dass Queries wirklich nützlich sind für eine komplexe Applikation, Query processing: bottom up, bis Anfrage voll beantwortet werden kann. Evtl. Unklare Details:
  • Roter Faden: Einige Qualitätsaspekte der Lösung und ein Bild zur Visualisierung des Baumes Evtl. Unklare Details:
  • Roter Faden: Anwendungsbeispiel, soll zeigen, dass ohne unsere Lösung die vielen Fragen schwer zu beantworten sind. Ein Effizienz Management System kann sie beantworten Evtl. Unklare Details: Text zu der Animation: Ein Objekt wird in der DHT gespeichert, der Peer fällt aus, Objekt ist weg -> Das ist unerwünscht, Replikation ist die Lösung um die Objekte im Netz zu halten. Abe es stellen sich Fragen (gerade bei vielen Objekten und Peers im Netz) …
  • Roter Faden: Visionäres Ende, keine Details mehr. Ausblick: Selbst-Analyze des P2P Systems, mit dem Ziel eines Selbst-Bewusstseins. Das System soll fehlende Ressourcen erkennen und sich mittels “Ressourcen-Handel” mit den Peers am Leben halten. Ein Service-Provider für die Peers. Evtl. Unklare Details:
  • Transcript

    • 1. Efficiency and Information Management in Peer-to-Peer Systems QuaP2P Kolloquium, 9. April 2008 Efficiency and Information Management 7.31.10.25 peer - to - peer.info 12.5.7.31 95.7.6.10 86.8.10.18 planet - lab.org berkeley.edu 89.11.20.15
    • 2. Overview
      • 1 The Peer-to-Peer Paradigm
      • 1.1 Trends in Peer-to-Peer Research
      • 1.2 Efficiency in Peer-to-Peer Systems
      • 2 Towards QoS & Emergency Call Handling
      • 2.1 Serious Application: Emergency Call Handling
      • 2.2 Towards QoS for Overlay Flows
      • 2.3 Overlay Bandwidth Management
      • 3 Lessons Learned for QoS in P2P Systems
      • 4 Towards a Kind of „Efficiency Management”
      • 4.1 Current State of Efficiency Management
      • 4.2 Our Vision of an Efficiency Management Lifecycle
      • 4.3 Efficiency Management Module
      • 4.4 Current Research Focus: Improved Architecture for Efficiency Management
      • 5 Lessons Learned for Efficiency Management in P2P Systems
    • 3. The Peer-to-Peer Paradigm
      • Peer-to-Peer Systems:
        • Users of a system provide the infrastructure of the system
        • Service is provided from users/peers to users/peers
        • Peer-to-Peer overlays:
          • virtual networks, providing new functionality
          • E.g. Distributed Hash Tables, Keyword-based Search
      • Evolution of applications
        • File sharing:
          • No Quality of Service (QoS) requirements
        • Voice over IP
          • Real-time requirements
        • Video-on-demand
          • Real-time and bandwidth requirements
      1 H(„ my data “) = 3107 2207 7.31.10.25 peer-to-peer.info 12.5.7.31 95.7.6.10 86.8.10.18 planet-lab.org berkeley.edu 2906 3485 2011 1622 1008 709 611 61.51.166.150 ?
    • 4. Trends in Peer-to-Peer Research
      • Quality aspects gain importance
        • Reliability: expected professionalism
        • Ease of Use: Multimedia and interactivity
      • Critical success factor for
        • complex P2P applications
        • modular P2P applications
      • Quality aspects:
        • Adaptability – to scenario, system scale
        • Validity – of stored data
        • Trust – of users and mechanisms
        • Efficiency – ratio between performance and costs
      1.1 Costs Security Quality of P2P Systems Retrievability Coherence Consistency Correctness Performance Scalability Flexibility Stability Dependability Service Provisioning Overlay Operations Individual Node Complete System IP Infrastructure Availability Reliability Robustness / Fault tolerance Integrity Confidentiality Authentication Non - repudiation Trust Validity Efficiency Adaptability Costs Security Quality of P2P Systems Retrievability Coherence Consistency Correctness Performance Scalability Flexibility Stability Dependability Service Provisioning Overlay Operations Individual Node IP Infrastructure Availability Reliability Robustness / Fault tolerance Integrity Non - Trust Validity Efficiency Adaptability
    • 5. Efficiency in Peer-to-Peer Systems
      • Future Peer-to-Peer based applications
        • Modular, component based composition
        • A module has to
          • be highly efficient
          • provide Quality of Service
      • Efficiency can be viewed in three aspects:
        • Resource usage
          • in the peer
          • in the overlay
          • in the network
      • Question in peer-centric view:
        • local knowledge vs. system wide solution
      1.2 Devices IP Infrastructure 3 $ $ P2P Overlay 2 ^ Peer IN OUT 10100110110001000110 1
    • 6. Efficiency Management in Peers Publikationen Systematic First Response Use Case Evaluation D. Bradler, J. Kangasharju, M. Mühlhäuser MODIES 2008 Evaluation of Peer-to-Peer Overlays for First Response D.Bradler, J. Kangasharju, M. Mühlhäuser, MP2P 2008 Taxonomy on Sched. and AQM Mechanisms in P2P Scenarios K. Graffi et al.Technical Report KOM-2007-1/2, 2007 Taxonomy of Scheduling and Active Queue Mechanisms on Overlay Flows Welche Strategien aus der Netzwerk-schicht sind auch im Overlay anwendbar? 01/2007 1. Message Scheduling Before: After : 2. Queue Management Before: After: Queue Limit Overlay Bandwidth Management – Scheduling and Active Queue Management of Overlay Flows Overlay Bandwidth Mana-gement: Sched. and AQM of Overlay Flows, K.Graffi et al., IEEE LCN ‘07 Wie kann man QoS für Overlay Flows erbring-en? Welche Verfahren sind geeignet? Sind lokale Lösungen aus-reichend? 10/2007 Wie kann man QoS (geringe Latenz, Loss) für P2P-basierte Notrufdienste erbringen? Anwendung der Erkenntnisse. ECHoP2P: Emergency Call Handling over P2P Overlays ECHoP2P: Emergency Call Handling over P2P Overlays K.Graffi, A.Kovacevic, et al. P2P-NVE ‘07 11/2007 Connect me to an emergency station! Emergency Call Handling QoS Provisioning 05/2007 P2P Forschung - Übersicht und Herausforderungen, K.Graffi, Quap2P, it-Informa-tion Technology, 2007 Peer-to-Peer Research – Overview and Challenges Was ist Peer-to-Peer? Gemeinschaftsarbeit von QuaP2P mit einem Überblick zu Peer-to-Peer und seinen Herausforder-ungen. Network Wrapper Overlay Layer User UDP TCP Online-time Model Behavior Package Loss Delay Model Bandwidth Kademlia Simulation Engine Application Layer Distribution Strategy Chord Replication Strategy
    • 7. Efficiency Management in P2P Networks 05/2008 Load Balancing for Multimedia Streaming in Heterogeneous Peer-to-Peer Systems K.Graffi,…,A.Kovacevic, et al. ACM NOSSDAV ‘08 Load Balancing for Multimedia Streaming in Heterogeneous Peer-to-Peer Systems Wie kann die Ressourc- en und Aufgabenver-teilung in P2P Netzen fair gestaltet werden? Wie kann man Lasten-verteilung und Hetero-genität unterstützen? Eine allgemeines Over-Overlay für Attribute-based Peer Search. Wie gestaltet man skalierende Informationsarchitekturen? Information and Efficiency Management: Attribute-based Peer Search for Structure P2P Overlays Towards and Information and Efficiency Management Arch… K. Graffi et al. Technical Report KOM-2008-2, 2008 4/2008 7.31.10.25 peer - to - peer.info 12.5.7.31 95.7.6.10 86.8.10.18 graffi.org - lifesocial.org 130.83.139.139 KBR Layer Unified ID Space with Overlay specific IDs KBR using Unified ID Space Coordinator Support Peer Peer From Cells to Organisms: Long-Term Guarantees on Service Provisioning in P2P Networks K. Graffi, A.Kovacevic, et al. ACM SIGAPP NOTERE ’08 From Cells to Orga-nisms: Long-term Guarantees on Service Provisioning in Peer-to-Peer Networks Wie kann man die Optimierung der Res- sourcenverteilung im Overlay als Aufgabe des Overlays gestalten? 06/2008 Information and Efficiency Management in the case of FreePastry Information and Efficiency Management in the Case of FreePastry, K.Graffi et al., to be published ‘08 Ein Statistik-Modul (für Overlay Service Provider) und Attribute-based Peer Search. 3/2008 7.31.10.25 peer - to - peer.info 12.5.7.31 95.7.6.10 86.8.10.18 graffi.org - lifesocial.org 130.83.139.139
    • 8. Towards QoS & Emergency Call Handling 2 Connect me to an emergency station! Emergency Call Handling QoS Provisioning
    • 9. Serious Future Peer-to-Peer Applications
      • Application Areas
        • To exploit self-organization abilities of P2P
      •  Catastrophe scenarios
          • require robust mechanisms
          • E.g. coping with churn
      •  Example: Emergency Call Handling
        • Hard QoS requirements
        • Peer-to-peer mechanisms provide
          • failure-tolerance
          • and Quality of Service
      • See how the efficiency can be improved in this example
        • Learn from it
    • 10. Serious Application: Emergency Call Handling
      • Emergency Call Handling is not supported in VoIP (Skype)
        • 2009: mandatory for VoIP providers
        • P2P fits: all-IP, scalable,
          • but Quality of Service?
      • Requirements
      • 1. Location critical service:
            • Find closest/responsible Emergency Station
      • 2. Quality of Service for P2P flows needed
          • QoS policy: low delay, low loss
            • contact Emergency Station as soon as possible
            • without message loss
      •  Goal:
        • How to solve problem locally ? OR do we need system wide management?
      Alabama Emergency Zones Snapshot of the simulated scenarios Source: US census Source: NENA  Paper at: K. Graffi et al., “ECHoP2P…”, Int. Workshop on P2P-NVE, Nov. 2007 Population density in Alabama 2.1
    • 11. Our Approach for P2P-based Emergency Call Handling
      • Challenge 1: Location-based search requirements
      • Approach: Globase.KOM - Geographical LOcation BAsed SEarch
        • Engineered for requirements of location based services
        • A logical neighbor is a geographical neighbor (like in CAN)
        • Tree structure enables search/lookup in O(log N)
      • Extended with following search mechanisms:
        • Closest peer (Emergency Station)
        • Peer fulfilling a specific criteria (responsibility)
      Search Query  Paper at: A. Kovacevic et al., “Location Awareness…”, Special Issue of the Proc. of the IEEE on Adv. In Distr. Multim. Comm., Jan. 2008 A B C D E G H I J K F A B C D E F G H I J K
    • 12. Towards QoS for Overlay Flows
      • Types of flows in P2P systems
      • Layer 4 traffic flows:
        • Underlay perspective
        • Out of scope
      • Direct P2P communication:
        • File transfer, application traffic, …
        • Few, but large data streams (elephants)
        • (Often) with low priority for the system
      • Overlay flows
        • Multi-hop: maintenance, user queries…
        • Many, small messages (mice)
        • Varying relevance for system
      •  Provide QoS to overlay flow according to its relevance
      2.2 P2P Overlay 2 ^ Peer IN OUT 10100110110001000110 1
    • 13. Scheduling and Active Queue Management
      • Learn from network layer
        • Scheduling
          • Reorder tasks/messages in queue
          • Reference: First-in-First-Out
        • Active Queue Management
          • Drop msg. upon congestion of queue
          • Reference: Drop Tail
      •  Research challenge: how to apply on overlay layer?
      Message Sched.+AQM  See: K. Graffi et al., “Taxonomy on Scheduling/AQM Strategies…” Tech. Report, KOM-TR-2007-1,2, TU Darmstadt 1. Message Scheduling Before: After: 2. Queue Management Before: After: Queue Limit
    • 14. Analyzing Overlay Flows in Kademlia
      • Simulation of Kademlia
      • Setup:
        • PeerfactSim.KOM
        • 10,000 peers, all join, random store/lookup
      • Metrics:
        • # of contacts per peer
        • # of msg per contact
      • Observation
        • Huge number of contacts with few messages each
        • Overlay “flows” and traditional flows differ significantly
      • Conclusion
        • Stateless scheduler needed
        • Quality of service information are stored in messages
       Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Loc. Comp. Networks, Oct. 2007
    • 15. Overlay Bandwidth Management
      • Novel substrate “Network Wrapper”
        • Between overlay and transport layer:
          • Queues messages
          • Applies Scheduling and AQM solution: HiPNOS.KOM
      • HiPNOS.KOM: Highest Priority First, No Starvation
        • Introduce message priorities for Loss and Delay
        • AQM: at congestion, drop message with lowest loss-prio.
        • Scheduling: at free bandwidth, send message with highest delay-prio.
        • Avoid starvation: Periodically increase delay-prio. of queued messages
      • Properties of HiPNOS.KOM
        • Focus on QoS for overlay flows
        • Easy to apply on existing overlays
       Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Loc. Comp. Networks, Oct. 2007 2.3
    • 16. Overlay Bandwidth Management Results
      • Observation:
        • Proportional relations:
          • Delay to delay-priority
          • Loss to loss-priority
      • Results:
        • HiPNOS.KOM provides QoS
          • Regarding delay and loss
          • According to chosen priorities
       Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Loc. Comp. Networks, Oct. 2007
    • 17. Lessons Learned for QoS in P2P Systems
      • Results for Scheduling and AQM
        • Delay and delay-priority, loss and loss-priority are proportional
        • Emergency Calls have always highest priority
        • All other messages have lower priority
        • Quality of service can be provided
      • Lessons learned:
      • IF … known:
          • Optimization criteria
          • Set of all alternatives
      • THEN mechanisms for Quality of Service are easy to adopt
      • Question from beginning:
      • How to solve problem locally ? OR do we need system wide management?
      • Solve it system wide  requires information
          • Necessary for efficient decisions in distributed systems
          • Often missing in Peer-to-Peer systems
      3
    • 18. Towards a Kind of „Efficiency Management” 4 Peers α β λ μ Parameters f( α , β )=…= x g( λ , μ )=…=y h( α , λ )=…=z Models Interpreted state Architecture Choose priorities Efficiency Management Architecture Analysis, Modeling and Interpretation Using Info. to Gain Efficiency
    • 19. Current State of Efficiency Management
      • Each functional layer has its own information/analysis architecture
          • To gather, analyze layer specific information
        • Examples
          • BitTorrent: for tit-for-tat peer selection
          • Replication: which data, on which peers
            • Give me 10 peers with specific availability
          • Skype: for Superpeer selection
            • Find one peer with specific bandwidth and online time
          • Network wrapper: underlay awareness
            • Identify most sent message type
          • Security: Find 5 trustworthy peers
      PAST PAST  See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt 4.1
    • 20. Desired State of Efficiency Management
      • Each functional layer has its own information/analysis architecture
          • To gather, analyze layer specific information
      • Common basic functionality can be “abstracted”, i.e. “extracted”
        • To gather layer specific information
          • Attribute-based peer search
        • To analyze information, (derive optimization goals)
        • To apply results for better decisions
      •  Separate Information/Efficiency Management Layer for this task
       See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt
    • 21. Our Vision of an Efficiency Management Lifecycle
      • Efficiency Management System:
      • To engineer & to build architecture
        • To gather information from peers
        • To retrieve system parameters
      • To analyze component
        • To use system model
        • To prepare statistics
        • To interpret system state
      • With application component
        • To Provide QoS
          • Based on above issues
       Paper at: K. Graffi et al., “From Cells to Organisms …” in Proc. of ACM SIGMM NOTERE, June 2008 4.2 Peers α β λ μ Parameters f( α , β )=…= x g( λ , μ )=…=y h( α , λ )=…=z Models Interpreted state Architecture Choose priorities Efficiency Management Architecture Analysis, Modeling and Interpretation Using Info. to Gain Efficiency
    • 22. Efficiency Management Module
      • Offers:
      • Statistics / Monitoring
          • Number of peers, avg. online times
          • Aggregable information
      • Attribute-based peer search
        • Give me:
          • 3 peers with
          • Storage space > 20Mb
          • Bandwidth > 100kb/s
      • Uses:
        • Can be used on all structured P2P overlay
        • Requires Key-based Routing (KBR)
       See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt 4.3 Efficiency Management Module KBR
    • 23. First Version of the Efficiency Mgmt. Module
      • First step: specific solution for FreePastry
        • For Monitoring / Statistics
        • Attribute-based peer search
      • Built a module for FreePastry
        • Using Scribe to spread data
        • Pub/sub principle
        •  Module can be used!
        • Valuable component for PIPE Architecture
      Underlay: The Internet Structured Overlay: DHT Efficiency Management System Monitoring, Statistics 7.31.10.25 peer - to - peer.info 12.5.7.31 95.7.6.10 86.8.10.18 graffi.org - lifesocial.org 130.83.139.139
    • 24. Current Research Focus: Improved Architecture for Efficiency Management
      • For all structured P2P overlays
        • Covered by common API KBR
        • Usable by all functional layers in a P2P system
      • Function:
        • Overlay-independence
        • Monitoring capabilities
        • Attribute-based peer search
      • Features:
        • Scalability (# of info and peers)
        • Robustness (double-churn)
        • Load-balancing
        • Considering peer heterogeneity
        • Adapting to usage patterns
        • Low overhead
       See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt Structured Overlay: DHT Underlay: The Internet KBR Layer Unified ID Space with Overlay specific IDs KBR using Unified ID Space 4.4 7.31.10.25 peer - to - peer.info 12.5.7.31 95.7.6.10 86.8.10.18 graffi.org - lifesocial.org 130.83.139.139 Efficiency Management System Coordinator Support Peer Peer
    • 25. Efficiency Management Architecture
      • Efficiency Management Architecture
        • Built on underlying structured overlay
        • Communicates via common API
          • Route to PeerID
        • Just an add-on, easy to deploy
      • Principle
        • Each node publishes information updates in the architecture (bottom up)
        • Update-tree is established
        • Each node knows where to send updates to
        • Queries are processed bottom up
       See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt Unified ID Space KBR using Unified ID Space Coordinator Support Peer Peer
    • 26. Efficiency Management Architecture Details
      • Over-overlay:
        • ID space separated in intervals (domains)
        • Peer responsible for a specific ID (e.g. middle) is responsible for ID domain
        • Peers in the domain send updates to this Coordinator
        • Updates propagated upwards the tree
      • Supporting Peers for Load Balancing
        • Coordinator may chose Supporting Peers
        • Good peers chosen by 50/50 ratio
          • Pick e.g. 20 best peers in the domain
          • Best 10 peers in domain advertised one level up
          • Second best 10 peers can be used as support
        • Workload can be delegated to supporting peers
        • Tree depth / peer load adjustable
      ID Space Peers Coordinator Supporting Peer  See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt
    • 27. Queries in the Efficiency Management System
      • Query Type:
        • Give me M peers
        • Fulfilling specific requirements on
          • Bandwidth, storage space, computational capabilities,
          • Online time, peer load, reputation
          • … (wide set of requirements definable)
      • Query processing
        • First sent to coordinator of lowest domain
        • Query traverses bottom-up, until M matching peers found
        • Result is sent then to requesting peer
        • Tradeoff:
          • Upper peers in tree know more
          • Load should be kept on lower levels of the tree
       See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt
    • 28. Structure of the Efficiency Management Arch.
      • Query Performance: O(log N) hops
      • Scalability:
        • Tree-structure of coordinators form information architecture
        • Supporting peers: Strong peers can take the load
      • Robustness:
        • No additional maintenance needed (done by structured overlay)
        • Any peer can fail, no unwanted effects
       See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt
    • 29. Example Application: Replication Layer
      • Content storage in P2P systems
          • Churn is a problem
            • Data may get lost
      •  Replication is a solution
      • Challenges
        • Which files to replicate?
          • Most requested, rarest?
        • At which peers?
          • Most reliable? Highest bandwidth?
        • How many replicas?
          • Depends on requirements on availability.
        • By which peers?
      •  Efficiency Management System allows for answers
    • 30. Lessons Learned for Efficiency Management in P2P Systems
      • Information Management is just ONE part of the Efficiency Management Lifecycle
      • Next steps:
        • Publish Technical Report on distributed Efficiency Management Architecture on IEEE P2P 2008
        • Fine tune on some aspects / limitations
      • Long-term vision:
        • P2P network regulates itself
          • According to QoS constraints towards efficiency
        • From self-organization of the peers to self-consciousness of the system*
        • Requires P2P modeling, data mining, distributed decision making, …
      • Upcoming Applications:
        • P2P-based Grid: Share resources, negotiate service in return with the system*
        • Modularized, layer-interactive, complex applications
      *  Paper at: K. Graffi et al., “From Cells to Organisms …” in Proc. of ACM SIGMM NOTERE, June 2008 5
    • 31. Questions? Kalman Graffi Peer-to-Peer Research Group Dept. of Electrical Engineering and Information Technology Multimedia Communications Lab · KOM Merckstr. 25 · 64283 Darmstadt · Germany Phone (+49) 6151 – 16 49 59 Fax (+49) 6151 – 16 61 52 [email_address] Further information: http://www.KOM.tu-darmstadt.de/ Publications: http://www.KOM.tu-darmstadt.de/Research/Publications/publications.html

    ×