Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems

500 views

Published on

Invited Talk by the EU Initiative EMANICS - in Zurich / Switzerland

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
500
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Kalman Graffi - Efficiency and Information Management in Peer-to-Peer Systems

  1. 1. Efficiency and Information Management in Peer-to-Peer Systems Invited Talk by the EU Initiative EMANICS @ 1st EMANICS Workshop on Peer-to-Peer Management University of Zürich, Switzerland, 3. March 2008 Efficiency and Information Management httc – Hessian Telemedia Technology Competence-Center e.V - www.httc.de 12.5.7.31 peerto-peer.info - berkeley.edu planetlab.org - 89.11.20.15 95.7.6.10 KOM - Multimedia Communications Lab 86.8.10.18 7.31.10.25 Prof. Dr.-Ing. Ralf Steinmetz (director) Dept. of Electrical Engineering and Information Technology Dept. of Computer Science (adjunct professor)Dipl.-Math. Dipl.-Inform. Kalman Graffi TUD – Technische Universität DarmstadtProf. Dr.-Ing. Ralf Steinmetz Merckstr. 25, D-64283 Darmstadt, Germany Tel.+49 6151 166150, Fax. +49 6151 166152Ralf.Steinmetz@KOM.tu-darmstadt.de www.KOM.tu-darmstadt.deP2P___resource-mgmt_Uni-Zuerich___080303-v.6.ppt 16. Februar 2011© author(s) of these slides 2008 including research results of the research network KOM and TU Darmstadt otherwise as specified at the respective slide
  2. 2. Overview1 The Peer-to-Peer Paradigm 1.1 Trends in Peer-to-Peer Research 1.2 Quality in Peer-to-Peer Systems 1.3 Serious Future Peer-to-Peer Applications2 Towards QoS & Emergency Call Handling 2.1 Serious Application: Emergency Call Handling 2.2 Our Approach for P2P-based Emergency Call Handling H(„my data“) = 3107 1622 1008 2011 709 2207 2.3 Quality of Service for Overlay Traffic 611 2906 34853 Lessons Learned for QoS in P2P Systems4 Towards a Kind of „Efficiency Management” 12.5.7.31 peer-to-peer.info berkeley.edu planet-lab.org 61.51.166.150 95.7.6.10 86.8.10.18 7.31.10.25 4.1 Current State of Efficiency Management 4.2 Our Vision of an Efficiency Management Lifecycle 4.3 Over-Overlay: Efficiency Management System 4.4 Queries in the Efficiency Management System 4.5 Example Application: Replication Layer5 Lessons Learned for Efficiency Management in P2P Systems KOM – Multimedia Communications Lab 2
  3. 3. Overviewhttp://www.p2p08.org/ H(„my data“) = 3107 1008 1622 2011 709 2207 611 2906 3485 12.5.7.31 peer-to-peer.info berkeley.edu planet-lab.org 61.51.166.150 95.7.6.10 86.8.10.18 7.31.10.25 KOM – Multimedia Communications Lab 3
  4. 4. 1 The Peer-to-Peer ParadigmPeer-to-Peer Systems:  Users of a system provide the infrastructure of the system  Service is provided from users/peers to users/peers  Peer-to-Peer overlays:  virtual networks, providing new functionality H(„my data“) = 3107 1008 1622 2011 709 2207  E.g. Distributed Hash Tables, Keyword-based Search 611 3485 2906 12.5.7.31 planet-lab.org peer-to-peer.info berkeley.edu 61.51.166.150 95.7.6.10 86.8.10.18 7.31.10.25Evolution of applications  File sharing:  No Quality of Service (QoS) requirements  Voice over IP  Real-time requirements  Video-on-demand  Real-time and bandwidth requirements KOM – Multimedia Communications Lab 4
  5. 5. 1.1 Trends in Peer-to-Peer ResearchQuality aspects gain importance Quality of P2P Systems  Reliability: expected professionalism  Ease of Use: Adaptability Efficiency Validity Trust Multimedia and interactivity Scalability Performance Retrievability Dependability Stability Service Coherence Availability ProvisioningCritical success factor for Flexibility Overlay Consistency Reliability  complex P2P applications Operations Robustness/ Correctness Fault tolerance  modular P2P applications Costs Security Individual Node Integrity Complete System ConfidentialityQuality aspects: IP Infrastructure Authentication Non- repudiation  Adaptability – to scenario, system scale  Validity – of stored data  Trust – of users and mechanisms  Efficiency – ratio between performance and costs KOM – Multimedia Communications Lab 5
  6. 6. 1.2 Quality in Peer-to-Peer Systems DFG Research Group FOR 733 @ TU Darmstadt QuaP2P  “Verbesserung der Qualität von Peer-to-Peer-Systemen durch die systematische Erforschung von Qualitätsmerkmalen und deren wechselseitigen Abhängigkeiten“ User Online-time Behavior Approach Model  Evaluation using simulation and Application Layer Simulation Engine prototypes Distribution Strategy Replication Strategy  PeerfactSim.KOM        Overlay Layer  Proof-of-Concept of investigated Kademlia Chord mechanisms using 2 scenarios Please visit Network Wrapper  www.quap2p.tu-darmstadt.de or www.quap2p.de TCP UDP  www.peerfactsim.com Package Loss Delay Model Bandwidth KOM – Multimedia Communications Lab 6 Visit PeerfactSim.KOM at CeBIT 2008, hall 9, stand C22
  7. 7. 1.3 Serious Future Peer-to-Peer ApplicationsFuture Peer-to-Peer based applications  Modular, component based composition  E.g. FreePastry and/with PAST, Scribe,  E.g. POST, SplitStream  A module has to  be highly efficient  provide Quality of ServiceApplication Areas  To exploit self-organization abilities of P2P Catastrophe scenarios  require robust mechanisms  E.g. coping with churnExample: Emergency Call Handling  Hard QoS requirements  Peer-to-peer mechanisms provide failure-tolerance (and QoS) KOM – Multimedia Communications Lab 7
  8. 8. 2 Towards QoS & Emergency Call Handling QoS Provisioning Emergency Call Handling Connect me to an emergency station! KOM – Multimedia Communications Lab 8
  9. 9. 2.1 Serious Application: Emergency Call Handling Emergency Call Handling is not supported in VoIP (Skype)  2009: mandatory for VoIP providers  P2P fits: all-IP, scalable,  but Quality of Service? Requirements 1. Location critical service:  Find closest/responsible Emergency Station 2. Quality of Service for P2P flows needed  QoS policy: low delay, low loss  contact Emergency Station as soon as possible  without message loss Goal:  How to solve problem locally ? OR Source: NENA Source: US census do we need system wide management? Alabama Emergency Zones Population density in Alabama KOM – Multimedia Communications Lab 9 Paper at: K. Graffi et al., “ECHoP2P…”, Int. Workshop on P2P-NVE, Nov. 2007 Snapshot of the simulated scenarios
  10. 10. 2.2 Our Approach for P2P-based Emergency Call Handling Challenge 1: Location-based search requirements Approach: Globase.KOM - Geographical LOcation BAsed SEarch  Engineered for requirements of location based services  A logical neighbor is a geographical neighbor (like in CAN)  Tree structure enables search/lookup in O(log N) Extended with following search mechanisms:  Closest peer (Emergency Station)  Peer fulfilling a specific criteria (responsibility) A B B C D E F G J H F G H I K J Search K E C I Query D A KOM – Multimedia Communications Lab 10 Paper at: A. Kovacevic et al., “Location Awareness…”, Special Issue of the Proc. of the IEEE on Adv. In Distr. Multim. Comm., Jan. 2008
  11. 11. 2.3 Quality of Service for Overlay Traffic Challenge 2: Providing Quality of Service for Overlay Traffic Approach: Scheduling and Active Queue Management (AQM)  Scheduling: Reordering of packets  AQM: to decide which message to drop at congestion 1. Message Scheduling 2. Queue Management Before: Before: After: After: Queue Limit Observation: Classical flows do not exist in P2P overlays Resource  Many small bursts, rarely from the same peers Scheduling &  Requires a stateless solution Queue Mgmt. Existing solutions mainly focus on classical flows Need for approaches for Peer-to-Peer systems KOM – Multimedia Communications Lab 11 See: K. Graffi et al., “Taxonomy on Scheduling/AQM Strategies…” Technical Report, KOM-TR-2007-1,2, TU Darmstadt
  12. 12. Overlay Bandwidth Management Novel substrate “Network Wrapper”  Between overlay and transport layer:  Queues messages  Applies Scheduling and AQM solution: HiPNOS.KOM HiPNOS.KOM: Highest Priority First, No Starvation  Introduce message priorities for Loss and Delay  AQM: at congestion, drop message with lowest loss-prio.  Scheduling: at free bandwidth, send message with highest delay-prio.  Avoid starvation: Periodically increase delay-prio. of queued messages Properties of HiPNOS.KOM  Focus on QoS for overlay flows  Easy to apply on existing overlays KOM – Multimedia Communications Lab 12 Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Local Computer Networks, Oct. 2007
  13. 13. Overlay Bandwidth Management Results Observation: Proportional relations:  Delay to  delay-priority KOM – Multimedia Communications Lab 13 Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Local Computer Networks, Oct. 2007
  14. 14. Overlay Bandwidth Management Results Observation: Proportional relations:  Loss  to  loss-priority KOM – Multimedia Communications Lab 14 Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Local Computer Networks, Oct. 2007
  15. 15. Overlay Bandwidth Management Results Observation:  Proportional relations:  Delay to delay-priority  Loss to loss-priority Results:  HiPNOS.KOM provides QoS  Regarding delay and loss  According to chosen priorities KOM – Multimedia Communications Lab 15 Paper at: K. Graffi et al., “Overlay Bandwidth Management …” in Proc. of IEEE Local Computer Networks, Oct. 2007
  16. 16. 3 Lessons Learned for QoS in P2P SystemsResults for Scheduling and AQM  Delay and delay-priority, loss and loss-priority are proportional  Emergency Calls have always highest priority  All other messages have lower priority  Quality of service can be providedLessons learned: IF … known:  Optimization criteria  Set of all alternatives THEN mechanisms for Quality of Service are easy to adopt  Required Information  Necessary for efficient decisions in distributed systems  Often missing in Peer-to-Peer systems KOM – Multimedia Communications Lab 16
  17. 17. 4 Towards a Kind of „Efficiency Management” Peers Architecture Parameters α Efficiency λ β Management μ Architecture Using Info. Analysis, to Gain Modeling and Efficiency Interpretation f(α, β)=…=x g(λ, μ)=…=y h(α, λ)=…=z Choose Interpreted Models priorities state KOM – Multimedia Communications Lab 17
  18. 18. 4.1 Current State of Efficiency ManagementEach functional layer has its own information/analysis architecture  To gather, analyze layer specific information  Examples PAST  BitTorrent: for Tit-for-tat peer selection  Replication: which data, on which peers  Skype: for Superpeer selection  Network wrapper: underlay awarenessCommon basic functionality can be “abstracted”, i.e. “extracted”  To gather layer specific information  To analyze information, (derive optimization goals)  To apply results for better decisions Separate Information/Efficiency Management Layer for this task KOM – Multimedia Communications Lab 18
  19. 19. 4.2 Our Vision of an Efficiency Management LifecycleEfficiency Management System: Peers ArchitectureTo engineer & to build architecture Parameters  To gather information from peers α  To retrieve system parameters Efficiency λ β Management μTo analyze component Architecture  To use system model  To prepare statistics Using Info. Analysis,  To interpret system state to Gain Modeling andWith application Component Efficiency Interpretation  To Provide QoS  Based on f(α, β)=…=x g(λ, μ)=…=y above issues h(α, λ)=…=z Interpreted Models Choose state priorities KOM – Multimedia Communications Lab 19
  20. 20. 4.3 Over-Overlay: Efficiency Management SystemFor all structured P2P overlays  Covered by common API  Usable by all functional layers in a P2P systemEnables query for: Efficiency Management  M peers with System ID  specific characteristics Space Common API for structured overlaysApplication Examples:  Super-peer choosing Structured  3 peer Overlay: DHT  Storage space > 20Mb 12.5.7.31  Bandwidth > 100kb/s peerto-peer.info - Underlay: berkeley.edu planetlab.org - 89.11.20.15 95.7.6.10 The Internet 86.8.10.18 7.31.10.25 KOM – Multimedia Communications Lab 20
  21. 21. Efficiency Management Architecture Efficiency Management Architecture  Built on underlying structured overlay  Communicates via common API  Route to PeerID ID Space  Just an add-on, easy to deploy Common API for structured overlays Principle  Each node publishes information updates in the architecture  Update-tree is established  Each node knows where to send updates to  Queries are processed bottom up KOM – Multimedia Communications Lab 21 See: K. Graffi et al., “Towards an Information and Efficiency Management Architecture…” Technical Report, KOM-TR-2008-2, TU Darmstadt
  22. 22. Efficiency Management Architecture DetailsOver-overlay:  ID space separated in intervals (domains)  Peer responsible for a specific ID (e.g. middle) is responsible for ID domain  Peers in the domain send updates to this Coordinator  Updates propagated upwards the tree Coordinator Supporting PeerSupporting Peers for Load Balancing  Coordinator may chose Supporting Peers  Good peers chosen by 50/50 ratio ID Space  Pick e.g. 20 best peers in the domain  Best 10 peers in domain advertised one level up Peers  Second best 10 peers can be used as support  Workload can be delegated to supporting peers  Tree depth / peer load adjustable KOM – Multimedia Communications Lab 22
  23. 23. 4.4 Queries in the Efficiency Management SystemQuery Type:  Give me M peers  Fulfilling specific requirements on  Bandwidth, storage space, computational capabilities,  Online time, peer load, reputation  … (wide set of requirements definable)Query processing  First sent to coordinator of lowest domain  Query traverses bottom-up, until M matching peers found  Result is sent then to requesting peer  Tradeoff:  Upper peers in tree know more  Load should be kept on lower levels of the tree KOM – Multimedia Communications Lab 23
  24. 24. Structure of the Efficiency Management Arch.Query Performance: O(log N) hopsScalability:  Tree-structure of coordinators form information architecture  Supporting peers: Strong peers can take the loadRobustness:  No additional maintenance needed (done by structured overlay)  Any peer can fail, no unwanted effects KOM – Multimedia Communications Lab 24
  25. 25. 4.5 Example Application: Replication LayerContent storage in P2P systems  Churn is a problem  Data may get lost Replication is a solutionChallenges  Which files to replicate?  Most requested, rarest?  At which peers?  Most reliable? Highest bandwidth?  How many replicas?  Depends on requirements on availability  By which peers? Efficiency Management System allows for answers KOM – Multimedia Communications Lab 25
  26. 26. Lessons Learned for Efficiency Management in P2P 5 SystemsInformation Management is just ONE part of the Efficiency Management LifecycleNext steps:  To build information analyzing quorum  To process and analyze gathered system parameters  Status determination and prediction  QoS policy determination based on identified QoS requirementsLong-term vision:  P2P network regulates itself  According to QoS constraints towards efficiency  From self-organization of the peers to self-consciousness of the systemUpcoming Applications:  P2P-based Grid: Share resources, negotiate service in return with the system  Modularized, layer-interactive, complex applications KOM – Multimedia Communications Lab 26
  27. 27. Fragen ? – Any Questions ?Mitglied des TechnologiebeiratsBeauftragter für Informations- undKommunikationstechnikdes Landes Hessen KOM – Multimedia Communications Lab 27

×