This document discusses monitoring and managing peer-to-peer systems. It aims to coordinate millions of autonomous peers to provide controlled quality of service. Specifically, it addresses how to monitor system-specific and peer-specific metrics to analyze the current system state. It then proposes a self-configuration framework where the root peer derives and distributes new parameter configurations to reach predefined quality goals. The evaluation shows this approach enables quick convergence to quality intervals while imposing low overhead.
IEEE P2P 2009 - Kalman Graffi - Monitoring and Management of Structured Peer-...Kalman Graffi
The peer-to-peer paradigm shows the potential to provide the same functionality and quality like client/server based systems, but with much lower costs. In order to control the quality of peer-to-peer systems, monitoring and management mechanisms need to be applied. Both tasks are challenging in large-scale networks with autonomous, unreliable nodes. In this paper we present a monitoring and management framework for structured peer-to-peer systems. It captures the live status of a peer-to-peer network in an exhaustive statistical representation. Using principles of autonomic computing, a preset system state is approached through automated system re-configuration in the case that a quality deviation is detected. Evaluation shows that the monitoring is very precise and lightweight and that preset quality goals are reached and kept automatically.
Kalman Graffi - Monitoring and Management of P2P Systems - 2010Kalman Graffi
This is the long presentation of the contributions made in the dissertation of Dr.-Ing. Kalman Graffi - Monitoring and Management of P2P Systems. The talk was given at 29. Sept. 2009 in Madrid / Spain.
IEEE P2P 2009 - Kalman Graffi - Monitoring and Management of Structured Peer-...Kalman Graffi
The peer-to-peer paradigm shows the potential to provide the same functionality and quality like client/server based systems, but with much lower costs. In order to control the quality of peer-to-peer systems, monitoring and management mechanisms need to be applied. Both tasks are challenging in large-scale networks with autonomous, unreliable nodes. In this paper we present a monitoring and management framework for structured peer-to-peer systems. It captures the live status of a peer-to-peer network in an exhaustive statistical representation. Using principles of autonomic computing, a preset system state is approached through automated system re-configuration in the case that a quality deviation is detected. Evaluation shows that the monitoring is very precise and lightweight and that preset quality goals are reached and kept automatically.
Kalman Graffi - Monitoring and Management of P2P Systems - 2010Kalman Graffi
This is the long presentation of the contributions made in the dissertation of Dr.-Ing. Kalman Graffi - Monitoring and Management of P2P Systems. The talk was given at 29. Sept. 2009 in Madrid / Spain.
RFEM is a powerful 3D FEA program helping structural engineers to meet requirements in modern civil engineering. Intuitive handling, user friendliness and efficient data input make working with RFEM easy.
The RFEM program family is based on a modular system. The main program RFEM is used to define structures, materials and loads for planar and spatial structural systems consisting of plates, walls, shells and members. Creating combined structures as well as modeling solid and contact elements is also possible. RFEM provides deformations, internal and support forces as well as soil contact stresses. Add-on modules facilitate the data input by creating structures as well as connections automatically and perform further analyses and designs. The modular approach allows you to combine all programs individually according to your needs. Upgrades at a later time are always possible. RFEM offering numerous interfaces represents the perfect tool for a smooth interaction between CAD and structural analysis in Building Information Modeling (BIM).
System model optimization through functional models execution methodology and...Daniele Gianni
Presentation delivered at the 3rd IEEE Track on
Collaborative Modeling & Simulation - CoMetS'12.
Please see http://www.sel.uniroma2.it/comets12/ for further details.
4.5.12, Lightning Talks, Main Hall: Performance Measurement of MEMS Elements for Information Security of G-Cloud Channels (Roumiana Ilieva, Silvia Bobeva) #CeDEM12
Research Overview about the Multimedia Communications Lab (KOM) - Technische Universität Darmstadt - Germany
Research areas towards Adaptive Seamless Multimedia Communications are: Knowledge & Educational Technologies, Multimedia Technologies & Serious Games, Mobile Systems & Sensor Networks, Self-organizing Systems & Overlay Communications, Service-oriented Computing
Redes de sensores sem fio autonômicas: abordagens, aplicações e desafiosPET Computação
Este curso tem como principal objetivo apresentar aos ouvintes conceitos sobre redes de sensores sem fio (RSSF), protocolos de comunicação para RSSF e conceitos de computação autonômica. Além disso, aplicações focadas nas áreas de monitoramento ambiental, agricultura de precisão, segurança e defesa também serão apresentados.
RFEM is a powerful 3D FEA program helping structural engineers to meet requirements in modern civil engineering. Intuitive handling, user friendliness and efficient data input make working with RFEM easy.
The RFEM program family is based on a modular system. The main program RFEM is used to define structures, materials and loads for planar and spatial structural systems consisting of plates, walls, shells and members. Creating combined structures as well as modeling solid and contact elements is also possible. RFEM provides deformations, internal and support forces as well as soil contact stresses. Add-on modules facilitate the data input by creating structures as well as connections automatically and perform further analyses and designs. The modular approach allows you to combine all programs individually according to your needs. Upgrades at a later time are always possible. RFEM offering numerous interfaces represents the perfect tool for a smooth interaction between CAD and structural analysis in Building Information Modeling (BIM).
System model optimization through functional models execution methodology and...Daniele Gianni
Presentation delivered at the 3rd IEEE Track on
Collaborative Modeling & Simulation - CoMetS'12.
Please see http://www.sel.uniroma2.it/comets12/ for further details.
4.5.12, Lightning Talks, Main Hall: Performance Measurement of MEMS Elements for Information Security of G-Cloud Channels (Roumiana Ilieva, Silvia Bobeva) #CeDEM12
Research Overview about the Multimedia Communications Lab (KOM) - Technische Universität Darmstadt - Germany
Research areas towards Adaptive Seamless Multimedia Communications are: Knowledge & Educational Technologies, Multimedia Technologies & Serious Games, Mobile Systems & Sensor Networks, Self-organizing Systems & Overlay Communications, Service-oriented Computing
Redes de sensores sem fio autonômicas: abordagens, aplicações e desafiosPET Computação
Este curso tem como principal objetivo apresentar aos ouvintes conceitos sobre redes de sensores sem fio (RSSF), protocolos de comunicação para RSSF e conceitos de computação autonômica. Além disso, aplicações focadas nas áreas de monitoramento ambiental, agricultura de precisão, segurança e defesa também serão apresentados.
HOMOGENEOUS MULTISTAGE ARCHITECTURE FOR REAL-TIME IMAGE PROCESSINGcscpconf
In this article, we present a new multistage architecture oriented to real-time complex processing applications. Given a set of rules, this proposed architecture allows the using of different communication links (point to point link, hardware router…) to connect unlimited number of parallel computing elements (software processors) to follow the increasing complexity of algorithms. In particular, this work brings out a parallel implementation of multihypothesis approach for road recognition application on the proposed Multiprocessor Systemon-Chip (MP-SoC) architecture. This algorithm is usually the main part of the lane keeping applications. Experimental results using images of a real road scene are presented. Using a low cost FPGA-based System-on-Chip, our hardware architecture is able to detect and recognize the roadsides in a time limit of 60 mSec. Moreover, we demonstrate that our multistage architecture may be used to achieve good speed-up in solving automotive applications.
LIVE media streaming applications have become more and more popular. IP multicast is the most efficient mechanism but Due to the practical issues of routers, IP multicast has not been widely deployed in the wide-area network infrastructure. The application-level solution build a peer-to-peer (P2P) overlay network out of unicast tunnels across cooperative participating users. P2P media streaming has become a promising approach to broadcast non interactive live media from one source to a large number of receivers. Design of a live P2P streaming system faces many challenges. Therefore, no single application-level multicast stream can meet the requirements of everyone. The proposed architecture aims to provide higher streaming quality and to provide robustness. In the Proposed System the Parallel efforts have been exerted in the media streaming field and networking field to avoid the problem of distributing LIVE video. The tree-based approaches are vulnerable for dynamic group variation but the gossip based mesh-like topology for overlay network systems allow peers to form multiple neighbors, so multilayered video contents are distributed among mesh-like network. Due to this multisource transmission scheme, packets can be exchanged among clients efficiently. In Proposed, The system can achieve improved performance on video delivery quality, bandwidth utilization, and service reliability when using the peer-assisted multipath transmission.
IEEE CRS 2014 - Secure Distributed Data Structures for Peer-to-Peer-based Soc...Kalman Graffi
Jens Janiuk, Alexander Mäcker, Kalman Graffi -
Secure Distributed Data Structures for Peer-to-Peer-based Social Networks
In IEEE CTS ’14: Proceedings of the IEEE International Conference on Collaboration Technologies and Systems, 2014.
Abstract—Online social networks are attracting billions of nowadays, both on a global scale as well as in social enter- prise networks. Using distributed hash tables and peer-to-peer technology allows online social networks to be operated securely and efficiently only by using the resources of the user devices, thus alleviating censorship or data misuse by a single network operator. In this paper, we address the challenges that arise in implementing reliably and conveniently to use distributed data structures, such as lists or sets, in such a distributed hash-table- based online social network. We present a secure, distributed list data structure that manages the list entries in several buckets in the distributed hash table. The list entries are authenticated, integrity is maintained and access control for single users and also groups is integrated. The approach for secure distributed lists is also applied for prefix trees and sets, and implemented and evaluated in a peer-to-peer framework for social networks. Evaluation shows that the distributed data structure is convenient and efficient to use and that the requirements on security hold.
LibreSocial - P2P Framework for Social Networks - OverviewKalman Graffi
Digital social networks promise to activate the social participants and to support them in their interactivity patterns. Private relationships evolve to friendships, professional contacts define competence networks and political opinions emerge to revolutionary trends. Social networks often act as driving force to intensify the social and global relationships.
In future, using the „Peer-to-Peer Framework for Social Networks“ everybody may host easily and out-of-the-box his personal online social network, without operating costs and without security risks. The framework offers a large set of interactive apps, which can be are freely combinable and technically limitless in their applicability.
The operating costs for such a social network are a revolutionary: no expenses arise. Whether a network for 10 users or for a global network of Millions of users, one aspect is common: due to the peer-to-peer technology used, no expenses arise. Researchers led by Dr.-Ing. Kalman Graffi at the University of Paderborn combined in the framework the advantages of decentralized peer-to-peer applications, of an app market as well as the cloud principle.
The social network is maintained in a peer-to-peer fashion through the computational power of the users’ devices, expensive servers are not needed. Still the availability, retrievability and security of the users‘ data are guaranteed. Each user keeps total control on the access control rights of his data. Similar to the main property of the cloud, the network’s capabilities grow elastically with the number of users. Further plugins can be developed easily. An app market that is included allows to provide these plugins in order to extend the capabilities and applications in the social network on the fly.
Enormous application opportunities without operating costs are the main reason to use the „P2P Framework for Social Networks“ emphasize the researchers of the corresponding project group at the University of Paderborn. The software as a prototype is already in use. Contact us for more information.
Timo Klerx and Kalman Graffi. Bootstrapping Skynet: Calibration and Autonomic Self-Control of Structured Peer-to-Peer Networks. In IEEE P2P ’13: Proceedings of the International Conference on Peer-to-Peer Computing, 2013.
Abstract—Peer-to-peer systems scale to millions of nodes and provide routing and storage functions with best effort quality. In order to provide a guaranteed quality of the overlay functions, even under strong dynamics in the network with regard to peer capacities, online participation and usage patterns, we propose to calibrate the peer-to-peer overlay and to autonomously learn which qualities can be reached. For that, we simulate the peer- to-peer overlay systematically under a wide range of parameter configurations and use neural networks to learn the effects of the configurations on the quality metrics. Thus, by choosing a specific quality setting by the overlay operator, the network can tune itself to the learned parameter configurations that lead to the desired quality. Evaluation shows that the presented self-calibration succeeds in learning the configuration-quality interdependencies and that peer-to-peer systems can learn and adapt their behavior according to desired quality goals.
Vitaliy Rapp and Kalman Graffi. Continuous Gossip-based Aggregation through Dynamic Information Aging. In IEEE ICCCN ’13: Proceedings of the International Conference on Computer Communications and Networks, 2013.
Abstract—Existing solutions for gossip-based aggregation in peer-to-peer networks use epochs to calculate a global estimation from an initial static set of local values. Once the estimation converges system-wide, a new epoch is started with fresh initial values. Long epochs result in precise estimations based on old measurements and short epochs result in imprecise aggregated estimations. In contrast to this approach, we present in this paper a continuous, epoch-less approach which considers fresh local values in every round of the gossip-based aggregation. By using an approach for dynamic information aging, inaccurate values and values from left peers fade from the aggregation memory. Evaluation shows that the presented approach for continuous information aggregation in peer-to-peer systems monitors the system performance precisely, adapts to changes and is lightweight to operate.
IEEE ICC 2013 - Symbiotic Coupling of P2P and Cloud Systems: The Wikipedia CaseKalman Graffi
Lars Bremer and Kalman Graffi. Symbiotic Coupling of P2P and Cloud Systems: The Wikipedia Case. In IEEE ICC ’13: Proceedings of the International Conference on Communications, 2013.
Abstract—Comparative evaluations of peer-to-peer protocols through simulations are a viable approach to judge the per- formance and costs of the individual protocols in large-scale networks. In order to support this work, we enhanced the peer- to-peer systems simulator PeerfactSim.KOM with a fine-grained analyzer concept, with exhaustive automated measurements and gnuplot generators as well as a coordination control to evaluate a set of experiment setups in parallel. Thus, by configuring all experiments and protocols only once and starting the simulator, all desired measurements are performed, analyzed, evaluated and combined, resulting in a holistic environment for the comparative evaluation of peer-to-peer systems.
Abstract—Cloud computing offers high availability, dynamic scalability, and elasticity requiring only very little administration. However, this service comes with financial costs. Peer-to-peer systems, in contrast, operate at very low costs but cannot match the quality of service of the cloud. This paper focuses on the case study of Wikipedia and presents an approach to reduce the operational costs of hosting similar websites in the cloud by using a practical peer-to-peer approach. The visitors of the site are joining a Chord overlay, which acts as first cache for article lookups. Simulation results show, that up to 72% of the article lookups in Wikipedia could be answered by other visitors instead of using the cloud.
IEEE HPCS 2013 - Comparative Evaluation of Peer-to-Peer Systems Using Peerfac...Kalman Graffi
Matthias Feldotto and Kalman Graffi. Comparative Evaluation of Peer-to-Peer Systems Using PeerfactSim.KOM. In IEEE HPCS’13: Proceedings of the International Conference on High Per- formance Computing and Simulation, 2013.
IEEE CCNC 2011: Kalman Graffi - LifeSocial.KOM: A Secure and P2P-based Soluti...Kalman Graffi
The phenomenon of online social networks reaches millions of users in the Internet nowadays. In these, users present themselves, their interests and their social links which they use to interact with other users. We present in this paper LifeSocial.KOM, a p2p-based platform for secure online social networks which provides the functionality of common online social networks in a totally distributed and secure manner. It is plugin-based, thus extendible in its functionality, providing secure communication and access-controlled storage as well as monitored quality of service, addressing the needs of both, users and system providers. The platform operates solely on the resources of the users, eliminating the concentration of crucial operational costs for one provider. In a testbed evaluation, we show the feasibility of the approach and point out the potential of the p2p paradigm in the field of online social networks.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
2. The Peer-to-Peer Paradigm
Evolution of applications / QoS demands
Single purpose applications:
File sharing (high bandwidth)
Voice over IP (real-time)
Video-on-demand (real-time and bandwidth)
In the future: Multi-purpose/goal applications
App-based online community platforms
Potential for high user interaction
Possible IT infrastructures:
H(„ my data“ )
Client/Server, server farms, SoA, Cloud, Grid, P2P = 3107
1008 1622 2011
709 2207
Peer-to-peer systems
?
Users build infrastructure 611
3485
2906
Service is provided from users to users
12.5.7.31
Scale well berkeley.edu planet-lab.org
peer-to-peer.info
95.7.6.10
89.11.20.15
Provide unreliable quality of service 86.8.10.18 7.31.10.25
See: K. Graffi, AsKo, et al. “Peer-to-Peer Forschung - Überblick und Herausforderungen” KOM – Multimedia Communications Lab 2
In: it - Information Technology (Methods and Applications of Informatics and Information Technology), vol. 46, no. 5, p. 272-279, July 2007
3. Dynamics in P2P System
Assume fixed combination of mechanisms and configuration
Challenges for providing quality in p2p systems:
Various applications
Distributed storage
Content delivery
User
Discovery and contacting of users
Peer heterogeneity Application
Manage-
Peer capacities ment
Connectivity Overlay
User behavior
Access patterns Devices
Churn
Network
Create a new overlay/mechanism for every case?
No, reuse existing overlays/mechanisms
Goal: Monitor and manage the quality of service of the p2p systems
KOM – Multimedia Communications Lab 3
4. Overview on my work
Goal: Monitor and manage the quality of service of the p2p systems
Target group: p2p system providers
Goal Monitor and managing the quality of service of the p2p systems
Controlled system metrics Reliable resource reservation
Monitoring
… of system-specific metrics, … of peer-specific metrics,
global view on system status capacity-based peer search
Management
Autonomic self-configuration cycle Distributed redundancy control for
to reach and hold preset quality goals guaranteed resource provisioning
Evaluation
Analytical model, simulations, Analytical model, simulations
testbed
Example P2P platform for app-based application composition with monitored QoS
Application
LifeSocial.KOM – a P2P-based Onlince Social Network
KOM – Multimedia Communications Lab 4
5. Management of P2P Systems through
Automated Self-Configuration
Quality of Service
“The well-defined and controllable behavior of a system with respect to quantitative
parameters”
Defined by system provider, aimed by system
Quality goals are predefined
Application and scenario specific
e.g. Metric intervals
Examples
Goal interval for hop count: [7,10]
Standard deviation of peer load: max 500%
Goal
Configuration should adapt to quality goals
Automated meeting of predefined metric intervals
Step 1: Monitor current system state
Step 2: Analysis state, plan new parameter configuration
Step 3: Distribute and adopt new parameter configuration on all peers
KOM – Multimedia Communications Lab 5
6. System- and Peer-specific Information
Global system statistics Peer-specific information
Statistics: Capacities:
Average CPU usage Max / current bandwidth
Average bandwidth utilization Operating System, Java version
Average hop count CPU power
Messages sent / received Free disk space
Number of peers Responsibility range
Message sizess Parent coordinator
… …
Statistical information: List-based concatenation
avg, min, max, standard dev., sum,... E.g. peer 101, up bandwidth 27kb/s, …
Information is aggragatable: Information is NOT aggragatable:
Size of information remains the same Size of information grows with number
Independent of number of peers of peers
Leads to overhead issues
K. Graffi et al. “SkyEye.KOM: An Information Management Over-Overlay for Getting the Oracle View on Structured P2P Systems”Communications Lab
KOM – Multimedia 6
IEEE International Conference on Parallel and Distributed Systems (IEEE ICPADS ‘08), December 2008
7. Gathering System-specific Information
Design decisions:
new layer, for structured p2p overlays, tree
topology, proactive information gathering,
static position assignment
Equal roles for all peers
Load similar for all peers in all positions
Aggregation of statistics
Sum, min, max, average
Standard deviation, count
[µ,σ,σ²,Σ,
min,max]
Statistic updates [µ,σ,σ²,Σ,
Periodically sent to parent peer min,max]
Aggregated in each node ( same size)
Global view in root [µ,σ,σ²,Σ,
Every update is ACKed with global min,max]
view from above
Root analyzes information and pushes a new
configuration to all peers
KOM – Multimedia Communications Lab 7
8. Gathering System-specific Information
Design decisions:
new layer, for structured p2p overlays, tree
topology, proactive information gathering,
static position assignment
Equal roles for all peers
Load similar for all peers in all positions
Aggregation of statistics
Sum, min, max, average
Standard deviation, count + new parameter [µ,σ,σ²,Σ,
configuration min, max]
Statistic updates
[µ,σ,σ²,Σ,
Periodically sent to parent peer min, max]
Aggregated in each node ( same size)
Global view in root
Every update is ACKed with global [µ,σ,σ²,Σ,
min, max]
view from above
Root analyzes information and pushes a new
configuration to all peers
KOM – Multimedia Communications Lab 8
9. Some Remarks on SkyEye.KOM and
Monitoring System Statistics
Why is it generally applicable on DHTs?
Unified ID space, using core DHT functions
(Key based Routing API)
Coordinator_ID
C 0 0,5
Why is it robust against churn? C_ID 0,25 C_ID
1 1 0,75
If peer fails: automatically replaced in the DHT C C
Updates are routed to new peer for aggregation C_ID 0,125 C_ID 0,625 C_ID 0,875
2 2
C2 C_ID 0,375 C C
2
C
Why are costs low?
C_ID
One update: ~1kb, 0,3125 C3
Out + in degree = 1 + tree degree (2 or 4)
Independent of position in the tree! 0,09 0,2 0,31 0,4 0,5 0,6 0,75 0,9
0 1
Age of information:
50 1
Limited by tree depth, O(log (N)) 10
45 DHT
Influenced by update period 15
40 20
30
Just two message types: Update, ACK
Assumed functions:
route(msg, key), isMyKey(key)
KOM – Multimedia Communications Lab 9
10. Gathering Peer-specific Information
Type of information
Individual Peer ID and peer specific information:
Free storage space, CPU power, bandwidth capabilities, online time, …
Responsibility range, node degree, Coordinator ID, …
Desired query
Capacity-based peer search:
Find N peers with e.g. node degree > 20, free storage space > 10MB, online time > 10h
Challenge:
Information cannot be aggregated grows in size
Costs may overload the Coordinators
Supporting Peers for Load Balancing
Each peer defines max. load
Coordinator may choose strong Supporting Peers
Workload delegated to supporting peer
Results in a tree with strong peers
KOM – Multimedia Communications Lab 10
11. Gathering Peer-specific Information: Protocol
Update information: Query format:
Peer 11, RAM = 700MB, Online = 12h 5_of_
RAM_>_1024_Int,CPU_>2048_Int
… Threshold
150MB
Query
C0 Match 1 C0
15MB Match 2
Match 3
Threshold
50MB
42MB
37MB
C1 C1
11MB 20MB
Query
Threshold 10MB
16MB Match 1
15MB
Match 2
C2 SP C2 SP
Threshold Query
200MB Query Query
10MB Match 1
Threshold Address Match 1
20MB 20MB of the Match 2
10MB
Support-Peer Match 3
Match 4
C3 C3 Match 5
10MB
KOM – Multimedia Communications Lab 11
12. System Monitoring Performance
Tree degree = 4
Update interval = 60sec
K.See: K.D. Stingl et al.“Monitoringand Management ofof Structured Peer-to-Peer Systems” IEEE P2P 2009
Graffi, Graffi et al., “Monitoring and Management Structured P2P Systems” submitted to KOM – Multimedia Communications Lab 12
In: IEEE Peer-to-Peer Computing '09 (IEEE P2P’09), September 2009.
13. System Monitoring Costs
Tree degree = 4
Update interval = 60sec
K.See: K.D. Stingl et al.“Monitoringand Management ofof Structured Peer-to-Peer Systems” IEEE P2P 2009
Graffi, Graffi et al., “Monitoring and Management Structured P2P Systems” submitted to KOM – Multimedia Communications Lab 13
In: IEEE Peer-to-Peer Computing '09 (IEEE P2P’09), September 2009.
14. SkyEye.KOM: Tree Growth and Depth
Logarithmic Tree Depth
Example tree
Tree degree (TD) = 2
Balanced, if ID space balanced
Peers may be Coordinators at various
levels not always 2 children
KOM – Multimedia Communications Lab 14
15. Testbed: Topology of the Tree
Topology
link to Coordinator
responsibility
range
With 44 Peers 8 tree
levels are used
(2 above minimum)
Minimum (=O(logN))
not reached due to
non uniform peer ID
distribution
KOM – Multimedia Communications Lab 15
16. SkyEye.KOM: Peer-specific Information
Observations Quality of Information:
Scope of view not complete Less useful peer information is dropped
Determined by individual load limits of Tradeoff: Completeness vs. load limits
Coordinators and Support Peers Peers in the results: >98% are online
Exchange of root
Load limit of new root = 270
Current load = 440
Support peer chosen
Actual screenshot of demo
KOM – Multimedia Communications Lab 16
17. SkyEye.KOM: Peer-specific Information
Query – originators and solvers Effect of query complexity
Scenario with 5000 peers Queries demanding better resources
Most peers around level 10 are solved higher in the tree
Most queries solved between root and “Good” peers bubble up in the tree
peers at level 5
KOM – Multimedia Communications Lab 17
18. A Self-Configuration Framework for P2P
Systems
Steps prepared Metric goals
Predefined quality goals given Metrics Analysis Parameter
Parameters and Plan
P2P overlay parametrizable
Monitoring reveals current system state
Steps needed
Root is deciding component
Derive new parameter configuration
Based on
Predefined metric intervals Metric
Current metrics and parameter goal
configuration Current
Distribute new configuration metric
timely to all peers
Parameters
Goal: System reaches and holds
predefined metric intervals
See: K. Graffi et al., “Monitoring and Management of Structured Peer-to-Peer Systems” KOM – Multimedia Communications Lab 18
In: IEEE Peer-to-Peer Computing '09 (IEEE P2P’09), September 2009.
19. Deriving a new Configuration
Prevent configuration oscillation
Give time for changes to take effect
Analyze slope of metric history
Act only if small, i.e. changes settled
Goal
Manage the p2p system so that it fulfills
our demands on a defined metric set
Evaluation in next step
Monitoring an overlay (Chord)
Set goal intervals: hop count [7;10]
Goal Metric
Static rules: Adapt finger table size by
10% (down) or 100% (up)
KOM – Multimedia Communications Lab 19
20. Starting with High Hop Count
Too large hop count is detected Quick convergence towards goal
Finger table size: increase by 100% One iteration: 12 update intervals
Initial FT size: 20, at end 80 Quality goal is reached and kept
KOM – Multimedia Communications Lab 20
21. Starting with Low Hop Count
Too small hop count is detected Quick convergence towards goal
Finger table size: decrease by 10% One iteration: 12 update intervals
Initial FT size: 160, at end 116 Quality goal is reached and kept
KOM – Multimedia Communications Lab 21
22. Summary on Management of P2P Systems
Main question:
How to control a p2p system so that it fulfills
our demands on a defined metric set?
Self-Configuration Framework
Uses SkyEye.KOM to monitor the system state and deploy new configuration
No additional protocol complexity
Extendable for more metrics and parameters
Evaluation shows:
Overhead is very small (piggybacking parameters in monitoring messages)
Preset quality intervals are quickly reached and hold
KOM – Multimedia Communications Lab 22
23. Summary
Management of P2P overlays
Reach and hold preset quality intervals
Through monitoring and self-configuration
Coordinated resource usage
Parameterization influences quality metrics
Identifying optimization goals needs monitoring
Monitoring: SkyEye.KOM
Global view on statistics of running system:
avg./std./min./max on all metrics
Gathering peer-specific information
Precise yet cost effective monitoring
Self-Configuration Cycle in Chord
Automated rule application
Preset quality intervals are reached and hold
KOM – Multimedia Communications Lab 23
24. Outlook on Reliable Resource Reservations
Reservation managers
Resource providers
KOM – Multimedia Communications Lab 24
25. Thank you for your attention! Questions?
KOM – Multimedia Communications Lab 25