The document provides an overview of community detection in networks. It defines what a community and partition are, and describes several algorithms for partitioning networks into communities:
1. Kernighan and Lin's algorithm from 1970 which iteratively swaps nodes between partitions to minimize the cost.
2. Newman and Girvan's algorithm from 2004 which removes the edge with the highest betweenness centrality on each iteration.
3. Bagrow and Bollt's algorithm from 2008 which expands a shell of nodes out from a seed node, looking for a drop in the total emerging degree to identify community boundaries.
The document also discusses different ways to define communities, assess partition quality, and represent partitioning results as a dendrogram
Quick introduction to community detection.
Structural properties of real world networks, definition of "communities", fundamental techniques and evaluation measures.
Action and content based Community Detection in Social Networksritesh_11
With the exponential growth and availability of data, the techniques for analyzing it are also
maturing. An emerging trend is to model entities and their interactions as a Social Network.
Extracting communities from these networks provides useful insights into the latent structure of
the underlying network as well as the dynamics of how the network evolves.
The modern science of networks has brought significant advances to our understanding of
complex systems. One of the most relevant features of graphs representing real systems is
community structure, or clustering i.e. the organization of vertices in clusters, with many edges
joining vertices of same clusters and few edges joining vertices of different clusters. Such
clusters can be considered as independent groups of graphs playing a similar role.
Clustering Methods and Community Detection with NetworkX. A slide deck for the NTU Complexity Science Winter School.
For the accompanying iPython Notebook, visit: http://github.com/eflegara/NetStruc
Graph Community Detection Algorithm for Distributed Memory Parallel Computing...Alexander Pozdneev
Community detection is an important problem that spans many research areas, such as social networks, systems biology, power grid optimization. The fine-grained communication and irregular access pattern to memory and interconnect limit the overall scalability and performance of existing algorithms. This talk presents a highly scalable parallel algorithm for distributed memory systems. The method employs a novel implementation strategy to store and process dynamic graphs. The scalability analysis of the algorithm was performed using two massively parallel machines, Blue Gene/Q and Power7-IH, for graphs with up to hundreds of billions of edges. Leveraging the convergence properties of the algorithm and the efficient implementation, it is possible to analyze communities of large-scale graphs in just a few seconds. The talk is based on a paper accepted for publication in IPDPS-2015 conference proceedings that was kindly provided by Dr. Fabrizio Petrini (IBM Research).
Quick introduction to community detection.
Structural properties of real world networks, definition of "communities", fundamental techniques and evaluation measures.
Action and content based Community Detection in Social Networksritesh_11
With the exponential growth and availability of data, the techniques for analyzing it are also
maturing. An emerging trend is to model entities and their interactions as a Social Network.
Extracting communities from these networks provides useful insights into the latent structure of
the underlying network as well as the dynamics of how the network evolves.
The modern science of networks has brought significant advances to our understanding of
complex systems. One of the most relevant features of graphs representing real systems is
community structure, or clustering i.e. the organization of vertices in clusters, with many edges
joining vertices of same clusters and few edges joining vertices of different clusters. Such
clusters can be considered as independent groups of graphs playing a similar role.
Clustering Methods and Community Detection with NetworkX. A slide deck for the NTU Complexity Science Winter School.
For the accompanying iPython Notebook, visit: http://github.com/eflegara/NetStruc
Graph Community Detection Algorithm for Distributed Memory Parallel Computing...Alexander Pozdneev
Community detection is an important problem that spans many research areas, such as social networks, systems biology, power grid optimization. The fine-grained communication and irregular access pattern to memory and interconnect limit the overall scalability and performance of existing algorithms. This talk presents a highly scalable parallel algorithm for distributed memory systems. The method employs a novel implementation strategy to store and process dynamic graphs. The scalability analysis of the algorithm was performed using two massively parallel machines, Blue Gene/Q and Power7-IH, for graphs with up to hundreds of billions of edges. Leveraging the convergence properties of the algorithm and the efficient implementation, it is possible to analyze communities of large-scale graphs in just a few seconds. The talk is based on a paper accepted for publication in IPDPS-2015 conference proceedings that was kindly provided by Dr. Fabrizio Petrini (IBM Research).
Despite significant recent advances in the field of face
recognition [10, 14, 15, 17], implementing face verification
and recognition efficiently at scale presents serious chal-
lenges to current approaches. In this paper we present a
system, called FaceNet, that directly learns a mapping from
face images to a compact Euclidean space where distances
directly correspond to a measure of face similarity. Once
this space has been produced, tasks such as face recogni-
tion, verification and clustering can be easily implemented
using standard techniques with FaceNet embeddings as fea-
ture vectors.
Our method uses a deep convolutional network trained
to directly optimize the embedding itself, rather than an in-
termediate bottleneck layer as in previous deep learning
approaches. To train, we use triplets of roughly aligned
matching / non-matching face patches generated using a
novel online triplet mining method. The benefit of our
approach is much greater representational efficiency: we
achieve state-of-the-art face recognition performance using
only 128-bytes per face.
On the widely used Labeled Faces in the Wild (LFW)
dataset, our system achieves a new record accuracy of
99.63%
. On YouTube Faces DB it achieves
95.12%
. Our
system cuts the error rate in comparison to the best pub-
lished result [15] by 30% on both datasets.
Physics Inspired Approaches to Community Detectionfuzzysphere
Community structure is one of the most relevant features of graphs in sociology, biology, computer science and so on. In this slide, the following methods for community detection are reviewed: (1) synchronization, and (2) spinglass.
References
[1] A. Arenas, A. D. Guilera, C. J. P. Vicente, Phys. Rev. Lett. 96, 114102 (2006) [arXiv:cond-mat/0511730]
[2] P. Ronhovde, Z. Nussinov, Phys. Rev. E 81, 046114 (2010) [arXiv:0803.2548]
[3] S.Fortunato, Phys. Rep. 486, 74 (2010) [arXiv:0906.0612]
Human mobility,urban structure analysis,and spatial community detection from ...Song Gao
In the age of Big Data, the widespread use of location-awareness devices has made it possible to collect spatio-temporal individual trajectory datasets for analyzing human activity patterns in both physical space and cyberspace. Aggregation of such data can also support the urban computing studies and the understanding of urban dynamics and spatial networks. The research results can be utilized by urban managers to understand the dynamic spatial interaction patterns between different parts of the city in real-time and may guide them to conduct the optimized transportation infrastructures based on projected demand.
Στην παρούσα εργασία γίνεται μελέτη της οντολογίας μεγάλης κλίμακας FOAF (Friend of a Friend). Η εργασία έγινε στα πλαίσια του Μεταπτυχιακού προγράμματος σπουδών στην "Επιστήμη του Διαδικτύου" του τμήματος Μαθηματικών του Αριστοτελείου Πανεπιστημίου Θεσσαλονίκης.
Big data matrix factorizations and Overlapping community detection in graphsDavid Gleich
In a talk at the Chinese Academic of Sciences Institute for Automation, I discuss some of the MapReduce and community detection methods I've worked on.
Despite significant recent advances in the field of face
recognition [10, 14, 15, 17], implementing face verification
and recognition efficiently at scale presents serious chal-
lenges to current approaches. In this paper we present a
system, called FaceNet, that directly learns a mapping from
face images to a compact Euclidean space where distances
directly correspond to a measure of face similarity. Once
this space has been produced, tasks such as face recogni-
tion, verification and clustering can be easily implemented
using standard techniques with FaceNet embeddings as fea-
ture vectors.
Our method uses a deep convolutional network trained
to directly optimize the embedding itself, rather than an in-
termediate bottleneck layer as in previous deep learning
approaches. To train, we use triplets of roughly aligned
matching / non-matching face patches generated using a
novel online triplet mining method. The benefit of our
approach is much greater representational efficiency: we
achieve state-of-the-art face recognition performance using
only 128-bytes per face.
On the widely used Labeled Faces in the Wild (LFW)
dataset, our system achieves a new record accuracy of
99.63%
. On YouTube Faces DB it achieves
95.12%
. Our
system cuts the error rate in comparison to the best pub-
lished result [15] by 30% on both datasets.
Physics Inspired Approaches to Community Detectionfuzzysphere
Community structure is one of the most relevant features of graphs in sociology, biology, computer science and so on. In this slide, the following methods for community detection are reviewed: (1) synchronization, and (2) spinglass.
References
[1] A. Arenas, A. D. Guilera, C. J. P. Vicente, Phys. Rev. Lett. 96, 114102 (2006) [arXiv:cond-mat/0511730]
[2] P. Ronhovde, Z. Nussinov, Phys. Rev. E 81, 046114 (2010) [arXiv:0803.2548]
[3] S.Fortunato, Phys. Rep. 486, 74 (2010) [arXiv:0906.0612]
Human mobility,urban structure analysis,and spatial community detection from ...Song Gao
In the age of Big Data, the widespread use of location-awareness devices has made it possible to collect spatio-temporal individual trajectory datasets for analyzing human activity patterns in both physical space and cyberspace. Aggregation of such data can also support the urban computing studies and the understanding of urban dynamics and spatial networks. The research results can be utilized by urban managers to understand the dynamic spatial interaction patterns between different parts of the city in real-time and may guide them to conduct the optimized transportation infrastructures based on projected demand.
Στην παρούσα εργασία γίνεται μελέτη της οντολογίας μεγάλης κλίμακας FOAF (Friend of a Friend). Η εργασία έγινε στα πλαίσια του Μεταπτυχιακού προγράμματος σπουδών στην "Επιστήμη του Διαδικτύου" του τμήματος Μαθηματικών του Αριστοτελείου Πανεπιστημίου Θεσσαλονίκης.
Big data matrix factorizations and Overlapping community detection in graphsDavid Gleich
In a talk at the Chinese Academic of Sciences Institute for Automation, I discuss some of the MapReduce and community detection methods I've worked on.
Finding Emerging Topics Using Chaos and Community Detection in Social Media G...Paragon_Science_Inc
In this talk, we describe our recent work in the analysis of Twitter-based network graphs, including the Ebola crisis in 2014 and the stock market in 2015.
Community detection in graphs with NetworKitBenj Pettit
This is a "lightning talk" I gave at the 22nd PyData London meetup on 5 April 2016. The accompanying demonstration code is at https://github.com/benjpettit/networkit-demo
Community detection from a computational social science perspectiveDavide Bennato
This is the talk I gave at the Lipari Summer School on Computational Social Science, 2014. Which are the sociological strategies to detect communities in social media? How we can define a community form a computational social science point of view?
In many scientific areas, systems can be described as interaction networks where elements correspond to vertices and interactions to edges. A variety of problems in those fields can deal with network comparison and characterization.
The problem of comparing and characterizing networks is the task of measuring their structural similarity and finding characteristics which capture structural information. In order to analyze complex networks, several methods can be combined, such as graph theory, information theory, and statistics.
In this project, we present methods for measuring Shannon’s entropy of graphs.
An introductory-to-mid level to presentation to complex network analysis: network metrics, analysis of online social networks, approximated algorithms, memorization issues, storage.
Notes taken while reading the paper "A Tutorial on Spectral Clustering" by Ulrike von Luxburg. Find original paper at http://www.informatik.uni-hamburg.de/ML/contents/people/luxburg/publications/Luxburg07_tutorial.pdf
This set of slides introduces the reader to a subset of the C++ Standard Library called the Standard Template Library (STL). The STL provides a collection of parameterized containers and algorithms, and it is the most successful example of an approach to programming called generic programming. In this presentation, we aim at studying the ideals and concepts of the STL by re-implementing small parts of the library. Specifically, we first show how we can discover requirements on types in order to devise generic algorithms. Then, we focus on how to make algorithms independent of containers through the pivotal abstraction of iterators. To this end, we replicate the standard algorithm for finding the minimum in a sequence (min_element), which we subsequently match with a custom forward iterator over intrusive linked lists of integers. Finally, we see how function objects can be used to customize containers and algorithms alike. This allows us to deepen our understanding of ordering relations, and, in particular, to introduce the concept of strict weak orderings.
This set of slides introduces the reader to the concept of regular types, i.e., user-defined types whose semantics closely resembles that of built-in types. The notion of regular types proved crucial for the development of the C++ Standard Template Library, and it is currently employed in next-gen C++ libraries such as Eric Niebler's range-v3. The presentation serves as a gentle introduction to the topic, and discusses which requirements must be satisfied for a type to be regular. In particular, the concept of equality-preserving copy and assignment is presented, as well as how to define ordering relations that satisfy the requirements of strictness, transitivity and comparability (i.e., that adhere to the trichotomy law).
This set of slides introduces the reader to the concept of resource wrappers, i.e., classes that are responsible for the correct handling of resources of some kind (e.g., memory). In particular, the presentation discusses the design and implementation of a simplified version of std::vector for the specific case of integer elements. In this regard, we first discuss the fundamental role of destructors as a deterministic, general-purpose undo mechanism. Second, we notice that providing an explicit destructor entails the need of a consequent explicit implementation for the copy constructor and copy assignment operator. We conclude with the formulation of the so-called "rule of three".
This set of slides introduces the reader to the basics of memory management in C++ (with elements of C++11 and C++14). Specifically, after presenting the essential idea of memory segmentation, the presentation contrasts automatic and dynamic objects. To this extent the critical issue of memory leaks is introduced. It follows a discussion on the more advanced topics, such as owning and non-owning (i.e., observer) pointers, as well as dynamic arrays and structures.
This set of slides introduces the reader to the concept of operator overloading for user-defined types in C++ (with elements of C++11 and C++14). The exemplary case of the complex class is introduced. It follows a discussion on how to implement mixed-mode arithmetic, which requires mixing member and non-member operator functions. Moreover, the technical tool of friend functions and access functions is discussed.
This set of slides introduces the reader to the concept of multidimensional arrays in C++ (with elements of C++11 and C++14). The true nature of multidimensional arrays is discussed by means of an intermediate type alias. The pheonomenon of array-to-pointer decay and pointer arithmetic is then generalized to arrays of arrays. The presentation proceeds with a discussion on how to pass multidimensional arrays to functions.
This set of slides introduces the reader to the concept of arrays in C++ (with elements of C++11 and C++14). After presenting the array data type, the concept of array-to-pointer decay is introduced. The presentation proceeds with a discussion on how to pass arrays to functions. To this extent, the reader is guided to the use of bounded ranges as the first step towards the use of the Standard Template Library (STL).
An introduction to pointers and references in C++ (with elements of C++11 and C++14). The presentation introduces the readers to the concepts of pointers and references through the pragmatic need of writing a swap function between integers. Generic programming notions (e.g., type constructors) are adopted when useful for the explanation.
Spring MVC - Wiring the different layersIlio Catallo
The presentation focuses on the problem of wiring the different layers of a Spring MVC Web application. More precisely, the presentation explains how to wire the presentation layer (made of Spring MVC controllers) with the service and persistence objects. To this end, the slides first explain how to create the so called back-end (a.k.a. root) container, which will host the persistence and the service objects. Second, the @Repository and @Service annotations are introduced, which permit the framework to auto-discover the application beans. Finally, it is discussed how to wire such beans, through the usage of either the Spring-specific @Autowired annotation or the JEE @Inject annotation.
This set of slides introduces the basic concepts of Java development. First, the difference between Java as the programming language and Java as a platform is stated. Then, the different Java platforms are presented, with a particular interest in Java Standard Edition and Java Enterprise Edition. With respect to the latter, the roles and differences between the many JEE containers are discussed.
How to process request parameters with the Spring MVC framework. Namely, the presentation tackles the three primary concerns when dealing with request parameters: data binding, data buffering and data validation. To this end, the Bean Validation API (JSR-303) is discussed, and the concept of MessageSource for localized error messages is introduced. Moreover, The Post/Redirect/Get (PRG) pattern is presented along with a possible implementation strategy.
An introduction to the basics of the Spring MVC Web framework. The concepts of front controller, controller (handler), model and view are introduced. The whole processing pipeline is discussed, with an in-depth description of the HandlerMapping and ViewResolver strategy interfaces. The alternative representations of the Model (Map, Model and ModelMap) are presented.
This set of slides deals with the problem of building robust Web applications by isolating different areas of concern into different layers. The typical four-layer architecture is presented (consisting of the presentation, service and persistence layer, as well as of the domain model) along with an in-depth discussion on the role and responsibilities of each single layer.
Introduzione al concetto di oggetto nel modello della memoria del C++ e ai suoi possibile tempi di vita (temporaneo, automatico, dinamico, ...). Relazione tra il tempo di vita e la visibilità (scope) di un oggetto. Gestione degli oggetti dinamici per tipi primitivi, strutture e array mediante l'utilizzo di puntatori (raw pointers).
Introduzione agli Array in C++. Argomenti trattati: decadimento a puntatore di un array; conseguenze del meccanismo di decadimento sul passaggio di array a funzioni; array multidimensionali e il concetto di puntatore ad array; come passare array multidimensionali a funzioni; gli iteratori come generalizzazione di un puntatore ad elemento di un array ed una breve scorsa di come usare gli iteratori con gli algoritmi standard (e.g., std::copy) del C++.
Introduzione ai puntatori e ai riferimenti in C++. Viene presentato il problema dello swap, come esempio motivante per l'introduzione della semantica di riferimento. Si procede con l'introduzione del concetto di puntatore, a cui segue una spiegazione dei basilare operatori di referenziazione e dereferenziazione. Il problema dello swap viene risolto mediante puntatori. Si procede con l'introduzione dei riferimenti, come alias di variabili esistenti. Il problema dello swap viene in ultimo risolto mediante riferimenti.
An introduction to the Java Persistence API 2.x (JPA). Namely, this set of slides first introduces the problem of impedance mismatch between the object model and the relation model. Then, the EntityManager's public interface is discussed, together with the parallelism between a persistence context and a L1 cache. The issue of encapsulating the different application's use cases (which are exposed by the service objects) is solved by means of the @Transactional annotation, which provides a declarative way of demarcating the application's transactional boundaries. It follows an in-depth explanation on how to integrate a local, container-managed EntityManager within an existing Spring MVC application, so as to correctly support the propagation of the persistence context throughout the different beans involved in the same transaction. We then turn our attention to the problem of mapping relationships, thus introducing the @OneToMany, @ManyToMany and @OneToOne annotations. The additional topics of how to cascade operations and how to support lazy loading are covered as well. The presentation concludes with a discussion on how to use both the Criteria and Query API (and JPQL) to query the underlying database.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
2. Outline
¡ Communities and Partitions
¡ What is a community?
¡ What is a partition?
¡ Partitioning algorithms
¡ Kerninghan and Lin, 1970
¡ Newman and Girvan, 2004
¡ Bagrow and Bollt, 2008
¡ Assess the quality of good partitions
¡ The impossibility theorem
¡ Quality functions
4. 4
What is a community?
Intuition
¡ Community: a set of tightly
connected nodes
¡ Examples:
¡ People with common
interests
¡ Papers on the same
topics
¡ Scholars working on the
same field
5. 5
What is a community?
Local definitions (1/3)
clique (complete subgraph)
¡ Too strict definition (what to
do if just one link is missing?)
¡ Cliques are hard to find
(exponential complexity in
the graph size)
6. 6
What is a community?
Local definitions (2/3)
Strong community: subgraph
V ⊆ G such that each vertex
has more connection within
the community than with the
rest of the graph
in out
ki (V ) > ki (V ) 8i 2 V
The number of edges The number of
connecting node i to connections toward
other nodes belonging nodes in the rest of the
to V graph
7. 7
What is a community?
Local definitions (3/3)
¡ Strong communitiy definition is too strict
¡ Unrealistic in many real cases
¡ Weak communities: subgraph V ⊆ G such that
the sum of all degrees within V in greater than
the sum of all degrees toward the rest of the
network
¡ A strong community is also weak, while the converse
is not generally true
P in
P out
i2V ki (V )> i2V ki (V )
number of edges connecting
number of edges connecting nodes in V toward nodes in the
nodes in V to other nodes rest of the graph
belonging to V
8. 8
What is a community?
Global definitions (1/2)
¡ Idea: the graph has a community structure if it is
different from the random graph
¡ Random graph: graph such that each pair of
vertices is connected with equal probability p,
independently on the other pairs
¡ Any two vertices have the same probability to be
adjacent
¡ No preferential linking involving
9. 9
What is a community?
Global definitions (2/2)
¡ The graph of interest is compared with the null
model
¡ Null model: a graph which matches the original
in some of its structural features, but which is
otherwise a random graph
¡ Used as term of comparison to verify whether the
graph of interest shows community structures
10. 10
What is a community?
Vertex-based definitions
¡ Idea: communities are subgraphs of vertices similar
to each other
¡ A measure of similarity needs to be defined
¡ If it is possible to embed the vertices in an n-
dimensional Euclidian space, possible (dis)similarity
measures are: q
PN 2
¡ Euclidian distance dA,B = j (ak bk )
PN 2
¡ Manhattan distance dA,B = j |(ak bk ) |
A·B
¡ Cosine similarity dA,B = kAkkBk
¡ With A = (a1, a2, …, aN) and B = (b1, b2, …, bN) vertex
feature vectors
11. 11
What is a community?
Vertex-based definitions
¡ If it is not possible to embed the vertices in
Euclidian space the similarity must be inferred
from the adjacency relationships
¡ Dissimilarity measure based on structural
equivalence:
qP
dij = k6=i,j (Aik Ajk )2
¡ Structural equivalence: two vertices are structural
equivalent if they have the same neighbors,
even if they are not adjacent themselves
¡ if i and j are structural equivalent then dij = 0
12. 12
What is a partition?
¡ Partition: a division of a
graph in clusters, such that
each vertex belongs to one
cluster
¡ If the vertices can be
shared among different
communities the division is
called cover
13. 13
How many partitions we
may have in a graph?
¡ Stirling number of second kind: the number of
possible partitions in k clusters of a graph with n
vertices
⇢
1 k = n, k = 1
S(n, k) =
kS(n 1, k) + S(n 1, k 1) otherwise
¡ Nth Bell number: the total number of possible
partitions n
X
Bn = S(n, k)
k=1
¡ The nth Bell number is huge, even for relatively
small graphs
15. 15
Kernighan and Lin, 1970:
Basic concepts (1/2)
¡ Given:
¡ A graph G = (N,A) of n vertices of weights wi > 0
¡ p a positive number s.t. wi ≤ p
¡ C = (cij) the weighted adjacency matrix (cost matrix)
¡ A k-way partition 𝚪 of G is a set of non-empty,
pairwise disjoint set 𝜐1, …, 𝜐k such that:
k
[
i =G
i=1
The sum of weights of
¡ A partition is admissible if: vertices in 𝜐i is less or
X equal to p
wj p 8i = 1, . . . , k
j2 i
16. 16
Kernighan and Lin, 1970:
Basic concepts (2/2)
¡ The cost T of a partition 𝚪 is the summation of cij over all i and j
such that i and j are in different clusters
5
b cb2
a 1
2
f cf 4
e
c 4
3
T ( ) = cb2 + cf 4
17. 17
Kernighan and Lin, 1970:
2-way uniform partitioning prob.
¡ 2-way uniform partitioning problem: finding a minimal cost
partition of a given graph of 2n vertices (of equal weights) into
two subsets of n vertices
5
b cb2
a 1
2
f cf 4
e
c 4
3
¡ The Kernighan and Lin algorithm is a heuristic for solving the
2-way uniform partitioning problem
18. 18
Kernighan and Lin, 1970:
Basic principle (1/2)
¡ Basic principle: starting with any arbitrary
partition 𝛤 = {A, B} of N try to decrease the initial
cost T by a series of interchanges of elements of
A and B
¡ When no further improvement is possible, the
resulting partition 𝛤’ is locally minimum with
respect to the algorithm
19. 19
Kernighan and Lin, 1970:
Basic principle (2/2)
¡ Given:
¡ 𝛤* = {A*, B*} is a minimum cost 2-way uniform
partition
¡ 𝛤 = {A, B} is a arbitrary 2-way uniform partition
¡ There are subsets X⊂A, Y⊂B with |X| = |Y| such
that interchanging X and Y produces A* and B*
X Y
A B A⇤ = A X +Y
B⇤ = B Y +X
Y X
A⇤ B⇤
20. 20
Kernighan and Lin, 1970:
Internal and external cost
¡ Let’s define for each a∈A :
X
¡ External cost: Ea = cay
y2B
X
¡ Internal cost: Ia = cax
x2A
¡ Cost difference: D a = Ea Ia
¡ Similarly, define Eb, Ib, Db for each b∈B
21. 21
Kernighan and Lin, 1970:
Cost reduction
¡ Lemma 1: Consider any a∈A, b∈B. If a and b
are interchanged, the reduction in cost (i.e., the
gain) is
g=T T 0 = Da + Db 2cab
¡ Lemma 2: Consider any a∈A, b∈B. If a and b
are interchanged, the variations in the cost
difference for all the other nodes are
0
Dx = Dx + 2cxa 2cxb x ⇥ A {a}
0
Dy = Dy + 2cyb 2cya y ⇥ B {b}
22. 22
Kernighan and Lin, 1970:
The algorithm
1. Compute the D values for all elements of N
2. A1 A, B1 B; X1 = ;, Y1 = ;; i 1
3. While i < n Lemma 1
(a) arg maxai 2A,bi 2B gi = Dai + Dbi 2cai bi
(b) Xi+1 Xi [ {ai }, Yi+1 Yi [ {bi };
Lemma 2
(c) Ai+1 Ai {ai }, Bi+1 Bi {bi }
(d) Recalculate the D values for the elements of Ai+1 , Bi+1
(e) i i+1
Pk
4. Choose k to maximize G = i gi k = 1, . . . , n
5. If G > 0 then swap Xk , Yk and go back to 1; if G = 0 exit
23. 23
Newman and Girvan, 2004:
Betweenness (1/2)
¡ All paths from any two
vertices in different
communities pass along the
few inter-community edges
¡ Betweenness: a measure
j
that favors edges that lie i
between communities and
disfavors those that lie inside
communities Bij ≫ 0
24. 24
Newman and Girvan, 2004:
Betweenness (2/2)
¡ Different implementation of betweenness:
¡ Shortest-path betweenness: find the shortest path
between all pairs of vertices and count how many
run along each edge
¡ Random-walk betweenness: expected number of
times that a random walk between a particular pair
of vertices will pass down a particular edge and sum
over all vertex pairs
¡ Current-flow betweenness: absolute value of current
along the edge summed over all source/sink pairs
25. 25
Newman and Girvan, 2004:
Basic principle
¡ Algorithm based on a divisive approach
¡ Basic principle: removes links with the highest
betweenness
26. 26
Newman and Girvan, 2004:
Algorithm
1. Calculate betweennes scores for all edges in
the network
2. Find the edge with the highest score and
remove it from the network
3. Recalculate betweennes for all remaining
edges
4. Repeat from step 2
27. 27
Newman and Girvan, 2004:
Dendrogram
¡ The output of the algorithms
is called dendrogram
¡ Cutting the diagram
horizontally at some height
displays a possible partition
of the graph
FIG. 2: A hierarchical tree or dendrogram illustrating the
type of output generated by the algorithms described here.
The circles at the bottom of the figure represent the indi- FIG. 3
vidual vertices of the network. As we move up the tree the at disc
vertices join together to form larger and larger communities, vertice
as indicated by the lines, until we reach the top, where all are even w
joined together in a single community. Alternatively, we the munity
28. 28
Bagrow and Bollt, 2008:
L-shell
¡ L-shell: given a starting
vertex i, the l-shell is the set
of all the i’s neighbors within
a shortest path distance i
d≤l
¡ Example: 1-shell from
starting vertex i
29. 29
Bagrow and Bollt, 2008:
Emerging degree (1/2)
1
¡ Emerging degree kj(i) of K0 = 6
internal vertex j: the number 0
of edges that connect j to
1
vertices external to the l-
2
shell
3
¡ Total emerging degree Kjl: 4
the total number of
emerging edges from that l-
shell k1 (0) = 1
k2 (0) = 2
¡ Leading edge Sil: the set of
all vertices exactly l steps k3 (0) = 1
away from vertex i k4 (0) = 2
30. 30
Bagrow and Bollt, 2008:
Emerging degree (2/2)
1
¡ Change in the total K0 = 6
emerging degree: for a shell 0
at depth l starting from
1
vertex i is
2
l
l Ki 3
Ki = l 1 4
Ki
k1 (0) = 1
k2 (0) = 2
k3 (0) = 1
k4 (0) = 2
31. 31
Bagrow and Bollt, 2008:
Basic principle
¡ Basic principle: expanding an l-shell outward from
some starting vertex i and comparing the change in
total emerging to some thresholdα
l
Ki < ↵
¡ There are many interconnections within a
community
¡ The total emerging degree tends to increase
¡ The edges connecting the community to the rest of
the graph are less in number
¡ The total emerging degree tends to decrease sharply
32. 32
Bagrow and Bollt, 2008:
Algorithm
1. Select starting vertex i; l 0
2. CM = ;
0
3. Compute Ki
l
4. While Ki < ↵
(a) l l+1
l l
(b) Compute Si ; CM CM [ Si
l l
(c) Compute Ki and Ki
33. 33
Bagrow and Bollt, 2008:
αas “Social acceptance”
¡ The performance of the algorithm is strictly
dependent on the value of α
¡ αcan be thought as a measure of social
acceptance
¡ α≪1 indicates people who are more welcoming of
their neighbors (the l-shell will spread to much of the
network)
¡ α≫1 indicates hermit-like people who are unwilling
to accept even their immediate neighbors into their
communities (the l-shell will stop growing
immediately)
35. 35
Expected properties of a
good partition (1/3)
¡ Problem: How to say that the partition my
algorithm found is good?
¡ Given:
¡ A set N of n ≥ 2 points
¡ A distance function d: N x N → ℝ
¡ A partitioning function f that takes a distance
function d on N and returns a partition 𝚪 on N
36. 36
Expected properties of a
good partition (2/3)
¡ A partition is “good” if it satisfies a set of basic
properties:
¡ Scale invariance: for any distance function d and
any α> 0, we have f(d) = f(α⋅d)
¡ Richness: every partition of N must be a possible
output of f(d)
¡ Consistency: if we produce a d’ by reducing
distances within the clusters and enlarging distance
between the clusters, the same same partition 𝚪
should arise from d’
37. 37
Expected properties of a
good partition (3/3)
¡ The impossibility theorem: for each n ≥ 2, there’s
no partitioning function f that satisfies Scale-
Invariance, Richness and Consistency at the
same time
38. 38
Quality functions
¡ Problem: In practical situations, the communities
are not know ahead of time.
¡ How to asses the quality of the partition the
algorithm found?
¡ It may be convenient to have a quantitative
criterion to assess the goodness of a graph
partition
¡ Quality function: a function that assigns a number
to each partition of a graph
¡ Partitions can be ranked
39. 39
Modularity:
Trace as a metric (1/2)
¡ Given a partition 𝛤 of G =
(V,E), the fraction of edges
that fall within the same
community is
P
Aij (ci , cj )
ij 1 X
P = Aij (ci , cj )
ij Aij 2m ij
red green blue
¡ Where: red 5 0 2
¡ A is the adjacency matrix green 0 9 2 x(1/27)
¡ 𝛿(ci, cj) equals 1 iff ci = cj,
0 otherwise
blue 2 2 11
matrix e
40. 40
Modularity:
Trace as a metric (2/2)
¡ The trace Tr(e) gives the fraction of edges in the
network that connect vertices in the same
community
¡ A good division in communities should have a
high value of trace
¡ Problem: the trace on its own it is not a good
indicator of the quality of the division
¡ Example: placing all vertices in a single community
would give maximal Tr(e) = 1
41. 41
Modularity:
Founding principle
¡ Solution: random graph is not expected to have a
cluster structure
¡ The possible existence of clusters is revealed by
the comparison between:
¡ The actual density of edges in a subgraph
¡ The density one would expect in the subgraph if the
vertices of the graph were attached randomly (null
model)
42. 42
Quality functions:
Modularity function
¡ The modularity is the number of edges falling
within groups minus the expected value of the
same quantity in the case of a randomized
network
1 X
Q= (Aij Pij ) (ci , cj )
2m ij
¡ Pij is the expected number of edges between
vertices i and j in the null model
43. 43
Quality functions:
Modularity’s null model (1/2)
¡ Modularity’s null model: the random graph has to
keep the same degree distribution of the original
graph
¡ A vertex can be attached to any other vertex
¡ It’s simple to compute Pij
44. 44
Quality functions:
Modularity’s null model (2/2)
¡ What is the expected
number of edges between i
and j in the null model?
¡ Given: (i) = ki (j) = kj
¡ Total number of edges m
¡ Degree of i (i) = ki
¡ Degree of j (j) = kj
¡ The number of possible
edges kikj out of 2m
¡ Expected number:
✓ ◆
ki kj 1 X ki kj
Pij = Q= Aij (ci , cj )
2m ij 2m
2m
45. 45
Quality functions:
Modularity function
¡ Modularity,
¡ It can be negative
¡ It equals to 0 if there’s no community division (i.e.,
the whole graph is a single cluster)
¡ It is size-dependent: graphs of different size cannot
be compared
46. 46
Bibliography
¡ F. Radicchi, C. Castellano, F. Cecconi, V. Loreto, D. Parisi - Defining and
identifying communities in networks, Proc. Natl. Acad. Sci. USA, 2004
¡ P. Erdős , A Rényi, On the evolution of random graphs, publication of
the mathematical institute of the Hungarian Academy of Sciences,
1960
¡ R.S. Burt, Positions in networks, Social Forces, 1976
¡ Wikipedia contributors, Stirling numbers of the second kind, Wikipedia,
The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 1 Aug.
2012. Web. 19 Sep. 201
¡ B.W. Kernighan, S. Lin, An Efficient Heuristic Procedure for Partitioning
Graphs, Bell System Tech Journal No. 49, 1970
¡ M.E. Newman, M. Girvan, Finding and evaluating community structure
in networks, Physical Review E, Vol. 69, No. 2.,11 Aug 2003
47. 47
Bibliography
¡ J.P. Bagrow, E.M. Bollt, Local method for detecting communities,
Physical Review E, 2005
¡ J. Kleinberg. An Impossibility Theorem for Clustering. Advances in
Neural Information Processing Systems (NIPS) 15, 2002