Network and IT Operations
•Download as PPTX, PDF•
3 likes•2,609 views
This webinar focuses on the particular use case of graph databases in Network & IT-Management. This webinar is designed for people who work with Network Management at telecom companies or professionals within industries that handle and rely on complex networks. We’ll start with an overview of Neo4j and Graph-thinking within Networks, explaining how Neworks are naturally modelled as graphs. We’ll explain how graph databases vastly help mitigate some of the major challenges the Network and Security Managers face on daily basis — including intrusions and other cyber crimes, performance optimization, outage simulations, fraud prevention and more.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Network and IT Operations
Similar to Network and IT Operations (20)
More from Neo4j
More from Neo4j (20)
Recently uploaded
Recently uploaded (20)
Network and IT Operations
- 1. Network & IT-operations LEVERAGING CONNECTIONS IN DATA WITH GRAPH DATABASES Webinar, September 15, 2016
- 2. Alessandro Svensson Solutions @ Neo Technology William Lyon Developer Relations @ Neo Technology
- 3. Agenda About Neo4j and the Property Graph Model How Networks are Naturally Graphs Network Graphs (demo) Security Graphs (demo) Wrap up
- 4. The Property Graph Model
- 5. Databases have evolved in order to handle large networks of connected data
- 6. Databases have evolved in order to handle large networks of connected data
- 8. The internet is a graph Huge networks of connected data
- 10. This is data modelled as graph!
- 16. A Graph Is
- 19. Network Graphs
- 20. Networks are Naturally Graphs! What does that mean?
- 23. The Network Operations Center (NOC)
- 24. Monitor health of an entire network Visualize and understand how different components correlate Troubleshoot issues Perform impact analysis Model outage scenarios Requirements Fragmented monitoring tools Inability to correlate problems in different network domains Stale or unreliable data in traditional correlation systems Inefficiencies and high support costs Key Challenges Main purpose of a NOC: Manage, Control, and Monitor for Reliability and Performance
- 25. Different Types of Workloads • Real time event correlation/enrichment/root cause • Real time network analysis & SPOF-detection Operational Analytical • “What if”—analysis for change management • Node centrality, usage analysis, traffic engineering validation • Monitoring strategic transitions (i.e. ATM->IP, 3G->LTE, NOC- >SOC)
- 26. Cross Domain Network & Services Topology “A single coherent, real-time view of customers, services and the network they depend upon” 🏦 💥 Optical & Switching layer Customer Service view IP-Routing layer
- 27. << Enriched event << PRIORITY 1, PLATINUM CUSTOMER IMPACT, LOC, interface AX2431 Example Architecture: Cross Domain Event Correlation/Enrichment >> Raw event >> LOC, interface AX2431 🏦 :DEPENDS_ON :DEPENDS_ON :DEPENDS_ON IF/AX2431
- 28. >> Raw event >> LOC, interface AX2431 << Enriched event << PRIORITY 1, PLATINUM CUSTOMER IMPACT, LOC, interface AX2431 Router 1 Router 2 Switch B SDH Node IFace B1 IFace B4 IFace S7 IFace 15 IFace 22 SDH NodeAX2431 Switch A IFace A1 IFace A4 Switch C IFace C1 IFace C4 IFace 27 Customer Example Architecture: Cross Domain Event Correlation/Enrichment
- 29. Fault Mgmnt System IBM Netcool, HP TeMIP… Event Collector NoSQL store… (1) Raw events (2A) Correlated/enriched/prioritized events (2B) Correlated/enriched/ prioritized events Cross Domain Topology Server (Cluster) Network Inventory Vendor EMS Vendor NMS CRM Device Config, Spreadsheets… Continuous data collection Event Store NoSQL store… Example Architecture: Cross Domain Event Correlation/Enrichment Send it back here Log / key value store
- 30. Change Schedule Change Manager Custom UI Change Planner Change Manager Cross Domain Topology Server (Cluster) Network Inventory Vendor EMS Vendor NMS CRM Device Config, Spreadsheets… Continuous data collection Example Architecture: Change & Impact Analysis
- 31. Why You Should Use Neo4j and Graph Technology in Networks Native Graph Storage • Fast writes for real time topology • Lightning speed traversals for real-time impact computation Schema-less Model: Flexibility / Agility • Ease of ingestion / integration of data from multiple sources • Easy to accommodate changes in a very dynamic environment Standard surfaces / API for integration with other solutions and middleware • Declarative query language (Cypher) • Extendable platform. Server side logic. (Stored Procedures, UEx)
- 32. Dem o “The use of a graph model to show dependencies in an IT network consisting of servers, virtual machines, database servers and application servers.”Network Graphs
- 34. Security Graphs
- 35. The Complex Nature of Network Security Data Siloed and unstructured Data coming from different sources, often evolving and incomplete Dynamic Constant flow of newly generated data Large Accumulated storage of raw data means huge data volumes
- 37. Visualize the entire cyber posture Identify vulnerabilities Prevent attacks Detect attacks Investigate and reduce zero-day losses Requirements Fragmented security tools including firewalls, intrusion detection, vulnerability assessment, SIEM systems Inability to visualize cyber posture Difficult to predict intrusion impact Harder to model scenarios Key Challenges Main purpose of a Security Operating Center: Protect, Detect and Investigate for Security and Loss Prevention
- 38. Common Security Tools Security Intelligence Intrusion Detection System Security Information and Event Management (SIEM) Firewall Manager Vulnerability Scanner Too Much Information, Too Little Context
- 39. Network Infrastructure • Segmentation • Topology • Sensors Cyber Threats • Campaigns • Actors • Incidents • Indicators • TTPs Cyber Posture • Configurations • Vulnerabilities • Policy Rules Mission Dependencies • Objectives • Activities • Tasks • Information
- 40. Network Topology Firewall Rules Host Vulnerabilities XML CSV Graphical Cisco ASA Cisco IOS Juniper JUNOS Juniper ScreenOS Fortinet McAfee Nessus Retina nCirlce Core Impact Foundscan Qualms SAINT nmap Attack Graph Analysis Source: https://neo4j.com/blog/big-data-architecture-cyber-attack-graphs/
- 41. Network Topology Firewall Rules Host Vulnerabilities XML CSV Graphical Cisco ASA Cisco IOS Juniper JUNOS Juniper ScreenOS Fortinet McAfee Nessus Retina nCirlce Core Impact Foundscan Qualms SAINT nmap Source: https://neo4j.com/blog/big-data-architecture-cyber-attack-graphs/ Attack Graph Analysis
- 42. Network Topology Firewall Rules Host Vulnerabilities XML CSV Graphical Cisco ASA Cisco IOS Juniper JUNOS Juniper ScreenOS Fortinet McAfee Nessus Retina nCirlce Core Impact Foundscan Qualms SAINT nmap Source: https://neo4j.com/blog/big-data-architecture-cyber-attack-graphs/ Attack Graph Analysis
- 43. “The little links between incidents, which on the surface look like random meaningless threats, are often what causes the largest problems” — Steve Ragan, CSO Online
- 45. Graphs in Telecommunications Security Operations Centers (SOC)
- 46. Neo4j is used to ensure network security and provides organizations to have a complete visibility of their networks, security rules, firewalls and all the vulnerable points in the network. Neo4j provides real-time query capability, which is required when providing security over huge and highly interconnected networks. Neo4j is used by telecommunication and cyber security firms for understanding a networks cyber posture, identify vulnerabilities and trace network intrusion. How Neo4j is used in Network Security
- 47. Dem o “Using a public dataset of network traffic commonly used for identifying malicious network requests we will see how to model and import data using Cypher.” Security Graphs
- 49. Government Commercial clients Who’s Using Neo4j? Institutions Local Governments Law Enforcement Military & Intelligence
- 50. Neo4j Adoption by Selected Verticals SOFTWARE FINANCIAL SERVICES RETAIL MEDIA & OTHER SOCIAL NETWORKS TELECOM HEALTHC ARE
- 52. “Graph analysis is possibly the single most effective competitive differentiator for organizations pursuing data-driven operations and decisions after the design of data capture. “By the end of 2018, 70% of leading organizations will have one or more pilot or proof-of-concept efforts underway utilizing graph databases.” Towards Graph Inevitability
- 53. “Forrester estimates that over 25% of enterprises will be using graph databases by 2017.” Towards Graph Inevitability
- 54. Valuable Resources! neo4j.com/developer neo4j.com/solutions neo4j.com/product Developers Solutions Product
- 56. Thank you!
Editor's Notes
- Hi, and welcome to this Neo4j-webinar on Network & IT-operations.
- My name is Alessandro Svensson, and I work with Solutions at Neo Technology. Together with me today I have Will from our Developer Relations-team, and he’s gonna provide us with some Neo4j and Cypher demonstrations.
- So thank you for joining us. This is today’s agenda: I’m going to start of with a brief intro to the Property Graph Model. Then we’ll move on to talk about Networks, and in particular what we call the “Network Graphs” and the “Security Graphs” before we wrap things up.
- Okay, so let’s begin with talking a little bit about The Property Graph Model. Some of you are probably familiar with this, but it doesn’t hurt to get into the world of graph thinking a little bit. I think to understand what graphs are and where they come from.
- We have to consider the way databases have evolved in order to handle large networks of…
- …connected data.
- While the traditional relational databases are great for storing data that is heavily structured. That’s just not how data looks anymore.
- What we see is a huge explosion of connectivity and value being created within huge networks of connected data.
- Another way to put it, is to say, data structured as tables, columns and rows… is evolving…
- …to be stored and modelled in a way that the data is relating to itself. And this is what modeling data as graph is.
- A graph is connected data. Which essentially means – datapoints (or nodes as we call it) that have relationships with other datapoints.
- This could be a person, that has checking account with a bank.
- Or a hotel that has rooms, which have availability
- Or it could be people who know other people – who know other people.. who studied together, who work at the same place – who studied with other people, who works somewhere else… etc.
- This is EXACTLY how data appears in Neo4j. One of the benefits with this model is that it is extremely intuitive to understand, very easy to model. The interesting thing is what happens when you start to add more and more relationships to these graphs, and these things start to take off at scale…
- …and forming an extremely powerful foundation from which you can derive value through analysis, perform real-time queries to build extremely powerful applications and algorithms.
- Today we’re going to talk about Network & IT-operations and, while there are many possible examples of how graphs can help in this space, we’re going to focus on two particular topics, The “Network graphs”, and the “Security graphs”.
- Let’s begin with the Network Graph.
- First of all, networks are interesting because they are naturally graphs. What does that mean.
- When we say that networks are graphs, we mean that networks by default are entities that are connected. If you do a quick search on “network topology” you basically end up with a display of a bunch of graphs…
- And if we zoom in on one of them, which seems to be a mesh network of some sort, with routers, gateways — this would be very easy to translate and model into a graph in Neo4j.
- So what are the challenges that managerns in an Network Operation Center would face on a daily bases?
- Well, the purpose of the NOC is to Manage, Control and Monitor for network Reliability and Perfomance. This requires an ability to monitor the health of the entire network, to visualize and understand how different components correlate in order to trouble shoot, perform impact analysis, and to plan and model outage scenarios. The key challenge here is that normally you would use a bunch of fragmented monitoring tools, which gives you an inability to correlate problems in different network domains. This leads to serious inefficiencies and often high support costs and penalties.
- So, what are some of the different workloads within network management. Well, they can be either operational, such as real-time event correlations, or real-time network analysis for Single point of failure detection. Or more analytical, where you would perform “What if”-analysis for change management, or monitoring for strategic transitions for example.
- So let’s look at an example of what a cross-domain Network and Services Topology would look like. Well, the goal here is to have a single coherent, real-time view of customers, services, and the physical network they depend on. Often though, that is not the case. Conceptually, this is how a network normally looks. You will have different domains in different layers. And the ability to map all of this into a unified view is very challenging and complicated.
- Let’s look at an example. Okay, so here we have an event that is being reported to the NOC. Apparently there’s a Loss of Conncetion with the interface AX2431. And in this scenario, you quickly want to find out about the impact of this LOC, and return an enriched event, that will help you priorities the action that you take. And here we seem to have a problem affecting a priority 1-customer. So what you need here is an accurate dependency model, that quickly traverse through your graph. Let’s see how that would look like when we model it as a graph.
- As you can see, the dependancies over the network appears very intuitively. The emphasis here is that Your action will only be as good as your model is. It has to be a real-time view of your topology in order for you to take appropriate measures, in this example a Priority 1-customer, that needs to fixed right away which needs to be reported back to you as you sit, which means that systems that require pre-computing is not an option. So apart form having the visibility of a network topology that is provided by graphs, you also need the real-time capability, that is really only possible with the use of graph databases.
- Another way to look at this is from an architecture point of view. So here we have a cluster of Cross Domain Topology Servers with Neo4j, that ingests data from your different existing sources, and in this example a registered raw event is sent back as an enriched event either to your fault management system, or is logged in your event store.
- Based on the same cross domain topology, there’s of course other types of uses. Your change manager will be able to perform “What if”-analysis and combine those with topology analysis for all the other planned changes within a network. And all this is made so much easier and accurate when modeling your network topology and storing it as graphs in a graph database.
- Transition from a Network Operations Centre to a Service Operations Centre requires reliable and integrated data. Fundamentally a SOC can’t function without accurate and up-to-date knowledge of the changing dependencies and state between a customer, their services and the entire end-to-end network upon which those services depend. So you will need an infrastructure that would look something like this.
- Let’s talk about what the reason is for why so many companies choose to use Neo4j in Network Operations. First of all, Neo4j, is a Native Graph Storage. • Fast writes of real time topology, and lightning speed traversals for real-time impact computation. Second, the graph model is extremely Flexible and Agile • Integration and ingestion of data from multiple sources is extremely easy. • And this makes it easy to accommodate changes in a very dynamic environment. A third reason is the standard surfaces and API for integration with other solutions. • And especially our declarative query language Cypher.
- I’m going to hand over to Will, who will give you a short demonstration on how to use a graph model to show dependencies in an IT network. So over to you Will. Demo 1: IT Network Management. We will use a graph model to show dependencies in an IT network consisting of servers, virtual machines, database servers and application servers. Using Cypher we'll see how to write queries for impact and dependency analysis, answering questions such as, "Do we have a single point of failure in the network?"
- Okay, so let’s talk a little bit about Security Graphs.
- The nature of modeling network security data is very complex.
- This is due the fact that you often have to deal with storage of huge volumes of raw data Also, the data in itself comes from different sources, it’s constantly evolving, and often incomplete At last but not least, you’re dealing with a constant flow of newly generated data.
- So let’s look at some of the challenges that the security operators face on a daily bases.
- The main purpose of a Security Operating Center: is of of course to protect, Detect and Investigate for Security and Loss Prevention. As with network management, This requires, an ability to visualize an entire cyber posture. Identify vulnerabilities, prevent attacks, detect attacks etc. The key challenges here are the fragmented range of security tools that are being used. That don’t provide you with a visibility of context, which makes it difficult to predict intrusions and harder to model different cyber attack scenarios.
- Companies today use a lot of different data-security tools. You’ll have one system for Firewall rules, one intrusion detection system, another system for Security Information and Event management etc leading to an almost overload in information, without providing the necessary context.
- Without a clear view of network topology, that will effect your ability to configure your cyber posture, policy rules etc. Which will affect your ability to respond to cyber threats, identify incidents and impact effectively — essentially affecting your ability to take the appropriate actions, set objectives, form activities etc.
- This is an example of an attack graph analysis, that you can read more about in a blogpost, from the Neo4j-blog. This is an example of a way to analyze data in a way that prevents cyber attacks. It first takes an expression for how a network is segmented and how those segments fit together. Next, it examines the connectivity at a logical level and looks at known vulnerabilities across the endpoints.
- Finally, it determines all the different ways in which an invasion would get routed through the network, including which firewalls it would pass and the rules applied to each of those firewalls. Each one of those source destinations could be single movement in a potential multiple stepping-stone attack moving through your environment. And one example of an attack graph analysis would be to map these steps, expose bottlenecks and show how an attacker could navigate throughout the environment.
- And one example of an attack graph analysis would be to map these steps, expose bottlenecks and show how an attacker could navigate throughout the environment.
- And this a interesting quote, form Steve Ragan — that in network security it’s the little links between incidents that seam almost meaningless, are often what causes the largest problems. And that’s why it’s so important to have a unified cross-domain view of you networks and security measures.
- This is an architecture from that same blog-post. And in this example you see how many different domains are ingested through a REST Web interface, and in imported to a document databases — then all that data is captured and stored in the way the data relates in Neo4j, to be able to get the holistic unified view of all the relationships and dependencies that you would want to analyze…. in a security operating center.
- “…so these are the screens you want to be looking at.” An holistic, unified view of your network topology and security measures.
- Okay, so the reason why Neo4j is used in Network Security is… …first of all, as with network management, in order to perform efficient security management, you need a complete visibility of your topology as it relates to your security rules, firewalls, and vulnerability. And you can’t really achieve that without graphs technology. Second, Noe4j provides real-time query capability, which is required when providing security over huge and highly interconnected networks.
- I’m going to hand over to Will again, who will give you a demonstration on a security graph in Neo4j. Over to you Will. Demo 2: Security Analysis. Network traffic can be modeled as a graph as well. Using a public dataset of network traffic commonly used for identifying malicious network requests we will see how to model and import data using Cypher. We will write queries to analyze patterns in the network traffic to identify malicious activity.
- Summing up, we would like to give you an overview of who’s using Neo4j.
- And we have a solid adoption from everything from government data-operations, to institutions like the world economic forum, ICIJ And of course many commercial clients, in the Fortune 100’s. Like Walmart, eBay, Linkedin, Cisco etc.
- “Neo4j is the most popular graph database around, so we have a privileged view on many use cases and usage of graph databases. Neo4j is today used in verticals as diverse as Software, Financial Services Retail, etc… across a wide range of use cases.”
- So graphs really are everywhere! As we like to put it — We’re moving towards a graph inevitability.
- Gartner talks about, graphs being the single most effective competitive differentiatior for data-driven operations today. And they predict that by the end of 2018, 70% of leading organizations will have at least on or more pilot or POC underway.
- Forrester goes so far as to say that 25% of enterprises will be using graph databases by next year even.
- I also want to give a shout out for som valuable resources for those of you who want to learn more. If you’re a developer, please visit our developers page If you’re interested in case studies and solutions, there’s a lot interesting use cases and case studies at our solutions web page. And if you want to learn more about the product and Neo4j, please visit our product page.
- Lastly, If you are in Bay Area around October 13, don’t forget register for GraphConnect, which is our yearly event in San Fransisco. There you you can sign up for training, listen to customers sharing their graph-projects, meet our Neo engineers and senior staff. It’s a lot of fun, and I strongly recommend it if you’re in town. Visit graphconnect.com for more information on that.
- Thank you for participating in this webinar. Thank you Will for demonstrating a bit of Cypher and Neo4j for us. I hope is was useful. Reach out to us if you have any questions. And I hope to see you again soon.