Presentation about using Neo4j from Django presented at OSCON 2010, Portland OR.
Sample code is available at: https://svn.neo4j.org/components/neo4j.py/trunk/src/examples/python/djangosites/blog/
Graph Database Meetup in Korea #4. 그래프 이론을 적용한 그래프 데이터베이스 활용 사례 bitnineglobal
Graph Database Meetup in Korea #4. Graph Database Usecases based on Graph Theory (그래프 이론을 적용한 그래프 데이터베이스 활용 사례)
국내 유일 그래프 데이터베이스 연구 개발 전문 기업, <비트나인> 주최로 진행된
그래프 데이터베이스 밋업(Meetup) 4번째! "그래프 이론을 적용한 그래프 데이터데이터베이스 활용 사례_ 그래프 이론 및 그래프 알고리즘 소개" 입니다.
10가지 그래프 이론 소개 및 활용 사례를 소개 드렸습니다. 발표자료를 통해 그래프 데이터베이스의 다양한 활용 사례를 확인해 보시기 바랍니다. :)
AgensGraph는 고객의 DB 수집 및 가공을 통한 저장/관리부터, 분석하여 시각화를 통한 구현 목표 기능과의 연동까지! "데이터 흐름의 모든 영역"에서 서비스를 제공하고 있습니다.
☞ 발표 영상 보러가기: https://www.youtube.com/playlist?list=PLGp3huJbWNDhQT4aBCS13udGhYnIc1GnO
☞ 밋업 참가 신청 하러가기: https://www.meetup.com/ko-KR/graphdatabase/
☞ 이메일 문의: hnkim@bitnine.net
☞ 그래프 데이터베이스 솔루션 AgensGraph 직접 사용해 보기: https://bitnine.net/
Graph Database Meetup in Seoul #1. What is Graph Database? (그래프 데이터베이스 소개)bitnineglobal
Graph Database Meetup in Seoul #1. What is Graph Database?
국내 유일 그래프 데이터베이스 연구 개발 전문 기업, <비트나인> 주최로 진행된
그래프 데이터베이스 밋업(Meetup) "그래프 데이터베이스 기본 개념 소개" 입니다.
그래프 데이터베이스의 기본 개념 및 특징, 활용 분야 등에 대해 간략하게 소개하였으며, 추후 진행되는 밋업에서 좀 더 자세한 실제 활용 사례 등을 소개드릴 예정입니다.
밋업 관련 정보는, https://www.meetup.com/ko-KR/graphdatabase/
관련 문의는 hnkim@bitnine.net으로 부탁드립니다.
https://bitnine.net/ 에서 그래프 데이터베이스 솔루션 AgensGraph를 직접 다운로드 하시어 사용해 보실 수 있습니다. :)
Elasticsearch is a very flexible and useful search tool for any application. It helps to store data in multiple scopes, customize search relevance, collect statistics, and improve overall user experience. Let us see how Elasticsearch has been integrated with OroCommerce to improve E-Commerce flows and achieve better results. We will use the OroCommerce application to show real life use cases and how to implement them.
Graph Database Meetup in Korea #4. 그래프 이론을 적용한 그래프 데이터베이스 활용 사례 bitnineglobal
Graph Database Meetup in Korea #4. Graph Database Usecases based on Graph Theory (그래프 이론을 적용한 그래프 데이터베이스 활용 사례)
국내 유일 그래프 데이터베이스 연구 개발 전문 기업, <비트나인> 주최로 진행된
그래프 데이터베이스 밋업(Meetup) 4번째! "그래프 이론을 적용한 그래프 데이터데이터베이스 활용 사례_ 그래프 이론 및 그래프 알고리즘 소개" 입니다.
10가지 그래프 이론 소개 및 활용 사례를 소개 드렸습니다. 발표자료를 통해 그래프 데이터베이스의 다양한 활용 사례를 확인해 보시기 바랍니다. :)
AgensGraph는 고객의 DB 수집 및 가공을 통한 저장/관리부터, 분석하여 시각화를 통한 구현 목표 기능과의 연동까지! "데이터 흐름의 모든 영역"에서 서비스를 제공하고 있습니다.
☞ 발표 영상 보러가기: https://www.youtube.com/playlist?list=PLGp3huJbWNDhQT4aBCS13udGhYnIc1GnO
☞ 밋업 참가 신청 하러가기: https://www.meetup.com/ko-KR/graphdatabase/
☞ 이메일 문의: hnkim@bitnine.net
☞ 그래프 데이터베이스 솔루션 AgensGraph 직접 사용해 보기: https://bitnine.net/
Graph Database Meetup in Seoul #1. What is Graph Database? (그래프 데이터베이스 소개)bitnineglobal
Graph Database Meetup in Seoul #1. What is Graph Database?
국내 유일 그래프 데이터베이스 연구 개발 전문 기업, <비트나인> 주최로 진행된
그래프 데이터베이스 밋업(Meetup) "그래프 데이터베이스 기본 개념 소개" 입니다.
그래프 데이터베이스의 기본 개념 및 특징, 활용 분야 등에 대해 간략하게 소개하였으며, 추후 진행되는 밋업에서 좀 더 자세한 실제 활용 사례 등을 소개드릴 예정입니다.
밋업 관련 정보는, https://www.meetup.com/ko-KR/graphdatabase/
관련 문의는 hnkim@bitnine.net으로 부탁드립니다.
https://bitnine.net/ 에서 그래프 데이터베이스 솔루션 AgensGraph를 직접 다운로드 하시어 사용해 보실 수 있습니다. :)
Elasticsearch is a very flexible and useful search tool for any application. It helps to store data in multiple scopes, customize search relevance, collect statistics, and improve overall user experience. Let us see how Elasticsearch has been integrated with OroCommerce to improve E-Commerce flows and achieve better results. We will use the OroCommerce application to show real life use cases and how to implement them.
Neo4j is a powerful and expressive tool for storing, querying and manipulating data. However modeling data as graphs is quite different from modeling data under a relational database. In this talk, Michael Hunger will cover modeling business domains using graphs and show how they can be persisted and queried in Neo4j. We'll contrast this approach with the relational model, and discuss the impact on complexity, flexibility and performance.
This developer-focused webinar will explain how to use the Cypher graph query language. Cypher, a query language designed specifically for graphs, allows for expressing complex graph patterns using simple ASCII art-like notation and offers a simple but expressive approach for working with graph data.
During this webinar you'll learn:
-Basic Cypher syntax
-How to construct graph patterns using Cypher
-Querying existing data
-Data import with Cypher
-Using aggregations such as statistical functions
-Extending the power of Cypher using procedures and functions
In this presentation, Raghavendra BM of Valuebound has discussed the basics of MongoDB - an open-source document database and leading NoSQL database.
----------------------------------------------------------
Get Socialistic
Our website: http://valuebound.com/
LinkedIn: http://bit.ly/2eKgdux
Facebook: https://www.facebook.com/valuebound/
Twitter: http://bit.ly/2gFPTi8
Neo4j is a powerful and expressive tool for storing, querying and manipulating data. However modeling data as graphs is quite different from modeling data under a relational database. In this talk, Michael Hunger will cover modeling business domains using graphs and show how they can be persisted and queried in Neo4j. We'll contrast this approach with the relational model, and discuss the impact on complexity, flexibility and performance.
This developer-focused webinar will explain how to use the Cypher graph query language. Cypher, a query language designed specifically for graphs, allows for expressing complex graph patterns using simple ASCII art-like notation and offers a simple but expressive approach for working with graph data.
During this webinar you'll learn:
-Basic Cypher syntax
-How to construct graph patterns using Cypher
-Querying existing data
-Data import with Cypher
-Using aggregations such as statistical functions
-Extending the power of Cypher using procedures and functions
In this presentation, Raghavendra BM of Valuebound has discussed the basics of MongoDB - an open-source document database and leading NoSQL database.
----------------------------------------------------------
Get Socialistic
Our website: http://valuebound.com/
LinkedIn: http://bit.ly/2eKgdux
Facebook: https://www.facebook.com/valuebound/
Twitter: http://bit.ly/2gFPTi8
Neo4j is a graph database (nodes and relationships) and is the perfect fit for some types of problem. Within that domain Neo4j is much, much faster than SQL and easier to query. Py2neo is a Python binding to Neo4j. The live presentation showed how to create word transformation puzzles e.g. getting from "stores" to "slaked" by one latter transformations where each intermediate step is a valid word. One solution is "stores"->"stored"->"stared"->"staked"->"slaked".
Beim Bauen von Software werden tolle Technologien, Programmiersprachen und Tools eingesetzt. Das ist gut und richtig. Aber leider wird dabei oft aus den Augen verloren, dass das Entscheidende nicht die Technik, sondern die *Fachlichkeit* ist. Wenn wir in der Software nicht das fachliche Modell abbilden, dann wird sie unseren Anwendern nicht bei Ihrer Arbeit helfen. Davor schützt uns keine Technologie der Welt. Um das zu verhindern, zeige ich in diesem Vortrag, wie man die Architektur von E-Commerce-Systemen so baut, dass sie die Fachlichkeit darstellt und wie Domain Driven Design (DDD) und Microservices uns dabei helfen können.
Compelling location-based services require more than simple “what’s near me?” operations. The Open Street Map dataset is a perfect example of a rich geographically-based wiki that can be used for much more than map rendering.
With the newly released Neo4j Spatial, any data can be adapted to complex queries with geographic components like “Select all streets in the Municipality of NYC where at least 2 of my friends are walking right now”.
The talk will demonstrate the important benefits of modeling geodata in a graph, the main components needed to expose data to geo stacks like map servers, and explain how the Open Street Map dataset is modeled in Neo4j. I’ll show how using Neo4j unlocks the full potential of the OSM data far beyond just rendering maps.
There will also be some cool examples of Neo4j Spatial, from Telecomms network planning, Web-based AJAX GIS systems, topology editing and routing to REST and Web Feature Service endpoints, all in a single stack.
This is Location-based Services on steroids!
Natural Language Processing and Graph Databases in LumifyCharlie Greenbacker
Lumify is an open source platform for big data analysis and visualization, designed to help organizations derive actionable insights from the large volumes of diverse data flowing through their enterprise. Utilizing both Hadoop and Storm, it ingests and integrates virtually any kind of data, from unstructured text documents and structured datasets, to images and video. Several open source analytic tools (including Tika, OpenNLP, CLAVIN, OpenCV, and ElasticSearch) are used to enrich the data, increase its discoverability, and automatically uncover hidden connections. All information is stored in a secure graph database implemented on top of Accumulo to support cell-level security of all data and metadata elements. A modern, browser-based user interface enables analysts to explore and manipulate their data, discovering subtle relationships and drawing critical new insights. In addition to full-text search, geospatial mapping, and multimedia processing, Lumify features a powerful graph visualization supporting sophisticated link analysis and complex knowledge representation.
Charlie Greenbacker, Director of Data Science at Altamira, will provide an overview of Lumify and discuss how natural language processing (NLP) tools are used to enrich the text content of ingested data and automatically discover connections with other bits of information. Joe Ferner, Senior Software Engineer at Altamira, will describe the creation of SecureGraph and how it supports authorizations, visibility strings, multivalued properties, and property metadata in a graph database.
I give a talk through my Graph Database and Python learning journey at PyCon Australia 2015. It should be up on PyVideo soon enough.
Note: A great question was asked regarding why I didn't cover Postgres on the "what should I use" slide. That was a great question. Definitely consider Postgres, especially if you've got existing expertise in it. Rhys Elsemores talk (Just Use Postgres) at the same conference is excellent.
Complex hierarchical relationships between entities can only be mapped with difficulty in a relational database and demanding queries are usually quite slow.
Graph databases are optimized for exactly these kinds of relationships and can provide high-performance results even with huge amounts of data. Moreover, not only the entities that are stored in the database, have attributes, but also their relationships. Queries can look at entities as well as their relationships.
Get to know the basics of graph databases, using Neo4j as an example, and see how it is used C# projects.
The trend nowadays is to represent the relationships between entities in a graph structure. Neo4j is a NOSQL graph database, which allows for fast and effective queries on connected data. Implementation of own algorithms is possible, which can improve the functionality of built in API. We make use of the graph database to model and recommend movies and other media content.
Social phenomena is coming. We have lot’s of social applications that we are using every day, let’s say Facebook, twitter, Instagram. Lot’s of such kind apps based on social graph and graph theory. I would like to share my knowledge and expertise about how to work with graphs and build large social graph as engine for Social network using python and Graph databases. We'll compare SQL and NoSQL approaches for friends relationships.
A NOSQL Overview And The Benefits Of Graph Databases (nosql east 2009)Emil Eifrem
Presentation given at nosql east 2009 in Atlanta. Introduces the NOSQL space by offering a framework for categorization and discusses the benefits of graph databases. Oh, and also includes some tongue-in-cheek party poopers about sucky things in the NOSQL space.
Spring Data Graph is an integration library for the open source graph database Neo4j and has been around for over a year, evolving from its infancy as brainchild of Rod Johnson and Emil Eifrem. It supports transparent AspectJ based POJO to Graph Mapping, a Neo4jTemplate API and extensive support for Spring Data Repositories. It can work with an embedded graph database or with the standalone Neo4j Server.
The session starts with a short introduction to graph databases. Following that, the different approaches using Spring Data Graph are explored in the Cineasts.net web-app, a social movie database which is also the application of the tutorial in the Spring Data Graph Guidebook. The session will also cover creating a green-field project using the Spring Roo Addon for Spring Data Graph and deploying the App to CloudFoundry.
Join Objectivity, Inc.’s VP of Product Management, Brian Clark, in a discussion of the latest trends in Big Data Analytics, defining what is Big Data and understanding how to maximize your existing architectures by utilizing NOSQL technologies to improve functionality and provide real-time results. There will be a focus on relationship analytics as well as an introduction to NOSQL data stores, object and graph databases, such as the architecture behind Objectivity/DB and InfiniteGraph.
While Hadoop is the dominant "Big Data" tool suite today, it's a first-generation technology. I discuss its strengths and weaknesses, then look at how we "should" be doing Big Data and currently-available alternative tools.
Big Data is used to store huge volume of both structured and unstructured data which is so large and is
hard to process using current / traditional database tools and software technologies. The goal of Big Data
Storage Management is to ensure a high level of data quality and availability for business intellect and big
data analytics applications. Graph database which is not most popular NoSQL database compare to
relational database yet but it is a most powerful NoSQL database which can handle large volume of data in
very efficient way. It is very difficult to manage large volume of data using traditional technology. Data
retrieval time may be more as per database size gets increase. As solution of that NoSQL databases are
available. This paper describe what is big data storage management, dimensions of big data, types of data,
what is structured and unstructured data, what is NoSQL database, types of NoSQL database, basic
structure of graph database, advantages, disadvantages and application area and comparison of various
graph database.
A Study on Graph Storage Database of NOSQLIJSCAI Journal
Big Data is used to store huge volume of both structured and unstructured data which is so large and is
hard to process using current / traditional database tools and software technologies. The goal of Big Data
Storage Management is to ensure a high level of data quality and availability for business intellect and big
data analytics applications. Graph database which is not most popular NoSQL database compare to
relational database yet but it is a most powerful NoSQL database which can handle large volume of data in
very efficient way. It is very difficult to manage large volume of data using traditional technology. Data
retrieval time may be more as per database size gets increase. As solution of that NoSQL databases are
available. This paper describe what is big data storage management, dimensions of big data, types of data,
what is structured and unstructured data, what is NoSQL database, types of NoSQL database, basic
structure of graph database, advantages, disadvantages and application area and comparison of various
graph database.
Big Data is used to store huge volume of both structured and unstructured data which is so large and is
hard to process using current / traditional database tools and software technologies. The goal of Big Data
Storage Management is to ensure a high level of data quality and availability for business intellect and big
data analytics applications. Graph database which is not most popular NoSQL database compare to
relational database yet but it is a most powerful NoSQL database which can handle large volume of data in
very efficient way. It is very difficult to manage large volume of data using traditional technology. Data
retrieval time may be more as per database size gets increase. As solution of that NoSQL databases are
available. This paper describe what is big data storage management, dimensions of big data, types of data,
what is structured and unstructured data, what is NoSQL database, types of NoSQL database, basic
structure of graph database, advantages, disadvantages and application area and comparison of various
graph database.
A Study on Graph Storage Database of NOSQLIJSCAI Journal
Big Data is used to store huge volume of both structured and unstructured data which is so large and is
hard to process using current / traditional database tools and software technologies. The goal of Big Data
Storage Management is to ensure a high level of data quality and availability for business intellect and big
data analytics applications. Graph database which is not most popular NoSQL database compare to
relational database yet but it is a most powerful NoSQL database which can handle large volume of data in
very efficient way. It is very difficult to manage large volume of data using traditional technology. Data
retrieval time may be more as per database size gets increase. As solution of that NoSQL databases are
available. This paper describe what is big data storage management, dimensions of big data, types of data,
what is structured and unstructured data, what is NoSQL database, types of NoSQL database, basic
structure of graph database, advantages, disadvantages and application area and comparison of various
graph database.
Similar to Django and Neo4j - Domain modeling that kicks ass (20)
Presented at JavaOne 2013, Tuesday September 24.
"Data Modeling Patterns" co-created with Ian Robinson.
"Pitfalls and Anti-Patterns" created by Ian Robinson.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
2. It all started with this guy.
Emil Eifrem, CEO of Neo
Technology. We picked him
apart, and inside his brain
we found the base for a
database that models the
connections in bet ween
entities.
2
3. It all started with this guy.
Emil Eifrem, CEO of Neo
Technology. We picked him
apart, and inside his brain
we found the base for a
database that models the
connections in bet ween
entities.
Image credits: US Army
2
4. It all started with this guy.
Emil Eifrem, CEO of Neo
Technology. We picked him
apart, and inside his brain
we found the base for a
database that models the
connections in bet ween
entities.
2
5. Neo4j
It all started with this guy.
Emil Eifrem, CEO of Neo
Technology. We picked him
apart, and inside his brain
we found the base for a
database that models the
connections in bet ween
entities.
2
7. The problems NOSQL focuses on
Relational database
Requirement of application
Focus area of many
NOSQL Databases ๏ Huge amounts of data
Performance
๏ (mostly) Disjoint data
Salary List
๏ Heavy load most focus on...
Majority of
Webapps ๏ Many concurrent writers
Social network
Semantic Trading
All NOSQL databases focus
on solving problems where
RDBMSes fail.
While this handles the
load, it lacks in “social”
} custom
Data complexity 4
8. The evolution of data
Giant
Global
Graph (GGG)
Ontologies
RDF
Folksonomies
Information connectivity
Tagging
Wikis User-generated
content
Blogs
... but it turns out that
RSS data evolves to become
MORE interconnected
Hypertext (as well as greater sizes)
Text documents
web 1.0 web 2.0 “web 3.0”
1990 2000 2010 2020 5
9. Neo4j is a Graph Database
Graph databases FOCUS
on the interconnection
bet ween entities.
6
10. IS_A
Neo4j Graph Database
Graph databases FOCUS
on the interconnection
bet ween entities.
6
11. Scaling to size vs. Scaling to complexity
Size
Key/Value stores
Bigtable clones
Document databases
Graph databases
Complexity
7
12. Scaling to size vs. Scaling to complexity
Size
Key/Value stores
Bigtable clones
Document databases
Graph databases
Billions of nodes
and relationships
> 90% of use cases
Complexity
7
13. What is Neo4j?
๏ Neo4j is a Graph Database
• Non-relational (“#nosql”), transactional (ACID), embedded
• Data is stored as a Graph / Network
‣Nodes and relationships with properties
‣“Property Graph” or “edge-labeled multidigraph”
• Schema free, bottom-up data model design
๏ Neo4j is Open Source / Free (as in speech) Software
Prices are available at
http://neotechnology.com/
• AGPLv3
Contact us if you have
questions and/or special
license needs (e.g. if you
• Commercial (“dual license”) license available
want an evaluation license)
‣First server is free (as in beer), next is inexpensive 8
14. More about Neo4j
๏ Neo4j is stable
• In 24/7 operation since 2003
๏ Neo4j is in active development
• Neo Technology received VC funding October 2009
๏ Neo4j delivers high performance graph operations
• traverses 1’000’000+ relationships / second
on commodity hardware
(1000~2500 traversals/ms)
9
15. Building business applications with Neo4j
๏ Try it out! It’s all open source!
• Build a prototype, find out your needs and how Neo4j matches
• AGPL this stage thisusers should have access to your code
- at
says all your
is your employees / co-workers
10
16. Building business applications with Neo4j
๏ Try it out! It’s all open source!
• Build a prototype, find out your needs and how Neo4j matches
• AGPL this stage thisusers should have access to your code
- at
says all your
is your employees / co-workers
๏ Put it in front of users! The license is free for the first server!
• Contact Neo Technology sales to get a free single server license
• You’ll (probably) not have massive load the first days
10
17. Building business applications with Neo4j
๏ Try it out! It’s all open source!
• Build a prototype, find out your needs and how Neo4j matches
• AGPL this stage thisusers should have access to your code
- at
says all your
is your employees / co-workers
๏ Put it in front of users! The license is free for the first server!
• Contact Neo Technology sales to get a free single server license
• You’ll (probably) not have massive load the first days
๏ As you grow, Neo4j grows with you!
• Aslicenseneeds and revenue increase you can by an advanced
your
(prices are resonable)
10
18. Graphs are all around us
A B C D ...
1 17 3.14 3 17.79333333333
2 42 10.11 14 30.33
3 316 6.66 1 2104.56
4 32 9.11 592 0.492432432432
5 Even if this spreadsheet looks
like it could be a fit for a RDBMS
2153.175765766
it isn’t:
•RDBMSes have problems with
... extending indefinitely on both
rows and columns
•Formulas and data
dependencies would quickly lead
to heavy join operations
11
19. Graphs are all around us
A B C D ...
1 17 3.14 3 = A1 * B1 / C1
2 42 10.11 14 = A2 * B2 / C2
3 316 6.66 1 = A3 * B3 / C3
4 32 9.11 592 = A4 * B4 / C4
5 = SUM(D2:D5)
With data dependencies
... the spread sheet turns
out to be a graph.
12
20. Graphs are all around us
A B C D ...
1 17 3.14 3 = A1 * B1 / C1
2 42 10.11 14 = A2 * B2 / C2
3 316 6.66 1 = A3 * B3 / C3
4 32 9.11 592 = A4 * B4 / C4
5 = SUM(D2:D5)
With data dependencies
... the spread sheet turns
out to be a graph.
12
21. Graphs are all around us If we add external data
sources the problem
becomes even more
interesting...
17 3.14 3 = A1 * B1 / C1
42 10.11 14 = A2 * B2 / C2
316 6.66 1 = A3 * B3 / C3
32 9.11 592 = A4 * B4 / C4
= SUM(D2:D5)
13
22. Graphs are all around us If we add external data
sources the problem
becomes even more
interesting...
17 3.14 3 = A1 * B1 / C1
42 10.11 14 = A2 * B2 / C2
316 6.66 1 = A3 * B3 / C3
32 9.11 592 = A4 * B4 / C4
= SUM(D2:D5)
13
23. The Neo4j Graph data model
•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties 14
24. The Neo4j Graph data model
•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties 14
25. The Neo4j Graph data model
LIVES WITH
LOVES
OWNS
DRIVES
•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties 14
26. The Neo4j Graph data model
LOVES
LIVES WITH
LOVES
OWNS
DRIVES
•Nodes
•Relationships bet ween Nodes
•Relationships have Labels
•Relationships are directed, but traversed at
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties 14
27. The Neo4j Graph data model
name: “Mary”
LOVES
name: “James” age: 35
age: 32 LIVES WITH
twitter: “@spam” LOVES
OWNS
DRIVES
•Nodes
•Relationships bet ween Nodes
•Relationships have Labels brand: “Volvo”
•Relationships are directed, but traversed at model: “V70”
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties 14
28. The Neo4j Graph data model
name: “Mary”
LOVES
name: “James” age: 35
age: 32 LIVES WITH
twitter: “@spam” LOVES
OWNS
item type: “car” DRIVES
•Nodes
•Relationships bet ween Nodes
•Relationships have Labels brand: “Volvo”
•Relationships are directed, but traversed at model: “V70”
equal speed in both directions
•The semantics of the direction is up to the
application (LIVES WITH is reflexive, LOVES is not)
•Nodes have key-value properties
•Relationships have key-value properties 14
29. Graphs are Whiteboard Friendly The domain I specify is the
domain I implement.
No mismatch, no ER-modeling.
15
30. Graphs are Whiteboard Friendly The domain I specify is the
domain I implement.
odin No mismatch, no ER-modeling.
thobe
dude
#17
#6
#14 Wardrobe Strength
Joe project
#32
Call site caching
Hello world
OSCON
Best panncakes
Optimizing Jython 15
31. Graphs are Whiteboard Friendly
odin
dude
#17
#6
#14 username: “thobe” Wardrobe Strength
Joe project
#32 name: “Tobias Ivarsson”
twitter: “@thobe”
password: “**********”
Call site caching
Hello world
OSCON
Best panncakes
Optimizing Jython 16
32. Graphs are Whiteboard Friendly
odin
thobe
dude
#17
#6
#14 address: “http://journal.thobe.org”
#32 title: “Wardrobe Strengthproject
Joe
”
tagline: “Good enough
thoughts”
Call site caching
Hello world
OSCON
Best panncakes
Optimizing Jython 17
33. Building a graph - the basic API
import neo4j
grapDb = neo4j.GraphDatabase( PATH_TO_YOUR_NEO4J_DATASTORE )
with graphDb.transaction: # All writes require transactions
# Create Thomas 'Neo' Anderson
mrAnderson = graphDb.node(name="Thomas Anderson", age=29)
# Create Morpheus
morpheus = graphDb.node(name="Morpheus", rank= "Captain",
occupation= "Total bad ass")
# Create relationship representing they know each other
mrAnderson.KNOWS( morpheus )
# ... similarly for Trinity, Cypher, Agent Smith, Architect
18
34. Graph traversals
name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
19
35. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ]
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
20
36. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ]
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
37. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ]
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
38. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
39. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
40. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
Cypher (@ depth=2)
returnable = neo4j.RETURN_ALL_BUT_START_NODE
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
41. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
Cypher (@ depth=2)
returnable = neo4j.RETURN_ALL_BUT_START_NODE
Agent Smith (@ depth=3)
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
42. Graph traversals name: “The Architect”
disclosure: “public”
name: “Thomas Anderson”
age: 29 name: “Cypher”
last name: “Reagan”
KNOWS name: “Morpheus”
KNOWS KNOWS
rank: “Captain” CODED BY
LOVES occupation: “Total badass” KNOWS
KNOWS
name: “Trinity” disclosure: “secret”
name: “Agent Smith”
version: “1.0b”
since: “meeting the oracle” since: “a year before the movie”
language: “C++”
cooperates on: “The Nebuchadnezzar”
import neo4j
class Friends(neo4j.Traversal): # Traversals ! queries in Neo4j
types = [ neo4j.Outgoing.KNOWS ] Morpheus (@ depth=1)
order = neo4j.BREADTH_FIRST Trinity (@ depth=1)
stop = neo4j.STOP_AT_END_OF_GRAPH
Cypher (@ depth=2)
returnable = neo4j.RETURN_ALL_BUT_START_NODE
Agent Smith (@ depth=3)
for friend_node in Friends(mr_anderson):
print "%s (@ depth=%s)" % ( friend_node["name"],
friend_node.depth )
20
43. Finding a place to start
๏ Traversals need a Node to start from
• QUESTION: How do I find the start Node?
• ANSWER:You use an Index
๏ Indexes in Neo4j are different from Indexes in Relational Databases
• RDBMSes use them for Joining
• Neo4j use them for simple lookup
index = graphDb.index["name"]
mr_anderson = index["Thomas Anderson"]
performTraversalFrom( mrAnderson )
21
44. Indexes in Neo4j
๏ The Graph *is* the main index
• Use relationship labels for navigation
• Build index structures *in the graph*
‣Search trees, tag clouds, geospatial indexes, et.c.
‣Linked/skip lists or other data structures in the graph
‣We have utility libraries for this
๏ External indexes used *for lookup*
• Finding a (number of) points to start traversals from
• Major difference from RDBMS that use indexes for everything
22
48. from neo4j.model import django_model as models
class User(models.NodeModel):
username = models.Property(indexed=True)
name = models.Property()
blog
comment blogs = models.Relationship(Blog,
type=models.Outgoing.member_of,
related_name="users")
def __unicode__(self):
return self.name
entry
25
49. from neo4j.model import django_model as models
user
class Blog(models.NodeModel):
identifier = models.Property(indexed=True)
comment title = models.Property()
def __unicode__(self):
return self.title
entry
26
50. from neo4j.model import django_model as models
user
class Entry(models.NodeModel):
title = models.Property()
text = models.Property()
blog
comment date = models.Property()
blog = models.Relationship(Blog,
type=models.Outgoing.posted_on,
single=True, optional=False,
related_name="articles")
author = models.Relationship(User,
type=models.Outgoing.authored_by,
single=True, optional=False,
related_name="articles")
27
51. models.py
from neo4j.model import django_model as models
class Blog(models.NodeModel):
identifier = models.Property(indexed=True)
title = models.Property()
The rest of the code for
working with the domain
class User(models.NodeModel): objects is (mostly) the same
username = models.Property(indexed=True) as you are used to in Django.
name = models.Property()
blogs = models.Relationship(Blog,
type=models.Outgoing.member_of,
related_name="users")
class Entry(models.NodeModel):
title = models.Property()
text = models.Property()
date = models.Property()
blog = models.Relationship(Blog,
type=models.Outgoing.posted_on,
single=True, optional=False,
related_name="articles")
author = models.Relationship(User,
type=models.Outgoing.authored_by,
single=True, optional=False,
28
related_name="articles")
52. Why not use an O/R mapper?
๏ Model evolution in ORMs is a hard problem
• virtually unsupported in most ORM systems
๏ SQL is “compatible” across many RDBMSs
• data is still locked in
๏ Each ORM maps object models differently
• Moving to another ORM == legacy schema support
‣except your legacy schema is a strange auto-generated one
๏ Object/Graph Mapping is always done the same way
• allows you to keep your data through application changes
• or share data between multiple implementations 29
53. What an ORM doesn’t do
๏Deep traversals
๏Graph algorithms
๏Shortest path(s)
๏Routing
๏etc.
30
54. Path exists in social network
๏ Each person has on average 50 friends The performance impact
in Neo4j depends only on
the degree of each node. in
Tobias an RDBMS it depends on
the number of entries in
the tables involved in the
join(s).
Emil
Johan
Peter
Database # persons query time
Relational database 1 000 2 000 ms
Neo4j Graph Database 1 000 2 ms
Neo4j Graph Database 1 000 000 2 ms
Relational database 1 000 000 way too long...
31
55. Path exists in social network
๏ Each person has on average 50 friends The performance impact
in Neo4j depends only on
the degree of each node. in
Tobias an RDBMS it depends on
the number of entries in
the tables involved in the
join(s).
Emil
Johan
Peter
Database # persons query time
Relational database 1 000 2 000 ms
Neo4j Graph Database 1 000 2 ms
Neo4j Graph Database 1 000 000 2 ms
Relational database 1 000 000 way too long...
31
56. On-line real time routing with Neo4j
๏ 20 million Nodes - represents places
๏ 62 million Edges - represents direct roads between places
• These edges have a length property, for the length of the road
๏ Average optimal route, 100 separate roads, found in 100ms
๏ Worst case route we could find:
• Optimal route is 5500 separate roads
• Total length ~770km There’s a difference
• Found in less than 3 seconds
bet ween least
number of hops and
least cost.
๏ Uses A* “best first” search
32
57. Jython vs. CPython
๏ Neo4j with the Python bindings work in both
• Requires no code modification in your code
๏ Neo4j at its core is an Embedded (in-process) database
• CPython manages concurrency by forking multiple processes
• Jython has full concurrency support in the same JVM
• Stand-alone Neo4jon
is being worked
server-process with (C)Python client
๏ Neo4j has a RESTful interface
• There are Python clients
• The API differs slightly (no transactions) 33
58. Finding out more
๏ http://neo4j.org/ - project website - main place for getting started
‣Contains screen casts, download links, et.c.
‣http://api.neo4j.org/ and http://components.neo4j.org/
‣Specifically http://components.neo4j.org/neo4j.py/
‣http://wiki.neo4j.org/ - HowTos, Tutorials, Examples, FAQ, et.c.
‣http://planet.neo4j.org/ - aggregation of blogs about Neo4j
‣http://github.com/neo4j-examples - small example applications
๏ https://lists.neo4j.org/ - community mailing list
๏ http://twitter.com/neo4j/team - follow the Neo4j team
๏ http://neotechnology.com/ - commercial licensing 34
59. Helping out!
๏ Neo4j and the Python integration is all Open Source
๏ The Python bindings in particular would benefit from more devs...
• Integrate more of the Neo4j components
‣Neo4j Spatial
‣The Graph Algorithms package
‣The Graph Matching component
• Trimming off the rough edges in the Django integration
• Native client for CPython
35