This document provides a summary of a talk on using Spring with NoSQL databases. The talk discusses the benefits and drawbacks of NoSQL databases, and how the Spring Data project simplifies development of NoSQL applications. It then provides background on the two speakers, Chris Richardson and Mark Pollack. The agenda outlines explaining why NoSQL, overviewing some NoSQL databases, discussing Spring NoSQL projects, and having demos and code examples.
A presentation at Twitter's official developer conference, Chirp, about why we use the Scala programming language and how we build services in it. Provides a tour of a number of libraries and tools, both developed at Twitter and otherwise.
To hit Ruby3x3, we must first figure out **what** we're going to measure, **how** we're going to measure it, in order to get what we actually want. I'll cover some standard definitions of benchmarking in dynamic languages, as well as the tradeoffs that must be made when benchmarking. I'll look at some of the possible benchmarks that could be considered for Ruby 3x3, and evaluate them for what they're good for measuring, and what they're less good for measuring, in order to help the Ruby community decide what the 3x goal is going to be measured against.
A presentation at Twitter's official developer conference, Chirp, about why we use the Scala programming language and how we build services in it. Provides a tour of a number of libraries and tools, both developed at Twitter and otherwise.
To hit Ruby3x3, we must first figure out **what** we're going to measure, **how** we're going to measure it, in order to get what we actually want. I'll cover some standard definitions of benchmarking in dynamic languages, as well as the tradeoffs that must be made when benchmarking. I'll look at some of the possible benchmarks that could be considered for Ruby 3x3, and evaluate them for what they're good for measuring, and what they're less good for measuring, in order to help the Ruby community decide what the 3x goal is going to be measured against.
Scalable and Flexible Machine Learning With Scala @ LinkedInVitaly Gordon
The presentation given by Chris Severs and myself at the Bay Area Scala Enthusiasts meetup. http://www.meetup.com/Bay-Area-Scala-Enthusiasts/events/105409962/
Scala - The Simple Parts, SFScala presentationMartin Odersky
These are the slides of the talk I gave on May 22, 2014 to the San Francisco Scala user group. Similar talks were given before at GOTO Chicago, keynote, at Gilt Groupe and Hunter College in New York, at JAX Mainz and at FlatMap Oslo.
The slides for a lecture about the Ruby programming language. This language was given at FEUP, on a course called "Laboratories of Object-Oriented Programming".
Finally Java SE 7 is GA and you can start using it. This talk will cover the most important new features of the language and the virtual machine. It will also cover some features that did not make it in to the SE 7 release. Finally we will discuss current state of Java as an ecosystem and my analysis and hopes for the future.
Scalable and Flexible Machine Learning With Scala @ LinkedInVitaly Gordon
The presentation given by Chris Severs and myself at the Bay Area Scala Enthusiasts meetup. http://www.meetup.com/Bay-Area-Scala-Enthusiasts/events/105409962/
Scala - The Simple Parts, SFScala presentationMartin Odersky
These are the slides of the talk I gave on May 22, 2014 to the San Francisco Scala user group. Similar talks were given before at GOTO Chicago, keynote, at Gilt Groupe and Hunter College in New York, at JAX Mainz and at FlatMap Oslo.
The slides for a lecture about the Ruby programming language. This language was given at FEUP, on a course called "Laboratories of Object-Oriented Programming".
Finally Java SE 7 is GA and you can start using it. This talk will cover the most important new features of the language and the virtual machine. It will also cover some features that did not make it in to the SE 7 release. Finally we will discuss current state of Java as an ecosystem and my analysis and hopes for the future.
Virtuoso - music retrieval and recommendationeanes pereira
Software for music retrieval and recommendation. It is able to classify music by genre and by elicited emotion. Besides, it is able to classify specific Brazilian music genres.
Schnelligkeit ist gesünder für alle dank der High Speed Datenbank EXASolution
Bei Interesse an der Präsentation können Sie sich gerne an marketing@exasol.com wenden.
NoSQL is not a buzzword anymore. The array of non- relational technologies have found wide-scale adoption even in non-Internet scale focus areas. With the advent of the Cloud...the churn has increased even more yet there is no crystal clear guidance on adoption techniques and architectural choices surrounding the plethora of options available. This session initiates you into the whys & wherefores, architectural patterns, caveats and techniques that will augment your decision making process & boost your perception of architecting scalable, fault-tolerant & distributed solutions.
Technical overview of three of the most representative KeyValue Stores: Cassandra, Redis and CouchDB. Focused on Ruby and Ruby on Rails developement, this talk shows how to solve common problems, the most popular libraries, benchmarking and the best use case for each one of them.
This talk was part of the Conferencia Rails 2009, Madrid, Spain.
http://app.conferenciarails.org/talks/43-key-value-stores-conviertete-en-un-jedi-master
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
1. Chicago, October 19 - 22, 2010
Using Spring with NoSQL
databases
Mark Pollack
Chris Richardson
2. Goal of this talk
• The benefits and drawbacks of
NoSQL databases
• How the Spring Data project
simplifies the development of
NoSQL applications
3. About Chris
•Grew up in England and live in Oakland, CA
•Over 25+ years of software development
experience including 14 years of Java
•Speaker at JavaOne, SpringOne, NFJS,
JavaPolis, Spring Experience, etc.
•Organize the Oakland JUG and the Groovy
Grails meetup
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
4. About Mark
• Spring committer since 2003
• Founder of Spring.NET
• Work on some other new open source projects
– Spring Gemfire
– Spring AMQP
• Used to work on large scale data collection/analysis in
High Energy Physics (~10yrs ago)
5. Agenda
• Why NoSQL?
• Some NoSQL Databases
• Spring NoSQL projects
• Demos/Code/Deep dive
7. Relational databases are great
• Well understood by developers
• Well supported by frameworks and tools, e.g. Spring JDBC,
Hibernate, JPA
• Well understood by operations
– Configuration
– Care and feeding
– Backups
– Tuning
– Failure and recovery
– Performance characteristics
• But….
8. The trouble with relational database
• Object/relational impedance mismatch
– Complicated to map rich domain model to relational schema
– Relational schema is rigid
• Extremely difficult/impossible to scale writes:
– Vertical scaling is limited/expensive
– Horizontal scaling is limited or requires $$
• Performance can be suboptimal
9. Why NoSQL?
Internet scale is the primary driver in the use of non-relational DBs
Large Data Size – TBs to PBs
High Transaction rates
Need for High Availability
Pictures courtesy of Gilt Group
Relational DBs
Harder to scale to larger size
Transactions and joins reduce access rates
10. Different priorities
• Relational Databases with ACID semantics
– Atomic, Consistent, Isolated, Durable
• Internet Systems with BASE semantics
– Basically Available, Scalable, Eventually Consistent
• NoSQL/BASE values remove standard DB behaviors
– No Joins
– Very limited "transactions"
– Different Durability/consistency guarantees
• NoSQL datastores emerge as “point solutions”
– Amazon/Google papers
– Facebook, LinkedIn …
11. The NoSQL Landscape
There are a variety of non-relational datastores
Many are Open Source
Many with companies behind them
Rapidly changing feature sets
Scalaris
Neo4J db4o Objectivity Amazon Dynamo
Terrastore Gemfire Sones
HBase CouchDB
InfoGrid RavenDB
Hadoop Scalaris Tokyo Cabinet
MongoDB
Hypertable Cassandra Azure Table Coherence
Mnesia Voldemort Gigaspaces
Riak
Amazon SimpleDB
Redis BerkeleyDB MemcacheDB
Tamino
InfiniteGraph Chordless
Expect consolidation over time
With so many choices, picking which one to use can be difficult
12. Stops on a tour through NoSQL-land
What is the data model?
How to I access my data?
What are the primary use cases?
What notable companies that are using it?
13. Data Model
Key Value
K1 V1
Key -> value K2 V2
Redis, Riak, Voldemort… K3 V2
“Amazon Dynamo inspired”
Column
Key -> ‘column families’
HBase, Cassandra
“Google Bigtable inspired”
14. Data Model
Document
Collections containing semi- { id: ‘4b2b9f67a1f631733d917a7b"),’
author: ‘joe’,
structured data tags : [‘example’, ‘db’],
comments : [ { author: 'jim', comment: 'OK' },
{ author: ‘ida', comment: ‘Bad' }
key-values , JSON, XML, ]
CouchDB, MongoDB { id: ‘4b2b9f67a1f631733d917a7c"),
author: ‘ida’, ...
{ id: ‘4b2b9f67a1f631733d917a7d"),
author: ‘jim’, ...
Graph
Graph: nodes and edges
Edges can be directional
Each can contain key-value
attributes
Neo4j, Sones, InfiniteGraph
16. How to I access my data?
Query Mechanism
Key lookup
Map-Reduce
Query by example
Query language
‘out of box’ query power usually linked to data model
Increasing power
KV -> Column -> Document/Graph
Counter examples
Riak is key-value and supports Map-Reduce
Spring Data Mapper…
.
17. Reality Check - Relational DBs are not going away
NoSQL usage small
by comparison…
But growing…
18. Future = multi-paradigm data storage for
enterprise applications
IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
19. Agenda
• Why NoSQL?
• Some NoSQL Databases
• Spring NoSQL projects
• Demos/Code/Deep dive
20. Redis
• Advanced key-value store
– Think memcached on steroids (the good kind)
– Values can be binary strings, Lists, Sets, Ordered K1 V1
Sets, Hash maps, ..
– Operations for each data type, e.g. appending K2 V2
to a list, adding to a set, retrieving a slice of a
K3 V2
list, …
• Very fast: ~100K operations/second on entry-
level hardware
• Persistent
– Periodic snapshots of memory OR append
commands to log file
– Limits are size of keys retained in memory.
• Has “transactions”
• Libraries – Many languages, all separate
open source projects
21. Redis CLI
redis> sadd myset a
(integer) 1
redis> sadd myset b
(integer) 1
redis> smembers myset
1. "a"
2. "b"
redis> srem myset a
(integer) 1
redis> smembers myset
1. "b"
22. Scaling Redis
• Master/slave replication
– Tree of Redis servers
– Non-persistent master can replicate to a persistent slave
– Use slaves for read-only queries
• Sharding
– Client-side only – consistent hashing based on key
– Server-side sharding – coming one day
• Run multiple servers per physical host
– Leverage multiple CPUs
– 32 bit more efficient than 64 bit
• Optional "virtual memory"
– Ideally data should fit in RAM
– Values (not keys) written to disc
23. Redis use cases
• Use in conjunction with another database as the SOR
• Drop-in replacement for Memcached
– Session state
– Cache of data retrieved from SOR
• Denormalized datastore for high-performance queries
• Hit counts using INCR command
• Randomly selecting an item – SRANDMEMBER
• Queuing – Lists with LPOP, RPUSH, ….
• High score tables – Sorted sets
• Notable users: github, guardian.co.uk, ….
24. Cassandra
• Originally developed by Facebook for inbox search
– Now an Apache open-source project
• Extremely scalable
• Data is replicated and sharded
• The data model will hurt your brain
• Column-oriented database
• 4 or 5-dimensional hash map
25. Cassandra data model – insert/update
My Column family (within a key space)
Keys Columns
a colA: value1 colB: value2 colC: value3 This can
be data!
b colA: value colD: value colE: value
Insert(key=a, columName=colZ, value=foo)
Keys Columns
a colA: value1 colB: value2 colC: value3 colZ: foo
b colA: value colD: value colE: value
26. Cassandra query – range slice
Keys Columns
a colA: value1 colB: value2 colC: value3 colZ: foo
b colA: value colD: value colE: value
rangeSlice(startKey=a, endKey=b,
startColumn=colA, endColumnName=colB)
Keys Columns
a colA: value1 colB: value2
b colA: value
27. Cassandra CLI
$ bin/cassandra-cli
cassandra> connect localhost/9160
Connected to "Test Cluster" on localhost/9160
cassandra> get Keyspace1.Standard1['foo']
Returned 0 results
cassandra> set Keyspace1.Standard1['foo']['a'] = '2'
Value inserted
cassandra> get Keyspace1.Standard1['foo']
=> (Column=61, value=1, timestamp...)
cassandra> set Keyspace1.Standard1['foo']['bar'] = 'abc'
Value inserted
cassandra> get Keyspace1.Standard1['foo']
=> (Column=626172, value=abc, timestamp...)
=> (Column=61, value=1, timestamp...)
cassandra> count Keyspace1.Standard1['foo']
2 columns
cassandra>
28. Scaling Cassandra through sharding/replication
Keys = [D, A] •Client connects to any node
•Dynamically add/remove
nodes
Node 1
•Configurable # replicas
Token = A •Reads/Writes specify how
many nodes
Replicates to Replicates to Keys = [A, B]
Node 4 Node 2
Token = D Token = B
Keys = [C, D] Replicates to Replicates to
Node 3
Token = C
Keys = [B, C]
29. Cassandra use cases
• Use cases
• Big data
• Persistent cache
• (Write intensive) Logging
• Who is using it
– Digg, Facebook, Twitter, Reddit, Rackspace
– Cloudkick, Cisco, SimpleGeo, Ooyala, OpenX
– “The largest production cluster has over 100 TB of data in
over 150 machines.“ – Casssandra web site
30. MongoDB
• JSON-style documents
– Lists, Maps, Primitive values
• Documents organized into collections (~table)
• Full or partial document updates
– Transactional update in place on one document
– Atomic Modifiers
• GridFS for efficiently storing large files
• Index support – secondary and compound
• Rich query language for dynamic queries
• Map/Reduce
• Replication and Auto Sharding
32. MongoDB query by example
• Find a restaurant that serves the 94619 zip code and is
open at 6pm on a Monday
{
serviceArea:"94619",
openingHours: {
$elemMatch : {
"dayOfWeek" : Monday,
"open": {$lte: 1800},
"close": {$gte: 1800}
}
}
} DBCursor cursor = collection.find(qbeObject);
while (cursor.hasNext()) {
DBObject o = cursor.next();
…
}
34. Scaling MongoDB
Shard 1 Shard 2
mongod mongod
mongod mongod mongod mongod
A shard
consists of a
Config Server replica set =
generalization
mongod mongos mongos of master slave
mongod
Collections
mongod spread over
client multiple shards
35. MongoDB use cases
• Use cases
– Real-time analytics
– Content management systems
– Single document partial update
– Caching
– High volume writes
• Who is using it?
– Shutterfly, Foursquare
– Bit.ly Intuit
– SourceForge, NY Times
– GILT Groupe, Evite,
– SugarCRM
36. CouchDB
• JSON documents
• Database is flat collection of documents
• Queries are implemented as views using Map/Reduce in
Javascript
– Queries typically defined up front
– Views are indexed and incrementally updated
• MVCC based
– Need to deal with reconciliation
• Multi-Master (bi) replication
• Scale out with replication
• Mobile version - Android
• Use as a Server-Side JavaScript application server
– “CouchApps”
• Cloudant provides Java VM based views
37. CouchDB with curl
$ curl -X PUT http://localhost:5984/contacts
{"ok":true}
$ curl -X PUT http://localhost:5984/contacts/markpollack -d
'{"firstName":"Mark",
"lastName":"Pollack",
"email":"mpollack@vmware.com"}‘
{"ok":true,"id":"markpollack","rev":"1-7bb43f219a8d4306614dbd526ed577fe"}
$curl -X GET http://localhost:5984/contacts/markpollack
{"_id":"markpollack",
"_rev":"1-7bb43f219a8d4306614dbd526ed577fe",
"firstName":"Mark",
"lastName":"Pollack",
"email":"mpollack@vmware.com"}
40. CouchDB use cases
• Use cases
– Synchronization
• master-master replication
• Offline replication for mobile devices: “Ground Computing”
– Web apps that consume JSON
• Server side JavaScript using CouchApps
– Content management systems
– Caching
• Who is using it?
– Mozilla, BBC, Palm, Apple,
– IBM, Ubuntu, meebo, Engine Yard
– Cloudant, UNICEF, Credit Suisse
41. Neo4j
• Graph Database
• DB is a collection of graph nodes, relationships
– Nodes and relationships have properties
• Querying is done via a traversal API
• Indexes on node/relationship properties
• Written in Java, can be embedded
– Also a REST API
• Transactional (ACID)
• Master-Slave replication in next release
43. Neo4j Use Cases
• Use Cases
– Social networking
– Bioinformatics
– Fraud detection
• Who is using it?
– Fanbox, Jayway
– Large companies that wish to remain anonymous
44. Agenda
• Why NoSQL?
• Some NoSQL Databases
• Spring NoSQL projects
• Demos/Code/Deep dive
45. Spring Data Project Goals
• Bring classic Spring value propositions to NoSQL
– Productivity
– Programming model consistency
• Support for a wide range of NoSQL databases
• Many entry points to use
– Low level data access APIs
– Opinionated APIs (Think JdbcTemplate)
– Object Mapping (Java and GORM)
– Cross Store Persistence Programming model
– Productivity support in Roo and Grails
– Vertical Applications
– Guidance
46. Spring Data
• Working together with various projects/companies
– Redis / VMware
– Riak / Basho
– CouchDB / CouchOne
– Mongo / 10gen
– Neo4j / Neo Technologies
47. Spring Data Projects
Data Commons Planned
Polyglot persistence Guidance Docs
Datastore Key-Value The big picture
Redis Datastore Column
Cassandra, HBase
Datastore Document
Hadoop
MongoDB, CouchDB
Datastore Key-Value
Datastore Graph
Riak
Neo4j
Datastore Mapping
Datastore Mapping
Additional DB Map
Redis, Gemfire
Blob storage
Amazon, Atmos, Azure
Reaching M1 release in Q4. SQL - Generic DAOs
Grails/Roo support
48. Spring Datastore Key-Value (Redis)
Interface based low-level Redis Client API
Method names are Redis command names
Pluggable implementations (Jedis, JRedis)
Translation to Spring DAO exception hierarchy
Connection Pooling
RedisTemplate
Descriptive method names, grouped into natural categories
ServerOps, SetOps
Convenience methods – ‘redis recipies’
getAndConvert, convertAndSet for payload/value serialization
Java, JSON, XML
Protocol Buffers, Avro, to come…
Redis backed java.util.Set, List, Map, Capped Collections
RedisPlatformTransactionManager
Redis Messaging
RedisMessageTemplate, RedisMessageListenerContainer
49. Spring Redis Examples
RedisClientFactory clientFactory = new JedisClientFactory();
RedisClient client = clientFactory.createClient();
client.sadd("s1", "1"); client.sadd("s1", "2");
client.sadd("s1", "3");
client.sadd("s2", "2"); client.sadd("s2", "3");
Set<String> intersection = client.sinter("s1", "s2");
System.out.println(intersection); // [3,2]
RedisTemplate redisTemplate = new RedisTemplate(client)
Person p = new Person("Joe", "Trader", 33);
redisTemplate.convertAndSet("trader:1", p);
p = redisTemplate.getAndConvert("trader:1", Person.class);
intersection =
redisTemplate.getSetOps().getIntersectionOfSets(“s1”,”s2”)
RedisSet s1 = new RedisSet(template, "s1");
s1.add("1"); s1.add("2"); s1.add("3");
RedisSet s2 = new RedisSet(template, "s2");
s2.add("2"); s2.add("3");
Set s3 = s1.intersection("s3", s1, s2); // [3,2]
50. Spring MongoDB
• MongoTemplate
– ‘RowMapper’ like functionality for mapping Mongo query
results.
– Basic POJO mapping support for simple CRUD
• Leverage Spring 3.0 TypeConverters and SpEL
– Connection affinity callback
• Support common map-reduce operations from Mongo
Cookbook
• Higher level integration
– File Upload
– Spring MVC activity monitoring and analysis
51. Spring MongoDB Example
Mongo mongo = new Mongo();
DB db = mongo.getDB("mvc");
MongoTemplate mongoTemplate = new MongoTemplate(db,
"people");
Person p = new Person("Joe", "Trader", 33);
mongoTemplate.save(p);
List<People> people =
mongoTemplate.queryForCollection(Person.class);
52. Spring CouchDB
CouchTemplate
RestTemplate based API for CouchDB
Basic JSON/POJO mapping support
CouchDbMappingJacksonHttpMessageConverter
Android implementation
.NET implementation planned
Repository base classes
Out of the box CRUD
Embedded View Definitions
Automatic view generation
Plans to work with ektrop OS project
http://code.google.com/p/ektorp/
53. Spring CouchDB Example
RestTemplate restTemplate = new RestTemplate();
restTemplate.setMessageConverters(couchDbConverter);
List<Restaurants> rests = (List<Restaurant>)
restTemplate.getForObject(getDbURL() +
"_design/demo/_view/all", Restaurant.class);
UserAccount userAccount = new UserAccount("u1");
restTemplate.put(getDbURL() + userAccount.getUserName(),
userAccount);
//Note, not handling response to get new revision string…
55. Spring Neo4j
• Easy POJO based programming model
• Support for graph Entities, Relationships, and Properties
– Properties leverage Spring 3.0 converters
• DAO finder support
– findAll, count, findById, findByPropertyValue,
findAllByTraversal
See Graph Database Persistence Using Neo4j with Spring and Roo
Tomorrow @ 10:15am in Lilac C
56. Spring Cross Store
• Use Case
– An existing application want to use NOSQL database as a
part of its existing SQL-based application
• Pick a problem space for which NOSQL is well suited
• How to link together the two worlds?
– Developer application code task
– No longer a single domain entity
• Spring Cross store allows for a single entity to be
persisted across multiple datastores
– AspectJ manages NOSQL entities/fields in change sets
– Change sets are flushed when JPA commits
– Interception to rehydrate when performing JPA queries
60. Spring Mapping
• GORM for Redis, Gemfire….
– Built on pure Java
• Mapping metadata is not tied to GORM
– Can use annotations, XML etc.
• Will be working on making this mapping framework
available to Java developers
63. Agenda
• Why NoSQL?
• Some NoSQL Databases
• Spring NoSQL projects
• Demos/Code/Deep dive
64. Redis meets POJOs in Action
• Finding restaurants that are open at the delivery time (e.g.
94619) and serve the delivery zip code (e.g. 6pm)
select r.*
from FTGO_RESTAURANT_ZIPCODE zip,
FTGO_RESTAURANT_TIME_RANGE tr, FTGO_RESTAURANT r
where r.RESTAURANT_ID = zip.RESTAURANT_ID
and tr.RESTAURANT_ID = r.RESTAURANT_ID;
and zip.ZIPCODE = '94619'
and tr.day_Of_week = 'Monday'
Joins and
and tr.open_time <= 1800
complex
and 1800<= tr.close_time where clause!
No way Redis can handle this! Right?
65. Redis – denormalizing the data is the key
Tr_id Day of Open_time Close_tim Zip code
Week e
Ajanta10 Monday 1130 1430 94707
Ajanta10 Monday 1130 1430 94619
Ajanta11 Monday 1730 2130 94707
Ajanta11 Monday 1730 2130 94619
Egg30 Monday 0700 1430 94707
… Two simpler queries
(SELECT tr_id
FROM time_range_zip_code
where day_of_week = ? and zip_code = ? open_time < ?)
INTERSECTION
(SELECT tr_id
FROM time_range_zip_code
where day_of_week = ? and zip_code = ? and ? < close_time)
66. Redis – storing the opening hours using sorted
sets
Sorted Set
Keys Members = TimeRangeId with opening time as score
open:94707:Monday Ajanta10:1130 Ajanta11:1730
open:94619:Monday Egg30:0700 Ajanta10:1130 Ajanta11:1730
ZRANGEBYSCORE open:94619:Monday 0 1800
{Egg30, Ajanta10, Ajanta11, …} Finds all time ranges that open
before 6pm
67. Redis – storing closing times with sorted sets
Ordered Set
Keys Members = TimeRangeId with closing time as score
close:94707:Monday Ajanta10:1430 Ajanta11:2130
close:94619:Monday Egg30:1430 Ajanta10:1430 Ajanta11:2130
ZRANGEBYSCORE close:94619:Monday 1800 2400
Finds all time ranges that close
{Ajanta11, …} after 6pm
68. Redis – computing the intersection
{Egg30, Ajanta10, Ajanta11, …}
Intersection
{Ajanta11, …}
=
{Ajanta11, …}
• This is super-cool!
• Illustrates the feasibility of using Redis as a queryable
datastore to improve scalability
69. Cassandra – Food to Go
AvailableRestaurants Column Family
Keys Columns
1430
94619_
Super
Monday_ Egg30: {…} column
0700
1430
94619_
Monday_ Ajanta10: {…}
1130
2130
94619_
Monday_ Ajanta11: {…}
1730
70. Cassandra – Food to Go
rangeSlice(startKey=94619_Monday_0000,
endKey=94619_Monday_1800,
startColumn=1800,
endColumnName=2359)
Keys Columns
2130
94619_
Monday_ Ajanta11: {…}
1730
71. Food to Go - MongoDB
{ {
"name" : "Ajanta", serviceArea:"94619",
"type" : "Indian", openingHours: {
"serviceArea" : [ $elemMatch : {
"94619", "dayOfWeek" : Monday,
"94618" "open": {$lte: 1800},
], "close": {$gte: 1800}
"openingHours" : [ }
{ }
"dayOfWeek" : Monday, }
"open" : 1730,
"close" : 2130
}
], Straightforward to model a
"_id" : restaurant aggregate as a MongoDB
ObjectId("4bddc2f49d1505567c document and execute queries
6220a0")
}
72. MongoDB – MVC Analytics
• Monitoring page view and web application activity is
critical to analyzing effectiveness of
– Marketing campaigns
– Site design / new features
• Standard solutions are problematic aside from scaling
– Google Analytics
• not real time, hard to setup
– MySQL
• Downtime for schema changes
• Solutions needs to handle high write volume, handle large
amounts of data, allow for flexible data exploration.
73. Web Analytics - SQL
Still difficult to
scale writes
Application MySQL
Server Master
Reporting
MySQL
Server MySQL
Slave
MySQL
Slave
Slave
74. Web Analytics - MongoDB
Application
Server
MongoD
Reporting
Server
75. Web Analytics - MongoDB
Application MongoD
Server
MongoS
Reporting
Server MongoD
76. Spring MVC based solution
• Custom Handler Adapter with a ActionInterceptor
• ActionInterceptor has full context of web action
– Controller class, method name, and parameters
– WebRequest
– ModelAndView
– Exception
• Similar to Ruby on Rails/ASP.NET MVC Action Filters
• Collect data in MongoDB
– MvcEvent
– Increment counters per controller and action method
• AnalyticsController to display results
– Action method parameter information is critical for reporting
• Also a good fit for an Insight plugin!
82. CouchDB – Mobile offline access and sync
• Smartphones have huge data capacity
• Can carry large data sets with you.
• Mobile devices are not always connected
• Data sync is a difficult problem to solve
– CouchOne Mobile handles the dirty bits of sync
– Available for Android
• Same programming model as on server side
• Spring RestTemplate based CouchDB API
– On Server and Phone
• Can also be used a full-stack environment on mobile
– Database, JavaScript, Web Server
83. CouchDB – Offline data access and sync
Continuous
bi-directional
replication Application
Server
85. Summary
• Many new options for persistence
– A particular use case in your application might be a good fit
– Polyglot persistence will be increasingly common
• Trade-offs with
– Normalization
– Transactions
– Familiarity
• The Spring Data project
– Make it easier to build Spring-powered applications that use
these new technologies
• More info at http://www.springframework.org/spring-data
– Welcome contributors, comments…
86. Q&A
Chris Richardson Mark Pollack
crichardson@vmware.com mpollack@vmware.com
@crichardson @markpollack