Taming NoSQL with Spring Data


Published on

Enterprise applications are complex making it difficult to fit everything in one model. NoSQL is taking a leading role in the next generation database technologies and polyglot persistence a good option to leverage the strength of multiple data stores. This talk will introduce the Spring Data project, an umbrella project that provides a familiar and consistent Spring-based programming model for a wide range of data access technologies such as Redis, MongoDB, HBase, Neo4j...while retaining store-specific features and capabilities.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • Most popular persistence choice today\nRelations, ACID guarantees, SQL, strict schema, difficult to scale, mismatch with OO lang\n
  • Online analytical processing that enable users to interactively analyze multidimensional data from multiple perspectives\n
  • \n
  • \n
  • \nC – for “Consistency” :ability of a system to remain in consistent state after an update or an operation\nA – for “Availability” : availability of a system even in the event of adversity or system issues\nP – for “Partition Tolerance” : ability of system to function in presence of network partitions even if partitions are added/deletedYou can't have the three at the same time and get an acceptable latency.\nFast, good and cheap\nYou cannot scale without partition tolerance, so to scale you have to drop consistency\n\n
  • Most of the systems compromise between consistency and availability\nBASE - Basic Availability Soft-state Eventual consistency\nYou drop consistency for eventual consistency\n\nFirst were web frameworks (Struts, Spring MVC, Tapestry, Wicket, Stripes...)\nThen Ajax and Javascript frameworks (jQuery, prototype, Dojo...) (backbone.js, Knokout, batman.js...)\nNot it’s time for persistence!!!\n
  • \nWhich one should I use for my use case?\n
  • Key-Value: like a globally distrubuted hasmap\nColumn: \n
  • Relational Databases ACID Principles (Atomic, Consistent, Isolated, Durable)\nPolyglot Programming 2006\nScalability, High availability, fault tolerance, distributability, flexibility\n\n
  • \n
  • Dynamo: Amazon’s Highly available key-value store (2007)\nExtremely fast\nUse CasesSession data Shopping cartsUser preferences\n\nWhen to avoid?\nYou have relationsYou have multi-operational transactions \nYou want to query the valuesYou want to operate on sets of entries\n
  • Atomic\nUse cases: Counting views, who is online, social activity feeds, caching\nContentious benchmarks (memcached vs redis)\n
  • \n
  • \n
  • Based on Bigtable from Google: A Distributed storage system for Structured Data (2006)\nLike a big table where every row can have its own schema (one row may have 3 columns and another one 50 columns)\nBig Data problems, large scale data processing\n
  • \n
  • Easy to get started with\nSQL like query capabilities\nSchema less -no schema migration but cannot have data integrity\n
  • Rich Document: Closer to the data model that we have in our code\nArray of values are much more convenient than many-to-many relationships\nEmbedded documents\n\n_id -> PK globally unique identifier, you can override that value\n
  • Eventual consistency\nGridFS - supports native storage of binary data\nObjects in MongoDB are limited in size, the GridFS spec provides a mechanism for transparently dividing a large file among multiple documents.\n
  • \n
  • 10gen nothing is gonna be more \n10gen education\nmost of the querying capabilities that you get with RDBSM, \n
  • \n
  • \n
  • 3 core abstractions in the graph model: Node, Relationship, Properties (key-value pairs)\nSchema free\nOther dbs can model graphs, graph dbs make it the primary data structure.\n\n
  • \n
  • Spring Data makes it easier to build Spring-powered applications that use new data access technologies\nIt is worth taking a look at spring data even if you are not using nosql\nPromote classic Spring value propositions: \nProductivity (make the easy stuff a on-liner), \nconsistency: wide broad of APIs\nportability: repository support\nCommons: Repositories, Object Mapping\n
  • QueryDSL project, type-safe query API\nfields managed by different stores\n
  • \n
  • Pageable ->offset, page number, page size, sort (accepts multiple properties)\nQuery methods use method naming conventions to define query\n
  • \n
  • \n
  • \n
  • Foursquare - geospatial\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Taming NoSQL with Spring Data

    1. 1. Taming NoSQL with Spring Sergi Almar @sergialmar
    2. 2. About Sergi Almar‣CTO @ PayTouch‣VMWare / SpringSource CertifiedInstructor‣javaHispano JUG core member‣Spring I/O organizer (hope to see you nextyear in Spain)
    3. 3. Database evolution 1990$ RDBS$
    4. 4. Database evolution 1990$ 2000# RDBS# RDBS$ OLAP#/# BI#
    5. 5. Database evolution 1990$ 2000# 2010$ RDBS$ RDBS# NoSQL$ RDBS$ OLAP$/$BI$ OLAP#/# BI# Hadoop$
    6. 6. Increasing Cost & Complexity
    7. 7. CAP Theorem
    8. 8. CAP Theorem Hypertable HBase BigTable MongoDB RDBMS Redis Memcache Couchbase Terrastore Dynamo Voldemort Cassandra Riak SimpleDB CouchDB
    9. 9. CAP Theorem Dynamo Voldemort Cassandra RDBMS Riak SimpleDB CouchDB Which one should I choose? Hypertable HBase BigTable MongoDB Redis Memcache Couchbase Terrastore
    10. 10. NoSQL Categories Key-Value Column Document Graph
    11. 11. Welcome to Polyglot persistence
    12. 12. Key-Value Stores
    13. 13. Key-value stores‣Based on Amazon’s Dynamo paper‣Data stored as key / value pairs‣Hard to query‣Mostly in memory K1 V1 K2 V2 K3 V2
    14. 14. ‣Redis is an advanced key-value store ‣Similar to Memcached but the dataset is not volatile. ‣Data types: string, lists, sets, hashes, sorted sets ‣Data expiration‣Master-slave replication‣Has “transactions” (batch operations)‣Libraries - Many languages (Java: Jedis,JRedis...)
    15. 15. Redis Quick Demo
    16. 16. Column stores
    17. 17. Column storesBased on Bigtable paperData model: big table with columnfamilies
    18. 18. Document store
    19. 19. Table / Documents { title: "Taming NoSQL with Spring Data", abstract: "NoSQL is taking a leading ...", speaker: "Sergi Almar", topics: ["nosql", "spring"]}
    20. 20. Documents { amount: 33, currency: "EUR", user: { first_name: "Sergi", last_name: "Almar" }, payment : { cc: "2342", expiracy: "11/12" }, fingerprints : [{ index: "right_index", score: 94}, { index: "right_middle", score: 89}] }
    21. 21. ‣JSON-style documents‣Full or partial document updates‣GridFS for efficiently storing large files‣Index support - secondary andcompound‣Rich query language for dynamic queries‣Map / Reduce‣Replication and auto sharding
    22. 22. RDBMS MongoDB• Table • Collection• Row • JSON Document• Index • Index• Join • Embedding &• Partition Linking• Partition Key • Shard • Shard Key
    23. 23. MongoDB Quick Demo
    24. 24. Graph stores
    25. 25. ‣DB is a collection of graph nodes,relationships ‣Nodes and relationships have properties‣Query is done via a traversal API‣Indexes on node / relationshipproperties‣Written in Java, can be embedded‣Transactions (ACID)
    26. 26. Spring Data
    27. 27. Spring Data http://www.springsource.com/spring- data‣An umbrella project for:‣JPA - Repositories‣JDBC Extensions‣MongoDB - Document Database‣Neo4J - Graph Database‣Redis, Riak - Key Value Database‣Gemfire - Distributes Data Grid‣Hadoop / HBase - Big Data Storage andAnalysis platform
    28. 28. Spring Data Building Blocks‣Mapping of POJOs to underlying datamodel‣Familiar Spring ‘Template’ ‣MongoTemplate, RedisTemplate, Neo4JTemplate...‣Generic Repository support
    29. 29. Spring Data Repositories IpublicinterfaceRepository<T,IDextendsSerializable>{!}publicinterfaceCrudRepository<T,IDextendsSerializable>extendsRepository<T,ID>{!Tsave(Tentity);Iterable<T>save(Iterable<?extendsT>entities);TfindOne(IDid);!booleanexists(IDid);!Iterable<T>findAll();!longcount();!voiddelete(IDid);!voiddelete(Tentity);voiddelete(Iterable<?extendsT>entities);!voiddeleteAll();}
    30. 30. Spring Data Repositories IIpublicinterfacePagingAndSortingRepository<T,IDextendsSerializable>extendsCrudRepository<T,ID>{Iterable<T>findAll(Sortsort);Page<T>findAll(Pageablepageable);}publicinterfacePersonRepositoryextendsCrudRepository<Person,BigInteger>{//FinderforasingleentityPersonfindByEmailAddress(StringemailAddress);//FinderformultipleentitiesList<Person>findByLastnameLike(StringlastName);//FinderwithpaginationPage<Person>findByFirstnameLike(StringfirstName,Pageablepage);}
    31. 31. Spring Data JPA I@EntitypublicclassPerson{@Id@GeneratedValue(strategy=GenerationType.AUTO)&privateBigIntegerid;privateStringfirstname,lastname;@Column(name="email")privateStringemailAddress;@OneToManyprivateSet<Person>colleagues;}By just defining the interface, Spring provides the implementation<jpa:repositories,base.package="com.java2days.repository"2/>!
    32. 32. Spring Data JPA II‣Query methods use method namingconventions ‣ Can override with Query annotation ‣ Or method name references a JPA namedpublicinterfacePersonRepositoryextendsCrudRepository<Person,BigInteger>{//previousmethodsomitted… query!@Query("select!p!from!Person!p!where!p.emailAddress!=!?1")!!Person!findByEmailAddress(String!emailAddress);!!!!!@Query("select!p!from!Person!p!where!p.firstname!=!:firstname!or!p.lastname!=!:lastname")!!Person!findByLastnameOrFirstname(@Param("lastname")!String!lastname,!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!@Param("firstname")!String!firstname);}
    33. 33. Spring Data Key-ValueKey-Value Column Document Graph MongoDB
    34. 34. Spring Data‣MongoTemplate‣Object-Document Mapping‣Repository Support‣QueryDSL‣JMX‣Logging‣GeoSpatial‣Spring XML namespace
    35. 35. Spring Data MongoDB - Entity Mapping
    36. 36. Spring Data MongoTemplate
    37. 37. Spring MongoDB Template usage
    38. 38. Spring Data MongoDB Repository{ location : { $near : [point.x, point.y], $maxDistance : distance}}
    39. 39. publicinterfacePersonRepositoryextendsPagingAndSortingRepository<Person,String>{List<Person>findByLastName(StringlastName);List<Person>findByFirstName(StringlastName);List<Person>findByByShippingAdresses(Addressaddress);List<Person>findByByFirstNameAndLastName(Stringfn,Stringln);List<Person>findByByFirstNameAndLastName(Stringfn,Stringln,PageRequestpagingInfo);}
    40. 40. publicinterfacePersonRepositoryextendsPagingAndSortingRepository<Person,String>{List<Person>findByLastName(StringlastName);List<Person>findByFirstName(StringlastName);List<Person>findByByShippingAdresses(Addressaddress);List<Person>findByByFirstNameAndLastName(Stringfn,Stringln);List<Person>findByByFirstNameAndLastName(Stringfn,Stringln,PageRequestpagingInfo);}Keyword Sample Logical resultGreaterThan findByAgeGreaterThan(int age) {"age" : {"$gt" : age}}LessThan findByAgeLessThan(int age) {"age" : {"$lt" : age}}Between findByAgeBetween(int from, int to) {"age" : {"$gt" : from, "$lt" : to}}NotNull findByFirstnameNotNull() {”firstname" : {"$ne" : null}}Null findByFirstnameNull() {”firstname" : null}Like findByFirstnameLike(String name) "firstname" : firstname} (regex)
    41. 41. Spring Data Mongo Repository usage
    42. 42. Spring Data Key-ValueKey-Value Column Document Graph Redis, Riak
    43. 43. Spring Data Redis‣Portable API across several Redis connectors‣RedisTemplate ‣Access all Redis functionality, dedicated interfaces for each data type ‣Value / Hash / Set / ZSet / List Operations ‣Handles serialization and type conversion automatically ‣String specific class through StringRedisTemplate (JSON, XML...) ‣Fluent Query API‣Async Pub / Sub support with MLC‣Spring 3.1 Cache Abstraction provider
    44. 44. Redis Configuration (Jedis connector)<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd<bean id="jedisConnectionFactory" class="org.springframework.data.keyvalue.redis.connection.jedis.JedisConnect p:use-pool="true" p:host-name="server" p:port="6379"/><!-- redis template definition --><bean id="redisTemplate" class="org.springframework.data.keyvalue.redis.core.RedisTemplate" p:connection-factory-ref="jedisConnectionFactory"/>...</beans>
    45. 45. Using RedisTemplateredisTemplate.opsForValue().get(“user:salmar:transactions”);redisTemplate.opsForValue().set(“user:salmar:transactions”, 20);redisTemplate.opsForValue().increment(“user:salmar:transactions”, 1);redisTemplate.opsForSets().add(“user:salmar:friends”, “jlong”);redisTemplate.opsForSets().intersect(“user:salmar:friends”, “user.jlong:friends”);
    46. 46. Thank you! Q&A @sergialmar