Storing billions of images in a hybrid relational and NoSQL database using Oracle Active Data Guard and Oracle NoSQL

  • 873 views
Uploaded on

Storing billions of images …

Storing billions of images
in a hybrid relational and NoSQL database
using Oracle Active Data Guard and Oracle NoSQL

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
873
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Storing billions of images in a hybrid relational and NoSQL database using Oracle Active Data Guard and Oracle NoSQL Aris Prassinos
  • 2. Who I am • Oracle ACE 2007, Oracle ACE Director 2009 • Oracle Magazine Content Management Architect of the Year (2005) • http://www.ArisPrassinos.com Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 2 © Aris Prassinos 2013
  • 3. Agenda • Introduce a real world use case • Quick recap of relevant Oracle NoSQL concepts • Present a solution that straddles relational and Oracle NoSQL databases • Discuss things to consider and lessons learned Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 3 © Aris Prassinos 2013
  • 4. The use case • Application storing large amounts of binary data • images, documents, short videos • High value data • data consistency and minimal data loss • Write Once Read Many / occasional updates, deletes • Random reads throughout database regardless of age of record Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 4 © Aris Prassinos 2013
  • 5. Assumptions / Constraints • High transaction volume with low latency • Reads dominate writes • High Availability using Active Standby datacenter • Low latency between Primary and Standby datacenters • Clients can arbitrarily connect to any datacenter • System can tolerate a temporary loss of availability to guarantee consistency Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 5 © Aris Prassinos 2013
  • 6. Traditional RDBMS approach • Oracle Active Data Guard / Synchronous replication to Standby(s) • Reader farm of Standby's to scale Reads • Real-time intelligent routing of calls between Primary and Standby(s) based on transaction requirements (e.g. read own writes) and Standby lag http://www.oracle.com/technetwork/database/features/availability/morphotrak-132603.pdf • Oracle Global Data Services to load balance Reads on Standbys • Oracle 12c feature • Minimal loss of availability during failover • Third Observer site needed when two datacenters are used Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 6 © Aris Prassinos 2013
  • 7. Why a hybrid solution • High licensing cost for traditional RDBMS solution • Other lower cost RDBMS not well suited for large scale image handling • Concerns with storing images as regular files on the filesystem: • durability / recovery • consistency / integrity • performance / scalability • high availability / replication • security • Store the binary data and minimal metadata in Oracle NoSQL and everything else in Oracle database Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 7 © Aris Prassinos 2013
  • 8. Consistency between RDBMS and Oracle NoSQL • Lack of transactions in Oracle NoSQL • possibility of orphaned records left in Oracle NoSQL • RDBMS is the source of truth • Only query Oracle NoSQL using unique id obtained from RDBMS • avoid queries on metadata in Oracle NoSQL • Make NoSQL inserts idempotent (upserts) • Oracle NoSQL may be ahead of an Active Data Guard Standby • create a new version for each NoSQL update and store version# in RDBMS • don't immediately delete from NoSQL but flag as deleted instead Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 8 © Aris Prassinos 2013
  • 9. Oracle NoSQL key-value store • Major key : id, version • Minor keys (columns): binary_data, binary_data.lob, mime_type, size, signature, delete_flag • Internal representation • id/version/binary_data => data • id/version/binary_data.lob => data • id/version/signature => data etc • Note that Oracle NoSQL offers native streaming of large LOBs but different columns (binary_data, binary_data.lob) are needed to store both non-streamed and streamed LOBs under the same major key • ACID semantics only for data within same Major key Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 9 © Aris Prassinos 2013
  • 10. Oracle NoSQL Scalability • Reader farm approach to scale Reads • single Master / multiple Replicas architecture • run-time load balancing of Reads based on real-time latency stats • Oracle NoSQL latency aware driver • Sharding to scale NoSQL Writes • Multiple Oracle NoSQL Replication Groups • Each shard (Replication Group) has its own Master and Replicas • Transparent to the client • A physical node may host the Master of one shard and the Replicas of other shards • replicas may later be moved to dedicated physical nodes for scaling out Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 10 © Aris Prassinos 2013
  • 11. Oracle NoSQL Master election • Master is elected using PAXOS algorithm • Master election held when a group of nodes forming a majority does not have a Master • when less than half of the nodes remain alive the number that constitutes a majority can be manually overridden using je.rep.electableGroupSizeOverride • possible to end up with two Masters if former Master is in group that consists of a minority of nodes after a network disconnect • multiple Master problem solved by always requiring quorum writes Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 11 © Aris Prassinos 2013
  • 12. Two datacenter configuration • Nodes split equally between the two sites • A third lightweight arbiter site needed to allow automatic master election otherwise Standby site nodes cannot form a majority when Primary site lost • Arbiter is an upcoming feature • Arbiter placed at same site as Data Guard Observer • otherwise manually set je.rep.electableGroupSizeOverride for manual failover • By default NoSQL Master may not be on the same site as Data Guard Primary • overridden using je.rep.node.priority for the preferred Master site nodes Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 12 © Aris Prassinos 2013
  • 13. NoSQL Writes • 1. Attempt a quorum write (Durability.ReplicaAckPolicy.SIMPLE_MAJORITY) In terms of CAP favor Latency over Consistency. Possible inconsistencies can be handled when reading 2. If quorum write fails retry a few times to allow for potential new master election 3. If quorum write still fails decision depends on number of nodes (RF) In terms of CAP favor Consistency over Availability. If RF=2 attempt to write only to Master (Durability.ReplicaAckPolicy.NONE). If RF>2 an administrator needs to intervene since there is a possibility of having two masters if network partitioned and writing without quorum could cause divergent transactions. Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 13 © Aris Prassinos 2013
  • 14. NoSQL Reads • 1. Read from any replica using unique id and version. Due to quorum writes some replicas may not yet have the requested data 2. If data not found then attempt to read from Master. We expect the data to exist since we query based on information in the RDBMS which is the source of truth. 3. If Master not reachable return error. • This approach achieves strong consistency, minimal latency, high availability • most Reads to proceed when Master is temporarily unavailable • only Reads resulting in no data found return error Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 14 © Aris Prassinos 2013
  • 15. NoSQL log structured storage and backups • Data always appended to log files up to a configurable size • inserts, updates, deletes go to the latest log file • Durability.SyncPolicy controls how often data written / flushed to disk • old log files are immutable and may contain deleted or overwritten data • Compaction process may move data from old log files to new ones • when user defined percentage of valid records remains in a log file • this is one of the primary candidates for tuning in a production system • Snapshots and RSYNC can be used to perform incremental NoSQL backups Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 15 © Aris Prassinos 2013
  • 16. NoSQL backups using replication • NoSQL data since time of last snapshot lost after a restore when only a single node used i.e. ReplicationFactor = 1 • Imperative to use NoSQL replication for loss-less recovery • Possible to setup a node as a dedicated backup server for one or more nodes • define each backup Rep Node as non electable master (i.e. always be Replica) plan change-parameters ... configProperties="je.rep.node.priority=0" Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 16 © Aris Prassinos 2013
  • 17. Conclusion • Hybrid approach superior to just keeping pointers to binary data stored on the filesystem • replication, sharding, adjustable consistency / durability • Strong data consistency achieved with careful application coding • keeping in mind the consistency / availability / latency tradeoffs • Very good architectural fit with Active Data Guard • master / replica architecture, observer / arbiter third site • Better suited for a replicated NoSQL configuration rather than a single node • minimum of two server nodes and if necessary a third arbiter Oracle Open World 2006 MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2005 Slide 17 © Aris Prassinos 2013