Developing ii advanced document design

Advanced Document Design Matt Ingenthron matt@couchbase.com @ingenthr

● Heading up Developer Solutions ● Make sure you guys have the right bits ● Have touched pretty much every component in the system● Long involved in distributed systems ● Worked with Hadoop before it was popular ● Came across memcached, participated/lead in hackathons-- initially lead by Facebook ● Worked on the Java client● Joined Membase in the very early days ● 3rd person aboard

Agenda● Document Databases ● Schema-less Data Modeling ● Inherent Constraints● Document Design Basics ● living in the world of Documents● Getting at your Documents (with Demos!) ● Document Editing Demo ● Creating Views and Indexes ● Querying those Indexes ● Reducing the output to values● Questions

Document DatabasesDocument Document DesignDatabases Basics Creating Demonstration Summary Views of View Editing

Schema-less Data Modeling● ad hoc data store ● No ALTER TABLE ● Add new "ﬁelds" to any object any time ● improves development speed● Structure matters at query time, not write time● JSON data is interoperable ● 2.0 Features ● dynamic queries with views ● “Web-native” format ● broad language support: json.org

Inherent Constraints● Document ID is the only (DB-side) way to enforce uniqueness● JSON doc key name restrictions ● top-level keys may not begin with either “_” or “$”● Small “gotchas” ● watch out for numbers as strings (Reduce will care) ● data format consistency ● unix time stamps ● some other JS parse-able, IETF codiﬁed format

Document Design BasicsDocument Document DesignDatabases Basics Creating Demonstration Summary Views of View Editing

Beer Document type name in ID{ "_id": "beer_1554_Enlightened_Black_Ale", "brewery": "New Belgium Brewing",used to reference "name": "1554 Enlightened Black Ale", brewery doc the "abv": "5.5", in MapReduce "description": "Born of a ﬂood...", "category": "Belgian and French Ale", "style": "Other Belgian-Style Ales", "updated": "2010-07-22 20:00:20"} “vintage” date from an SQL dump >_<

Brewery Document type name in ID{"_id": "brewery_New_Belgium_Brewing", "name": "New Belgium Brewing", "address": ["500 Linden Street"], JSON array "city": "Fort Collins", "state": "Colorado", "code": "80524", "website": "http://www.newbelgium.com/", "description": "Well set the scene...”, "geo": { GeoCouch FTW! "loc": ["-105.07", "40.5929"], "accuracy": "RANGE_INTERPOLATED" }, "updated": "2010-07-22 20:00:20"}

Beernique Demo

Creating Views Couchbase Server 2.0: Querying and Aggregation with ViewsDocument Document DesignDatabases Basics Creating Demonstration Summary Views of View Editing

Document Editing Demo

Couchbase Server 2.0:Querying and Aggregation with Views

MapReduce Basics● Building indexes, not querying ● Eventually Consistent indexed view ● “Secondary Index”● Output keys are used to reference ranges of the index● Document IDs can also be used for referencing● “Querying” is done via the Query String ● the SDKs wrap this up for you● Values can be anything● Reduce is an additional layer

Map function to build the Index of Breweries andtheir Beers_design/beers/_view/brewery_beersfunction(doc) { if (doc.geo) { emit([humanize(doc._id)], 1); } else if (doc.brewery && typeof doc.brewery === string) { emit([doc.brewery, humanize(doc._id)], 1); } // handy function for building IDs from names function humanize(name) { return name.replace(/^(brewery_|beer_)/, ) .replace(/_/g, ); }}

Built-in Reduce function for the Brewery andtheir Beers_design/beers/_view/brewery_beers?reduce=true_count• Built-in reducers are the fastest option for Index reduction• _count• _sum• _stats (min, max, count, avg)

View EditingDocument Document DesignDatabases Basics Creating Demonstration Summary Views of View Editing

Views Editing Demo

Results from the brewery_beers index_design/beers/_view/brewery_beers{"rows": [ {"id": "brewery_Abhi_Brewery", "key": ["Abhi Brewery"], "value": 1 }, {"id": "beer_Abhi_beer", "key": ["Abhi Brewery","Abhi beer"], "value": 1 }]}

Wait...wheres the data? Meet ?include_docs=true

Results from the brewery_beers index_design/beers/_view/brewery_beers?include_docs=true{"rows": [ {"id": "brewery_Abhi_Brewery", "key": ["Abhi Brewery"], "value": 1, "doc": { "_id": "brewery_Abhi_Brewery", "name": "Abhi Brewery", .... } }, {"id": "beer_Abhi_beer", "key": ["Abhi Brewery","Abhi beer"], "value": 1, "doc": { "_id": "beer_Abhi_beer", "name": "Abhi beer", ....} }]}

Results from the brewery_beers index_design/beers/_view/brewery_beers?startkey=["NewBelgium Brewing"]&endkey=["New Belgium Brewing", {}]{"rows": [ {"id": "brewery_New_Belgium_Brewing", "key": ["New Belgium Brewing"], "value": 1, "doc": { "_id": "brewery_New_Belgium_Brewing", "name": "New Belgium Brewing", .... } }, {"id": "beer_Fat_Tire_Amber_Ale", "key": ["New Belgium Brewing", "Fat Tire Amber Ale"], "value": 1, "doc": { "_id": "beer_Fat_Tire_Amber_Ale", "name": "Fat Tire Amber Ale", ....} } ]}

Number of Beers per Brewery usingReduce_design/beers/_view/brewery_beers?group_level=1{"rows": [ {"id": "brewery_New_Belgium_Brewing", "key": ["New Belgium Brewing"], "value": 9, {"id": "brewery_Other_Brewery", "key": ["Other Brewery"], "value": 3]}• Values are the full count of docs beginning in this range • subtract one (in this case) to ﬁnd the # of beer docs

Creating Views Couchbase Server 2.0: Querying and Aggregation with ViewsDocument Document DesignDatabases Basics Creating Demonstration Summary Views of View Editing

Summary● Schema-less Document Databases● Document Design ● you deal in docs daily● View Index Building ● schema on the "way out"

Questions?

Thanks! Matt Ingenthronmatt@couchbase.com @ingenthr

Developing ii advanced document design

by Couchbase on May 29, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Developing ii advanced document design Presentation Transcript