Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Breaking down data silos with the open data protocol

2,519 views

Published on

Published in: Business, Technology
  • Be the first to comment

Breaking down data silos with the open data protocol

  1. 1. BREAKING DOWN “DATA SILOS”•WITH THE OPEN DATA PROTOCOL (ODATA) Chris Woodruff
  2. 2. WHO AM I? CHRIS WOODRUFF MVP, Data Platform Development Chris.Woodruff@rcmt.com http://chriswoodruff.com http://deepfriedbytes.com http://nplus1.org twitter @cwoodruff
  3. 3. Today, the tools for capturing data both at the mega-scale and at the milli-scale are just dreadful. After you have captured the data, you need to curate it before you can start doing any kind of data analysis, and we lack good tools for both data curation and data analysis. Jim Gray QUOTE
  4. 4. WHAT IS DATA NOW?
  5. 5. WHAT COULD DATA BE TOMORROW?
  6. 6.  An architectural style for building loosely coupled systems  defined by a set of very general constraints (principles)  the Web (URI/HTTP/HTML/XML) is an instance of this style  The Web used correctly (i.e., not using the Web as transport)  HTTP is built according to RESTful principles  services are built on top of Web standards without misusing them  most importantly, HTTP is an application protocol (not a transport protocol)  Anything that uses HTTP and XML (XML without SOAP)  XML-RPC was the first approach for this  violates REST because there is no uniform interface REST: 3 DEFINITIONS
  7. 7. The name Atom applies to a pair of related standards. The Atom Syndication Format is an XML language used for web feeds, while the Atom Publishing Protocol (AtomPub) is a simple HTTP-based protocol for creating and updating web resources. ATOMPUB
  8. 8. http://services.odata.org/OData/OData.svc _______________________________________/ | service root URI http://services.odata.org/OData/OData.svc/Category(1)/Products?$top=2&$orderby=name _______________________________________/ __________________/ _________________/ | | | service root URI resource path query options WHAT IS A URI?
  9. 9. Creation of standard Web API’s Allow internal Information Workers to be more productive Give IT staff more time to work on meaningful projects Monetize/Open Data for others to use and benefit from WHAT ARE THE BENEFITS OF ODATA?
  10. 10. THE BEAUTY OF METADATA
  11. 11. THE BASICS • Feeds, which are Collections of typed Entries • OData services can expose Service Operations • OData services expose all these constructs via URIs • OData service may also expose a Service Metadata Document
  12. 12. Type Example Description Binary <d:foo m:type="Edm.Binary">FRwvAAI...</d:foo> Fixed or variable length binary data in Base64 encoded format. Boolean <d:foo m:type="Edm.Boolean">true</d:foo> Represent the mathematical concept of binary valued logic. Byte <d:foo m:type="Edm.Byte">1</d:foo> An unsigned 8-bit integer value DateTime <d:foo m:type="Edm.DateTime">2010-02- 26T17:08:53.0900752-08:00</d:foo> A date and time with value ranging from 12:00:00 midnight, January 1, 1753 A.D. through 11:59:59 P.M, December 9999 A.D. DateTimeOffset <d:foo m:type="Edm.DateTimeOffset">2002-10- 10T17:00:00Z</d:foo> Date and time as an Offset in minutes from GMT, with values ranging from 12:00:00 midnight, January 1, 1753 A.D. through 11:59:59 P.M, December 9999 A.D. Time <d:foo m:type="Edm.Time">P120D</d:foo> Time duration. Decimal <d:foo m:type="Edm.Decimal">3.3</d:foo> A numeric value with fixed precision and scale from -10^255 + 1 to 10^255 - 1. Double <d:foo m:type="Edm.Double">4.4</d:foo> A floating point number with 15 digits precision that can represent values with approximate range of ± 2.23e -308 through ± 1.79e +308. Single <d:foo m:type="Edm.Single">4.4</d:foo> A floating point number with 7 digits precision that can represent values with approximate range of ± 1.18e -38 through ± 3.40e +38. Guid <d:foo m:type="Edm.Guid">223b00d9-e617-4a80- bf18-31bfd6e8e312</d:foo> A 16-byte (128-bit) unique identifier value Int16 <d:foo m:type="Edm.Int16">1</d:foo> Signed 16-bit integer value. Int32 <d:foo m:type="Edm.Int32">2</d:foo> Signed 32-bit integer value. Int64 <d:foo m:type="Edm.Int64">3</d:foo> Signed 64-bit integer value. String <d:foo>Beverages</d:foo> <d:foo m:type="Edm.String">Beverages</d:foo> Fixed or variable length character data. String is the default type. DATATYPES
  13. 13. FULL SQL LIKE QUERY “LANGUAGE” HTTP Command (Verb) SQL Command GET SELECT PUT UPDATE POST INSERT DELETE DELETE
  14. 14. How we get around OData QUERY NAVIGATION
  15. 15.  Addressing entities and sets  Presentation options Entity-set /Bookmarks Single entity /Bookmarks(123) Member access /Bookmarks(123)/Name Link traversal /Bookmarks(123)/Tags Deep access /Bookmarks(123)/Tags('abc')/Name Raw value access /Bookmarks(123)/Names/$value Sorting /Bookmarks?$orderby=Name Filtering /Bookmarks?$filter=Created gt '2007-05-07' Paging /Bookmarks?$top=10&$skip=30 Inline expansion /Bookmarks?$expand=Tags
  16. 16. Query Option Description $expand Expand related data inline with the results, e.g. Categories/$expand=Products would expand Product data inline with each Category entry. $filter A Boolean expression for whether a particular entry should be included in the feed, e.g. Categories?$filter=CategoryName eq 'Produce'. The Query Expression section describes OData expressions. $format One of “atom” (the default), “json” or “xml” depending on how you’d like your data returned to you. $inlinecount Includes the number of entries without the filter applied as a count element on the feed itself, e.g. Categories?$top=4&$inlinecount=allpages will return 8 with the default Northwind database installed. $orderby One or more comma-separated expressions with an optional “asc” (the default) or “desc” depending on the order you’d like the values sorted, e.g. Categories?$orderby=CategoryName desc. $select Limit the properties on each entry to just those requested, e.g. Categories?$select=CategoryName,Description. $skip How many entries you’d like to skip, e.g. Categories?$skip=4. $skiptoken Skips entries up to one past the one that matches the unique key. Used for paged access into a feed. Best used with $orderby, e.g. Categories?$orderby=CategoryID&$skiptoken=5 will skip everything ‘til it reaches the CategoryID of 5, then return entries past that. $top Return entries from the top of the feed, e.g. Categories?$top=4 QUERY OPTIONS
  17. 17. Operation Example Description Grouping (x add 4) eq 3 Used to make clear or override default operator precedence (which I can never remember anyway). Member access Categories(3)/Description Access of entity by ID or property by name. Method call startswith(Description, “Bread”) Call a built in method. Negate -x Change numeric sign. Not not x Logical not. And x and y Conditional and. Or x or y Conditional or. Multiply x mul y Multiplication. Divide x div y Division. Modulo x mod y Remainder. Add x add y Addition. Subtract x sub y Subtraction. Less than x lt y Comparison. Greater than x gt y Comparison. Less than or equal x le y Comparison. Greater than or equal x ge y Comparison. Equals x eq y Comparison. Not equals x ne y Comparison. FILTER EXPRESSIONS
  18. 18. Method Example Description endswith endswith(x, 'foo') Whether the end of the first parameter value matches the second parameter value. indexof indexof(x, 'foo') Index of the first occurrence of the second parameter value in the first parameter value or -1 otherwise. replace replace(x, 'foo', 'bar') String value with all occurrences of the second parameter value replaced by the third parameter value in the first parameter value. startswith startswith(x, 'foo') Whether the beginning of the first parameter values matches the second parameter value. tolower tolower(x) String value with the contents of the parameter value converted to lower case. toupper toupper(x) String value with the contents of the parameter value converted to upper case. trim trim(x) String value with the contents of the parameter value with all leading and trailing white-space characters removed. substring substring(x) substring(x, y) String value starting at the character index specified by the second parameter value in the first parameter string value. substringof substringof(x) substringof(x, y) Whether the second parameter string value occurs in the first parameter string value. concat concat(x) concat(y) String value which is the first and second parameter values merged together with the first parameter value coming first in the result. length length(x) The number of characters in the specified parameter value. FILTER METHODS
  19. 19. Examples http://.../OData.svc/GetProductsByRating?rating=5 Identifies the "GetProductsByRating" Service Operation and specifies a value of 5 for the "rating" input parameter. SERVICE OPERATIONS (FUNCTIONS)
  20. 20. WHERE CAN WE PRODUCE ODATA?
  21. 21. Data Access Layer Entity Framework Custom LINQ provider Relational database Other sources Service Runtime Hosting/HTTP listener HTTP IQueryable (+ IUpdatable) Create services directly from Visual Studio Various data sources Entity Framework LINQ providers Model-driven Structural description in Entity Data Model Metadata shapes service
  22. 22. Visual Studio 2010 SQL Server 2008 SP1 ASP.NET 4.0 WCF Data Services Framework uses ASP.NET for Authentication and Authorization WHAT DO YOU NEED FOR WCF DATA SERVICES?
  23. 23. WHERE CAN WE CONSUME ODATA?
  24. 24. CONSUMING FROM .NET APPLICATIONS Demo
  25. 25. OPEN YOUR DATA FOR OTHERS
  26. 26. MORE INFORMATION WCF Data Services Framework Home -- http://tinyurl.com/wcfdataservices WCF Data Services Framework Team Blog -- http://blogs.msdn.com/astoriateam ODataPrimer.com – http://odataprimer.com
  27. 27. NOTE ABOUT INTERNET EXPLORER Note, Internet Explorer by default renders ATOM in a friendly format that makes OData feeds unreadable. To fix this, turn off IE feed viewing by turning off the option Tools -> Internet Options -> Content - > Feeds (settings) -> Turn on feed viewing
  28. 28. Required Reading WRAP UP The Fourth Paradigm: Data-Intensive Scientific Discovery
  29. 29. MORE REQUIRED READING
  30. 30. CONTACT ME!! cwoodruff@live.com Twitter @cwoodruff MSN IM cwoodruff@live.com

×