What linq is about


Published on

Linq & MongoDB

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

What linq is about

  1. 1. What LINQ is aboutor, Playingwith LINQ, MongoDB, and someold new patterns for developers.or, How to injectfunctionalparadigms in imperativelanguagesor, parallelism made easy.<br />WebWorkersCamp, 03/07/2010<br />Pierre Couzy – Microsoft France<br />
  2. 2. Who are you ?<br />Worked in the Microsoft space for 20 years<br />Speaker, trainer, developer, architect, …<br />Primarilyinterested in Web topics<br />Identity, architecture, WS-*, interop, cloud, …<br />Recentinterop/OSS activity<br />Worked on drupal for Sql server project<br />Works on automateddrupaltesting on Windows<br />Speaker at major PHP events<br />DrupalCon, PHP Forum, Symfony live<br />blog.couzy.com / pierre.couzy@microsoft.com<br />
  3. 3. Agenda<br />LINQ<br />More LINQ<br />LINQ for MongoDB<br />GoingParallel<br />GoingDistributed<br />Q&A<br />Disclosure: This session is heavily Microsoft-Biased (which is to be expected, I work there)<br />
  4. 4. LINQ : Why ?<br />C# is an imperativelanguage<br />Whichisnice<br />Except for a few things<br />Splitting data manipulation betweenyour code and your data store<br />Coding data manipulation<br />Applying new manipulation patterns to existing data<br />
  5. 5. Clumsyc#example<br />
  6. 6. Whatwe’dlikeis<br />Express more intent, lessimplementation<br />Easy composition<br />Strongtyping<br />This issubject to hot debate<br />Be as declarative as possible within the limitsimposed by C# <br />
  7. 7. From C# to LINQ<br />
  8. 8. Things to notice<br />Lazyevaluation<br />If westepintothis code, wesee LINQ triggers itsmagic on demand<br />Easy composition<br />If weadd new clauses after the LINQ expression isbuilt but beforeit’sconsumed, wesimplymerge the additions to the existing expression<br />So far, we’veworked on objects<br />Whichis how manynaive LINQ implementationswork : synchronously pipelining methods calls on collections<br />LINQ has an interesting twist<br />Expression trees<br />
  9. 9. Expression Trees and IQueryable<br />Let’stake the sameexample and introduceAsQueryable()<br />Uponexecution, wecan notice the compiler didn’teatour LINQ expression<br />It’sstill in there, using a tree format (an AbstractSymbolTree)<br />That treewillberesolvedatruntime<br />In the Linq to Objects case, wegetexactly the samebehaviour. <br />The iteration pattern maystillbeyield-based, but thisisleft to IQueryable’simplementationdetails.<br />
  10. 10. Playingwith expression trees<br />Let’stackleLinq To SQL<br />My expression treeisnowpushed to the database layer<br />If I want to block the flow to SQL, I just have to insert a new node in the tree.<br />Whichallows me to balance things<br />Using C# patterns and methodswhen I wantthem<br />Pushing to the databasewhatitdoes best<br />
  11. 11. IQueryable goodies<br />So, if I wantmythings to be LINQ aware, I caneither<br />Buildsomemethods and let the IEnumerable<T> do its pure objectmagic (I stillgetlazyeval)<br />Or, playwithIQueryable<br />And do myowninterpretation of the tree<br />Let’sseeMongoDB in LINQ<br />
  12. 12. Strongly-typed interaction when querying and updating collections.<br />Improved interface to send common Mongo commands (creating indices, getting all the existing dbs and collections, etc.).<br />Ultra-fast de/serialization of BSON to .Net CLR types and back.<br />LINQ-to-Mongo<br />Support for Mono<br />
  13. 13. LINQ Patterns<br />Having a LINQ Expression isnice: youbasicallydelegate the execution to someoneelse’s code<br />Plus, your LINQ Expressions have every good aspect youusuallyexpectfromfunctionalprogramming (ie, they are closures).<br />So, they are perfectlysuited to parallelisation<br />All you have to do isaddanothernode to the expression tree<br />Demo : AsParallel()<br />
  14. 14. WritingMapReduce in LINQ <br />public static IQueryable<R>  MapReduce<S,M,K,R>(this IQueryable<S> source,                                 Expression<Func<S,IEnumerable<M>>> mapper,                                 Expression<Func<M,K>> keySelector,                                 Expression<Func<K,IEnumerable<M>,R>> reducer){    return source.SelectMany(mapper).GroupBy(keySelector, reducer);}<br />Notice that the callingsyntax : Source.MapReduce(lMapper, lKeySelector, lReducer)issimply a new nodethatcanbeinserted in a larger expression tree.<br />This MapReducemethodcanrun on N coresusing PLINQ, or on X machines (and N cores per machine) usingDryadLINQ<br />
  15. 15. Collection<T> collection;<br />boolIsLegal(Key k); <br />string Hash(Key);<br />var results = from c in collection where IsLegal(c.key) select new { Hash(c.key), c.value};<br />DryadLINQ = LINQ + Dryad<br />Vertexcode<br />Queryplan<br />(Dryad job)<br />Data<br />collection<br />C#<br />C#<br />C#<br />C#<br />results<br />
  16. 16. Parallel execution<br />var logentries =<br /> from line in logs<br />where !line.StartsWith("#")<br />select new LogEntry(line);<br />var user = <br /> from access in logentries<br />where access.user.EndsWith(@"ulfar")<br />select access;<br />var accesses =<br />from access in user<br />group access by access.page into pages<br />select new UserPageCount("ulfar", pages.Key, pages.Count());<br />var htmAccesses =<br />from access in accesses<br />where access.page.EndsWith(".htm")<br />orderby access.count descending<br />select access; <br />
  17. 17. Thanks.<br />