Tech Days Paris Intoduction F# and Collective Intelligence

F# on the Road to Visual Studio 2010 Robert Pickering – LexiFi (ALTi) With thanks to Don Syme, Leon “secretGeek” Bambrick, Chris Smith and the puppies

Session Overview • Why was F# created? Why learn F#? • A taste of the F# language – Especially the functional side! • See a complete working F# “Collective Intelligence” application

Part 1 F#... Do not be afraid.

I’m F#

I'll let you in on a secret: I'm doing F# simply because it's lots and lots of fun. In a very broad sense of the word: functional programming is fun, OO programming with F# is fun, watching people use F# is fun. One of the wonderful things about F# is that you can actually end up working in your domain. In the zone. With F#, you're not necessarily quot;justquot; a programmer! You're likely to also be a probabilistic modeller, or an AutoCAD engineer, or a finance engineer, or a symbolic programmer, or one of many other things. - Don Syme, F#’s creator

F# is unique amongst both imperative and declarative languages in that it is the golden middle road where these two extremes converge. F# takes the best features of both paradigms and tastefully combines them in a highly productive and elegant language that both scientists and developers identify with. F# makes programmers better mathematicians and mathematicians better programmers. - Eric Meijer, Forward to Expert F#

A Few Words From … Julien Laugel Head of Research & Development EuroPerformance

F#: Combining Paradigms I've been coding in F# lately, for a production task. F# allows you to move smoothly in your programming style... I start with pure functional code, shift slightly towards an object-oriented style, and in production code, I sometimes have to do some imperative programming. I can start with a pure idea, and still finish my project with realistic code. You're never disappointed in any phase of the project! Julien Laugel, Head of Research & Development, EuroPerformance

F#: Influences OCaml F# C#/.NET Similar core Similar object language model

Road Map • Current Release: September CTP for VS 2008 • Next Release: as part of VS 2010 beta 1 • Plugin for VS Shell and CodePlex release in VS 2010 timeframe • More: http://blogs.msdn.com/dsyme/archive/2008/12/10/fsharp-to-ship-as-part-of-visual-studio-2010.aspx

Part 2 F# the Language ... ... and the Functional Point of View

Code! //F# //C# open System using System; let a = 2 namespace ConsoleApplication1 Console.WriteLine a { class Program { static int a() { return 2; } static void Main(string[] args) { Console.WriteLine(a); } } }

More Code! //F# open System let a = 2 Console.WriteLine a a 2 Console.WriteLine a More Noise Than Signal!

Let Bindings & Literals Declaration An identifier A comment of a binding let aString = quot;Stringyquot; // a string Like “var” in A literal C#

Lists Let binding The empty // the empty list let emptyList = [] list Literal Concatenate The empty list // adding to a list let conact = quot;Onequot; :: []

Lists let list = 1 :: 2 :: 3 :: [] These are the same let list = [1; 2; 3]

Functions Parameter // a function let addTen = fun x -> x + 10 Let binding Result

Functions Parameter // shorter version let addTen x = x + 10 Let binding Result

Functions Let binding Parameters // multi parameters and // intermediate results let addThenTimesTwo x y = let temp = x + y Intermediate temp * 2 Value Whitespace Sensitive Result

Pattern Matching // simple pattern matching let matching n = match n with | 1 -> quot;Onequot; Pattern | 2 -> quot;Twoquot; Result | _ -> quot;Otherquot;

Pattern Matching // pattern matching Recursive // over lists let rec addOne list = function Pattern match list with | head :: tail -> head + 1 :: addOne tail | [] -> [] Result Recursive call // more pattern matching // over lists let rec subOne list = match list with | head :: tail -> head - 1 :: subOne tail | [] -> []

DRY! D on’t R eapt Y ourself (ML is 23 years older than Ruby)

Functions as Parameters // pattern matching Function // over lists let rec map func list = parameter match list with | head :: tail -> Call function func head :: map tail | [] -> [] let addOne list = List.map (fun x -> x + 1) list let subOne list = List.map (fun x -> x + 1) list

Part 3 A “Collective Intelligence” Example Application Classifying the Bloggers

Representing a Blog • A blog can be represented by the frequency that words appear in it • Most and least common words are cut – words like “the” are ignored – words like “woah” are ignored Phone Email Book People Affiliate Heckler Spray 0 0 1 2 0 Shiny Shiny 3 0 0 5 0 Chris Garret on New Media 1 75 12 60 24

Measuring “Closeness” of Blogs People Chris Garret on New Media Heckler Spray Email Book

Measuring “Closeness” of Blogs • Pervious slide shows “Euclidean Distance” • While this could be use we use “Pearson correlation” – “Pearson correlation” corrects for the problem that some blogs are longer than others where

Pearson correlation in F# let pearson (wc1: seq<float>) (wc2: seq<float>) = let sum = PSeq.reduce (+) let sum1 = sum wc1 let sum2 = sum wc2 let sumSq1 = sum (Seq.map (fun x -> x * x) wc1) let sumSq2 = sum (Seq.map (fun x -> x * x) wc2) let pSum = sum (Seq.map2 (fun x y -> x * y) wc1 wc2) let len = float (Seq.length wc1) let num = pSum - ((sum1 * sum2) / len) let den = sqrt ((sumSq1 - (sum1 * sum1) / len) * (sumSq2 - (sum2 * sum2) / len)) if den = 0. then 0. else num / den

Hierarchical Clustering A B A B A B A B C C C C D E D E D E D E A B C D E

Viewing the Hierarchical Clustering A B C D E

How we implement hierarchical clustering efficiently in F# THE ALGORITHM

Map Reduce Fetch Data Process Data Build Tree Start End Clean Data Calculate Words Used Split Data Make HTTP Request Build Tree Count Word Create Master Word List

Map Reduce Fetch Data Process Data Build Tree Start End Parallelized by the Parallelized by Asynchronous Workflows Parallel Task Library

3 Layer Architecture Data Access – dataAccess.fs Helpers extentions.fs Business Logic – algo.fs UI – Program.fs Tests – test.fsx

Hierarchical clustering application in action DEMO

Application Metrics • Helper/Extension Code: ~60 lines • Data Access Code: ~30 lines • Main algorithm: ~220 lines • UI code: ~150 lines (F#) / ~40 lines (XAML)

Other Things To Cluster • Any other kind of text: – Books from project Gutenberg – Twitter! – Messages boards or other web pages • People in social networks • Reviewers on Amazon • Companies based on metrics

Collective Intelligence By Toby Segaran

Part 3 End Bit

msdn.microsoft.com/fsharp/

F# Resources • MDSN Resource center: http://msdn.microsoft.com/fsharp/ • User forums: http://cs.hubfs.net/forums • Blogs (there are lots of others!): • http://blogs.msnd.com/dsyme • http://strangelights.com/blog • Samples on the web: • http://code.msdn.microsoft.com/fsharpsamples • http://code.google.com/hosting/search?q=label:fsharp • http://codeplex.com/Project/ProjectDirectory.aspx?TagName=F%23 • Source available with the distribution: %ProgramFiles%FSharp-1.9.6.2source

Current Books about F#

Books to come about F# Second Edition

alt.net.france Program: • 18 February: Aspectize – Frédéric Fadel – Winwise, 16 rue Gaillon 75002 Paris • 18 March: TDD – Djamel Zouaoui – chez OCTO Technology, 50 avenue des champs Elysées • 22 April: Entity Framework – Matthieu MEZIL • More info: http://groups.google.com/group/parisaltnet

Questions?

Tech Days Paris Intoduction F# and Collective Intelligence

by Robert Pickering on May 26, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 7

Statistics

Tech Days Paris Intoduction F# and Collective Intelligence — Presentation Transcript