Super Spike

Yep. It’s Super
Who am I?

            Who I am?
Indoctrination
                 Reduction in fear

 Super awesome MongoDB non-relational database tips
An introduction on how to use the MongoDB C# Driver


    Super awesome F# functional programming tips
      How to integrate F# into your apps today
      Looks at some functional techniques




     BONUS: (I hope) some inspiration
1. Analyze transactional data


                  2. Create models for setting prices


3. Work with sales team to raise prices


                         4. Measure results
Narrow




         Deep
• Mongo
  – Unstructured data
  – Flexible query language…using JavaScript!
  – Feels…”light”
• F#
  – Reasoning
       • Less code
       • No side effects
  – Productivity
       • fsi
       • Focus on what, not where
• Data Access
  – Mostly read-only
  – Data processing heavy
  – Non-standard formats from client to client
• Data Processing
  – Calculate metrics on different slices of data
  – Different formulas for various metrics
View (razor)


            Controller (C#)


Model
         Data Processing (F#)
 (C#)


           Data Access (F#)




         MongoDB
• Current Situation
  – Unknown schema
  – Complex transformation scripts
  – Complex loading scripts
static member Initialize() =
    BsonClassMap.RegisterClassMap<Customer>(fun cm ->
      cm.AutoMap()
      cm.SetIdMember(cm.GetMemberMap("Number"))
    ) |> ignore

   BsonClassMap.RegisterClassMap<Product>(fun cm ->
     cm.AutoMap()
     cm.SetIdMember(cm.GetMemberMap("SKU"))
   ) |> ignore
private static void ImportFromFile<T>(
        string filename, Dictionary<string, int> fieldMappings)
where T: CoreObject {
    var datafile = DataFile.FindOneById(filename);
    var record = datafile.ReadLine();
    while (record != null) {
        var o = Activator.CreateInstance<T>();
        o.Load(record, fieldMappings);
        o.Save();

        record = datafile.ReadLine();
    }
}
{
    "_id": "1234567",
    "_t": "Customer",
    "Attributes": {
              "Parent": "Mike's Chocolates",
              "Salesman": "Willy Wonka",
              "Industry": "Candy"
    },
    "Name": "Choc a lot"
}

                            • Invoice
                                • Date
                                • Quantity
                                • Amount
                                • Customer
                                • Product
• Current Situation
  – Hard to find
  – Hard to test
  – Calculations are often performed “on top of”
    other calculations
  – Technical team needs to explain how formulas
    work
let GetCollection<'T>() =
  (new MongoRepository())
    .GetDatabase()
    .GetCollection<'T> typeof<'T>.Name

let Invoices = GetCollection<Invoice>

let GetInvoices (fromDate:DateTime) (toDate:DateTime) includeFields =
  let query = Query
               .GTE("Date", BsonDateTime.Create(fromDate))
               .LTE(BsonDateTime.Create(toDate))
  let fields = Fields.Exclude("_id").Include(includeFields)
  Invoices().Find(query).SetFields(fields) :> seq<Invoice>
static member GetRevenueByYear (fromDate:DateTime) (toDate:DateTime) =
   GetInvoices fromDate toDate [|"Date"; "Amount"|]
     |> Seq.groupBy (fun i -> i.Date.Year)
     |> Seq.map (fun (year, invoices) -> (year, (sumRevenue invoices)))
     |> Seq.sortBy (fun (year, _) -> year)




   let sumRevenue (invoices:seq<Invoice>) =
     invoices |> Seq.sumBy (fun i -> i.Amount)
let IndexByCustomer fromDate toDate operation attribute =
    Index “Customer”
        (fun i -> i.Customer.Attributes.[attribute])
       fromDate toDate
       operation

let IndexByProduct fromDate toDate operation attribute =
    Index “Product”
       (fun i -> i.Product.Attributes.[attribute])
       fromDate toDate
       operation



  let Index type groupF fromDate toDate indexF

  string -> (Invoice -> string) -> DateTime -> DateTime -> (seq<Invoice> -> float) ->
     seq<string * float * float>
string -> (Invoice -> string) -> DateTime -> DateTime -> (seq<Invoice> -> float) ->
   seq<string * float * float>

let IndexCustomers attribute =
  Index "Customer" (fun (i:Invoice) -> i.Customer.Attributes.[attribute])

let IndexProducts attribute =
  Index "Product" (fun (i:Invoice) -> i.Product.Attributes.[attribute])


string -> (DateTime -> DateTime -> (seq<Invoice> -> float) ->
  seq<string * float * float>)



let IndexByCustomer fromDate toDate operation attribute =
    IndexCustomers attribute fromDate toDate operation

let IndexByProduct fromDate toDate operation attribute =
    IndexProducts attribute fromDate toDate operation
type Entity =
| Customer
| Product


let IndexEntity t attribute fromDate toDate indexOp =
  let f =
    match t with
      | Customer -> IndexCustomers
      | Product -> IndexProducts

 f attribute fromDate toDate indexOp
OPERATIONAL            DEVELOPMENT            USER PERCEIVED
From days to minutes   Reasoning about code   “OK”
                       Closer to the data
mongodb.org

https://github.com/mongodb/mongo-csharp-driver




http://www.tryfsharp.org

FsUnit (via NuGet)




http://www.jqplot.com/
www: michaelfalanga.com

E-mail: me@michaelfalanga.com

Twitter: @mfalanga

Super spike

  • 1.
  • 2.
    Who am I? Who I am?
  • 3.
    Indoctrination Reduction in fear Super awesome MongoDB non-relational database tips An introduction on how to use the MongoDB C# Driver Super awesome F# functional programming tips How to integrate F# into your apps today Looks at some functional techniques BONUS: (I hope) some inspiration
  • 4.
    1. Analyze transactionaldata 2. Create models for setting prices 3. Work with sales team to raise prices 4. Measure results
  • 5.
    Narrow Deep
  • 6.
    • Mongo – Unstructured data – Flexible query language…using JavaScript! – Feels…”light” • F# – Reasoning • Less code • No side effects – Productivity • fsi • Focus on what, not where
  • 7.
    • Data Access – Mostly read-only – Data processing heavy – Non-standard formats from client to client • Data Processing – Calculate metrics on different slices of data – Different formulas for various metrics
  • 8.
    View (razor) Controller (C#) Model Data Processing (F#) (C#) Data Access (F#) MongoDB
  • 9.
    • Current Situation – Unknown schema – Complex transformation scripts – Complex loading scripts
  • 12.
    static member Initialize()= BsonClassMap.RegisterClassMap<Customer>(fun cm -> cm.AutoMap() cm.SetIdMember(cm.GetMemberMap("Number")) ) |> ignore BsonClassMap.RegisterClassMap<Product>(fun cm -> cm.AutoMap() cm.SetIdMember(cm.GetMemberMap("SKU")) ) |> ignore
  • 13.
    private static voidImportFromFile<T>( string filename, Dictionary<string, int> fieldMappings) where T: CoreObject { var datafile = DataFile.FindOneById(filename); var record = datafile.ReadLine(); while (record != null) { var o = Activator.CreateInstance<T>(); o.Load(record, fieldMappings); o.Save(); record = datafile.ReadLine(); } }
  • 14.
    { "_id": "1234567", "_t": "Customer", "Attributes": { "Parent": "Mike's Chocolates", "Salesman": "Willy Wonka", "Industry": "Candy" }, "Name": "Choc a lot" } • Invoice • Date • Quantity • Amount • Customer • Product
  • 15.
    • Current Situation – Hard to find – Hard to test – Calculations are often performed “on top of” other calculations – Technical team needs to explain how formulas work
  • 16.
    let GetCollection<'T>() = (new MongoRepository()) .GetDatabase() .GetCollection<'T> typeof<'T>.Name let Invoices = GetCollection<Invoice> let GetInvoices (fromDate:DateTime) (toDate:DateTime) includeFields = let query = Query .GTE("Date", BsonDateTime.Create(fromDate)) .LTE(BsonDateTime.Create(toDate)) let fields = Fields.Exclude("_id").Include(includeFields) Invoices().Find(query).SetFields(fields) :> seq<Invoice>
  • 17.
    static member GetRevenueByYear(fromDate:DateTime) (toDate:DateTime) = GetInvoices fromDate toDate [|"Date"; "Amount"|] |> Seq.groupBy (fun i -> i.Date.Year) |> Seq.map (fun (year, invoices) -> (year, (sumRevenue invoices))) |> Seq.sortBy (fun (year, _) -> year) let sumRevenue (invoices:seq<Invoice>) = invoices |> Seq.sumBy (fun i -> i.Amount)
  • 19.
    let IndexByCustomer fromDatetoDate operation attribute = Index “Customer” (fun i -> i.Customer.Attributes.[attribute]) fromDate toDate operation let IndexByProduct fromDate toDate operation attribute = Index “Product” (fun i -> i.Product.Attributes.[attribute]) fromDate toDate operation let Index type groupF fromDate toDate indexF string -> (Invoice -> string) -> DateTime -> DateTime -> (seq<Invoice> -> float) -> seq<string * float * float>
  • 20.
    string -> (Invoice-> string) -> DateTime -> DateTime -> (seq<Invoice> -> float) -> seq<string * float * float> let IndexCustomers attribute = Index "Customer" (fun (i:Invoice) -> i.Customer.Attributes.[attribute]) let IndexProducts attribute = Index "Product" (fun (i:Invoice) -> i.Product.Attributes.[attribute]) string -> (DateTime -> DateTime -> (seq<Invoice> -> float) -> seq<string * float * float>) let IndexByCustomer fromDate toDate operation attribute = IndexCustomers attribute fromDate toDate operation let IndexByProduct fromDate toDate operation attribute = IndexProducts attribute fromDate toDate operation
  • 21.
    type Entity = |Customer | Product let IndexEntity t attribute fromDate toDate indexOp = let f = match t with | Customer -> IndexCustomers | Product -> IndexProducts f attribute fromDate toDate indexOp
  • 22.
    OPERATIONAL DEVELOPMENT USER PERCEIVED From days to minutes Reasoning about code “OK” Closer to the data
  • 24.
  • 25.

Editor's Notes

  • #3 Principles: 1) software is a means to an end; 2) the customer is the architect
  • #5 Principles: 1) software is a means to an end; 2) the customer is the architect
  • #6 INVESTIGATION of a solutionNarrow: focused on particular objectivesDrives deep: feasibility and “how it works”
  • #8 Not unique…
  • #9 SOLUTION EXPLORER- Models, Lib, Web- Show References- Order of files in F# project
  • #10 Schema: wastexform: ripe for error, low bus numberload: complex, business rules data driven during load, can only be changed by reloading, “further from the data”Show C# code – fields we need versus “things that are interesting”How much effort did we just eliminate?
  • #11 So What? Can do this with RDBMSMapping to database tables and columns vs. mapping to properties of an object
  • #14 Field names: Parent, Salesman, Industry
  • #18 Map-ReduceSELECT … GROUP BY!!!!Within a single toolUnit testing
  • #22 So many ways to compose functions. Make DSLs