1. Users as Data
Paul Ingles
@pingles

4. Users as Data

5. Everything as Data

6. Code as Data
(conj others me)

7. Data as Data
{:name "Paul Ingles"
:mood (constantly :happy)}

8. Data as Data
{:name "Paul Ingles"
:mood (constantly :hot)}

9. Builds as Data
(defproject energy-comparison "1.0.0"
:description "pricing service"
:dependencies
[[org.clojure/clojure "1.4.0"]
...)

10. Everything as Data

11. Everything as Data
• Data, in its most essential form

12. Everything as Data
• Data, in its most essential form
• Few simple abstractions
• map, seq, fn

13. Everything as Data
• Data, in its most essential form
• Few simple abstractions
• map, seq, fn
• General library
• map, filter, sort, take

14. Logs as Data?

15. Logs as Data?
0.0.0.0 - - [22/May/2012:09:00:03 +0000] "GET / HTTP/1.1" 304 0
"SomeUserAgent"

16. Logs as Files
/var/log/nginx/access.log

17. current power < potential power

18. log crunching event processing

19. Users as Data

20. current power < potential power

22. Energy Replication, ETL
Insurance
Credit Cards
Session DB Warehouse
Broadband

23. Data Muddling
Energy Replication, ETL
Insurance
Credit Cards
Session DB Warehouse
Broadband

24. Source: http://www.irregulartimes.com/panopt.html

25. Energy Insurance
Integration DB
Credit Cards Broadband

26. data in its essential
form?

27. simple abstractions?

28. The Joy of Clojure

29. The Joy of Clojure
• Functional
• Separation of computation and data

30. The Joy of Clojure
• Functional
• Separation of computation and data
• Immutable state

31. The Joy of Clojure
• Functional
• Separation of computation and data
• Immutable state
• Data as data

32. The Joy of Clojure
• Functional
• Separation of computation and data
• Immutable state
• Data as data
• Time, state and identity

33. Users as Data?

34. user = fn(interactions)

35. fn fn
S0 S1 S2
User
(Identity)
Time

36. User
fn fn
S0 S1 S2
User
(Identity)
Time

37. data > current state

39. data outlives
applications

40. current state
event processing
snapshots

41. Source: http://www.irregulartimes.com/panopt.html

42. “It is better to skip the massive up-
front time and expense, focusing
instead on making it very fast and easy
to add new data sources or new
elements to existing sources.”
Source: http://thinkrelevance.com/blog/2012/04/04/big-data-reference-model

43. data
data ecosystem
warehousing

44. Data Ecosystem

45. Producer
Producer Producer
Aggregator
Consumer Consumer
Consumer

46. events carrying smaller,
simpler pieces of data

47. {:user-id 1234
:email "paul@forward.co.uk"}
{:method :GET
:uri “/”
{:user-id 1234 :user-id 1234}
:postcode "NW3 2LB"}
{:user-id 1234
:energy-consumption {:gas {:units :kwh
:value 20500}}}

48. Users as Data

49. How we’ve used this...

51. http://github.com/pingles/clj-esper

52. $ ./print.sh -u <user-id>
(->> (events aggregator)
(filter (criteria :user-id "pingles")))

56. Users as Data

57. Thanks!

58. Thanks!
@pingles
h irin g!
We ’re