From Business Intelligence to BigData

1,647 views
1,530 views

Published on

Slid

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,647
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
78
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Presentation: BI/Big Data Futures - Is it really all about the Cloud?In this survey session, Lynn will bring you up-to-date on what's happening in the world of enterprise Business Intelligence.  BigData, NoSQL, Hadoop, Big Analytics, Cloud Storage, what does all of this mean to you as a data professional?  Which products and technologies are mature enough for enterprise adoption and which ones are not?  Which vendors should you be trying out and why? What is the reality of hosting enterprise data on the cloud? What are the business reasons to explore these new technologies?  How do you learn to implement them?Lynn frames this talk with the three major trends that she sees in the Enterprise BI space, highlighting products and technologies that warrant a deeper look.  
  • http://www.amazon.com/Lynn-Langit/e/B00335W5OS
  • http://hadoop.apache.org/ & http://www.mongodb.org/
  • http://www.oracle.com/technetwork/bdc/hadoop-loader/overview/index.html
  • http://www.microsoft.com/download/en/details.aspx?id=27584
  • https://www.hadooponazure.com/Account
  • https://www.hadooponazure.com/Account
  • http://windows.azure.com
  • https://datamarket.azure.com/
  • http://aws.amazon.com/
  • http://aws.amazon.com/http://docs.amazonwebservices.com/amazondynamodb/latest/developerguide/Introduction.html
  • http://code.google.com/appengine/http://code.google.com/appengine/articles/datastore/overview.html
  • http://code.google.com
  • http://lynnlangit.wordpress.com/2011/11/09/relational-cloud-storage-is-50x-more-expensive-than-nosql/
  • http://www.splunk.com/product
  • http://www.romymisra.com/the-new-job-market-rulers-data-scientists/
  • http://www.quora.com/Career-Advice/How-do-I-become-a-data-scientistSlide from- http://www.huffingtonpost.com/roger-ehrenberg/data-driven-startup_b_1088124.html?ref=tw
  • http://hortonworks.com/technology/hortonworksdataplatform/http://www.cloudera.com/
  • http://www.freebase.com/http://code.google.com/p/google-refine/
  • http://www.microsoft.com/en-us/sqlazurelabs/default.aspx andhttp://www.microsoft.com/en-us/sqlazurelabs/labs/dataexplorer.aspx
  • http://www.web-designers-directory.org/articles/top-rated-android-applications-for-2011-20.html
  • Part of Microsoft's Hadoop efforts include an ODBC driver for Hive, the Hadoop query engine, which will provide “direct realtime querying from business intelligence tools into Hadoop,” Leland said. The Azure Hadoop service and the Hive ODBC driver will be available in preview before the end of 2011, with a community technology preview for Hadoop on Windows Server arriving next year.
  • http://research.microsoft.com/en-us/um/cambridge/projects/infernet/
  • http://www.r-project.org/
  • http://www.youtube.com/watch?v=gjsMDAcI1Mo
  • http://dennyglee.com/
  • http://www.predixionsoftware.com/predixion/
  • http://www.qlikview.com/us
  • PowerView YouTube video - http://www.youtube.com/watch?v=75szAtMrkNs
  • http://www.romymisra.com/the-new-job-market-rulers-data-scientists/
  • From the blog - http://www.thisisthegreenroom.com/2011/data-science-vs-business-intelligence/
  • Lynn
  • From Business Intelligence to BigData

    1. 1. BI/Big Data Futures – the Cloud? “psst…it’s about Data Mining” Lynn Langit Practioner, Author, Instructor Jan 2012- for SoCalCodeCamp
    2. 2. BI = ‘Current State’ Questions • What did we sell? Collecting • When did we sell it?Transactional data • Where did we sell it? • What did we sell with it?
    3. 3. Current State • I define my OLAP Let’s all • Maybe it’s a read-only copy of my OLTP –OR- OLAP • Maybe it’s a cube • Maybe it uses some data mining too • I’ll keep it on premisesIt’s my data • I’ll secure it, tune queries, back it up, etc. Data Mining • Too difficult, expensive, proprietary…. really?
    4. 4. Do you use Data Mining?
    5. 5. Current State QuestionsWhy did this happen?When did this happen?Where did this happen?Who is responsible?What might happen to this one value in the future?Can you write me a report for…?
    6. 6. BI Data Landscape Storage Processing Query Presentation
    7. 7. Mix-in #1 -- the Cloud and…• Host Data in the Cloud• Process & Query Data in the Cloud – Click to query and (data) mine – Return the data locally – Use Self-service BI visualizers• Mash-up Cloud data – Combine with local data
    8. 8. NoSQL and Cloud-based BI• The Elephant in the room…Hadoop• Over 120+ types of noSQL databases – http://nosql-database.org/
    9. 9. Oracle Loader for Hadoop
    10. 10. SQL Server Connector for Hadoop
    11. 11. Hadoop on Azure
    12. 12. Hadoop on Azure
    13. 13. Comparing RDBMS and MapReduce Reference: Tom White’s Hadoop: The Definitive Guide Traditional RDBMS MapReduceData Size Gigabytes (Terabytes) Petabytes (Hexabytes)Access Interactive and Batch BatchUpdates Read / Write many times Write once, Read many timesStructure Static Schema Dynamic SchemaIntegrity High (ACID) LowScaling Nonlinear LinearDBA Ratio 1:40 1:3000
    14. 14. Microsoft Cloud Data I
    15. 15. Microsoft Cloud Data 2 -DataMarket
    16. 16. Amazon AWS
    17. 17. Amazon AWS
    18. 18. Google App Engine Data
    19. 19. Google – MySQL & Cloud Storage
    20. 20. BTW…NoSQL is 50x CHEAPER
    21. 21. BigData = ‘Next State’ Questions • What could happen?Collecting • Why didn’t this happen? • When will the next new thingbehavioral happen? data • What will the next new thing be?
    22. 22. Splunk
    23. 23. Mining Log Files
    24. 24. Presenting the results
    25. 25. Mix-in #2 - Data Scientists• Who prepares (processes and cleans) the data?• Who asks the ‘right’ questions now?• Who understands the languages?• Who can understand the results?
    26. 26. Is Data Science your next Career?
    27. 27. Becoming a Data Scientist• Conferences – Strata – Data Scientist Summit – CloudCamps• Practice – here
    28. 28. Hadoop --HortonWorks, Cloudera…
    29. 29. Google – Freebase & Refine
    30. 30. Microsoft – Data Explorer
    31. 31. Mix-in #3 - Presentation• New Devices – iPad, Kindle Fire• New User Experiences – touch, Kinect• EVERYTHING on the phone
    32. 32. Some BI Query Languages • MDX, DMX, T-SQL, DAX, XMLA -- data Microsoft • Infer.Net --programmer • R, Hive (SQL-like) –data • HQL, GQL, MQL – specialized dataOpen Source • MapReduce (Java) --programmer
    33. 33. R-Language
    34. 34. Karmasphere Studiofor Amazon Elastic MapReduce
    35. 35. Excel PowerPivot
    36. 36. Power Pivot in action
    37. 37. More PowerPivot
    38. 38. Hadoop Connector to Excel
    39. 39. Self-Service Data MiningPredixion
    40. 40. QlikView
    41. 41. QlikView on iPad
    42. 42. BI vNext for MicrosoftSQL Server 2012 -New BI tools andsemantic model Connectors for• Data Quality Services Hadoop• Master Data Services • SQL Server• Semantic Search • Excel• PowerView • Power Pivot SQL Azure vNext - Full BI IN the Federations, featu cloud (SSAS, SSIS, res, max. size SSRS) increase • Data Explorer – ETL for all
    43. 43. BI >BigData ‘To Do ListStore some (more) data on the cloud• Relational and non-relationalProcess some data in the cloud• Try data mining• Learn about Data ScienceUpdate your client tools• New UI (touch, gestures)• Click to Query• New form factors (phone, tablet)
    44. 44. Is Data Science your next Career?
    45. 45. Deeper Comparison Chart
    46. 46. www.TeachingKidsProgramming.org• Do a Recipe  Teach a Kid (Ages 10 ++)• Microsoft SmallBasic  Free Courseware (recipes)
    47. 47. Keep up with Big Data Follow me @LynnLangit RSS my blog www.LynnLangit.com Hire me • To help build your BI/Big Data solution • To teach your team next gen BI

    ×