Big Data and Data Science for traditional Swiss companies


Published on

This talk was held at the 10th meeting on February 3rd 2014 by Daniel Fasel.

Many traditional Swiss companies, such as banks, insurance companies and government agencies, are highly interested in Big Data and Data Science but don’t know exactly what the business value of Big Data is for them. Often Big Data is misinterpreted as large amounts of data and companies are unaware of the innovation behind the new technologies of Big Data and how these technologies can be profitable to them. In this presentation, I discuss sample cases that demonstrate a set of these new technologies and how they can be applied not only for large web scale data but also for data sets of traditional companies. First, I demonstrate how multi-structured data can be indexed and searched using Autonomy. I show how fast new analytical application can be built based on a real-time streaming example using STORM, Redis and Node.js. And the last demonstration shows how machine learning algorithms and visualization can be applied for improving analytics using AsterData.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Big Data and Data Science for traditional Swiss companies

  1. 1. Scigility! Scire propter agere! Big Data and Data Science for traditional Swiss companies! Dr. Daniel Fasel! Scigility AG !!
  2. 2. Who are we?! –  Dr. Daniel Fasel has been the first data scientist on the business intelligence team at Swisscom and was key in implementing NoSQL technologies for explorative analytics during his time at Swisscom! ! –  Prof. Dr. Philippe Cudré-Mauroux is the director of the eXascale Infolab and Professor at the University of Fribourg in Switzerland. Previously, he was a postdoctoral associate working in the Database Systems group at MIT. He also worked on distributed information and media management for HP, IBM Watson Research (NY), and Microsoft Research Asia.!   2!
  3. 3. Our services! •  We provide consulting, software development, operations and training in the areas of large-scale information systems, NoSQL technologies and real time streaming solutions. Technologies commonly known as Big Data.! •  We provide the optimal solution to your specific IT problems, based on the latest and most appropriate technologies available!!   3!
  4. 4. Content! •  Short overview of Big Data! •  Techniques & Technologies! •  3 Demonstrations how to use these new Techniques and Technologies!   4!
  5. 5. Big Data! •  Big – Volume! –  Big is relative! •  Google > 400PB ! •  Traditional Swiss companies < 400 PB ;-)! –  But Big Data (Volume) is a fact! •  Data constantly grow! •  More and more systems produce data! •  Classical relational schemas already do a pre-selection of data! •  There is more interesting data in your company than you potentially assume (dark data on Intranet or File Shares, application silos, etc.)!   5!
  6. 6. Big Data! •  Variety! –  Multi-structured data!   6!
  7. 7. Big Data! •  Variety! –  Structures change over time! •  Think about your legacy system! –  The structure of data is determined at the time on analysis and not at the time of storing! •  Schema on purpose / schema on read! –  Combination of classical and new data sources! 7!
  8. 8. Big Data! •  Velocity! –  Data is produced faster! –  Data becomes more and more ephemeral! –  Analytics gets real time! –  Data flows are as interesting as data itself!   8!
  9. 9. Big Data! •  What is the innovation of Big Data?! •  The real new innovations of Big Data are! –  optimized techniques and technologies! –  that address the specific characteristics which are commonly summarized as Big Data! •  Big Data is not a new kind of data!! –  You all have Big Data! !   9!
  10. 10. Techniques! •  New techniques! –  MapReduce for analysis of highly distributed data! –  Combination of linear algebra, statistics, computer science, visualization for broader users groups! –  Improved agility and rapid prototyping! –  Explorative analytics!   10!
  11. 11. Technologies! •  New technologies! –  Massive horizontal scalable / elastic! –  Optimized for specific types of problems! –  Not necessarily ACID compliant! –  Follow BASE & CAP concepts! –  Integration and combination of classical and new technologies (like SQL-MR)!   11!
  12. 12. Demonstration 1! •  Indexing and search on multi-structured data with Autonomy!   12!
  13. 13. Demonstration 2! •  Real Time Streaming with Storm! STORM Spout Twitter Redis Node.js / Bolt   13!
  14. 14. Demonstration 3! •  Collaborative filtering, path analysis and visualization with AsterData!   14!
  15. 15. Thank you!! •  Questions?! •  You can contact us:! –  Tel.: +41 79 202 47 89! –! –! !   15!