Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Houston Hadoop Meetup Series - Hybrid Big Data Solutions

874 views

Published on

This is the slide deck presented by Victor Cintron of Wisemen Consulting at a Houston Hadoop Meetup on September 11, http://www.meetup.com/Houston-Hadoop-Meetup-Group/events/133682522/

Published in: Technology
  • Be the first to comment

Houston Hadoop Meetup Series - Hybrid Big Data Solutions

  1. 1. HYBRID BIG DATA SOLUTIONS Houston Hadoop Meetup September 11, 2013
  2. 2. Agenda • Introduction • InfoSphere BigInsights • Hadoop • InfoSphere Streams • real-time analytics • Parse.com • mobility • Perl • integration and automation
  3. 3. Introductions •Victor Cintron • Analytics Practice Manager for Wise Men Consultants •Daniel Lopez • Software Engineer at Wise Men Consultants
  4. 4. InfoSphere BigInsights •Management console •BigSheets •Accelerators (machine, social, telco) •Jaql (like a blend of Pig and Hive) •Add your own Hadoop flavor •Basic edition
  5. 5. InfoSphere BigInsights
  6. 6. InfoSphere Streams •Gov (NSA) + IBM as System S •Real-time analytics •Telecoms, Gov and others •Visual development environment •Program with SPL and Java
  7. 7. InfoSphere Streams •Thousands of real-time sources •Millions of events per second •Microsecond latency •Can use R, SPSS and BigInsights/Hadoop •Structured and unstructured data •RHEL or CentOS
  8. 8. Parse.com •Mobile PaaS •MongoDB •Dashboard •Cloud Code •Geo Queries •Multiple Platform API’s
  9. 9. Parse.com http://aws.amazon.com/solutions/case-studies/parse/
  10. 10. Perl •Available for almost every OS •17K+ Modules on CPAN •MediaWiki::DumpFile •Geo::Parse::OSM •Streaming MapReduce •PDL
  11. 11. Perl MapReduce •Map • While(<>){ • Process each line • Print out for reduce • } •Reduce • While(<>){ • Aggregate lines • Print output • }
  12. 12. Perl MapReduce $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input myInputDirs -output myOutputDir -mapper /usr/’perl map.pl’ -reducer /usr/’perl reduce.pl’
  13. 13. Thank You •Questions?

×