Houston Hadoop Meetup Series - Hybrid Big Data Solutions

729 views
623 views

Published on

This is the slide deck presented by Victor Cintron of Wisemen Consulting at a Houston Hadoop Meetup on September 11, http://www.meetup.com/Houston-Hadoop-Meetup-Group/events/133682522/

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
729
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Houston Hadoop Meetup Series - Hybrid Big Data Solutions

  1. 1. HYBRID BIG DATA SOLUTIONS Houston Hadoop Meetup September 11, 2013
  2. 2. Agenda • Introduction • InfoSphere BigInsights • Hadoop • InfoSphere Streams • real-time analytics • Parse.com • mobility • Perl • integration and automation
  3. 3. Introductions •Victor Cintron • Analytics Practice Manager for Wise Men Consultants •Daniel Lopez • Software Engineer at Wise Men Consultants
  4. 4. InfoSphere BigInsights •Management console •BigSheets •Accelerators (machine, social, telco) •Jaql (like a blend of Pig and Hive) •Add your own Hadoop flavor •Basic edition
  5. 5. InfoSphere BigInsights
  6. 6. InfoSphere Streams •Gov (NSA) + IBM as System S •Real-time analytics •Telecoms, Gov and others •Visual development environment •Program with SPL and Java
  7. 7. InfoSphere Streams •Thousands of real-time sources •Millions of events per second •Microsecond latency •Can use R, SPSS and BigInsights/Hadoop •Structured and unstructured data •RHEL or CentOS
  8. 8. Parse.com •Mobile PaaS •MongoDB •Dashboard •Cloud Code •Geo Queries •Multiple Platform API’s
  9. 9. Parse.com http://aws.amazon.com/solutions/case-studies/parse/
  10. 10. Perl •Available for almost every OS •17K+ Modules on CPAN •MediaWiki::DumpFile •Geo::Parse::OSM •Streaming MapReduce •PDL
  11. 11. Perl MapReduce •Map • While(<>){ • Process each line • Print out for reduce • } •Reduce • While(<>){ • Aggregate lines • Print output • }
  12. 12. Perl MapReduce $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input myInputDirs -output myOutputDir -mapper /usr/’perl map.pl’ -reducer /usr/’perl reduce.pl’
  13. 13. Thank You •Questions?

×