The document is an introduction to big data infrastructure presented at a meetup in Vancouver, highlighting key components like volume, velocity, and variety. It discusses various technologies such as Amazon Kinesis, Apache Kafka, and Elasticsearch, while contrasting Silota's analytics services with Mixpanel. Silota offers an API for data ingestion, analysis, and export, emphasizing its low-level power compared to more user-friendly analytics solutions.

1. Introduction to
Big Data Infrastructure
Vancouver
SMAC (Social, Mobile, Analytics & Cloud) Meetup
Oct 22, 2014
Ganesh Swami
www.silota.com

2. Hi
• Programming professionally for 10+ years
• x86 assembly, STL, boost, python-boost, python
• Built emacs-­‐wiki-­‐blog: first blogging engine for
Emacs!

3. What is Big Data?

4. What is Big Data?
3V: Volume, Velocity, Variety

5. The Big Data Zoo
Amazon Kinesis Riak Cassandra Hive
Apache Spark Apache Hadoop Pig Apache Storm
Kibana Tableu Apache Kafka
Elasticsearch Amazon EMR Redshift
Dynamo DB Riak HBase

6. The Zoo Organized
Ingest Store
Process/
Enrich
Visualize
Kafka S3 Hive/Pig/EMR Tableu
Data Answers
Kinesis DynamoDB Spark Kibana
Flume HDFS Storm
Scribe Redshift

8. Data Ingestion
Ingest
Layer
Mobile Apps
Websites
Internet of Things

9. Elasticsearch
Open-source search and analytics solution

10. Kibana

11. Amazon Redshift
Petabyte-scale data warehouse solution

12. What is Silota
• Building blocks of analytics
• A simple REST API
• to ingest
• to analyze
• to export
• based on Kafka, Storm and Elasticsearch

13. Silota -vs- Mixpanel
• Mixpanel for product people
• great UI
• cookie-cutter analysis for verticals (gaming, e-commerce)
• Silota is an API
• more low-level, full-power
• first class API: responses, pagination, errors, etc.

14. Keep in Touch!
Ganesh Swami
ganesh@silota.com
www.silota.com