Big Data Perspective (Now part of NinjaMSP) is unified Operational Analytics Appliance for Big Data. Perspective360, unified operational analytics appliance for big data supports i.e. Hadoop, HDFS, Cassandra, Kafka, Hive, Pig, HBase, MongoDB, H2O, Docker, MapReduce, Spark, YARN & Flink. It is designed to have non-invasive micro service architecture to collect application & system data from Hadoop. It has anomaly detection for systems (CPU, memory, disk) data with auto encoder deep learning using H2O library and UI is equipped with floating widget & dynamic dashboard engine with interactive graphs using D3/DC & slicing-dicing data management.
2. The TeamNinja is the SolutionOur TeamWho we are?
Avkash Chauhan, Founder- Deep expertise in distributed data platforms including Hadoop, with various publications
and speaking engagements. Worked at Microsoft, Platfora & experience includes:
§17+ years in tech industry, worked with 15+ fortune #100 companies worldwide
§Large scale distributed application development and performance expert
§Core Developer and product implementation specialist for Microsoft Azure, HDInsight & Platfora.
§Microsoft distinguished engineer, key architect on Hadoop implementation for Azure.
Henry Ohara, Architect (Backend engineer)- Has a career of designing and developing scalable backend
systems for over 21+ years.
§Founding member of Yahoo’s mobile search team servicing over 30 million users.
§Developed a recommendation system for Yahoo's media site.
§Developed top #1 application to manage secure enterprise communication in Japan for mobile
Hamid Behnam, UX Engineer –
• Over 7+ years of front end engineering and development experience with a strong interest in JavaScript
• Expert in creating complex & multilayer web application using JavaScript frameworks
• Amazing work ethic
Sal Sferlazza, principle investor & chairman of board – LaunchCapital.co
• Overall 21+ years in tech industry from developer to CTO to CEO
• Worked at Anderson Consulting, Quest Software, SonicWALL, Dell
• Total 5 exits in last 10 years, 3 of the exits are in MSP space
Board
Engineering
3. Ninja is the SolutionWhat we have built?
Operational Analytics
for Big Data
Heterogeneous Monitoring
Smart Thresholds
Unified Alerting
Forecasting
Recommendations
BIG DATA APM
4. Ninja is the SolutionWhy?
There is no APM for
big data stack
Enterprise
BI Vendors
Valley Startups
IT Organizations
5. 3rd Party Applications
Ninja is the SolutionTrue IntegrationsMarket Placement
Data Perpetration
Data Quality
Data Lake
Hadoop
ETL
Batch Processing
Business
Intelligence
Machine Learning
Big Data
Analytics
Platform Monitoring & Analytics
Big Data Platforms
Environments where Big Data platform is deployed
Big Data
6. Ninja is the SolutionTrue IntegrationsSupported Components
Platform Monitoring & Analytics
Ec2 Container
Service
7. • Multi Vendor Monitoring
• Smart Thresholds
• Unified Alerting
• Forecasting
• Recommendations
Ninja is the Solution
Proprietary And Confidential
Platform Features
8. The TeamNinja is the SolutionSummaryArchitecture Complete
Hadoop Cluster
System
Collection
Server
Hadoop V1
Collection
Server
Hadoop V2
Collection
Server
Web Server Time Series DB
Scheduler
Server
Analysis
Server
RESTful Interface
AJAX
• Major Hadoop Distributions
• Amazon & HDInsight
• Cluster Deployment platform
• On Premise
• Any Hadoop Farm
HTML5, CSS,
JavaScript,
Knockout
And D3
Postgres DB
Server interaction over REST (admin/communication)
KAFKA
Streams
Messaging
Server
Non invasive appliance design & Self serving deployment model
Jetty + JAX-RS
9. Ninja is the SolutionIntegrations Complete
Hadoop
Distributions
Cloud Based
Hadoop
Distributed File
Systems
Distributed data
Processing
Distributed
Services
Cluster & container
Management
Machine
Learning
*
*Prototyping Phase
*
*
10. Ninja is the SolutionOur TeamFeatures Completed
• Data Collection Micro-services connected over REST interface
• Micro servers for data collection
• Micro servers for tunneling over SSH
• Distributed server architecture to support distributed deployment
• Hadoop Distributions
• Apache Hadoop
• Hortonworks
• Pivotal
• Dynamic Views
• Cross-cluster and individual cluster views through deep-dive navigation
• Dynamic widgets creation and deployment for any collected data
• Dynamic page generation for various Hadoop distributions and versions
• Dynamic graph details for both system and Hadoop data combined
• Dashboards
• Dynamic dashboards generation from any data point
• Dashboard sharing
• Dashboard scheduled delivery as PDF/Image over email
• Reporting
• Report generation from any page or dashboard in PDF or image format
• Custom Complex reporting for multi point data across multi cluster environment
• Scheduled delivery for any report over email in PDF/Image format
• Process data analysis per process
• Example process i.e. Kafka, Zookeeper etc
11. Ninja is the SolutionOur TeamFeatures Completed (Cont..)
• Query engine to access any data from any table at any duration, example query includes:
• Display System CPU load average last 2 hours
• Display System Memory Heap Memory top last 1 day
• Display System Network Interface rx Bytes eth0 today
• Display System Disk Space Usage Average this week
• Display System Disk IO Time Spent IO xvda1 last 2 hours
• MapReduce
• MapReduce jobs analysis individual or batch
• MapReduce jobs batch mode SLA
• Data collection for MapReduce 2 and Spark over YARN*
• Cluster Utilization
• In-depth cluster utilization metrics
• Dynamic scheduled delivery for cluster utilization reports and alert
• HDFS
• HDFS data analysis, visualization & reporting at folder level
• Alerting
• Scheduled alerts and notification for any data point collected
• Deployment
• Self serving deployment & Remote Patching
• System Data Analysis per machine
• CPU
• Memory
• Network data at interface level
• Disk I/O data at at partition level
• Disk Space at mounted disk level * Only data collection
12. Ninja is the SolutionIntegrations Roadmap
Hadoop
Distributions
Cloud Based
Hadoop
Distributed File
Systems
Distributed data
Processing
Distributed
Services
Cluster & container
Management
Machine
Learning
*
*Prototyping Phase
*
*
13. Ninja is the SolutionOur TeamFeatures Roadmap
• Use ODPi (http://www.odpi.org) to connect with supported big data platform
• Self service model to slice & dice any data collection
• MapReduce & Spark Support for YARN
• EMR/AMI Support
• Other Hadoop vendors Support (Cloudera & MapR)
• Cartridge design for Cassandra data collection
• Recommendation
• Docker based application deployment
• Cost Analysis Matrix
• Forecasting
14. The TeamNinja is the SolutionSummaryDemo
Cluster Monitoring HDFS Monitoring
MapReduce Job
Monitoring & Analysis
Data Analysis, Graphs
(D3/C3)
Alerts & Notifications
On-demand &
Scheduled Reporting
Dashboards, HOD,
UI, Control Panel,
Time Span
Data Collection
System
(Total 500+ data
points)
Server design and
communication
Cluster Utilization Self Management Troubleshooting
15. Ninja is the SolutionSummaryContact
Founder & Principal
Big Data Perspective LLC
657 Mission St. Suite 602, San Francisco, CA 94105
E: avkash@bigdataperspective.com M: 650-713-9055
Avkash Chauhan