Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
BigData Use-
cases
- Prepared by
- Vishal Shukla
- Pranav Shukla
- Krishna Meet
Brevitaz Overview
● Founded in 2014
● Small team of technocrats delivering Big Data Solutions
● Global client-base in Euro...
Agenda
➔ Big Data & Analytics
➔ Full-text search
➔ Log analytics
➔ Big Data Analytics
➔ Real-time Analytics
➔ IoT Analytic...
Analytics
Data is growing!
Full-text
Search
Spot the right data
quickly
“
It’s all about being able to spot
Right Information at Right Time
◎ Relevance search in near real-time
○ Find results matching “iphone”. Please don’t show me
Iphone chargers in first page....
What are you talking! I am here for
BigData!
Elasticsearch - does all of these for massive
volume, variety and velocity
◎ Crawl third-party websites
◎ Aggregate and classify the data
◎ Develop custom application on top of classified
data
Use-...
◎ Google’s “Did you mean?”
◎ Search suggestions as you type
◎ Text analytics
◎ McGrowHill - Transform text-books into digi...
Log
Analytics
Collect, analyze and
Improvise
“
Transform your dumb logs into
actionable insights
● Use machine generated logs to get operational
insights
● Sensors, application servers, web servers or any
IoT device log...
What’s the big deal!
Use-case - Network Logs Analysis
◎ High velocity
◎ High volume
◎ Collect, analyze and improvise
◎ Analyze click stream data to provide
personalized offers and user experience
◎ Interactive drill-down analysis
◎ Complia...
BigData
Analytics
Make your data speak
“
Combine all sources of data to
uncover hidden patterns and
unknown relations in your data
● Take your transactional data from various
sources
● Take operational and user behaviour logs data
● Collect social data
...
Usecase - Supply chain management
◎ RFID labels can indicate which product is where
at what time
◎ Get more accurate busin...
◎ Social media sentiment analysis to get end-user
feedback on launched products
◎ Identify market trends
◎ Predict employe...
Real-time
Analytics
Analyse instantaneously
as you collect data
“
Lag of seconds can make a
fraudster and you
● Ingest streaming data, possibly at high velocity
● Analyse and react immediately
To solve problems like...
◎ Identify ch...
Use-case - Enrich Customer Experience
◎ Get real-time feeds about customer location or
products being browsed
◎ Combine wi...
◎ Hospitality Industry
○ Bad weather reduces travel, which then
reduces overnight lodging
○ Combine weather data with flig...
◎ Fraud detection
◎ Predict and enrich customer experience based on
location, lifestyle
◎ Real-time process visibility acr...
IoT
Analytics
Let machines communicate
● Use sensors to detect low level data
● Report the captured data to server
● Analyse and get back to user
To provide smar...
Use-case
◎ Performance measurement & maintenance
schedule
DIAGRAM
◎ In agriculture, Sensors can detect crop health
along with geo data and based on that alert can
be sent to farmers where ...
Machine
Learning
on
BigData
Make the machines
learn from data
What is machine learning?
◎ Machine learning is not programming a machine to
do stuff
◎ Machine learning is making the mac...
Where is machine learning used?
● Identify similarities between products, users
● Predict values from past data
● Classify...
“
Recommending the right products
makes the difference between
selling or not selling a product
Use-case - Recommending Products
◎ Compare thousands of
users/products with each other
to find similar “clusters”
◎ Conten...
Use-case - Optimise team combination in Sports
◎ Choose best performing team with limited
budget
◎ It was first applied in...
Use-case - Sports
What they achieved?
◎ Average 90 wins in each
season in less than 30M $
◎ Same number of wins in
1/3rd of budget than
anot...
Other use-cases
◎ Fraud detection in banking and other sectors
◎ Fine grained customer segmentation for targeted
products
...
Big Data
Warehousing
Catch all that you can so, you
can analyze it later
Why modernize Data Warehouse with Big Data?
Traditional Enterprise Data Warehouse (EDW) can only
◎ Store only structured d...
Use-case - Modernizing Data Warehouse
◎ Low cost storage for years of data
◎ Data lake for structured, unstructured and se...
◎ Online archival with reporting
○ Make years of data available
◎ ETL off-loading
○ Spark jobs to reduce ETL job time from...
But we are just a startup !
“
Start small. Then scale.
Next steps
ry, evaluate and adopt in risk-
free manner
◎ Identify sources of your unused data
○ like server logs
○ social streams
◎ Collect and store on cloud to minimize initia...
◎ https://aws.amazon.com/public-data-sets/
◎ https://data.gov.in/
◎ https://open-data.europa.eu/en/data/
◎ https://www.dat...
Woo-ha! I am feeling empowered!
Thanks!
Any questions?
Contact Us
@pranavshukla81
http://in.linkedin.
com/in/pranavshukla81
pranav.shukla@brevitaz.com
@vishal1shukla2
https://in...
Big Data Usecases
Big Data Usecases
Upcoming SlideShare
Loading in …5
×

Big Data Usecases

1,646 views

Published on

Big Data Use-cases - Startup Saturday Ahmedabad. Presented by Vishal Shukla & Pranav Shukla from Brevitaz Systems.

Published in: Technology

Big Data Usecases

  1. 1. BigData Use- cases - Prepared by - Vishal Shukla - Pranav Shukla - Krishna Meet
  2. 2. Brevitaz Overview ● Founded in 2014 ● Small team of technocrats delivering Big Data Solutions ● Global client-base in Europe and Asia-pacific region ● Expertise ○ Full-text search ○ Real-time analytics ○ Log analytics ○ BigData analytics ○ IoT based solutions ○ Machine learning ○ BigData warehousing ● Technologies ○ Spark, Hadoop, Kafka, Flume, Storm ○ Elasticsearch, Logstash, Kibana ○ MongoDB, Cassandra, HBase, Apache Titan ○ Impala, Spark SQL, Hawk ○ Java & Spring stack, Typesafe stack (Scala, Akka, Spray, Slick) ○ AngularJS
  3. 3. Agenda ➔ Big Data & Analytics ➔ Full-text search ➔ Log analytics ➔ Big Data Analytics ➔ Real-time Analytics ➔ IoT Analytics ➔ Machine Learning on BigData ➔ Big Data Warehousing ➔ Big Data is for Everyone
  4. 4. Analytics
  5. 5. Data is growing!
  6. 6. Full-text Search Spot the right data quickly
  7. 7. “ It’s all about being able to spot Right Information at Right Time
  8. 8. ◎ Relevance search in near real-time ○ Find results matching “iphone”. Please don’t show me Iphone chargers in first page. ◎ Fuzzy search and search suggestions ○ Find results matching “iphne" ◎ Faceted search ○ Filters in amazon after searching a keyword ◎ Complex search with multiple criteria ○ Find me products matching “iphone” with in price range 30000 INR to 50000 INR and color “Space grey” ◎ Geo-spatial search ○ Find restaurants within 10 km radius from my current location. And yes, I want to see closer ones on top. Full-text search - What it is?
  9. 9. What are you talking! I am here for BigData!
  10. 10. Elasticsearch - does all of these for massive volume, variety and velocity
  11. 11. ◎ Crawl third-party websites ◎ Aggregate and classify the data ◎ Develop custom application on top of classified data Use-case - Information Aggregator
  12. 12. ◎ Google’s “Did you mean?” ◎ Search suggestions as you type ◎ Text analytics ◎ McGrowHill - Transform text-books into digital learning resource ◎ SoundCloud - Quickly find music that interests them Other use-cases
  13. 13. Log Analytics Collect, analyze and Improvise
  14. 14. “ Transform your dumb logs into actionable insights
  15. 15. ● Use machine generated logs to get operational insights ● Sensors, application servers, web servers or any IoT device logs To interactively answer questions like... ◎ How many users signed up this week? ◎ How users are using your website / mobile app ◎ How successful is our advertising campaign? ◎ Why is the database slow? ◎ Which are the websites categories my team is spending the most time at? ◎ Who are the potential employees to resign next? Log Analytics - What it is?
  16. 16. What’s the big deal!
  17. 17. Use-case - Network Logs Analysis ◎ High velocity ◎ High volume ◎ Collect, analyze and improvise
  18. 18. ◎ Analyze click stream data to provide personalized offers and user experience ◎ Interactive drill-down analysis ◎ Compliance reporting through interactive dashboard ◎ Real-time alerts on invalid login attempts ◎ Detect outages ◎ Multi-channel funnel reporting for your Advertising campaigns to find out which channels contribute the most for conversions Other use-cases
  19. 19. BigData Analytics Make your data speak
  20. 20. “ Combine all sources of data to uncover hidden patterns and unknown relations in your data
  21. 21. ● Take your transactional data from various sources ● Take operational and user behaviour logs data ● Collect social data ● Combine data collected from various sources to To interactively answer questions like... ◎ What is increase or decrease in sales over the years? ◎ How many unique customers are acquired this year? ◎ Which products are trending disproportionately this year? Big Data Analytics - What it is?
  22. 22. Usecase - Supply chain management ◎ RFID labels can indicate which product is where at what time ◎ Get more accurate business insights ◎ Theft detection
  23. 23. ◎ Social media sentiment analysis to get end-user feedback on launched products ◎ Identify market trends ◎ Predict employees attrition ◎ Customer churn analysis ◎ Influencer analysis ◎ Lead generation ◎ Proactive issues monitoring ◎ For insurance companies, identify potential customers by combining birth, marriage and health data Other use-cases
  24. 24. Real-time Analytics Analyse instantaneously as you collect data
  25. 25. “ Lag of seconds can make a fraudster and you
  26. 26. ● Ingest streaming data, possibly at high velocity ● Analyse and react immediately To solve problems like... ◎ Identify changing trends in real-time ◎ Detect fraud ◎ Analyse policy violations and react immediately ◎ Reduce downtimes ◎ Provide better and quicker business decisions Real-time Analytics - What it is?
  27. 27. Use-case - Enrich Customer Experience ◎ Get real-time feeds about customer location or products being browsed ◎ Combine with historical user behaviours ◎ Roll out offers in real-time
  28. 28. ◎ Hospitality Industry ○ Bad weather reduces travel, which then reduces overnight lodging ○ Combine weather data with flight cancellation to identify stranded travellers ○ Offer hotel coupons based on near by location. Other use-cases
  29. 29. ◎ Fraud detection ◎ Predict and enrich customer experience based on location, lifestyle ◎ Real-time process visibility across an enterprise ◎ Suggest optimal routes based on current traffic data ◎ Get player performance metrics in real-time to substitute players at right time Other use-cases
  30. 30. IoT Analytics Let machines communicate
  31. 31. ● Use sensors to detect low level data ● Report the captured data to server ● Analyse and get back to user To provide smart alerts and suggestions like ◎ Schedule maintenance of machines ◎ Your pulse rate is disproportionately increasing ◎ Medicines manufactured in a batch is not complying to standards IoT Based Smart Solutions - What it is?
  32. 32. Use-case ◎ Performance measurement & maintenance schedule DIAGRAM
  33. 33. ◎ In agriculture, Sensors can detect crop health along with geo data and based on that alert can be sent to farmers where they need to focus ◎ In retail, smart-shelves can detect and send alerts on when to replenish ◎ Smart home can analyze the patterns of each family member and optimize energy usage Other use-cases
  34. 34. Machine Learning on BigData Make the machines learn from data
  35. 35. What is machine learning? ◎ Machine learning is not programming a machine to do stuff ◎ Machine learning is making the machine learn and adapt based on the observed data
  36. 36. Where is machine learning used? ● Identify similarities between products, users ● Predict values from past data ● Classify items into categories, like an email is spam or not spam in order to ... ◎ Predict expected outcome ◎ Categorize large amounts of data ◎ Optimize algorithms or paths ◎ Find similarities ◎ Improve quality of predictions continuously
  37. 37. “ Recommending the right products makes the difference between selling or not selling a product
  38. 38. Use-case - Recommending Products ◎ Compare thousands of users/products with each other to find similar “clusters” ◎ Content-based filtering - Recommend similar products to what customer has already bought ◎ Find similar customers to the current customer and recommend him what they have bought ◎ Apply what is known as Clustering algorithms in machine learning on Big Data
  39. 39. Use-case - Optimise team combination in Sports ◎ Choose best performing team with limited budget ◎ It was first applied in Baseball, now many professional games use these techniques ◎ Choose a team consisting of players who could win at least enough games to make to the play- offs ◎ Use data analysis techniques to find undervalued players
  40. 40. Use-case - Sports
  41. 41. What they achieved? ◎ Average 90 wins in each season in less than 30M $ ◎ Same number of wins in 1/3rd of budget than another team ◎ 20 more wins than another team with similar budget
  42. 42. Other use-cases ◎ Fraud detection in banking and other sectors ◎ Fine grained customer segmentation for targeted products ◎ Predicting next product failure and sending a replacement part in advance ◎ Predict best candidates
  43. 43. Big Data Warehousing Catch all that you can so, you can analyze it later
  44. 44. Why modernize Data Warehouse with Big Data? Traditional Enterprise Data Warehouse (EDW) can only ◎ Store only structured data ◎ Extremely expensive license cost per TB of storage ◎ Capacity constrained with ETL and query workloads big data will help to... ◎ Store unstructured, semi-structured data ◎ Combine your structured data with other sources ◎ Run interactive SQL queries on big data ◎ Offload ETL workload from your EDW ◎ Offload less frequently used data from your EDW ◎ Save licensing costs
  45. 45. Use-case - Modernizing Data Warehouse ◎ Low cost storage for years of data ◎ Data lake for structured, unstructured and semi- structured data ◎ Interactive queries on historic data
  46. 46. ◎ Online archival with reporting ○ Make years of data available ◎ ETL off-loading ○ Spark jobs to reduce ETL job time from hours to minutes ◎ Batch reports off-loading ○ Reduce load on your warehouse by off- loading batch reports ◎ Big Data Discovery ○ Proactively find patterns guided by the system Other use-cases
  47. 47. But we are just a startup !
  48. 48. “ Start small. Then scale.
  49. 49. Next steps ry, evaluate and adopt in risk- free manner
  50. 50. ◎ Identify sources of your unused data ○ like server logs ○ social streams ◎ Collect and store on cloud to minimize initial investment ◎ Many cloud options like Amazon EC2, Databricks, Altiscale... ◎ Use open-source analytics engines like Elasticsearch, Kibana. They are free to use. ◎ Experience the success ◎ Automate using sensors or IoT devices to add more sources of useful data Start small and then scale
  51. 51. ◎ https://aws.amazon.com/public-data-sets/ ◎ https://data.gov.in/ ◎ https://open-data.europa.eu/en/data/ ◎ https://www.data.gov/ ◎ https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public Some open datasets to play with
  52. 52. Woo-ha! I am feeling empowered!
  53. 53. Thanks! Any questions?
  54. 54. Contact Us @pranavshukla81 http://in.linkedin. com/in/pranavshukla81 pranav.shukla@brevitaz.com @vishal1shukla2 https://in.linkedin.com/in/vishalshu vishal.shukla@brevitaz.com

×