Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases

6,979 views

Published on

Financial institutions today are under intense pressure to provide more value add to the customers, reduce IT costs and also grow year to year. This challenge has been further complicated by huge amounts of data being generated as well as mandatory federal compliances in place.
Similarly, Manufacturing industry today also is facing the challenge to process huge amount of data in real time and predict failures as early as possible to reduce cost and increase production efficiency.
The session will cover some high level Big Data use cases applicable to financial and manufacturing domain and how big data technologies are being used successfully to solve these challenges using some examples in credit card/banking industry in financial domain and semi-conductor production in manufacturing domain.

Published in: Technology
  • Be the first to comment

Global Big Data Conference Hyderabad-2Aug2013- Finance/Manufacturing Use Cases

  1. 1. 1 Finance and Manufacturing Big Data Use Cases and Solutions Sanjay Sharma Principal Architect August 2013
  2. 2. Impetus Big Data Services 2 Big Data Platform Implementation Operations and Visualization Business Analytics and Data Science Solution Architecture, POC and Production planning Technology strategy, Use Case development & Validation BUSINESS PROCESS MANAGEMENT Assessment Solution Modeling Solution Analysis People, Proc ess, Technol ogy Impact Analyze & Optimize Objectives & Strategy Model
  3. 3. Big Data : Value Drivers © 2013 Impetus Technologies
  4. 4. Financial World: Key Challenges © 2013 Impetus Technologies Business Challenges • Fraud • Regulatory Compliances • Customer Insights • Risk Handling Technical Challenges • Reduce IT costs • Do more with LESS • Unstructured data
  5. 5. Financial World: Some Use Cases © 2013 Impetus Technologies Fraud Detection and Analysis Risk Management Customer Insights/Management Micro targeting Trade/Payments Analytics Long‐Term Storage & Analytics New Opportunities Reputational Risk Marketing Campaigns
  6. 6. Financial World: Technical Solution Challenges © 2013 Impetus Technologies • Regulatory Compliances • Machine Learning based supervised and unsupervised analytics Large Data Storage and Advanced Analytics • Mash up structured and unstructured data • Enrich TX with NLP/Text analytics Unstructured Data • Customer level Profiling/ Recommendations • Transaction/Trade Ticker level analytics Individual Level Analytical Processing
  7. 7. Big Data Solution: Building Blocks © 2013 Impetus Technologies
  8. 8. Big Data Use Case Solution: Example © 2013 Impetus Technologies • Major Credit Card Company • Traditionally Oracle/ DB2 + SAS + Informatica • Huge data – Existing solutions/ infrastructure inadequate to meet new business requirements and data growth cost effectively – Successfully revamping to Big Data Platform
  9. 9. Big Data Use Case Solution: Example © 2013 Impetus Technologies Step 1: Hadoop POC with Sentiment Analysis Step 2: Hadoop Distribution Selection Step 3: Production-ized Recommendation Engine Step 4: Production-ized Negative Fraud Detection ($100+mlln) Step 5: … Hive Hadoop Platform HBas e Oozie Mahout + NLP Transactions Unstructured Sources (Email, Surveys, CRM notes, Social Feeds etc.) latform Data a a er CEP ata alytics g on Data Visualization, Exploration and Discovery Tools Tools Reporting Tools Analytical Tools Business User Business Analyst Advanced Analytics User Sqoop Flume MapRed Unified Analytics Platform (UAP) Unstructured Data Internal Data External Data Twitter Facebook Voice to Text Call Center Notes CEP WEP Structured Data In-Database Analytics Predictive Data Mining Segmentation Data Visualization, Exploration and Discovery Tools S A N D B O X Domain Specific Tools Reporting Tools Analytical Tools Applications Business User Business Analyst Advanced Analytics User BI/ Visualization Analytics- SAS/R Real Time Delivery RDBMS/API RDBMS/MPP Informatica ETL/ELT L Advantages = PB scale = ETL and Analytical Offloading = Prescriptive Analytics than Predictive
  10. 10. Manufacturing Domain: Challenges © 2013 Impetus Technologies Business Challenges • Early preventive maintenance/repair • Real time/ near real time actionable response • Improve productivity/Margin Recovery • Reduce wastes, improve efficiency • Improve Yield • Supply Chain • Optimize supply chain Technical Challenges • High Ingestion Rates • Sensor/ tool data with sub second ingestion requirements • Millions of read/writes per second • Complex log formats • Semi-structured data • Huge amount of data • TB/PB of data storage for deeper analytics
  11. 11. Manufacturing Domain: Architectural Building Blocks © 2013 Impetus Technologies NoSQL + Search Machine Data
  12. 12. © 2013 Impetus Technologies Machine Data Ingestion Engine (Real time + Batch components) Real Time Processing Engine (CEP/Analytics/ Rule Engine) Real Time Data Storage Engine (Store + Indexing/Searc h) Business Process Engine (Business Process+ Rule management) Kafka/ Storm Storm + Esper Cassandra + Solr JBoss Drools/jBPM Manufacturing Domain: Data Ingestion/ Streaming – Customer Example
  13. 13. © 2013 Impetus Technologies NoSQL + Search Machine Data Manufacturing Domain: Reference Architecture-Cloud– Customer Example
  14. 14. • Software Solutions and Services Company • Leader in Innovation led Technology services • 17 years of customer success, 1500 people across US/ India • Big Data, Enterprise Mobility, Test and Performance Engineering, Carrier Grade Large Systems • Vendor neutral, open source contributor with Big data accelerator products © 2013 Impetus Technologies Impetus Technologies
  15. 15. Q&A ssharma@impetus.com bigdata@impetus.com http://bigdata.impetus.com © 2013 Impetus Technologies

×