0
Aureus Claims Solution

Implementation
Challenges in Big Data
Footer Option 2
Analytics
• Dr. Nilesh N. Karnik

Copyright ...
What we will discuss

The Challenge of BIG Data

ADVANCED Analytics
SOLUTIONS in the Pipeline

Copyright 2013 RESTRICTED C...
Big Data : Distributed Processing

Aureus Claims Solution

!
Footer Option 2

OLD IDEA
Copyright 2013 RESTRICTED CIRCULATI...
EXAMPLE 1: Task of storing books on a shelf

Aureus Claims Solution

Footer Option 2
Image source Flickr. Image copyright ...
EXAMPLE 1: Task of storing books on a shelf

Aureus Claims Solution

And now?

Footer Option 2

Image source Flickr. Image...
Aureus Claims Solution

Footer Option 2

Image source Flickr. Image copyright belongs with original artist.
Copyright 2013...
EXAMPLE 2 : Summarizing a Report

Aureus Claims Solution

SUMMER PROJECT
REPORT

Footer Option 2

Simple, right?
Copyright...
EXAMPLE 2 : Summarizing a Report

Aureus Claims Solution

Footer Option 2

And now?
Copyright 2013 RESTRICTED CIRCULATION
...
EXAMPLE 3 : Baking a Cake

Simple, right?

Aureus Claims Solution

And now?

Footer Option 2

Image source PINTEREST. Imag...
Advanced Analytics
•

Well developed tool set for “small data” environment

•

Aureus Claims Solution

Challenges in Big D...
Advanced Analytics: MapReduce Difficulties

Aureus Claims Solution

Footer Option 2
ITERATIVE
Image source Flickr. Image c...
Advanced Analytics: MapReduce Difficulties

Aureus Claims Solution

Footer Option 2
INCREMENTAL PROCESSING REQUIRES RESTAR...
Advanced Analytics: MapReduce Difficulties

Aureus Claims Solution

Footer Option 2

BATCH LEARNING SCANS ALL DATA IN ONE ...
Some Solutions Data Scientists are working on

Aureus Claims Solution

New frameworks
• E.g., HaLoop*, PrIter# (Extensions...
Some Solutions Data Scientists are working on

Aureus Claims Solution

Smarter algorithms / Different
implementations
• Ra...
@nilesh_karnik
Nilesh@aureusanalytics.com
Copyright 2013 RESTRICTED CIRCULATION

17
Aureus Claims Solution

ThankOption 2
Footer You!
SINGAPORE

INDIA

Aureus Analytics Pte. Ltd.
17, Phillip Street,
#05-01,...
Upcoming SlideShare
Loading in...5
×

Implementation challenges in Big Data - Dr. Nilesh Karnik

400

Published on

In todays competitive environment companies are faced with different types of challenges. Implementation of Big Data is one of them. Dr. Nilesh Karnik takes us through some of them.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
400
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Implementation challenges in Big Data - Dr. Nilesh Karnik"

  1. 1. Aureus Claims Solution Implementation Challenges in Big Data Footer Option 2 Analytics • Dr. Nilesh N. Karnik Copyright 2013 RESTRICTED CIRCULATION
  2. 2. What we will discuss The Challenge of BIG Data ADVANCED Analytics SOLUTIONS in the Pipeline Copyright 2013 RESTRICTED CIRCULATION 2
  3. 3. Big Data : Distributed Processing Aureus Claims Solution ! Footer Option 2 OLD IDEA Copyright 2013 RESTRICTED CIRCULATION NEW IDEA 3
  4. 4. EXAMPLE 1: Task of storing books on a shelf Aureus Claims Solution Footer Option 2 Image source Flickr. Image copyright belongs with original artist. Simple, right? Copyright 2013 RESTRICTED CIRCULATION 5
  5. 5. EXAMPLE 1: Task of storing books on a shelf Aureus Claims Solution And now? Footer Option 2 Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 6
  6. 6. Aureus Claims Solution Footer Option 2 Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 7
  7. 7. EXAMPLE 2 : Summarizing a Report Aureus Claims Solution SUMMER PROJECT REPORT Footer Option 2 Simple, right? Copyright 2013 RESTRICTED CIRCULATION 8
  8. 8. EXAMPLE 2 : Summarizing a Report Aureus Claims Solution Footer Option 2 And now? Copyright 2013 RESTRICTED CIRCULATION 9
  9. 9. EXAMPLE 3 : Baking a Cake Simple, right? Aureus Claims Solution And now? Footer Option 2 Image source PINTEREST. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 10
  10. 10. Advanced Analytics • Well developed tool set for “small data” environment • Aureus Claims Solution Challenges in Big Data environment Footer Option 2 Copyright 2013 RESTRICTED CIRCULATION 11
  11. 11. Advanced Analytics: MapReduce Difficulties Aureus Claims Solution Footer Option 2 ITERATIVE Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 12
  12. 12. Advanced Analytics: MapReduce Difficulties Aureus Claims Solution Footer Option 2 INCREMENTAL PROCESSING REQUIRES RESTART Image source Flickr. Image copyright belongs with original artist. Copyright 2013 RESTRICTED CIRCULATION 13
  13. 13. Advanced Analytics: MapReduce Difficulties Aureus Claims Solution Footer Option 2 BATCH LEARNING SCANS ALL DATA IN ONE GO Copyright 2013 RESTRICTED CIRCULATION 14
  14. 14. Some Solutions Data Scientists are working on Aureus Claims Solution New frameworks • E.g., HaLoop*, PrIter# (Extensions of Hadoop) • Percolator$ (Proprietary Google framework) Footer Option 2 * Y. Bu, B. Howe, M. Balazinska, and M. Ernst, “HaLoop: Efficient iterative data processing on large clusters”, VLDB, 2010. # Y. Zhang, Q. Gao, L. Gao and C. Wang, “PrIter: A distributed framework for prioritized iterative computations”, SoCC, 2011. $ D. Peng and F. Dabek, “Large-scale incremental processing using distributed transactions and notifications”, OSDI, 2010 Copyright 2013 RESTRICTED CIRCULATION 15
  15. 15. Some Solutions Data Scientists are working on Aureus Claims Solution Smarter algorithms / Different implementations • Random forest • Parallelized Stochastic Gradient Descent Footer Option 2 Copyright 2013 RESTRICTED CIRCULATION 16
  16. 16. @nilesh_karnik Nilesh@aureusanalytics.com Copyright 2013 RESTRICTED CIRCULATION 17
  17. 17. Aureus Claims Solution ThankOption 2 Footer You! SINGAPORE INDIA Aureus Analytics Pte. Ltd. 17, Phillip Street, #05-01, Grand Building Singapore (048695) Aureus Analytics Pvt. Ltd. 706, Powai Plaza Hiranandani Gardens, Powai Mumbai – 400076 info@aureusanalytics.com Copyright 2013 RESTRICTED CIRCULATION www.aureusanalytics.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×