Successfully reported this slideshow.

Big Data Agile Analytics by Ken Collier - Director Agile Analytics, Thoughtworks

26

Share

•
Apr. 01, 2014
• 26 likes • 35,813 views
Upcoming SlideShare
Agile Analytics: Delivering on Promises by Atif Abdul Rahman
Agile Analytics: Delivering on Promises by Atif Abdul Rahman
Loading in …3
×
1 of 35
1 of 35

Big Data Agile Analytics by Ken Collier - Director Agile Analytics, Thoughtworks

Apr. 01, 2014
• 26 likes • 35,813 views

26

Share

Download to read offline

Technology

We are in the midst of an exciting time. There is an explosion of very interesting data, and emergence of powerful new technologies for harnessing data, and devices that enable humans to receive tremendous benefits from it. What is required are innovative processes that enable the creation and delivery of value from all of that data. More often than not, it is the predictive (what will happen?) and prescriptive (how to make it happen!) analytics that produces this value, not the raw data itself.

Agile software teams are continuously involved in projects that involve rich, complex, and messy data. Often this data represents innovative analytics opportunities. Being analytics-aware gives these teams the opportunity to collaborate with stakeholders to innovate by creating additional value from the data. This session is aimed at making Agile software teams more analytics-aware so that they will recognize these innovation opportunities.

The trouble with conventional analytics (like conventional software development) is that it involves long, phased, sequential steps that take too long and fail to deliver actionable results. This talk will examine the convergence of the following elements of an exciting emerging field called Agile Analytics:
•sophisticated analytics techniques, plus
•lean learning principles, plus
•agile delivery methods, plus
•so-called "big data" technologies
Learn:
•The analytical modeling process and techniques
•How analytical models are deployed using modern technologies
•The complexities of data discovery, harvesting, and preparation
•How to apply agile techniques to shorten the analytics development cycle
•How to apply lean learning principles to develop actionable and valuable analytics
•How to apply continuous delivery techniques to operationalize analytical models

We are in the midst of an exciting time. There is an explosion of very interesting data, and emergence of powerful new technologies for harnessing data, and devices that enable humans to receive tremendous benefits from it. What is required are innovative processes that enable the creation and delivery of value from all of that data. More often than not, it is the predictive (what will happen?) and prescriptive (how to make it happen!) analytics that produces this value, not the raw data itself.

Agile software teams are continuously involved in projects that involve rich, complex, and messy data. Often this data represents innovative analytics opportunities. Being analytics-aware gives these teams the opportunity to collaborate with stakeholders to innovate by creating additional value from the data. This session is aimed at making Agile software teams more analytics-aware so that they will recognize these innovation opportunities.

The trouble with conventional analytics (like conventional software development) is that it involves long, phased, sequential steps that take too long and fail to deliver actionable results. This talk will examine the convergence of the following elements of an exciting emerging field called Agile Analytics:
•sophisticated analytics techniques, plus
•lean learning principles, plus
•agile delivery methods, plus
•so-called "big data" technologies
Learn:
•The analytical modeling process and techniques
•How analytical models are deployed using modern technologies
•The complexities of data discovery, harvesting, and preparation
•How to apply agile techniques to shorten the analytics development cycle
•How to apply lean learning principles to develop actionable and valuable analytics
•How to apply continuous delivery techniques to operationalize analytical models

Technology

Recommended

More Related Content

Viewers also liked

H2O World - Advanced Analytics at Macys.com - Daqing Zhao
Sri Ambati
H2O World - What you need before doing predictive analysis - Keen.io
Sri Ambati
Agile Analytics
Atif Shaikh
H2O World - Intro to Data Science with Erin Ledell
Sri Ambati
Domino and AWS: collaborative analytics and model governance at financial ser...
Domino Data Lab
Pdf analytics-and-witch-doctoring -why-executives-succumb-to-the-black-box-me...
OrateTeam
H2O World - Collaborative, Reproducible Research with H2O - Nick Elprin
Sri Ambati
1440 track 2 boire_using our laptop
Rising Media, Inc.
Data Science Salon: Quit Wasting Time – Case Studies in Production Machine Le...
Formulatedby
20151016 Data Science For Project Managers
Tze-Yiu Yong
H2O World - Migrating from Proprietary Analytics Software - Fonda Ingram
Sri Ambati
1645 track 1 bress_using his laptop
Rising Media, Inc.
940 diamond sponsor sengupta
Rising Media, Inc.
CRISP-DM - Agile Approach To Data Mining Projects
Michał Łopuszyński
How to Build Data Science Teams
Ganes Kesari
Predictive Analytics - Big Data Warehousing Meetup
Caserta
Data Quality Analytics: Understanding what is in your data, before using it
Domino Data Lab
1000 track 1 groves_using our laptop
Rising Media, Inc.
Introduction to Data Science
LivePerson

Similar to Big Data Agile Analytics by Ken Collier - Director Agile Analytics, Thoughtworks

7 Dimensions of Agile Analytics by Ken Collier
Thoughtworks
An Agile Approach to Machine Learning
Randy Shoup
Predictive Intelligence
CharlesSayers.com
Game Changing Quality Strategies that Drive Organizational Excellence
kushshah
Insurance value chain
Shivanna Madalabhavi
From Business Intelligence to Predictive Analytics
Decision Management Solutions
Data Driven Engineering 2014
Roger Barga
The digital transformation of CPG and manufacturing
Cloudera, Inc.
Define, describe, deploy how to build an analytical framework
Peter Spangler
Bridging the AI Gap: Building Stakeholder Support
Peter Skomoroch
Crm and analytics modules
Aditya Madiraju
Four Trends Impacting Demand Marketing Leaders in 2018
Marketo
Amp Up Your Testing by Harnessing Test Data
TechWell
LIVE DEMO: Fraud Analytics
VMware Tanzu
The Why And How Of Machine Learning And AI: An Implementation Guide For Healt...
Health Catalyst
3rd Midlands BI Forum Presentation sample
Rory Allen
AI: A risk and way to manage risk
Karan Sachdeva
Creating a DevOps Practice for Analytics -- Strata Data, September 28, 2017
Caserta
Data Infrastructure for Your Retail Digital Strategy
Atif Shaikh
Customer loyalty and technology as crm tool
Ajit Singh

More from Thoughtworks

Design System as a Product
Thoughtworks
Designers, Developers & Dogs
Thoughtworks
Cloud-first for fast innovation
Thoughtworks
More impact with flexible teams
Thoughtworks
Culture of Innovation
Thoughtworks
Dual-Track Agile
Thoughtworks
Developer Experience
Thoughtworks
When we design together
Thoughtworks
Hardware is hard(er)
Thoughtworks
Customer-centric innovation enabled by cloud
Thoughtworks
Amazon's Culture of Innovation
Thoughtworks
When in doubt, go live
Thoughtworks
Don't cross the Rubicon
Thoughtworks
Error handling
Thoughtworks
Your test coverage is a lie!
Thoughtworks
Docker container security
Thoughtworks
Redefining the unit
Thoughtworks
Technology Radar Webinar UK - Vol. 22
Thoughtworks
A Tribute to Turing
Thoughtworks
Rsa maths worked out
Thoughtworks

Featured

What to Upload to SlideShare
SlideShare
Be A Great Product Leader (Amplify, Oct 2019)
Adam Nash
Trillion Dollar Coach Book (Bill Campbell)
Eric Schmidt
APIdays Paris 2019 - Innovation @ scale, APIs as Digital Factories' New Machi...
apidays
A few thoughts on work life-balance
Wim Vanderbauwhede
Is vc still a thing final
Mark Suster
The GaryVee Content Model
Gary Vaynerchuk
Mammalian Brain Chemistry Explains Everything
Loretta Breuning, PhD
Blockchain + AI + Crypto Economics Are We Creating a Code Tsunami?
Dinis Guarda
The AI Rush
Jean-Baptiste Dumont
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
Carol Smith
10 facts about jobs in the future
Pew Research Center's Internet & American Life Project
Harry Surden - Artificial Intelligence and Law Overview
Harry Surden
Inside Google's Numbers in 2017
Rand Fishkin
Pinot: Realtime Distributed OLAP datastore
Kishore Gopalakrishna
How to Become a Thought Leader in Your Niche
Leslie Samuel
Visual Design with Data
Seth Familian
Designing Teams for Emerging Challenges
Aaron Irizarry
UX, ethnography and possibilities: for Libraries, Museums and Archives
Ned Potter
Winners and Losers - All the (Russian) President's Men
Ian Bremmer

Related Books

Free with a 14 day trial from Scribd

See all
The Future Is Faster Than You Think: How Converging Technologies Are Transforming Business, Industries, and Our Lives Peter H. Diamandis
(4.5/5)
Free
Talk to Me: How Voice Computing Will Transform the Way We Live, Work, and Think James Vlahos
(4/5)
Free
SAM: One Robot, a Dozen Engineers, and the Race to Revolutionize the Way We Build Jonathan Waldman
(5/5)
Free
From Gutenberg to Google: The History of Our Future Tom Wheeler
(3.5/5)
Free
So You Want to Start a Podcast: Finding Your Voice, Telling Your Story, and Building a Community That Will Listen Kristen Meinzer
(3.5/5)
Free
Bezonomics: How Amazon Is Changing Our Lives and What the World's Best Companies Are Learning from It Brian Dumaine
(4/5)
Free
No Filter: The Inside Story of Instagram Sarah Frier
(4.5/5)
Free
Autonomy: The Quest to Build the Driverless Car—And How It Will Reshape Our World Lawrence D. Burns
(5/5)
Free
Future Presence: How Virtual Reality Is Changing Human Connection, Intimacy, and the Limits of Ordinary Life Peter Rubin
(4/5)
Free
Live Work Work Work Die: A Journey into the Savage Heart of Silicon Valley Corey Pein
(4.5/5)
Free
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are Seth Stephens-Davidowitz
(4.5/5)
Free
Life After Google: The Fall of Big Data and the Rise of the Blockchain Economy George Gilder
(4/5)
Free
System Identification: Tutorials Presented at the 5th IFAC Symposium on Identification and System Parameter Estimation, F.R. Germany, September 1979 Elsevier Books Reference
(4/5)
Free
Energy Conservation in Buildings: The Achievement of 50% Energy Saving: An Environmental Challenge? Elsevier Books Reference
(4/5)
Free
Young Men and Fire: Twenty-fifth Anniversary Edition Norman Maclean
(4.5/5)
Free
Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time Dava Sobel
(4/5)
Free

Related Audiobooks

Free with a 14 day trial from Scribd

See all
If Then: How the Simulmatics Corporation Invented the Future Jill Lepore
(4.5/5)
Free
A Brief History of Motion: From the Wheel, to the Car, to What Comes Next Tom Standage
(4/5)
Free
An Ugly Truth: Inside Facebook’s Battle for Domination Sheera Frenkel
(4.5/5)
Free
Spooked: The Trump Dossier, Black Cube, and the Rise of Private Spies Barry Meier
(4/5)
Free
Second Nature: Scenes from a World Remade Nathaniel Rich
(5/5)
Free
Driven: The Race to Create the Autonomous Car Alex Davies
(4.5/5)
Free
Test Gods: Virgin Galactic and the Making of a Modern Astronaut Nicholas Schmidle
(5/5)
Free
Einstein's Fridge: How the Difference Between Hot and Cold Explains the Universe Paul Sen
(4.5/5)
Free
Dignity in a Digital Age: Making Tech Work for All of Us Ro Khanna
(4/5)
Free
After Steve: How Apple Became a Trillion-Dollar Company and Lost its Soul Tripp Mickle
(4.5/5)
Free
User Friendly: How the Hidden Rules of Design Are Changing the Way We Live, Work, and Play Cliff Kuang
(4.5/5)
Free
Blockchain: The Next Everything Stephen P. Williams
(4/5)
Free
Uncanny Valley: A Memoir Anna Wiener
(4/5)
Free
Lean Out: The Truth About Women, Power, and the Workplace Marissa Orr
(4.5/5)
Free
A World Without Work: Technology, Automation, and How We Should Respond Daniel Susskind
(4.5/5)
Free
The Players Ball: A Genius, a Con Man, and the Secret History of the Internet's Rise David Kushner
(4.5/5)
Free

Big Data Agile Analytics by Ken Collier - Director Agile Analytics, Thoughtworks

  1. 1. BIG DATA AGILE ANALYTICS Ken Collier, PhD Director, Agile Analytics @theagilist #thoughtworks 1
  2. 2. Value Complexity What happened? Descriptive Analytics Why did it happen? Diagnostic Analytics What will happen? Predictive Analytics How can we make it happen? Prescriptive Analytics
  3. 3. Value Complexity What happened? Descriptive Analytics Why did it happen? Diagnostic Analytics What will happen? Predictive Analytics How can we make it happen? Prescriptive Analytics 3 Traditional Business Intelligence Advanced Analytics
  4. 4. Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Advanced Analytics
  5. 5. Agile Analytics Big Data Solutions Thinking Ethics Agile Delivery Lean Learning Impact Advanced Analytics Volume Velocity Variety NoSQL Complexity Polyglot Persistence
  6. 6. Big Data Analytics Pipeline Modeling Data Operational Data External Data Data Integration Reporting Engine Dimension Mapping Clean Data Report Report Report Dimensional Data Data Sampling Feature Selection Data Partitioning Test Data Training Data Analytical Modeling Candidate Model Model Validation Accepted Model
  7. 7. Agile Analytics Big Data Solutions Thinking Ethics Agile Delivery Lean Learning Impact Advanced Analytics Volume Velocity Variety NoSQL Complexity Polyglot Persistence
  8. 8. Advanced Analytics Agile Analytics Big Data Solutions Thinking Ethics Agile Delivery Lean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence
  9. 9. Discover & Explore Analyze & Act Data Convergence Analytical Divergence Discover Harvest Filter Integrate Augment Analyze Act Analytical Opportunities How Advanced Analytics Works If we knew X, we could do Y
  10. 10. Typical Timeline 3-6 months 2 months 2-4 months 10 Data Convergence Analytical Divergence Discover Harvest Filter Integrate Augment Analyze Act Analytical Opportunities Traditional Analytics If we knew X, we could do Y
  11. 11. Advanced Analytics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery
  12. 12. Advanced Analytics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure
  13. 13. Analytical Divergence Analytical Opportunities If we knew X, we could do Y Data Convergence Discover Harvest Filter Integrate Augment Analyze Act Repeat this cycle solving small problems every few days LEARN MEASURE BUILD Agility in Analytics
  14. 14. Retain high value customers High value business goal Like this example…
  15. 15. What’s the smallest, simplest thing we can do? Retain high value customers Like this example… Common features of defectors?
  16. 16. Is it useful & actionable? Retain high value customers Like this example… Common features of defectors?
  17. 17. Repeat!Retain high value customers Like this example… Common features of defectors? Shopping behaviors of defectors?
  18. 18. Retain high value customers Like this example… Common features of defectors? What leads to customers leaving? Shopping behaviors of defectors? What do defectors say about us? Customers’ sentiment before defecting? What encourages customers to stay? Do incentives reduce defection rates?
  19. 19. Problem solved or continue? What leads to customers leaving? Like this example… Common features of defectors? Shopping behaviors of defectors? What do defectors say about us? Customers’ sentiment before defecting? What encourages customers to stay? Do incentives reduce defection rates?
  20. 20. Advanced Analytics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure Data Science Machine Learning Statistics
  21. 21. THE “DATA SCIENTIST” Machine Learning Statistical Modeling Artificial Neural Networks Decision Tree Learning Support Vector Machines Clustering …and many more… Bayesian Classification Monte Carlo Simulation Logistic Regression K-Nearest Neighbor …and many more… Domain Knowledge Data Semantics Business Understanding Business Communication Programming Skills Functional Programming Data “Wrangling” Map/Reduce, SQL, & NoSQL
  22. 22. Advanced Analytics Data Science Visual Storytelling Machine Learning Statistics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure
  23. 23. drones.pitchinteractive.com Data Visualization
  24. 24. Advanced Analytics Data Science Visual Storytelling Machine Learning Statistics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure Data Reduction
  25. 25. Objective Truth Discoverable Truth Uninterpretable Irrelevant Noise Not Actionable Impactful New Insights “Little Data”
  26. 26. Advanced Analytics Data Science Visual Storytelling Machine Learning Statistics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure Data Reduction Insight Knowledge Action Disruption
  27. 27. Advanced Analytics Data Science Visual Storytelling Machine Learning Statistics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure Data Reduction Insight Knowledge Action Disruption Business vs. IT Focus vs. Platform Monitor & Measure
  28. 28. Advanced Analytics Data Science Visual Storytelling Machine Learning Statistics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure Data Reduction Insight Knowledge Action Disruption Business vs. IT Focus vs. Platform Monitor & Measure Privacy Controls Radical Transparency Data Democracy Open Data
  29. 29. Advanced Analytics Data Science Visual Storytelling Machine Learning Statistics Agile Analytics Big Data Solutions Thinking Ethics Agile DeliveryLean Learning Impact Volume Velocity Variety NoSQL Complexity Polyglot Persistence Continuous Integration Collaboration Evolve Continuous Delivery Hypothesis Build Learn Measure Data Reduction Insight Knowledge Action Disruption Business vs. IT Focus vs. Platform Monitor & Measure Privacy Controls Radical Transparency Data Democracy Open Data
  30. 30. Ken Collier, Director, Agile Analytics kcollier@thoughtworks.com Value Creation Cool New Technologies + Sophisticated Analytics + Lean Learning Principals + Fast Agile Delivery =

×