Your SlideShare is downloading. ×
Mike keating - News Int - 18th BDL meetup
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Mike keating - News Int - 18th BDL meetup


Published on

Slides from Mike Keating's, Big Data Project lead at News International, talk on Big Data at News. 18th Big Data London meetup.

Slides from Mike Keating's, Big Data Project lead at News International, talk on Big Data at News. 18th Big Data London meetup.

Published in: Technology, Business

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Big Data at News International!Welcome !Big Data at News International!!Big Data Event - 29/05/2013!Mike Keating: Product Owner!@mikerkeating!  Slide 01/14!
  • 2. Big Data at News International!Introduction!•  Where to start!•  Data and decisions!•  Our technology choices!•  Lessons!!Big Data Event - 29/05/2013!Slide 02/14!
  • 3. Big Data at News International!Where to start…!Big Data Event - 29/05/2013!Slide 03/14!Lots of data, suppliers, technology and teams!!Take control, bring digital data into one place!Link data sets and look for new approaches!!Make data/ information/ knowledge/ insight available!Build awareness and decision making capability!  
  • 4. Big Data at News International!Making the basics available via dashboards!Big Data Event - 29/05/2013!Slide 04/14!
  • 5. Big Data at News International!Understanding our content and consumption!Big Data Event - 29/05/2013!Slide 07/14!Visits per VisitorViewsperVisitBehaviours Across Website Sections
  • 6. Big Data at News International!Understanding our content in a social world!Big Data Event - 29/05/2013!Slide 08/14!
  • 7. Big Data at News International!Tracking  subscrip.on  growth  across  products!Big Data Event - 29/05/2013!Slide 05/14!Days Following First SubscriptionSubscriptionsProduct Growth Following Launch
  • 8. Big Data at News International!Analysing attributes that indicate churn!Big Data Event - 29/05/2013!Slide 06/14!
  • 9. Big Data at News International!Designing products based on patterns in navigation !The  iPhone  –  Octopus  Naviga4on  The  Website  –  Flower  Petal  Naviga4on  The  iPad  Edi4on  -­‐  Linear  Naviga4on  Big Data Event - 29/05/2013!Slide 10/14!
  • 10. Big Data at News International!Our technology choices – what?!Big Data Event - 29/05/2013!Slide 11/14!Infrastructure: AWS EC2, S3, RDS, EMR, Cloudformation, Vagrant!Ops: Jenkins, Anthill Pro, Maven, Nexus, Zabbix, CloudWatch!Code & Config: Puppet, Github!Data Retrieval: Java & Python!Data Pipeline: Java Map Reduce, Apache Crunch, !Spark, MRUnit, Celery, RabbitMQ, Python, Flume!Data Storage: HDFS, HBase, AWS S3, MySQL, Redis!Data Schema: Avro!Data Access: Python APIs, Tornado, S3, Hive & AWS EMR!Analysis: R, Pandas, Excel -> analyst’s choice!Visualisation: R, D3, Highcharts, Google Charts!Products: APIs, JS+HTML+CSS!
  • 11. Big Data at News International!Our technology choices – why?!Big Data Event - 29/05/2013!Slide 11/14!•  Team: Build on team’s skillsets and knowledge. !•  Recruitment: Be conscious of “hire-ability”!•  Open Source: Big wins from usage; greatcommunities; contribute back!•  Versions: use what works; work with alpha releasesbut not as production code!•  Consistency: Try and use what you do today, e.g.AWS!•  Flexibility: Use a “better” product where it makessense!
  • 12. Big Data at News International!Our technology choices – who?!Big Data Event - 29/05/2013!Slide 11/14!•  Tech Lead: Architecture; Design; Hands-On!•  Delivery Manager: Project Management w Agile!•  Hadoop: build from Java Map Reduce !•  Python: Tornado, Native Python, real-time processing!•  Data Science: Hive, R, Modeling!•  DevOps: AWS, Vagrant, Puppet!•  Experience: Practical experience of Hadoop in production!•  Capability: Ability to learn new tech, design and build!•  Demo: contributions to projects, working examples!
  • 13. Big Data at News International!Lessons!Big Data Event - 29/05/2013!Slide 12/14!•  Building awareness & common knowledge!•  Building on existing teams, systems and their work!•  Looking for extra capability and output!•  Focus on visuals – it needs to be sharable/ visible!•  Working with range of teams to share outputs!•  Making good tech choices!
  • 14. Big Data at News International!Thanks!Big Data Event - 29/05/2013!Slide 13/14!Big Data Team – DevOps, Hadoop, Python, UI,Analysts, Test!!Technology Teams – Design, Production Ops, PerfTesting, Security, Products, Platforms, Service Desk!!Editorial, Marketing, Commercial, Finance Teams!  
  • 15. Big Data at News International!!!Thanks!Big Data at News International              Contact  Us  News  Interna.onal  Technology  @techatni  Mike;  Product  Owner  Jobs  via  hBp://  Big Data Event - 29/05/2013!Slide 14/14!