Recommended
More Related Content
What's hot
What's hot (20)
Similar to Business with Big data
Similar to Business with Big data (20)
Recently uploaded
Recently uploaded (20)
Business with Big data
- 1. BUSINESS WITH BIG DATA Bruno Curtarelli curtarelli@gmail.com
- 2. 1865 The term “business intelligence” is used by Richard Millar Devens in his Encyclopaedia of Commercial and Business Anecdotes, describing how a banker achieved an advantage over competitors by collecting and analyzing information 18,000 ac First evidences of data storage in Uganda 2400 ac Abacus in Babylon 2003 Hadoop e Map Reduce – Google 1970 Edgar F Codd presents his framework for a “relational database” 300 ac - 48 dc Alexandria’s Library 1663 John Graunt: early days of statistics, trying to predict bubonic plague in Europe 1880 Herman Hollerith creates the machine that would reduce US Census work from 10 years to 3 months 1962 The first steps are taken towards speech recognition, when IBM engineer William C Dersch presents the Shoebox Machine at the 1962 World Fair 1926 N. Tesla states that when wireless technology is “perfectly applied the whole Earth will be converted into a huge brain...and A man will be able to carry one in his vest pocket.” 1965 The US Government plans the world’s first data center to store 742 million tax returns and 175 million sets of fingerprints on magnetic tapes 1991 Tim Berners-Lee announced the birth of what would become the Internet as we know it today 1997 Michael Lesk publishes his paper theorizing that the existence of 12,000 petabytes is “perhaps not an unreasonable guess”. He also points out that even at this early point in its development, the web is increasing in size 10-fold each year 2001 Doug Laney – 3Vs Volume, Velocity, Variety 2005 Web 2.0 - the user-generated web where the majority of content will be provided by users of services, rather than the service providers themselves 2008 The world’s servers process 9.57 zettabytes of information 2013 IoT evolves from internet to wireless applications and from micro components to embedded systems, leveraging automation 2014 For the first time, more people are using mobile devices to access digital data, than office or home computers 2018 Big Data Software and Services must increase from 42Bi usd to 103Bi usd in 2027 (Statista) https://www.weforum.org/agenda/2015/02/a-brief-history-of-big-data-everyone-should-read/ http://www.dataversity.net/brief-history-internet-things/
- 4. Originally defined by the 3v’s - Doug Laney + Veracity (reliability) + Variability (Analytics/Statistics) + Validity (how it is used) + Vulnerability (security/privacy - GDPR) + Value (making business) + Visualization (Very large amounts) + Volatility (depreciation) 3 V’s VARIETY VOLUME VELOCITY New V’s
- 5. More than 4Mi videos are watched on YouTube Google process more than 2.4Mi searches - 3.5 Bi/day 90% of all data generated in the world were produced in the last 2 years 456K Tweets on Twitter In one minute... 47K photos on Instagram 528K Photos on Snapchat 120 new professionals on LinkedIn https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create-every-day-the-mind-blowing-stats-everyone-should-read/#6e2bed4760ba
- 6. 156 Mi e-mails 154K Skype calls 90% of all data generated in the world were produced in the last 2 years 15K Gifs on messenger In one minute... 103 Mi Spams (e-mails) ~1Mi Tinder Swipes 16 Mi text messages https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create-every-day-the-mind-blowing-stats-everyone-should-read/#6e2bed4760ba
- 7. Weather Channel receives +18Mi requests for forecasts ~46K Uber trips 90% of all data generated in the world were produced in the last 2 years In one minute... 600 new edits on Wikipedia 13 new musics in Spotify $51 Mi USD are processed by Venmo https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create-every-day-the-mind-blowing-stats-everyone-should-read/#6e2bed4760ba
- 8. How to storage and process all this data? Originally posted by www.mozy.com Found it here: https://gizmodo.com/5309889/how-large-is-a-petabyte
- 9. Main layers from Big Data Environment https://dzone.com/articles/a-quick-look-at-big-data-architecture-landscape-la 1 Insights: Decisions are taken and strategies can be drawn based on information 3 Data Analysis: This is where the data is processed. Data management models, analytics tools, data engineering and access to distribution system 5 Data Sources: This is where the data comes from. Different channels, such as apps, sensors, internal databases, softwares, mobile collectors, social network among others 2 Data Consumption: This is where data becomes information. Reports, dashboards among other data visualization tools 4 Storage and Distribution: It is where Big Data lives. Data Acquisition from different sources, applying formatting policies, privacy settings and security levels Data Platform Data Science Analytics Data Sources BI Dashboard Insights
- 10. For each problem a different framework Open Source!
- 12. Main Challenges
- 13. 1. Scalability: How to handle exponentially increasing data? 2. Taking insights without losing the timing for execution 3. Recruitment and talent retention: scarce qualified professionals 4. Data integration from different sources and channels: Data Lakes 5. Data Validation: Governance 6. Security and Privacy: How to prevent that data don’t get exposed (GDPR) 7. Organizational resistance, cultural changes, decisions over data - DATA IS THE BOSS! Main Challenges on Big Data https://www.datamation.com/big-data/big-data-challenges.html
- 17. How companies are using BIG DATA?
- 18. Client conversion 03 Using navigation behaviour to predict a client leaving an ecommerce website before they purchase the desired products, enabling the company to offer something to convert the sales Consumer Segmentation 02 Finding patterns in transactional data from a retailer to segment customers and tailor better services, promotions and strategies for retention, among other actions 360º client view 01 Data collected from different sources and channels, like call centers, apps, social network, company website, cell phone network, among others can be used to understand client behaviour and improve servicing and product offers Price optimization 04 Monitoring competitors through data crawlers and web scraping, combining with internal sources to build price elasticity models that would help to better define product prices Sentiment Analysis 05 Assessment from social network posts, activities, comments, hashtags and emojis to understand people sentiment over subject or discussion
- 19. Security 08 Smart Cities - Using IoT (internet of things) to collect video images from streets, weather, traffic, signals etc to support government on doing the right allocation of resources or automating systems Fraud Prevention 09 Analyzing banking transactions, social networks, among other data to detect suspect behaviour and fraudulent actions Demand Prediction 10 Analyzing daily sales to predict the amount of products needed, avoiding ruptures, data loss Recommendation Systems 06 Evaluation of consumer behaviour on any kind of service or purchase to find patterns and predict what these customers would like based on their preferences of preferences from people that behaves like them Improving Logistic Systems 07 Using GPS and telemetry to assess drivers behaviour to optimize car fleet, deliveries, among other logistic services, like ambulances, garbage collectors, police etc
- 21. Data Monetization INDIRECT ● Improve efficiency ● Reduce/manage risks ● Fraud prevention ● Products/services development ● Customer relationship development ● Performance measurement DIRECT ● Data Sales (database enrichment) ● Improving third party products and services ● Exchanging information ● API’s licensing subscriptions ● Dashboard and report sales
- 22. What about you?