Uploaded bySteven Miller
The exciting new world of code & data
The document discusses trends in data science and analytics jobs. It summarizes that data science and analytics jobs are growing rapidly, with over 230,000 such jobs in New York alone and an expected growth of 40,000 additional jobs in the next 5 years. It also discusses the emerging roles of data engineers, data scientists, data analysts, and data-driven decision makers and their typical skills, responsibilities, and salaries.
More Related Content
Similar to The exciting new world of code & data
![[DSC Europe 24] Jesse Anderson - The State of Data Engineering](https://cdn.slidesharecdn.com/ss_thumbnails/jesseanderson-thestateofdataengineeringbigdata-241219143316-5fdffa3a-thumbnail.jpg?width=640&height=640&fit=bounds)
The exciting new world of code & data
- 1. © 2017 IBMCorporation The exciting new world of code & data Steven Miller June 6th 2017
- 2. © 2016 IBMCorporation2 Where code goes, where data flows, cognition will follow.
- 3. © 2017 IBMCorporation3
- 4. © 2017 IBMCorporation4 Clinical Data Analyst 54%
- 5. © 2017 IBMCorporation5
- 6. © 2017 IBMCorporation6 1 AI and Advanced Machine Learning 2 Intelligent Apps 3 Intelligent Things 4 Virtual and Augmented Reality 5 Digital Twin 6 Blockchain and Distributed Ledgers 7 Conversational System 8 Mesh App & Service Architecture 9 Digital Technology Platforms 10 Adaptive Security Architecture Gartner’s Top 10 Trends Shaping the future – 2017 http://www.gartner.com/newsroom/id/3482617
- 7. © 2017 IBMCorporation7
- 8. © 2017 IBMCorporation8 Internet of Things We can sense & measure anything Did I shut my oven off? Lock my front door? Check on my elderly relative Find my lost keys Identify equipment about to fail Track Assets Monitor Pollution Levels … We’re just getting started
- 9. © 2017 IBMCorporation9 Tremendous growth of sensors generating data McKinsey - The Internet of Things: Sizing up the opportunity
- 10. © 2017 IBMCorporation10 How many sensors in your smart phone?
- 11. © 2017 IBMCorporation11
- 12. © 2017 IBMCorporation12 Sensors in the iPhone 6 front & rear Cameras multi-touch touchscreen display triple microphone GPS Apple M8 motion coprocessor 3-axis gyroscope 3-axis accelerometer digital compass iBeacon proximity sensor ambient light sensor touch ID fingerprint reader barometer connect any sensor w/Bluetooth or headphone jack
- 13. © 2017 IBMCorporation13
- 14. © 2017 IBMCorporation14 ‘Customer Curated Data’ Platforms Illustration by Lisa Larson-Walker/Slate
- 15. © 2017 IBMCorporation15 Extracting value from ‘Data Exhaust’
- 16. © 2017 IBMCorporation16 Ever been taken for ‘a ride’ in a taxi? I challenged the first night’s fare Uber refunded the difference Monday Night Tuesday Night
- 17. © 2017 IBMCorporation17 open data Data that can be freely used, re-used and redistributed by anyone - subject only, at most, to the requirement to attribute and share alike.
- 18. © 2017 IBMCorporation18 http://bit.ly/trimetandgoogle General Transit Feed Specification Reference https://developers.google.com/transit/gtfs/reference
- 19. © 2017 IBMCorporation19 Smart city A smart city gathers data from smart devices and sensors embedded in its roadways, power grids, buildings, waterways, anything that can be sensed. To deliver improved health, more efficient infrastructure systems, cleaner environments, and safer and more inclusive societies.
- 20. © 2017 IBMCorporation20 People + Data + Infrastructure + Technologies = Platform
- 21. © 2017 IBMCorporation21 Putting Sensors to work in your city IBM Center for Applied Insights
- 22. © 2017 IBMCorporation22 Data & Work
- 23. © 2017 IBMCorporation23 Talent: New cognitive & analytics professions are emerging Data Scientist Data Policy Professional Chief Data Officer Data Engineer Machine Learning Developer Cyber-Physical Systems Engineer
- 24. © 2017 IBMCorporation24 The Citizen Analyst The data driven economy places data in the hands of every professional.
- 25. © 2017 IBMCorporation25
- 26. © 2017 IBMCorporation26 bit.ly/quantcrunch
- 27. © 2017 IBMCorporation27
- 28. © 2017 IBMCorporation28 Manufacturing 16%
- 29. © 2017 IBMCorporation29 Data Science & Analytics (DSA) Jobs Framework
- 30. © 2017 IBMCorporation30 Defining the DSA Jobs Landscape Inconsistent nomenclature prevents stakeholders from answering key questions. Over 300 core analytical skills were identified, which were used to score occupations based on their level of analytical rigor. Occupation NYC Analytical Score (Q3 2015 – Q2 2016) Data Scientist 100 Data Engineer 97 Biostatistician 96 Data Architect 91 Statistician 90 Top Analytical Occupations in New York
- 31. © 2017 IBMCorporation31 DSA in New York: By the Numbers 231,562 DSA JOBS Number of jobs in the New York area in a 12 month period (2015-2016) 40,462 NEW POSTINGS Number of additional DSA job postings projected in New York in 5 years 45 DAYS Average time to fill a DSA role in New York $97,300 PER ANNUM Average annual salary of a DSA job in New York 83% PERCENT Proportion of jobs in New York requesting experienced workers (3 or more years of experience)
- 32. © 2017 IBMCorporation32 Framework Category Postings Projected 5 Year Growth Estimated Postings in 5 Years Average Time to Fill (days) Average Advertised Salary % Postings requiring MA or higher % Requesting Experienced Workers (3 years or more) All 231,562 17% 272,024 45 $97,300 9% 83% Data-Driven Decision Makers 82,618 17% 96,963 47 $105,672 7% 91%* Functional Analysts 72,472 18% 85,430 41 $83,265 8% 71% Data Systems Developers 51,093 17% 59,837 49 $98,957 5% 87% Data Analysts 11,987 16% 13,860 44 $90,007 8% 76% Data Scientists & Advanced Analysts 7,170 24% 8,890 45 $117,432 46% 79% Analytics Managers 6,222 13% 7,045 44 $126,840 17% 94%* Quantifying the NYC DSA Job Market
- 33. © 2017 IBMCorporation33
- 34. © 2017 IBMCorporation34 Data Science is a diverse field Human Data Scientist Primary focus is advising the business Make sense of any dataset(s) Apply any form of analytics from descriptive to cognitive Visualization experts Data Storytellers Machine Data Scientist Primary focus is writing advanced algorithms for: Advanced Robotics Self driving cars Recommendation engines Virtual Assistants IBM Watson
- 35. © 2017 IBMCorporation35 The Data Engineer https://www.linkedin.com/job/data-engineer-jobsThe Data Engineer builds the - modern data systems needed by data scientists & developers. - Complex - Scale, often extreme scale - Near-real time performance - Diverse data sources & types - Secure
- 36. © 2017 IBMCorporation36 Engineering ? Data Engineer a trending job title. 295 open positions in Texas (LinkedIn June 6th) Data Engineers are responsible for building COMPLEX SYSTEMS. But… does the average person hired into one of theses have training in SYSTEMS Engineering? Likely NOT.
- 37. © 2017 IBMCorporation37 What is the best path to creating an engineering DISCIPLINE? Data Systems Engineer Systems Software Sensors & Things Data Database Algorithms Mission Critical Cyber Physical Systems Engineer Systems, not technologies Empowering, not determining Future by Design Design Values as a feature, not a bug (Source: 4th Industrial Revolution Workshop Proceedings)
- 38. © 2017 IBMCorporation38 Data What data could solve this problem? What existing business data is relevant? What can I attach a sensor to? Is there available government open data? Can data be acquired from other sources? Analyze What models & methods can I apply to solve this problem? Descriptive Predictive Prescriptive Cognitive / Machine Learning Visualization / Story telling Operations How to apply to our business? • Create / amend business processes? • Create new businesses? Close existing ones? • Enter new markets? … Design Thinking Communication T-shaped Skills The Data Driven Decision Maker
- 39. © 2017 IBMCorporation39 The Chief Data Officer Chief Data Officers understand how to use data to create strategic opportunities; responsible for organizational data strategy & governance
- 40. © 2017 IBMCorporation40 Growing Demand for Data Policy Skills Data is a core business asset! • Curated • Consistent • High Quality • Protected • Available • Complies • Preserved • Managed Lifecycle
- 41. © 2017 IBMCorporation41 Data literacy Understanding how to use & program computers is not enough in a data driven world. Every profession. Every career is impacted by data, by analytics, by cognitive. Everyone must become data literate.
- 42. © 2017 IBMCorporation42
- 43. © 2017 IBMCorporation43 In the Cognitive Era everyone must be data literate Define problems Wrangle Data Self-Manage Data Choose Methods and Tools Analyze Data Communicate findings Engage in Lifelong Learning
- 44. © 2017 IBMCorporation44
- 45. © 2017 IBMCorporation45 Reimagining Ethics Education … moving beyond traditional compliance and ethics courses: MAKE IT PRACTICAL!!! Values & Ethical dilemmas Interactive materials, group-based work and action-learning initiatives https://www.weforum.org/agenda/2017/03/why-we-have-to-reimagine-education-to-fight-corruption
- 46. © 2017 IBMCorporation46
- 47. © 2017 IBMCorporation47 Data Literacy for Everyone: To achieve widespread data literacy, next gen students must be exposed to data and its relevance and applicability early. New Education Programs Accelerated development of new learning pathways which are specifically targeted at preparing candidates for DSA roles. Continuous development of competences and skills: We need new type of workforce and attitude from both employers and employees around continuous learning and mastering skills that will enable employees to be prepared for not-yet-arrived jobs of the future. Initiate data labs: Data labs are creative spaces bringing together learners, experts from industry and academia, open data, and data systems to collaborate on challenging problems.
- 48. © 2017 IBMCorporation48
- 49. © 2017 IBMCorporation49 http://bit.ly/dataskillsgap @brandsteve
Editor's Notes
- #2 1
- #4 Which was the fasted growing data role in the United States last year? https://www.polleverywhere.com/multiple_choice_polls/i3g3frmK1a6t0Ig
- #6 Data is the new oil. Our ability to make use of ALL data anytime anywhere is transforming our world.
- #7 Data plays a key role in every single one of these trends. AI and Advanced Machine Learning Artificial intelligence (AI) and advanced machine learning (ML) are composed of many technologies and techniques (e.g., deep learning, neural networks, natural-language processing [NLP]). The more advanced techniques move beyond traditional rule-based algorithms to create systems that understand, learn, predict, adapt and potentially operate autonomously. This is what makes smart machines appear "intelligent." "Applied AI and advanced machine learning give rise to a spectrum of intelligent implementations, including physical devices (robots, autonomous vehicles, consumer electronics) as well as apps and services (virtual personal assistants [VPAs], smart advisors), said Mr. Cearley. "These implementations will be delivered as a new class of obviously intelligent apps and things as well as provide embedded intelligence for a wide range of mesh devices and existing software and service solutions." Intelligent Apps Intelligent apps such as VPAs perform some of the functions of a human assistant making everyday tasks easier (by prioritizing emails, for example), and its users more effective (by highlighting the most important content and interactions). Other intelligent apps such as virtual customer assistants (VCAs) are more specialized for tasks in areas such as sales and customer service. As such, these intelligent apps have the potential to transform the nature of work and structure of the workplace. "Over the next 10 years, virtually every app, application and service will incorporate some level of AI," said Mr Cearley. "This will form a long-term trend that will continually evolve and expand the application of AI and machine learning for apps and services." Intelligent Things Intelligent things refer to physical things that go beyond the execution of rigid programing models to exploit applied AI and machine learning to deliver advanced behaviors and interact more naturally with their surroundings and with people. As intelligent things, such as drones, autonomous vehicles and smart appliances, permeate the environment, Gartner anticipates a shift from stand-alone intelligent things to a collaborative intelligent things model. Virtual and Augmented Reality Immersive technologies, such as virtual reality (VR) and augmented reality (AR), transform the way individuals interact with one another and with software systems. "The landscape of immersive consumer and business content and applications will evolve dramatically through 2021," said Mr. Cearley. "VR and AR capabilities will merge with the digital mesh to form a more seamless system of devices capable of orchestrating a flow of information that comes to the user as hyperpersonalized and relevant apps and services. Integration across multiple mobile, wearable, Internet of Things (IoT) and sensor-rich environments will extend immersive applications beyond isolated and single-person experiences. Rooms and spaces will become active with things, and their connection through the mesh will appear and work in conjunction with immersive virtual worlds." Digital Twin A digital twin is a dynamic software model of a physical thing or system that relies on sensor data to understand its state, respond to changes, improve operations and add value. Digital twins include a combination of metadata (for example, classification, composition and structure), condition or state (for example, location and temperature), event data (for example, time series), and analytics (for example, algorithms and rules). Within three to five years, hundreds of millions of things will be represented by digital twins. Organizations will use digital twins to proactively repair and plan for equipment service, to plan manufacturing processes, to operate factories, to predict equipment failure or increase operational efficiency, and to perform enhanced product development. As such, digital twins will eventually become proxies for the combination of skilled individuals and traditional monitoring devices and controls (for example, pressure gauges, pressure valves). Blockchain and Distributed Ledgers Blockchain is a type of distributed ledger in which value exchange transactions (in bitcoin or other tokens) are sequentially grouped into blocks. Each block is chained to the previous block and recorded across a peer-to-peer network, using cryptographic trust and assurance mechanisms. Blockchain and distributed-ledger concepts are gaining traction because they hold the promise to transform industry operating models. While the current hype is around the financial services industry, there are many possible applications including music distribution, identity verification, title registry and supply chain. "Distributed ledgers are potentially transformative but most initiatives are still in the early alpha or beta testing stage," said Mr. Cearley. Conversational System The current focus for conversational interfaces is focused on chatbots and microphone-enabled devices (e.g., speakers smartphones, tablets, PCs, automobiles). However, the digital mesh encompasses an expanding set of endpoints people use to access applicatons and information, or interact with people, social communities, governments, and businesses. The device mesh moves beyond the traditional desktop computer and multiple devices to encompass the full range of endpoints with which humans might interact. As the device mesh evolves, connection models will expand and greater cooperative interaction between devices will emerge, creating the foundation for a new continuous and ambient digital experience. Mesh App and Service Architecture In the mesh app and service architecture (MASA), mobile apps, web apps, desktop apps and IoT apps link to a broad mesh of back-end services to create what users view as an "application." The architecture encapsulates services and exposes APIs at multiple levels and across organizational boundaries balancing the demand for agility and scalability of services with composition and reuse of services. The MASA enables users to have an optimized solution for targeted endpoints in the digital mesh (e.g., desktop, smartphone, automobile) as well as a continuous experience as they shift across these different channels. Digital Technology Platforms Digital technology platforms provide the basic building blocks for a digital business and are a critical enabler to become a digital business. Gartner has identified the five major focal points to enable the new capabilities and business models of digital business — information systems, customer experience, analytics and intelligence, the IoT, and business ecosystems. Every organization will have some mix of these five digital technology platforms. The platforms provide the basic building blocks for a digital business and are a critical enabler to become a digital business. Adaptive Security Architecture The intelligent digital mesh and related digital technology platforms and application architectures create an ever-more-complex world for security. "Established security technologies should be used as a baseline to secure Internet of Things platforms," said Mr. Cearley. "Monitoring user and entity behavior is a critical addition that is particularly needed in IoT scenarios. However, the IoT edge is a new frontier for many IT security professionals creating new vulnerability areas and often requiring new remediation tools and processes that must be factored into IoT platform efforts."
- #8 This next section provides clear examples of innovating with data.
- #12 How many sensors does a Smart phone have? https://www.polleverywhere.com/multiple_choice_polls/32oWx9gkU3OGUDM
- #13 https://en.wikipedia.org/wiki/IPhone_6 The Apple iWatch has a heart rate sensor
- #15 An entire industry has arisen that profits from the data we provide for free. Our resume, our food & beer checkins, our yelp & trip advisor reviews.
- #16 New relic is a leader helping cloud providers maintain 24x7 availability. The old-school call support when an outage occurs, wait for the maytag repairman to show up, fix it and reboot is no longer viable. Now we must predict failures and correct them BEFORE they happen. Kabbage is the new leader in small business lending. If your business is real, we can figure it out in seconds. Got a Yelp profile. People like you. People talk about you on Twitter and other social properties. Good. Here is the 10,000 line of credit you asked for.
- #17 Uber is using DATA to remake an old stodgy business that customers hate. Need a ride at 2am… no problem. Uber will find you a car and it will show up in minutes. Don’t have $20 in cash… no problem. Thank the driver get out of the car and go on your way. Have trouble explaining to the taxi driver where you are? No problem here’s my exact address, or at least my GPS coordinates. Tired of getting ripped off by taxi drivers? No problem. Uber drtivers who deliberately run up the fare are tracked every step of the way. Get your money back America. And and you get to rate your driver and your driver gets to rate you. It pays to be a good driver and to be a good customer.
- #18 Open data, is a wave of opportunity to remake our world.
- #19 Portland Tri-Met partnered with Google to create the Transit Feed which is now in use all over the globe. Now it’s real time too. Put a sensor on your bus, on your train. Never miss a train or bus again.
- #20 Want to learn more – tell your audience about the City as Platform report -- http://bit.ly/cityasplatform
- #22 Everything can be measured. Cities are actively measuring air quality, water quality, traffic, transit, noise, weather, tides, people, anything that be sensed & measured.
- #23 Data is the new oil. Our ability to make use of ALL data anytime anywhere is transforming our world.
- #25 Every professional needs to become a citizen analyst in the data driven economy
- #26 Free stock photo from: https://unsplash.com/collections/164957/crowd?photo=TZCppMjaOHU
- #28 Fastest growing industry for data savvy pros? Pro Svcs & Finance are #1, #2. Which is #3? https://www.polleverywhere.com/multiple_choice_polls/7JDADZUJU62JD6g
- #35 Human data scientists support the business or lines of business helping them understand and make better decisions. Machine data scientists drive the machines that run much of the cognitive world.
- #36 With the data engineer data scientists will accomplish little.
- #39 A business leader needs a comprehensive view of data, analytics, and putting it to work
- #40 http://bit.ly/ibmcdostudy The chief data officer is growing fast… not sure what drove the plunge in Indeed’s data but instead focus on the trend line.
- #41 Other possible adjectives: Controlled? Pedigreed? Grey Data? Black Data? Version control? The Data Self by Rob Horning http://thenewinquiry.com/blogs/marginal-utility/dumb-bullshit/ http://www.gartner.com/newsroom/id/2506315 “The underlying message of all these examples is that information is an asset in its own right. It has value. Gartner calls this emerging discipline of valuating information "Infonomics.It is not something of the far future, in fact, this is happening today in various industries, in commerce and public sector, in large and small enterprises.” However, Mr. Buytendijk underlined the fact that as exciting as all new business opportunities are, there are also reasons for concern. Concerning the ethics of big data, a recent Gartner Circle study showed that "governance and privacy" was the most important concern around big data – clearly there is a fine line between superior customer insight and being "creepy."
- #44 In partnership with Oceans of Data we brought a group of experts together from industry & academia to define data & analytics literacy. These are the top level recommendations. Michael Bowen Associate Professor, Science Education, Mount Saint Vincent University, Halifax, Nova Scotia Ben Davison Quantitative User Experience Researcher, Google Rob Gould Faculty, UCLA Department of Statistics Ryan Kapaun Crime Analyst, Eden Prairie Police Department Cliff Konold Director, Scientific Reasoning Research Institute, University of Massachusetts, Amherst Juan Miguel Lavista Ferres Principal Data Scientist at Bing/Microsoft Odette Merchant Project Manager, Nova Scotia Community College (NSCC), Halifax, Nova Scotia, Canada Andrew Schaffner Professor of Statistics, California Polytechnic State University, San Luis Obispo Hunter Whitney Consultant, Author, and Instructor; UX and Data Visualization Sponsored by Steven Miller IBM Moderated by the Oceans of Data Team
- #49 What will you invent?
- #51 50