Presentation at Big Data World Asia Singapore 2017. A brief introduction to strategies for digitization transformation and introduction to Artificial Intelligence.
Building an AI Startup: Realities & TacticsMatt Turck
AI is all the rage in tech circles, and the press is awash in tales of AI entrepreneurs striking it rich after being acquired by one of the giants. As always, the realities of building a startup are different, and the path to success requires not just technical prowess but also thoughtful market positioning and business excellence.
In a talk of interest to anyone building or implementing an AI product, Matt Turck and Peter Brodsky leverage hundreds of conversations with AI (and big data) founders and hard-learned lessons building companies from the ground up to highlight successful strategies and tactics.
Topics include:
Successful data acquisition strategies
Data network effects
Competing with the giants
A pragmatic approach to building an AI team
Why social engineering is just as important to success as groundbreaking AI technology
This document discusses different types of analytics used for predictive modeling. Descriptive analytics describe what happened in the past using data, diagnostic analytics analyze why something happened, predictive analytics forecast what could happen in the future based on patterns in data, and prescriptive analytics determine the best course of action based on predictions. The document also provides examples of using property data and other variables to build a predictive model for property sale timing and valuation.
This document provides a charter and roadmap for a Computing, Data, and Informatics Working Group. It discusses their vision of enabling data, computing, and identity services at unlimited scale. It highlights how information technology has been critical but also a source of tension in large projects like the Human Genome Project. The document outlines current enabling technologies like machine learning, blockchain, and DevOps practices. It identifies key challenge areas the working group will focus on, including identity and authorization, information security and privacy, and issues around data storage in multi-cloud environments. The working group members are then listed.
This document discusses big data in an Internet of Things (IoT) world. It describes how IoT analytics can answer any question as long as the source data is digital, dealing with issues of data volume, velocity, variety and veracity. It also discusses how IoT adoption has progressed from vendor-controlled single device/single app models to models with customer-owned data across many devices and apps. The document envisions a world where IoT analytics are delivered in real-time at the point where data is created.
The document discusses emerging technologies including quantum computing, artificial intelligence, machine learning, blockchains, the internet of things, cloud computing, edge computing, and data analytics. While these technologies show promise, the document cautions against hype and notes that many technologies do not yet work as described or have practical applications. The author offers experience supporting genomics and building IT infrastructure and is available for consulting work.
Presentation that I delivered at "Accelerate AI, Europe 2018" in London on Sept 19, 2018. My focus is on socio-cultural perspective as well as proving information about various tools, vendors and partners available to help companies get started using AI.
How to make business using AI and Big Data?Ivano Digital
This document discusses artificial intelligence (AI), big data, the Internet of Things (IoT), and their applications and implications. It covers topics like AI reasoning, machine learning algorithms, data processing infrastructure, collecting data from IoT sensors, and potential AI applications in business, personal life, and society. It also addresses some challenges and barriers to adopting AI, as well as career opportunities working with data.
Top 5 Deep Learning and AI Stories - August 30, 2019NVIDIA
Read the top five news stories in artificial intelligence and learn how innovations in AI are transforming business across industries like healthcare and finance and how your business can derive tangible benefits by implementing AI the right way.
Building an AI Startup: Realities & TacticsMatt Turck
AI is all the rage in tech circles, and the press is awash in tales of AI entrepreneurs striking it rich after being acquired by one of the giants. As always, the realities of building a startup are different, and the path to success requires not just technical prowess but also thoughtful market positioning and business excellence.
In a talk of interest to anyone building or implementing an AI product, Matt Turck and Peter Brodsky leverage hundreds of conversations with AI (and big data) founders and hard-learned lessons building companies from the ground up to highlight successful strategies and tactics.
Topics include:
Successful data acquisition strategies
Data network effects
Competing with the giants
A pragmatic approach to building an AI team
Why social engineering is just as important to success as groundbreaking AI technology
This document discusses different types of analytics used for predictive modeling. Descriptive analytics describe what happened in the past using data, diagnostic analytics analyze why something happened, predictive analytics forecast what could happen in the future based on patterns in data, and prescriptive analytics determine the best course of action based on predictions. The document also provides examples of using property data and other variables to build a predictive model for property sale timing and valuation.
This document provides a charter and roadmap for a Computing, Data, and Informatics Working Group. It discusses their vision of enabling data, computing, and identity services at unlimited scale. It highlights how information technology has been critical but also a source of tension in large projects like the Human Genome Project. The document outlines current enabling technologies like machine learning, blockchain, and DevOps practices. It identifies key challenge areas the working group will focus on, including identity and authorization, information security and privacy, and issues around data storage in multi-cloud environments. The working group members are then listed.
This document discusses big data in an Internet of Things (IoT) world. It describes how IoT analytics can answer any question as long as the source data is digital, dealing with issues of data volume, velocity, variety and veracity. It also discusses how IoT adoption has progressed from vendor-controlled single device/single app models to models with customer-owned data across many devices and apps. The document envisions a world where IoT analytics are delivered in real-time at the point where data is created.
The document discusses emerging technologies including quantum computing, artificial intelligence, machine learning, blockchains, the internet of things, cloud computing, edge computing, and data analytics. While these technologies show promise, the document cautions against hype and notes that many technologies do not yet work as described or have practical applications. The author offers experience supporting genomics and building IT infrastructure and is available for consulting work.
Presentation that I delivered at "Accelerate AI, Europe 2018" in London on Sept 19, 2018. My focus is on socio-cultural perspective as well as proving information about various tools, vendors and partners available to help companies get started using AI.
How to make business using AI and Big Data?Ivano Digital
This document discusses artificial intelligence (AI), big data, the Internet of Things (IoT), and their applications and implications. It covers topics like AI reasoning, machine learning algorithms, data processing infrastructure, collecting data from IoT sensors, and potential AI applications in business, personal life, and society. It also addresses some challenges and barriers to adopting AI, as well as career opportunities working with data.
Top 5 Deep Learning and AI Stories - August 30, 2019NVIDIA
Read the top five news stories in artificial intelligence and learn how innovations in AI are transforming business across industries like healthcare and finance and how your business can derive tangible benefits by implementing AI the right way.
This document discusses the Industrial Internet of Things (IIOT) and provides an overview of what IIOT does, IIOT platforms, and how to build an IIOT strategy. Specifically, it notes that IIOT brings together disparate data formats, provides true real-time information, filters out repetitive data, integrates with multiple applications, gathers and distributes data, and provides benefits like reduced downtime. It also discusses challenges like operating environments, immaturity, lack of management capabilities, and vast amounts of useless data. The document outlines key capabilities of IIOT platforms and recommends defining desired business outcomes, integrating with existing applications and strategies, tearing down data silos, considering edge computing, and linking IIOT with predictive AI.
The document discusses three AI projects:
1) An algorithm to quickly assemble disaster relief teams for Team Rubicon by matching volunteers to tasks based on their skills and experience.
2) An attempt to prioritize customer service call queues based on customer lifetime value that was shelved due to shifting goals and lack of collaboration.
3) A contract intelligence assistant to recommend contract amendments that showed early success but was overtaken by changing priorities and lack of legal expertise.
It provides lessons learned from each, including the importance of clear project boundaries, collaboration between technical and business teams, and focusing on proven technologies versus hype. Bias and privacy are also discussed as ongoing challenges for AI.
AI Panel: AI in Practice- the Good, the Bad, and the UglyLucidworks
The document discusses AI in practice from three perspectives - the good, the bad, and the ugly. It outlines an agenda for a panel discussion on AI that will explore where AI has been successfully applied ("the good"), where it has broken or failed in practice ("the bad"), and how to avoid bias in developing and applying AI systems ("the ugly"). The panelists are experts from companies like Reddit, Slack, Github, and Lucidworks who will share their experiences and take questions from the audience.
(1) The document discusses several computer science topics including data science, artificial intelligence, and cloud computing. (2) It notes that data science has grown in popularity from 2012-2017 due to an ability to better process large volumes of data using statistics, specialized hardware, and contributions from companies. (3) Artificial intelligence aims to develop machines that can think and learn like humans, and this field has accelerated in recent years with improved data processing and hardware.
Any truly important technology creates fear and uncertainty. By this measure AI is going to become truly significant. However to achieve this we must close the cognitive concept gap.
The document discusses the industry buzz around big data and the cloud. It provides an agenda for a webinar on these topics, including challenges of big data, architectural solutions using the cloud, and case studies. The document notes that data is growing exponentially and coming from more sources faster, creating challenges around complexity, validity, and linking diverse data sources. It argues the cloud can help address these challenges by providing vast, correlated, high confidence data to drive real-time predictions and recommendations.
Decentralized AI: Convergence of AI + Blockchain geetachauhan
Santa Clara IoT Expo talk slides - convering convergence of of AI and Blockchain and how it solves challenges for IoT, Ai@Edge and Data Ethics and User Data Monetization
This Week in Data Science - Top 5 News - April 26, 2019NVIDIA
What's new in data science? Flip through this week's Top 5 to read a report on the most coveted skills for data scientists, top universities building AI labs, data science workstations for AI deployment, and more.
This document discusses the synergy between artificial intelligence (AI) and blockchain technology. It describes how blockchain provides a means of distributed data storage and consensus that can enable more efficient multi-agent AI systems. Blockchain allows for open data exchange in trustless environments and its distributed ledger is well-suited for applications like machine learning. The document also outlines how AI could power technologies like the Internet of Things that blockchain relies on.
Smart Data Webinar: The Road to Autonomous ApplicationsDATAVERSITY
Autonomous systems are like teenagers. The decision to trust one to complete a task without strict supervision depends on the individual and the task. Performance and trust are variables on spectra. As with teens, some autonomous systems will be ready before we are ready to trust them, and some will take a little longer.
As we get comfortable with delegating routine domestic tasks to home robots and prepare for a world with self-driving cars and beyond, it is important to understand the opportunities and limits for autonomous systems. Participants in this webinar will learn about the technologies that enable autonomous systems, and how to critically assess design constraints for independent and collaborating autonomous solutions.
Top 5 Deep Learning and AI Stories April 7th NVIDIA
Learn the state of AI technology, Wall Street predictions for AI investments, and how deep learning is quickly advancing medicine in this week's top 5.
Bridging the Gap: Analyzing Data in and Below the CloudInside Analysis
The Briefing Room with Dean Abbott and Tableau Software
Live Webcast July 23, 2013
http://www.insideanalysis.com
Today’s desire for analytics extends well beyond the traditional domain of Business Intelligence. That’s partly because business users are realizing the value of mixing and matching all kinds of data, from all kinds of sources. One emerging market driver is Cloud-based data, and the desire companies have to analyze this data cohesively with their on-premise data sets.
Register for this episode of The Briefing Room to learn from Analyst Dean Abbott, who will explain how the ability to access data in the cloud can play a critical role for generating business value from analytics. He’ll be briefed by Ellie Fields of Tableau Software who will tout Tableau’s latest release, which includes native connectors to cloud-based applications like Salesforce.com, Amazon Redshift, Google Analytics and BigQuery. She’ll also demonstrate how Tableau can combine cloud data with other data sources, including spreadsheets, databases, cubes and even Big Data.
MIT Enterprise Forum of Cambridge Connected Things 2017 panel discussion on "IoT Analytics: Using Analytics to Generate High Value from IoT in the Real World"
The document discusses inspiration for Internet of Things (IoT) solutions. It provides 10 examples of IoT solutions, including a smart baby monitor, smart bike, smart garbage cans, smart tennis racket, smart home devices, smart transportation, smart agriculture, smart healthcare, smart industry, and smart cities. It emphasizes that successful IoT requires a combination of products, applications, and services within a business ecosystem and cannot be done by any single player alone. IoT adoption may be limited more by difficulties finding the right business models than by technology challenges.
The document provides information about Ahmed Banafa's background and experience, including extensive experience in operations and management with a research background in various techniques and analysis. It also lists that he has taught at several universities and has received several awards for his work. It provides a brief introduction to big data and defines it as large and complex structured and unstructured data that cannot be processed by traditional database tools. It also discusses some of the roots and key aspects of big data like volume, velocity, and variety.
5 Ways a Digital-Ready Network can Transform your BusinessNatalie Andrusyk
The document discusses how a digital-ready network can transform a business in 5 ways: 1) enhance performance at the network edge, 2) use insights to create better experiences, 3) stay secure without compromising agility, 4) react and adapt faster to market trends, and 5) make the network the primary platform for business transformation. It also analyzes where organizations fall on a scale of digital readiness from "best effort" to "self-driving".
Computer Applications and Systems - Workshop VRaji Gogulapati
This document provides an overview of emerging technologies and their impact on businesses. It discusses how businesses are using new approaches like online collaborative communities and technologies to solve problems. It also covers topics like Enterprise 2.0, cloud computing, big data, analytics, social networking, collaboration tools, search engines, platforms, open source, e-learning and MOOCs. The document suggests that connectivity and data are driving new applications and experiences for consumers, and technologies are becoming the drivers of business success by enabling new ways of working and finding insights.
1) In-memory computing is growing rapidly, with the total data market expected to grow from $69 billion in 2015 to $132 billion in 2020.
2) In-memory databases are gaining popularity for applications that require fast response times, like telecommunications and mobile advertising, as memory access is faster than disk access.
3) Modern applications are driving adoption of in-memory solutions as they generate more data from more users and transactions and require faster performance to handle growing traffic.
4) Two examples presented were DellEMC using MemSQL for a real-time customer 360 application and an IoT logistics application called MemEx that processes sensor data from warehouses for predictive analytics.
This document discusses the Industrial Internet of Things (IIOT) and provides an overview of what IIOT does, IIOT platforms, and how to build an IIOT strategy. Specifically, it notes that IIOT brings together disparate data formats, provides true real-time information, filters out repetitive data, integrates with multiple applications, gathers and distributes data, and provides benefits like reduced downtime. It also discusses challenges like operating environments, immaturity, lack of management capabilities, and vast amounts of useless data. The document outlines key capabilities of IIOT platforms and recommends defining desired business outcomes, integrating with existing applications and strategies, tearing down data silos, considering edge computing, and linking IIOT with predictive AI.
The document discusses three AI projects:
1) An algorithm to quickly assemble disaster relief teams for Team Rubicon by matching volunteers to tasks based on their skills and experience.
2) An attempt to prioritize customer service call queues based on customer lifetime value that was shelved due to shifting goals and lack of collaboration.
3) A contract intelligence assistant to recommend contract amendments that showed early success but was overtaken by changing priorities and lack of legal expertise.
It provides lessons learned from each, including the importance of clear project boundaries, collaboration between technical and business teams, and focusing on proven technologies versus hype. Bias and privacy are also discussed as ongoing challenges for AI.
AI Panel: AI in Practice- the Good, the Bad, and the UglyLucidworks
The document discusses AI in practice from three perspectives - the good, the bad, and the ugly. It outlines an agenda for a panel discussion on AI that will explore where AI has been successfully applied ("the good"), where it has broken or failed in practice ("the bad"), and how to avoid bias in developing and applying AI systems ("the ugly"). The panelists are experts from companies like Reddit, Slack, Github, and Lucidworks who will share their experiences and take questions from the audience.
(1) The document discusses several computer science topics including data science, artificial intelligence, and cloud computing. (2) It notes that data science has grown in popularity from 2012-2017 due to an ability to better process large volumes of data using statistics, specialized hardware, and contributions from companies. (3) Artificial intelligence aims to develop machines that can think and learn like humans, and this field has accelerated in recent years with improved data processing and hardware.
Any truly important technology creates fear and uncertainty. By this measure AI is going to become truly significant. However to achieve this we must close the cognitive concept gap.
The document discusses the industry buzz around big data and the cloud. It provides an agenda for a webinar on these topics, including challenges of big data, architectural solutions using the cloud, and case studies. The document notes that data is growing exponentially and coming from more sources faster, creating challenges around complexity, validity, and linking diverse data sources. It argues the cloud can help address these challenges by providing vast, correlated, high confidence data to drive real-time predictions and recommendations.
Decentralized AI: Convergence of AI + Blockchain geetachauhan
Santa Clara IoT Expo talk slides - convering convergence of of AI and Blockchain and how it solves challenges for IoT, Ai@Edge and Data Ethics and User Data Monetization
This Week in Data Science - Top 5 News - April 26, 2019NVIDIA
What's new in data science? Flip through this week's Top 5 to read a report on the most coveted skills for data scientists, top universities building AI labs, data science workstations for AI deployment, and more.
This document discusses the synergy between artificial intelligence (AI) and blockchain technology. It describes how blockchain provides a means of distributed data storage and consensus that can enable more efficient multi-agent AI systems. Blockchain allows for open data exchange in trustless environments and its distributed ledger is well-suited for applications like machine learning. The document also outlines how AI could power technologies like the Internet of Things that blockchain relies on.
Smart Data Webinar: The Road to Autonomous ApplicationsDATAVERSITY
Autonomous systems are like teenagers. The decision to trust one to complete a task without strict supervision depends on the individual and the task. Performance and trust are variables on spectra. As with teens, some autonomous systems will be ready before we are ready to trust them, and some will take a little longer.
As we get comfortable with delegating routine domestic tasks to home robots and prepare for a world with self-driving cars and beyond, it is important to understand the opportunities and limits for autonomous systems. Participants in this webinar will learn about the technologies that enable autonomous systems, and how to critically assess design constraints for independent and collaborating autonomous solutions.
Top 5 Deep Learning and AI Stories April 7th NVIDIA
Learn the state of AI technology, Wall Street predictions for AI investments, and how deep learning is quickly advancing medicine in this week's top 5.
Bridging the Gap: Analyzing Data in and Below the CloudInside Analysis
The Briefing Room with Dean Abbott and Tableau Software
Live Webcast July 23, 2013
http://www.insideanalysis.com
Today’s desire for analytics extends well beyond the traditional domain of Business Intelligence. That’s partly because business users are realizing the value of mixing and matching all kinds of data, from all kinds of sources. One emerging market driver is Cloud-based data, and the desire companies have to analyze this data cohesively with their on-premise data sets.
Register for this episode of The Briefing Room to learn from Analyst Dean Abbott, who will explain how the ability to access data in the cloud can play a critical role for generating business value from analytics. He’ll be briefed by Ellie Fields of Tableau Software who will tout Tableau’s latest release, which includes native connectors to cloud-based applications like Salesforce.com, Amazon Redshift, Google Analytics and BigQuery. She’ll also demonstrate how Tableau can combine cloud data with other data sources, including spreadsheets, databases, cubes and even Big Data.
MIT Enterprise Forum of Cambridge Connected Things 2017 panel discussion on "IoT Analytics: Using Analytics to Generate High Value from IoT in the Real World"
The document discusses inspiration for Internet of Things (IoT) solutions. It provides 10 examples of IoT solutions, including a smart baby monitor, smart bike, smart garbage cans, smart tennis racket, smart home devices, smart transportation, smart agriculture, smart healthcare, smart industry, and smart cities. It emphasizes that successful IoT requires a combination of products, applications, and services within a business ecosystem and cannot be done by any single player alone. IoT adoption may be limited more by difficulties finding the right business models than by technology challenges.
The document provides information about Ahmed Banafa's background and experience, including extensive experience in operations and management with a research background in various techniques and analysis. It also lists that he has taught at several universities and has received several awards for his work. It provides a brief introduction to big data and defines it as large and complex structured and unstructured data that cannot be processed by traditional database tools. It also discusses some of the roots and key aspects of big data like volume, velocity, and variety.
5 Ways a Digital-Ready Network can Transform your BusinessNatalie Andrusyk
The document discusses how a digital-ready network can transform a business in 5 ways: 1) enhance performance at the network edge, 2) use insights to create better experiences, 3) stay secure without compromising agility, 4) react and adapt faster to market trends, and 5) make the network the primary platform for business transformation. It also analyzes where organizations fall on a scale of digital readiness from "best effort" to "self-driving".
Computer Applications and Systems - Workshop VRaji Gogulapati
This document provides an overview of emerging technologies and their impact on businesses. It discusses how businesses are using new approaches like online collaborative communities and technologies to solve problems. It also covers topics like Enterprise 2.0, cloud computing, big data, analytics, social networking, collaboration tools, search engines, platforms, open source, e-learning and MOOCs. The document suggests that connectivity and data are driving new applications and experiences for consumers, and technologies are becoming the drivers of business success by enabling new ways of working and finding insights.
1) In-memory computing is growing rapidly, with the total data market expected to grow from $69 billion in 2015 to $132 billion in 2020.
2) In-memory databases are gaining popularity for applications that require fast response times, like telecommunications and mobile advertising, as memory access is faster than disk access.
3) Modern applications are driving adoption of in-memory solutions as they generate more data from more users and transactions and require faster performance to handle growing traffic.
4) Two examples presented were DellEMC using MemSQL for a real-time customer 360 application and an IoT logistics application called MemEx that processes sensor data from warehouses for predictive analytics.
Big Data brings big promise and also big challenges, the primary and most important one being the ability to deliver Value to business stakeholders who are not data scientists!
Platform for Big Data Analytics and Visual Analytics: CSIRO use cases. Februa...Tomasz Bednarz
Presented at the ACEMS workshop at QUT in February 2015.
Credits: whole project team (names listed in the first slide).
Approved by CSIRO to be shared externally.
Sr. Jon Ander, Internet de las Cosas y Big Data: ¿hacia dónde va la Industria? INACAP
This document provides an introduction to Apache Spark, a cluster computing framework for large-scale data processing. It describes Spark as being faster and easier to use than Hadoop MapReduce, supporting a wide range of applications including SQL queries, streaming, machine learning, and graph processing. Key components of Spark include its core for scheduling tasks across a cluster, Spark SQL for structured data, Spark Streaming for real-time data, and MLlib for machine learning algorithms.
The document discusses Dell Technologies' artificial intelligence (AI) and data analytics solutions portfolio. It provides an overview of Dell's solutions for AI/machine learning, IoT/streaming data, augmented analytics/data warehousing, data lakes, and high-performance computing (HPC). The solutions leverage Dell infrastructure along with partner technologies and are designed to address various analytical use cases such as digital manufacturing, life sciences research, and retail loss prevention.
Moving Targets: Harnessing Real-time Value from Data in Motion Inside Analysis
The Briefing Room with David Loshin and Datawatch
Live Webcast Feb. 17, 2015
Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=4a053043c45cf0c2f6453dfb8577c72a
Patience may be a virtue, but when it comes to streaming analytics, waiting is no option. Between Big Data and the Internet of Things, businesses are faced with more data and greater complexity than ever before. Traditional information architectures simply cannot support the kind of processing necessary to make use of this fast-moving resource. The modern context requires a shorter path to analytics, one that narrows the gap between governance and discovery
Register for this episode of The Briefing Room to hear veteran Analyst David Loshin as he explains how the prevalence of streaming data is changing business pace and processes. He’ll be briefed by Dan Potter of Datawatch, who will tout his company’s real-time data discovery platform for data in motion. He will show how self-service data preparation can lead to faster insights, ultimately fostering the ability to make precise decisions at the right time.
Visit InsideAnalysis.com for more information.
This document provides an introduction to big data and Hadoop. It discusses the three V's of big data: volume, variety, and velocity. Examples are given of the large amounts of data generated daily from various sources. The growth and market opportunity for big data technologies is also discussed. Common use cases for big data in different industries are outlined. The document then covers Hadoop components and how Hadoop HDFS and MapReduce work. Other Hadoop technologies like Hive, Pig, and Zookeeper are introduced. Benefits of Hadoop and commercial Hadoop distributions are summarized. Finally, technologies alternative to Hadoop like HPCC and SAP HANA are briefly described.
This document provides an overview of big data presented by five individuals. It defines big data, discusses its three key characteristics of volume, velocity and variety. It explains how big data is stored, selected and processed using techniques like Hadoop and MapReduce. Examples of big data sources and tools are provided. Applications of big data across various industries are highlighted. Both the risks and benefits of big data are summarized. The future growth of big data and its impact on IT is also outlined.
AI in the Enterprise: Past, Present & Future - StampedeCon AI Summit 2017StampedeCon
This document discusses AI in the enterprise from past, present, and future perspectives. It provides an overview of the history and recent developments in AI and deep learning, including improved performance on tasks like image recognition. Case studies are presented showing how various large companies have successfully applied deep learning techniques like convolutional neural networks to problems in different industries involving computer vision, predictive maintenance, fraud detection, and more. The importance of data quantity for deep learning performance is highlighted. The final sections discuss challenges in AI adoption and the importance of piloting models before full production deployment.
This document provides an introduction and overview of big data technologies. It begins with defining big data and its key characteristics of volume, variety and velocity. It discusses how data has exploded in recent years and examples of large scale data sources. It then covers popular big data tools and technologies like Hadoop and MapReduce. The document discusses how to get started with big data and learning related skills. Finally, it provides examples of big data projects and discusses the objectives and benefits of working with big data.
In his keynote presentation titled Advanced Analytics for Any Data at Real-Time Speed, Dan Potter will discuss how a new-found ability to prepare, incorporate, enrich and visualize streaming data for advanced visual analysis is essential for making timelier, high-impact business decisions in tough competitive markets.
Advanced Analytics for Any Data at Real-Time Speeddanpotterdwch
The kenyote presentation from Predictive Analytics World entitled "Advanced Analytics for Any Data at Real-Time Speed" Dan Potter, CMO from Datawatch, presents a new approach to prepare, incorporate, enrich and visualize streaming data for advanced visual analysis is essential for making timelier, high-impact business decisions in tough competitive markets.
This document provides an overview of big data. It begins by defining big data and noting that it first emerged in the early 2000s among online companies like Google and Facebook. It then discusses the three key characteristics of big data: volume, velocity, and variety. The document outlines the large quantities of data generated daily by companies and sensors. It also discusses how big data is stored and processed using tools like Hadoop and MapReduce. Examples are given of how big data analytics can be applied across different industries. Finally, the document briefly discusses some risks and benefits of big data, as well as its impact on IT jobs.
This document discusses how big data can enable the travel and tourism industries. It defines big data as large datasets characterized by their volume, velocity, variety, and veracity. Big data comes from a variety of sources as people leave digital traces online and through mobile technologies. The benefits of big data for businesses include improved customer experience personalization, optimized marketing and products, predictive analytics, and risk management. The big data market is expected to double from 2014 to 2018. Future developments include improvements in data processing, centralized data repositories, and analytics solutions in the public cloud to reduce costs and security risks. Big data can deliver business insights, innovation, better customer relationships, and continuously improved experiences for the tourism industry.
Data science and its potential to change business as we know it. The Roadmap ...InnoTech
The document summarizes a presentation on data science and its potential to change business. It discusses how organizations can increase their data science maturity and capabilities to gain more value from data. As data volumes continue growing exponentially, data science can help organizations move from simple reporting to predictive analytics in order to make real-time decisions. The presentation examines how data science is an emerging field that incorporates techniques from many areas and how organizations can assess their analytics maturity.
Big data is still relatively new and it is very exciting. The opportunities, if not necessarily endless, are are at least incredibly rich and varied. Aiming to bridge the link between Big Data as a Technology and Big Data as Business Value, we hope our presentation will help frame some of your thinking on how to use and benefit from this topical development.
The document discusses knowledge graphs and provides examples of how Neo4j has been used by customers for knowledge graph and graph database applications. Specifically, it discusses how Neo4j has helped organizations like Itau Unibanco, UBS, Airbnb, Novartis, Columbia University, Telia, Scripps Networks, and Pitney Bowes with fraud detection, master data management, content management, smart home applications, investigative journalism, and other use cases by building knowledge graphs and connecting diverse data sources.
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...orcsab
This document discusses big data technologies, jobs, and opportunities in Hong Kong. It defines big data and its key components such as volume, velocity, variety and veracity. It outlines the major technologies involved like servers, storage, databases, visualization tools, and platforms like Hadoop. It discusses why big data is now possible due to cheap storage, abundant computing power, and data accessibility. It also examines career opportunities for data scientists and developers in both Hong Kong and the US and provides advice on getting involved in the big data community in Hong Kong.
The document discusses big data, defining it as large volumes of data from various sources that cannot be analyzed using traditional methods. It outlines three key characteristics of big data - volume, velocity and variety. Volume refers to the huge amount of data, velocity to the speed at which data is generated and processed, and variety to the different data types. The document also discusses how big data is stored, processed using tools like Hadoop, and analyzed to provide insights. It highlights some applications and risks of big data as well as its impact on IT.
Similar to Big Data World Singapore 2017 - Moving Towards Digitization & Artificial Intelligence (20)
Open Source Contributions to Postgres: The Basics POSETTE 2024ElizabethGarrettChri
Postgres is the most advanced open-source database in the world and it's supported by a community, not a single company. So how does this work? How does code actually get into Postgres? I recently had a patch submitted and committed and I want to share what I learned in that process. I’ll give you an overview of Postgres versions and how the underlying project codebase functions. I’ll also show you the process for submitting a patch and getting that tested and committed.
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...Kaxil Naik
Navigating today's data landscape isn't just about managing workflows; it's about strategically propelling your business forward. Apache Airflow has stood out as the benchmark in this arena, driving data orchestration forward since its early days. As we dive into the complexities of our current data-rich environment, where the sheer volume of information and its timely, accurate processing are crucial for AI and ML applications, the role of Airflow has never been more critical.
In my journey as the Senior Engineering Director and a pivotal member of Apache Airflow's Project Management Committee (PMC), I've witnessed Airflow transform data handling, making agility and insight the norm in an ever-evolving digital space. At Astronomer, our collaboration with leading AI & ML teams worldwide has not only tested but also proven Airflow's mettle in delivering data reliably and efficiently—data that now powers not just insights but core business functions.
This session is a deep dive into the essence of Airflow's success. We'll trace its evolution from a budding project to the backbone of data orchestration it is today, constantly adapting to meet the next wave of data challenges, including those brought on by Generative AI. It's this forward-thinking adaptability that keeps Airflow at the forefront of innovation, ready for whatever comes next.
The ever-growing demands of AI and ML applications have ushered in an era where sophisticated data management isn't a luxury—it's a necessity. Airflow's innate flexibility and scalability are what makes it indispensable in managing the intricate workflows of today, especially those involving Large Language Models (LLMs).
This talk isn't just a rundown of Airflow's features; it's about harnessing these capabilities to turn your data workflows into a strategic asset. Together, we'll explore how Airflow remains at the cutting edge of data orchestration, ensuring your organization is not just keeping pace but setting the pace in a data-driven future.
Session in https://budapestdata.hu/2024/04/kaxil-naik-astronomer-io/ | https://dataml24.sessionize.com/session/667627
Codeless Generative AI Pipelines
(GenAI with Milvus)
https://ml.dssconf.pl/user.html#!/lecture/DSSML24-041a/rate
Discover the potential of real-time streaming in the context of GenAI as we delve into the intricacies of Apache NiFi and its capabilities. Learn how this tool can significantly simplify the data engineering workflow for GenAI applications, allowing you to focus on the creative aspects rather than the technical complexities. I will guide you through practical examples and use cases, showing the impact of automation on prompt building. From data ingestion to transformation and delivery, witness how Apache NiFi streamlines the entire pipeline, ensuring a smooth and hassle-free experience.
Timothy Spann
https://www.youtube.com/@FLaNK-Stack
https://medium.com/@tspann
https://www.datainmotion.dev/
milvus, unstructured data, vector database, zilliz, cloud, vectors, python, deep learning, generative ai, genai, nifi, kafka, flink, streaming, iot, edge
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
We are pleased to share with you the latest VCOSA statistical report on the cotton and yarn industry for the month of March 2024.
Starting from January 2024, the full weekly and monthly reports will only be available for free to VCOSA members. To access the complete weekly report with figures, charts, and detailed analysis of the cotton fiber market in the past week, interested parties are kindly requested to contact VCOSA to subscribe to the newsletter.
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
Big Data World Singapore 2017 - Moving Towards Digitization & Artificial Intelligence
1. BIG DATA WORLD ASIA 2017
MOVING TOWARDS DIGITIZATION & ARTIFICIAL INTELLIGENCE
CONNECTING PEOPLE THROUGH THE WORLD OF RENOVATION
2. A few decades ago….
1. Saw an advertisement for a pair of
jeans you like on your way to work
(billboards, tv ads, newspapers).
2. Head to the store after work to make
the purchase.
3. Tell your friends about the new jeans
that you bought.
What has
changed now?
3. AGENDA
1. DIGITAL TRANSFORMATION
a) People
b) Infrastructure
c) Data
2. UNDERSTANDING ARTIFICIAL INTELLIGENCE
a) A Hype or Real
b) Domains
c) Phases of Deployment
3. CASE STUDIES
a) FinTech
b) InsurTech
c) RenoTalk
4. QUESTIONS AND ANSWERS
4. DIGITIZATION & TRANSFORMATION - PEOPLE
Data Scientist – tells the future
‘Geeks’ the brain for your AI
Design ML / Deep Learning Algorithms
Interact more with computer than humans
Data Analyst – big picture
Fact finders or Storytellers
Simple stats
Interact more with humans than computers
Data Engineer – make things happen
Deploy and integrate your AI solutions
Resource planner
Interact more with IT teams, especially backend
5. DIGITIZATION & TRANSFORMATION - INFRASTRUCTURE
• Near real time updates and monitoring. (e.g. Fraud Detection, Recommendation Engine)
• Periodic updates. (Churn Analysis, Marketing Prediction, Sales Forecast, Cancer/Disease Risk)
• Predict-On-Demand. (Credit Risk/Scoring, Leads Conversion)
• Storage:
• Hadoop Distributed File System (HDFS), Traditional RDBMS, AWS Redshift, AWS RDS/S3
instance, HBase.
• Architecture:
• Apache Spark (Near Real Time Analytics) e.g. SparkR, PySpark, H2O.
• HDInsights, HortonWorks, SpringXD
• Computational:
• Computational power – Number of CPU cores, GPUs, RAM memory
• Access & Governance:
• Who owns the data and who has access to the insights?
7. DIGITIZATION & TRANSFORMATION
Non Transactional
Transactional
Public and Social Media
4 Vs of Data
Data Scientist
Data Analyst
Data Engineer
Data Storage
Data Processing & ETLs
Data Access & Governance
Computational Resource
Real Time Processing
Visualization Tools
Data Modelling Tools
Deployment Tools
8. AI – A HYPE OR REAL?
A 2016 study by HubSpot Research, found that nearly 90 percent of
consumers around the globe are either interested in AI tools or are willing
to try them.
It is not about using or deploying AI that matters but the impact that the AI
initiatives can bring about to the business.
There are many variants of AI, diversity of domains, complexity of AI, and
the amount of data required to make AI happen.
Visualization and reporting tools are definitely not AI!
9. AI - DOMAINS
Supervised
Learning
• Predict what you
know
Unsupervised
Learning
• Predict what you
don’t know
Video Images
Analytics
• Specific ML/DL
technique
Natural
Language
Processing
• Specific processing
required
10. AI - PHASES OF DEPLOYMENT
• What has happened?
• Visualizations
• Exploratory Data
Analysis
DATA
COLLECTION
• What will happen?
• Machine Learning
• Deep learning
ARTIFICIAL
INTELLIGENCE
• What should happen?
• Actionable Insights
• Business Decisions
COGNITIVE
11. CASE STUDIES - FINTECH
TRADITIONAL CARD/LOAN APPLICATION PROCESS
Fill up form Submit form and docs Verifications Approve loans/CC
DIGITIZING CARD/LOAN APPLICATION PROCESSES WITH AI
Online form Take selfie with docs Credit/Risk scoring E-Wallet payments
12. CASE STUDIES - INSURTECH
Underwriters Actuarial Tables Broad profiles of risk premium for each persona
Lifestyle, Driving behaviors
Database
Fitbits
Car sensors
Premium Risk Bot
13. CASE STUDIES - RENOTALK
RenoTalk is Singapore’s leading online social networking platform connecting
people through the world of renovation.
Online portal with 13 years of accumulated user’s generated content on their
experiences, knowledge and journey on home renovation and interior designs.
Online
presence
since 2004
70,000
Forum members
682,000
Online user posts
80,000
Facebook fans
47,000
Total discussion topics
1.2 million
Monthly page views
180,000
Unique monthly visitors
53.3%
Repeat visitors
400
Industry leaders
14. CASE STUDIES - RENOTALK
How do home owners
find the right
information for their
question?
ChatBot
Natural
Language
Processing &
Deep Learning
m.me/renotalksg
16. THANK YOU
BIG DATA WORLD ASIA 2017
GARRETT TEOH HOR KEONG
CHIEF DATA OFFICER, RENOTALK
Editor's Notes
A very good afternoon ladies and gentleman, how is everyone doing? I hope everyone had enjoyed your lunch so please stay with me for the next 20 minutes as I will be walking you through on how to move towards digitization & artificial intelligence.
My name is Garrett and I am a data science professional, have been in the data science industry for more than 13 years, delivered data science projects for companies across various verticals which includes biomedical, government, social services and the financial institutions.
I am currently driving Renotalk’s digital transformation journey and exploring artificial intelligence initiatives to help them build their business up to the next level.
There has been a radical shift towards the commerce industry over the past few decades and this trend keeps on evolving from time to time.
I remembered my first experience as a shopper for a pair of jeans started off with the advertisement on magazines and billboards. After viewing the advertisement I head down to one of the retailer’s shop to make the purchase. A few years later after the internet boom era, I am able make purchase online without having the hassle of making a trip down to the store and I can buy anything and anytime I want. Whenever I need anything – from the most trivial item like a mouse, wireless keyboards, business suits when I need to present, dinner or lunch whenever I am hungry and too lazy to go out, or groceries if I feel like cooking. Take a look at the shopping cart, payment method, and the laptop
Over the time, these platforms seems so much smarter when they even make recommendations about the items I may need, for example, if I check out barbeque food items, the website will recommend me to get the grill mesh and charcoal wood. Sometimes, they will prompt me for my preferred delivery times and delivery address – whether home or office.
I will try to keep the agenda simple and easy to digest. First, I will introduce what are the key considerations to go digital in terms of the resources that are required for the transformation – which is the People, Technology and Data.
Next, it is very important to know if you are using the right AI for the right problem. I like to give an analogy to this with administering the right medicine to the symptom, if I caught a flu, all I need is a flu medicine, not 101 different pills which could make it worse.
Towards the end of my talk, I will present the case studies from the Fintech and Insurtech industry and round it up with a short preview of Renotalk’s digital transformation and AI journey. Last but not least, 5 minutes will be allocated for questions and answers.
The majority of the costs in moving towards digitization and AI is the people that the business invest in – with the assumption that you are going to build this project in house and not considering COTS. Risks are higher for COTS and I will explain why later. Due to the significant amount of costs invested in people, businesses need to choose and hire the right talent.
The first person you want to associate with AI and Deep Learning algorithms are the Data Scientists and this is hyped as the sexiest job of the 21st century by Forbes/Harvard Business Review. Why? For 2 main reasons 1) They are rare talents – someone who can talk in maths, stats, and equations language. They are the ‘mechanic’ for all your machines and they derive patterns from your data.
You need someone who can translate insights into business context and tell a story that is easy to digest. Meet your Data Analyst who tells you the big picture. They don’t use fanciful mathematics and algorithms but use simple elementary stats to represent your data and insights.
Finally, there will be someone who is able to put your AI machines and visualizations into production and integrate them with your business processes.
Understanding the different domains of Artificial Intelligence is key for a successful Big Data project. Not knowing the domains and using them inappropriately is one of the most common cause for failing.
With so much information on our portal, how do home owners deal with finding the right ‘answer’ for their question in the quickest time? The response time in answering the user’s questions is of paramount for retention and it will affects your website’s bounce rate and repeated visitors statistics. How do Renotalk leverage on digitization and AI to solve this problem? Will having a customer service officer to answer questions solve this problem? Yes, it will solve this problem with a caveat: that this CSO is able to handle all type of questions that the users ask and the possibility to simultaneously chat with a few home owners.
Can we automate this task by training an chatbot to handle the FAQs? The answer is yes, but it comes with a caveat too: it takes time to train the chat bot. The AI application domain is known as the Natural Language Processing.
What is NLP? It is the application of computational and linguistics analysis for computers to understand human language and thus capable to interact with humans. Due to the complexity of language semantics and corpora, deep learning algorithms are fundamentally essential for a chatbot to be robust and appear ‘human’. For the sake of time, I wont go into the details of how NLP and Deep Learning is used in the cognitive process of training the chatbots.
We are currently developing RenoBot in the pipeline, you can find our chatbot from our website on the bottom right of the screen. To initiate a conversation from your mobile phone, simply just ask a question and it will respond with a reply to your FB messenger. As this is work in progress, please try this out with a pinch of salt.
If there are no more questions I would like to wrap up the session with a few key takeaways. To make your move towards digitization and AI, you will need to consider first,
The business problem and objectives
Whether you have the right data
Then bring in the people and infrastructure
Know your AI domain and build the right solution
How to act upon the insights