Bridging the Gap Between Data and Insight using Open-Source Tools
•Download as PPT, PDF•
0 likes•640 views
O
odscDespite the proliferation of open-source tools for analysis (such as Python and R) and those used for visualization (such as Javascript / D3), there often exist significant gaps between these areas, and those of us trying to navigate the complete arc from data to insight can encounter many obstacles along the way. Fortunately, in recent years there have been many efforts to fill these needs, and today distilling a meaningful visualization from raw data is faster and easier than ever before. In this talk we will use will use examples in geospatial analysis and visualization to illustrate how to open-source tools like Python, geopandas, and TileMill work together. Using examples from the RunKeeper mobile app we will show how we currently use these tools to understand better our customers and their data, and to communicate with our colleagues, external partners, and the data community at large.
Report
Share
Report
Share
Recommended
Top 10 Signs of the Textpocalypse
Human-generated text may be the next frontier for big data analysis, but we humans are complicated beasts and the text we generate is messy and complicated in ways that can confound analysis. We’ll describe the top ten mistakes people make when they start doing text analysis, and hopefully save you from making a few of these mistakes yourself.
Machine Learning with scikit-learn
scikit-learn has emerged as one of the most popular open source machine learning toolkits, now widely used in academia and industry.
scikit-learn provides easy-to-use interfaces to perform advanced analysis and build powerful predictive models.
The tutorial will cover basic concepts of machine learning, such as supervised and unsupervised learning, cross validation, and model selection. We will see how to prepare data for machine learning, and go from applying a single algorithm to building a machine learning pipeline.
We will also cover how to build machine learning models on text data, and how to handle very large datasets.
Uplift Modeling Workshop
Traditional randomized experiments allow us to determine the overall causal impact of a treatment program (e.g. marketing, medical, social, education, political). Uplift modeling (also known as true lift, net lift, incremental lift) takes a further step to identify individuals who are truly positively influenced by a treatment through data mining / machine learning. This technique allows us to identify the “persuadables” and thus optimize target selection in order to maximize treatment benefits. This important subfield of data mining/data science/business analytics has gained significant attention in areas such as personalized marketing, personalized medicine, and political election with plenty of publications and presentations appeared in recent years from both industry practitioners and academics.
In this workshop, I will introduce the concept of Uplift, review existing methods, contrast with the traditional approach, and introduce a new method that can be implemented with standard software. A method and metrics for model assessment will be recommended. Our discussion will include new approaches to handling a general situation where only observational data are available, i.e. without randomized experiments, using techniques from causal inference. Additionally, an integrated modeling approach for uplift and direct response (where it can be identified who actually responded, e.g., click-through or coupon scanning) will be discussed. Last but not least, extension to the multiple treatment situation with solutions to optimizing treatments at the individual level will also be discussed. While the talk is geared towards marketing applications (“personalized marketing”), the same methodologies can be readily applied in other fields such as insurance, medicine, education, political, and social programs. Examples from the retail and non-profit industries will be used to illustrate the methodologies.
Understanding the Chief Data Officer
As the importance of having a data strategy in place is sinking in, many organizations have added a chief data officer (CDO) to their executive team to help create and implement that strategy. But every organization is doing this a little bit differently. This talk will describe how a variety of industries and organizations are using CDOs and will make recommendations for best practices.
Machine-In-The-Loop for Knowledge Discovery
I’ll present the new knowledge discovery tools we are building at Diffeo. Unlike traditional search engines that use keywords, Diffeo provides an in-browser knowledge base that accelerates information gathering about people, companies, chemical compounds, cyber events, or other real world entities. I’ll describe how Diffeo uses active learning to encourage long and deep user interactions in order to recommend new content for in-progress articles. As you write, the search results get better and more interesting, because the system can see more precisely which entity you mean and which you don’t (disambiguation) and also what you don’t know yet about the entity (discovery).
Finally in this presentation I’ll describe our experience organizing the Text REtrieval Conference (TREC) on Knowledge Base Acceleration (KBA) and Dynamic Domain (DD) which are pushing the state of the art in knowledge discovery on large streams. I’ll show you how to access the largest corpus of streaming text data ever released for public evaluations.
API Driven Development
An exposé on human-centered design, as related to data science and “medium data”. Examples of great API design will be showcased, as well as other end-user facing tools that can enable data scientists to share their observations with the world.
Mobile technology Usage by Humanitarian Programs: A Metadata Analysis
CommCare, developed by Dimagi Inc., is an open-source mobile technology platform that supports hundreds of humanitarian frontline programs worldwide. The objective of this analysis is to demonstrate how CommCare metadata contains a wealth of information that can inform humanitarian programs in their use of mobile technology. This understanding can help programs determine the most effective way to implement CommCare or other mobile technology in resource-poor settings. A typical CommCare user is a frontline worker, such as a community health worker who provides outreach to pregnant women and children. An important feature of CommCare is that it supports case management, allowing users to register, update, and close cases in their CommCare application. A case is usually a user’s client, e.g., a pregnant woman who is supported by the CommCare user. While using CommCare, the user fills out electronic forms which eventually get submitted to the CommCare cloud server. The cumulative number of forms submitted by CommCare users as of December 2014 was just over 10 million. Metadata for each form submitted through CommCare are stored in Dimagi’s data platform; included in a form’s metadata are date and time stamps for when each form was started and ended by the user and when the form was eventually received by the cloud server.
Recommended
Top 10 Signs of the Textpocalypse
Human-generated text may be the next frontier for big data analysis, but we humans are complicated beasts and the text we generate is messy and complicated in ways that can confound analysis. We’ll describe the top ten mistakes people make when they start doing text analysis, and hopefully save you from making a few of these mistakes yourself.
Machine Learning with scikit-learn
scikit-learn has emerged as one of the most popular open source machine learning toolkits, now widely used in academia and industry.
scikit-learn provides easy-to-use interfaces to perform advanced analysis and build powerful predictive models.
The tutorial will cover basic concepts of machine learning, such as supervised and unsupervised learning, cross validation, and model selection. We will see how to prepare data for machine learning, and go from applying a single algorithm to building a machine learning pipeline.
We will also cover how to build machine learning models on text data, and how to handle very large datasets.
Uplift Modeling Workshop
Traditional randomized experiments allow us to determine the overall causal impact of a treatment program (e.g. marketing, medical, social, education, political). Uplift modeling (also known as true lift, net lift, incremental lift) takes a further step to identify individuals who are truly positively influenced by a treatment through data mining / machine learning. This technique allows us to identify the “persuadables” and thus optimize target selection in order to maximize treatment benefits. This important subfield of data mining/data science/business analytics has gained significant attention in areas such as personalized marketing, personalized medicine, and political election with plenty of publications and presentations appeared in recent years from both industry practitioners and academics.
In this workshop, I will introduce the concept of Uplift, review existing methods, contrast with the traditional approach, and introduce a new method that can be implemented with standard software. A method and metrics for model assessment will be recommended. Our discussion will include new approaches to handling a general situation where only observational data are available, i.e. without randomized experiments, using techniques from causal inference. Additionally, an integrated modeling approach for uplift and direct response (where it can be identified who actually responded, e.g., click-through or coupon scanning) will be discussed. Last but not least, extension to the multiple treatment situation with solutions to optimizing treatments at the individual level will also be discussed. While the talk is geared towards marketing applications (“personalized marketing”), the same methodologies can be readily applied in other fields such as insurance, medicine, education, political, and social programs. Examples from the retail and non-profit industries will be used to illustrate the methodologies.
Understanding the Chief Data Officer
As the importance of having a data strategy in place is sinking in, many organizations have added a chief data officer (CDO) to their executive team to help create and implement that strategy. But every organization is doing this a little bit differently. This talk will describe how a variety of industries and organizations are using CDOs and will make recommendations for best practices.
Machine-In-The-Loop for Knowledge Discovery
I’ll present the new knowledge discovery tools we are building at Diffeo. Unlike traditional search engines that use keywords, Diffeo provides an in-browser knowledge base that accelerates information gathering about people, companies, chemical compounds, cyber events, or other real world entities. I’ll describe how Diffeo uses active learning to encourage long and deep user interactions in order to recommend new content for in-progress articles. As you write, the search results get better and more interesting, because the system can see more precisely which entity you mean and which you don’t (disambiguation) and also what you don’t know yet about the entity (discovery).
Finally in this presentation I’ll describe our experience organizing the Text REtrieval Conference (TREC) on Knowledge Base Acceleration (KBA) and Dynamic Domain (DD) which are pushing the state of the art in knowledge discovery on large streams. I’ll show you how to access the largest corpus of streaming text data ever released for public evaluations.
API Driven Development
An exposé on human-centered design, as related to data science and “medium data”. Examples of great API design will be showcased, as well as other end-user facing tools that can enable data scientists to share their observations with the world.
Mobile technology Usage by Humanitarian Programs: A Metadata Analysis
CommCare, developed by Dimagi Inc., is an open-source mobile technology platform that supports hundreds of humanitarian frontline programs worldwide. The objective of this analysis is to demonstrate how CommCare metadata contains a wealth of information that can inform humanitarian programs in their use of mobile technology. This understanding can help programs determine the most effective way to implement CommCare or other mobile technology in resource-poor settings. A typical CommCare user is a frontline worker, such as a community health worker who provides outreach to pregnant women and children. An important feature of CommCare is that it supports case management, allowing users to register, update, and close cases in their CommCare application. A case is usually a user’s client, e.g., a pregnant woman who is supported by the CommCare user. While using CommCare, the user fills out electronic forms which eventually get submitted to the CommCare cloud server. The cumulative number of forms submitted by CommCare users as of December 2014 was just over 10 million. Metadata for each form submitted through CommCare are stored in Dimagi’s data platform; included in a form’s metadata are date and time stamps for when each form was started and ended by the user and when the form was eventually received by the cloud server.
Big Data Infrastructure: Introduction to Hadoop with MapReduce, Pig, and Hive
The main objective of this workshop is to give the audience hands on experience with several Hadoop technologies and jump start their hadoop journey. In this workshop, you will load data and submit queries using Hadoop! Before jumping in to the technology, the Founders of DataKitchen review Hadoop and some of its technologies (MapReduce, Hive, Pig, Impala and Spark), look at performance, and present a rubric for choosing which technology to use when.
Think Breadth, Not Depth
We’ve all been told to “work smarter, not harder.” But what does working smarter really mean? In the world of finance and trading, working smarter means working differently. None of us can compete against computers stacked inches away from the stock exchange or blue chip companies with multi-million dollar marketing campaigns. The key to winning is to go where the big guys haven’t and the way to do that is through diverse datasets. In this talk, you will discover the theory and tools to discover new datasets from unexpected sources in order to gain an upper-hand in both finance and business. So whether you’re a quant that trades in his bedroom or a restaurateur looking to grow his business, you’ll learn how the diversity of data can be the sharpest knife if your set.
Data Science at Dow Jones: Monetizing Data, News and Information
In this presentation I will describe the way Data Science supports the business of information and news at Dow Jones. Specifically, I will describe how we are introducing innovative and advanced large-scale information mining and analytic approaches not only into Dow Jones’ products but also into our strategy and decision making processes.Our goal is to impact every aspect of Dow Jones: from the way journalism is produced in the newsroom, to the way we create and deliver institutional products, to the way we improve retention and acquisition of subscribers. While the task seems broad and daunting, we have already achieved various successes through the application of machine learning, data mining, advanced analytics and big data approaches.In this presentation I will describe how we have achieved this, including our tools, data, approaches and mechanisms as well as describe what our plans are going forward.
Spark, Python and Parquet
Have you been in the situation where you’re about to start a new project and ask yourself, what’s the right tool for the job here? I’ve been in that situation many times and thought it might be useful to share with you a recent project we did and why we selected Spark, Python, and Parquet. My plan is take you through a use case that involves loading, transforming, aggregating, and persisting the dataset. We’ll use an open dataset consisting of full fund holdings graciously provided by Morningstar. My goal in presenting this use case are to have the audience learn about how these technologies can be applied to a real world problem and to inspire members of the audience to start learning these technologies and applying them to their own projects.
Building a Predictive Analytics Solution with Azure ML
Create and operationalize a predictive model using Microsoft Azure Machine Learning.
– Perform the typical steps involved in building a predictive analytics solution such as data ingestion, data cleansing, data exploration, feature engineering, model selection and evaluation of model results
–learn how to use machine learning with big data scenarios using tools like Hadoop and SQL Server to process and work with such data.
Beyond Names
Finding and classifying the mentions of the things named in text, often called Named Entity Recognition or NER, is a fundamental task in many search and analysis applications. Mature, robust NER technology is available for many languages and domains, from people, places, and products, to diseases, genes, and molecules. However, for emerging tasks like knowledge-base construction, mentions alone are insufficient.
In this presentation we’ll explore techniques that go beyond names to:
link mentions to one another and to rich knowledge sources like Wikidata
discover and characterise the relationships between entities that are explicit in the text
And we’ll discuss some of the most important practical implications of these advancements for open data science.
How Woman are Conquering the S&P 500
According to Credit Suisse’s Gender 3000 report, at the end of 2013, women accounted for 12.9% of top management in 3000 companies across 40 countries. However, since 2009, companies with women as 25-50% of their management team
returned 22-29%. If companies with women in management outperform so dramatically, what would happen if you invested in women-led companies? Karen Rubin will explore this question and share her findings after running a 12 year investment simulation.
Domain Expertise and Unstructured Data
Data science allows us to turn a dark forest into a world of
perpetual twilight by giving us the tools to better understand the data that surrounds us. Unfortunately, in this world of twilight we still need a flashlight to get a clean crisp image of our immediate surroundings. We will talk about how to use deep domain expertise as that flashlight shedding light on our understanding of data. Our focus will be on using text analysis as a means to examine qualitative information in a structured, quantitative way. We will draw heavily from examples in complex central bank policy and financial regulation.
Open Source Tools & Data Science Competitions
This talk shares the presenter’s experience with open source tools in data science competitions. In the past several years Kaggle and other competitions have created a large online community of data scientists. In addition to competing with each other for fame and glory, members of this community also generously share knowledge, insights using forum and open source code. The open competition and sharing have resulted in rapid progress in the sophistication of the entire community. This presentation will briefly cover this journey from a competitor’s perspective, and share hands on tips on some open source tools proven popular and useful in recent competitions.
Feature Engineering
One of the most important, yet often overlooked, aspects of predictive modeling is the transformation of data to create model inputs, better known as feature engineering (FE). This talk will go into the theoretical background behind FE, showing how it leverages existing data to produce better modeling results. It will then detail some important FE techniques that should be in every data scientist’s tool kit.
Agile Data
To rephrase an old saying: ‘It takes a village to raise an Analyst.’ Data Analysts and Scientists are working in teams delivering insight and analysis on an ongoing basis. So how do you get the team to support experimentation and insight delivery without ending up in an IT Engineer vs Analyst vs Data Governance war? We present 5 shocking steps to get these teams of people working together with practical, doable steps that can help you achieve data agility. The speaker has decades of hands on and executive management experience in data, analytics, and software development.
Using your powers for good: Data science in the social sector
Just like every major corporation today, nonprofits and governments have more data than ever before. And just like those corporations, they are eager to tap into the power of their data. But the social sector doesn’t have the same resources to attract talent. Jeff Hammerbacher, Chief Scientist at Cloudera, put it best: “The best minds of my generation are thinking about how to make people click ads. That sucks.” At DrivenData our goal is to make the world suck a little less by empowering impact organizations to get the most from their data.
Peter Bull, co-founder at DrivenData, will speak on the ways in which statistics, computer science, and machine learning can be applied to the challenges in the social sector. The talk will address both the big-picture context of the data for good movement, and an in-depth case study of the methods which won DrivenData’s recent machine learning competitions.
It’s an exciting time for people who love data: methods are improving, computational costs are decreasing, storage and transport are cheaper, and the talent pool is growing. It’s up to the data geeks to use these powers for good.
Machine Learning for Suits
You will learn the basic concepts of machine learning – such as Modeling, Model Selection, Loss or Profit, overfitting, and validation – in a non-mathematical way, so that you can ask for data analysis and interpret the results of a model in the context of making business decisions. The concepts behind machine learning are actually quite simple, so expect to take away not just words and acronyms, but rather, a deep understanding. We will work in the context of concrete examples from different domains, including finance and medicine.
What is probability? What is a model? Supervised vs unsupervised learning. Regression and Classification.
Models and Data: Bias, Variance, Noise, Overfitting, and how to solve Overfitting with Regularization and Validation
Risk and Cost, Data snooping, asymmetry in data, experiments.
How good is a model? Profit Curves, ROC curves, and the expected value formalism.
Recurrent Neural Networks for Text Analysis
Recurrent Neural Networks hold great promise as general sequence learning algorithms. As such, they are a very promising tool for text analysis. However, outside of very specific use cases such as handwriting recognition and recently, machine translation, they have not seen wide spread use. Why has this been the case?
In this presentation, we will first introduce RNNs as a concept. Then we will sketch how to implement them and cover the tricks necessary to make them work well. With the basics covered, we will investigate using RNNs as general text classification and regression models, examining where they succeed and where they fail compared to more traditional text analysis models. A straightforward open-source Python and Theano library for training RNNs with a scikit-learn style interface will be introduced and we’ll see how to use it through a tutorial on a real world text dataset
Predictive Modeling Workshop
The workshop is an overview of creating predictive models using R. An example data set will be used to demonstrate a typical workflow: data splitting, pre-processing, model tuning and evaluation. Several R packages will be shown along with the caret package which provides a unified interface to a large number of R’s modeling functions and enables parallel processing. Participants should have a basic understanding of R data structures and basic language elements (i.e. functions, classes, etc).
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
From social networks to protein networks to financial transactions, graphs are everywhere. Graph Analytics represent a key tool for data science to take advance of this type of network information. Many “Bigdata” and NoSQL techniques for analysis and data science that work well for relational and structured data, do not scale effectively when applied to challenges in graph analytics and traversal algorithms. The data locality and graph access patterns challenge existing HW architectures and place a premium on bandwidth to main memory.GPUs currently have 10X advantage over CPUs in this area.
The advantage is projected to grow to 100X by 2016. This talk will discuss why GPUs are game-changer by dramatically improving the price-performance ratio for very large graph analytics over existing technologies. It will present results for work in GPU Acceleration of graph analytics within both research and industry applications.
Can We Automate Predictive Analytics
Recent news about the pending shortage of data scientists prompts speculation about automation: will machines replace human analysts? We propose a model of automation, and briefly review progress in automated machine learning over the past twenty years. Summarizing the current state of the art, we look at some of the remaining challenges, and the implications for practicing data scientists.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
More Related Content
More from odsc
Big Data Infrastructure: Introduction to Hadoop with MapReduce, Pig, and Hive
The main objective of this workshop is to give the audience hands on experience with several Hadoop technologies and jump start their hadoop journey. In this workshop, you will load data and submit queries using Hadoop! Before jumping in to the technology, the Founders of DataKitchen review Hadoop and some of its technologies (MapReduce, Hive, Pig, Impala and Spark), look at performance, and present a rubric for choosing which technology to use when.
Think Breadth, Not Depth
We’ve all been told to “work smarter, not harder.” But what does working smarter really mean? In the world of finance and trading, working smarter means working differently. None of us can compete against computers stacked inches away from the stock exchange or blue chip companies with multi-million dollar marketing campaigns. The key to winning is to go where the big guys haven’t and the way to do that is through diverse datasets. In this talk, you will discover the theory and tools to discover new datasets from unexpected sources in order to gain an upper-hand in both finance and business. So whether you’re a quant that trades in his bedroom or a restaurateur looking to grow his business, you’ll learn how the diversity of data can be the sharpest knife if your set.
Data Science at Dow Jones: Monetizing Data, News and Information
In this presentation I will describe the way Data Science supports the business of information and news at Dow Jones. Specifically, I will describe how we are introducing innovative and advanced large-scale information mining and analytic approaches not only into Dow Jones’ products but also into our strategy and decision making processes.Our goal is to impact every aspect of Dow Jones: from the way journalism is produced in the newsroom, to the way we create and deliver institutional products, to the way we improve retention and acquisition of subscribers. While the task seems broad and daunting, we have already achieved various successes through the application of machine learning, data mining, advanced analytics and big data approaches.In this presentation I will describe how we have achieved this, including our tools, data, approaches and mechanisms as well as describe what our plans are going forward.
Spark, Python and Parquet
Have you been in the situation where you’re about to start a new project and ask yourself, what’s the right tool for the job here? I’ve been in that situation many times and thought it might be useful to share with you a recent project we did and why we selected Spark, Python, and Parquet. My plan is take you through a use case that involves loading, transforming, aggregating, and persisting the dataset. We’ll use an open dataset consisting of full fund holdings graciously provided by Morningstar. My goal in presenting this use case are to have the audience learn about how these technologies can be applied to a real world problem and to inspire members of the audience to start learning these technologies and applying them to their own projects.
Building a Predictive Analytics Solution with Azure ML
Create and operationalize a predictive model using Microsoft Azure Machine Learning.
– Perform the typical steps involved in building a predictive analytics solution such as data ingestion, data cleansing, data exploration, feature engineering, model selection and evaluation of model results
–learn how to use machine learning with big data scenarios using tools like Hadoop and SQL Server to process and work with such data.
Beyond Names
Finding and classifying the mentions of the things named in text, often called Named Entity Recognition or NER, is a fundamental task in many search and analysis applications. Mature, robust NER technology is available for many languages and domains, from people, places, and products, to diseases, genes, and molecules. However, for emerging tasks like knowledge-base construction, mentions alone are insufficient.
In this presentation we’ll explore techniques that go beyond names to:
link mentions to one another and to rich knowledge sources like Wikidata
discover and characterise the relationships between entities that are explicit in the text
And we’ll discuss some of the most important practical implications of these advancements for open data science.
How Woman are Conquering the S&P 500
According to Credit Suisse’s Gender 3000 report, at the end of 2013, women accounted for 12.9% of top management in 3000 companies across 40 countries. However, since 2009, companies with women as 25-50% of their management team
returned 22-29%. If companies with women in management outperform so dramatically, what would happen if you invested in women-led companies? Karen Rubin will explore this question and share her findings after running a 12 year investment simulation.
Domain Expertise and Unstructured Data
Data science allows us to turn a dark forest into a world of
perpetual twilight by giving us the tools to better understand the data that surrounds us. Unfortunately, in this world of twilight we still need a flashlight to get a clean crisp image of our immediate surroundings. We will talk about how to use deep domain expertise as that flashlight shedding light on our understanding of data. Our focus will be on using text analysis as a means to examine qualitative information in a structured, quantitative way. We will draw heavily from examples in complex central bank policy and financial regulation.
Open Source Tools & Data Science Competitions
This talk shares the presenter’s experience with open source tools in data science competitions. In the past several years Kaggle and other competitions have created a large online community of data scientists. In addition to competing with each other for fame and glory, members of this community also generously share knowledge, insights using forum and open source code. The open competition and sharing have resulted in rapid progress in the sophistication of the entire community. This presentation will briefly cover this journey from a competitor’s perspective, and share hands on tips on some open source tools proven popular and useful in recent competitions.
Feature Engineering
One of the most important, yet often overlooked, aspects of predictive modeling is the transformation of data to create model inputs, better known as feature engineering (FE). This talk will go into the theoretical background behind FE, showing how it leverages existing data to produce better modeling results. It will then detail some important FE techniques that should be in every data scientist’s tool kit.
Agile Data
To rephrase an old saying: ‘It takes a village to raise an Analyst.’ Data Analysts and Scientists are working in teams delivering insight and analysis on an ongoing basis. So how do you get the team to support experimentation and insight delivery without ending up in an IT Engineer vs Analyst vs Data Governance war? We present 5 shocking steps to get these teams of people working together with practical, doable steps that can help you achieve data agility. The speaker has decades of hands on and executive management experience in data, analytics, and software development.
Using your powers for good: Data science in the social sector
Just like every major corporation today, nonprofits and governments have more data than ever before. And just like those corporations, they are eager to tap into the power of their data. But the social sector doesn’t have the same resources to attract talent. Jeff Hammerbacher, Chief Scientist at Cloudera, put it best: “The best minds of my generation are thinking about how to make people click ads. That sucks.” At DrivenData our goal is to make the world suck a little less by empowering impact organizations to get the most from their data.
Peter Bull, co-founder at DrivenData, will speak on the ways in which statistics, computer science, and machine learning can be applied to the challenges in the social sector. The talk will address both the big-picture context of the data for good movement, and an in-depth case study of the methods which won DrivenData’s recent machine learning competitions.
It’s an exciting time for people who love data: methods are improving, computational costs are decreasing, storage and transport are cheaper, and the talent pool is growing. It’s up to the data geeks to use these powers for good.
Machine Learning for Suits
You will learn the basic concepts of machine learning – such as Modeling, Model Selection, Loss or Profit, overfitting, and validation – in a non-mathematical way, so that you can ask for data analysis and interpret the results of a model in the context of making business decisions. The concepts behind machine learning are actually quite simple, so expect to take away not just words and acronyms, but rather, a deep understanding. We will work in the context of concrete examples from different domains, including finance and medicine.
What is probability? What is a model? Supervised vs unsupervised learning. Regression and Classification.
Models and Data: Bias, Variance, Noise, Overfitting, and how to solve Overfitting with Regularization and Validation
Risk and Cost, Data snooping, asymmetry in data, experiments.
How good is a model? Profit Curves, ROC curves, and the expected value formalism.
Recurrent Neural Networks for Text Analysis
Recurrent Neural Networks hold great promise as general sequence learning algorithms. As such, they are a very promising tool for text analysis. However, outside of very specific use cases such as handwriting recognition and recently, machine translation, they have not seen wide spread use. Why has this been the case?
In this presentation, we will first introduce RNNs as a concept. Then we will sketch how to implement them and cover the tricks necessary to make them work well. With the basics covered, we will investigate using RNNs as general text classification and regression models, examining where they succeed and where they fail compared to more traditional text analysis models. A straightforward open-source Python and Theano library for training RNNs with a scikit-learn style interface will be introduced and we’ll see how to use it through a tutorial on a real world text dataset
Predictive Modeling Workshop
The workshop is an overview of creating predictive models using R. An example data set will be used to demonstrate a typical workflow: data splitting, pre-processing, model tuning and evaluation. Several R packages will be shown along with the caret package which provides a unified interface to a large number of R’s modeling functions and enables parallel processing. Participants should have a basic understanding of R data structures and basic language elements (i.e. functions, classes, etc).
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
From social networks to protein networks to financial transactions, graphs are everywhere. Graph Analytics represent a key tool for data science to take advance of this type of network information. Many “Bigdata” and NoSQL techniques for analysis and data science that work well for relational and structured data, do not scale effectively when applied to challenges in graph analytics and traversal algorithms. The data locality and graph access patterns challenge existing HW architectures and place a premium on bandwidth to main memory.GPUs currently have 10X advantage over CPUs in this area.
The advantage is projected to grow to 100X by 2016. This talk will discuss why GPUs are game-changer by dramatically improving the price-performance ratio for very large graph analytics over existing technologies. It will present results for work in GPU Acceleration of graph analytics within both research and industry applications.
Can We Automate Predictive Analytics
Recent news about the pending shortage of data scientists prompts speculation about automation: will machines replace human analysts? We propose a model of automation, and briefly review progress in automated machine learning over the past twenty years. Summarizing the current state of the art, we look at some of the remaining challenges, and the implications for practicing data scientists.
More from odsc (20)
Big Data Infrastructure: Introduction to Hadoop with MapReduce, Pig, and Hive
Big Data Infrastructure: Introduction to Hadoop with MapReduce, Pig, and Hive
Data Science at Dow Jones: Monetizing Data, News and Information
Data Science at Dow Jones: Monetizing Data, News and Information
Building a Predictive Analytics Solution with Azure ML
Building a Predictive Analytics Solution with Azure ML
Using your powers for good: Data science in the social sector
Using your powers for good: Data science in the social sector
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
Recently uploaded
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
The Art of the Pitch: WordPress Relationships and Sales
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
UiPath Test Automation using UiPath Test Suite series, part 4
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
“AGI should be open source and in the public domain at the service of humanity and the planet.”
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Accelerate your Kubernetes clusters with Varnish Caching
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Recently uploaded (20)
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Bridging the Gap Between Data and Insight using Open-Source Tools
- 1. BRIDGING THE GAP BETWEEN DATA & INSIGHT USING OPEN SOURCE TOOLS Nicholas Arcolano O P E N D A T A S C I E N C E C O N F E R E N C E_ BOSTON 2015 @opendat asci