Making Investing A Science
•Download as PPTX, PDF•
2 likes•239 views
The document discusses the benefits of meditation for reducing stress and anxiety. Regular meditation practice can help calm the mind and body by lowering heart rate and blood pressure. Studies have shown that meditating for just 10-20 minutes per day can have significant positive impacts on both mental and physical health.
Report
Share
Report
Share
Recommended
What's in your workflow? Bringing data science workflows to business analysis...
While business analysis rapidly grows more data-driven, the analyst community is slow to adapt the best practices of data science workflows. Many parallels exists between data science “top topics” (e.g. reproducibility) and business pain points, but these common needs are obscured by the different “languages” of these two communities. The opportunity cost is greatest in heavily regulated industries such as finance and insurance where documentation and compliance are paramount.
In this talk, we will review our experience transitioning Capital One business analysts from legacy systems to open-source workflows by developing user-friendly tools. We incentivized business analysts to adopt the data science mindset by curating open-source tools and developing code packages which simplify workflows and eliminate pain points.
Our internal R package, tidycf, reimagines cumbersome Excel cashflow statements as dataframes and uses RMarkdown templates and the RStudio IDE for an intuitive, user-friendly experience without the overhead of maintaining a custom GUI. We tackle challenges in documentation and communication while immersing new users in the R language.
We will share best practices and lessons learned from our experience designing tools for non-technical end-users, standardizing workflows based on the RStudio IDE’s infrastructure, and evangelizing data science methods.
The Proliferation of New Database Technologies and Implications for Data Scie...
In this talk, we’ll describe NoSQL (“not-only SQL”) and document-oriented databases and the value they provide for data science companies like Uptake. We will walk through the unique challenges such datastores pose for data science workflows. To make these challenges and lessons learned concrete, we’ll explore data science workflows through a discussion of the development efforts that led to “uptasticsearch”, an R package released by the Uptake Data Science team to reduce friction in interacting with a document store called Elasticsearch. The talk will conclude with a discussion of recent developments in NoSQL technologies and implications for data scientists.
Racial Bias in Policing: an analysis of Illinois traffic stops data
Since 2004, Illinois has collected demographic information about traffic stops conducted by police in an effort to identify racial bias. This data has been used by groups such as the ACLU and the Stanford Open Policing Project to identify key markers that infer racial bias in policing. We have applied exploratory data analysis to investigate whether systemic racial bias may appear and to what extent. This talk will walk the audience through the insights gleaned from the exploration of this data along with the challenges posed and ongoing questions raised.
Data Quality Analytics: Understanding what is in your data, before using it
Analytics and data science are ever growing fields, as business decision makers continue to use data to drive decisions. The pinnacle of these fields are the models and their accuracy/fit,; what about the data? Is your data clean, and how do you know that? Our discussion will focus on best practices for data preprocessing for analytic uses. Beginning with essential distributional checks of a dataset to a propose method for automated data validation process during ETL for transactional data.
Supporting innovation in insurance with randomized experimentation
Recent technological advances, a dynamic competitive landscape, and an evolving regulatory environment have led to a period of rapid innovation for many insurance providers. Here, we’ll explore how data scientists may use randomized experiments to rigorously assess the causal impact of innovations on business outcomes. Particular emphasis will be placed on experimentation in “offline” channels, with some of the challenges and mitigation strategies highlighted.
Leveraging Data Science in the Automotive Industry
Cars.com Inc. is a decision engine for car buyers and a growth engine for our partners. Data Science is the bread and butter of any decision engine and Cars is no different. In this talk, I will discuss how we quantify various parameters of a car and plan to make use of all the data in hand to put predictive models at various stages of a users’ automobile lifecycle. This talk will also cater to students looking to gain knowledge on how data science is utilized at scale while still following certain processes and leading the way for business and product partners.
Summertime Analytics: Predicting E. coli and West Nile Virus
Lake Michigan and outdoor recreation are enjoyable aspects of summers in Chicago, but it can come with risk of potential E. coli in Lake Michigan or West Nile Virus from mosquitos. This summer, the City of Chicago launched two new predictive analytics projects to forecasts the risks and to proactively limit these risks. Members of the research team, Gene Leynes and Nick Lucius discuss the projects and how they’re being used as part of city operations.
Reproducible Dashboards and other great things to do with Jupyter
Mac Rogers of Domino Data Lab discusses Jupyter
Recommended
What's in your workflow? Bringing data science workflows to business analysis...
While business analysis rapidly grows more data-driven, the analyst community is slow to adapt the best practices of data science workflows. Many parallels exists between data science “top topics” (e.g. reproducibility) and business pain points, but these common needs are obscured by the different “languages” of these two communities. The opportunity cost is greatest in heavily regulated industries such as finance and insurance where documentation and compliance are paramount.
In this talk, we will review our experience transitioning Capital One business analysts from legacy systems to open-source workflows by developing user-friendly tools. We incentivized business analysts to adopt the data science mindset by curating open-source tools and developing code packages which simplify workflows and eliminate pain points.
Our internal R package, tidycf, reimagines cumbersome Excel cashflow statements as dataframes and uses RMarkdown templates and the RStudio IDE for an intuitive, user-friendly experience without the overhead of maintaining a custom GUI. We tackle challenges in documentation and communication while immersing new users in the R language.
We will share best practices and lessons learned from our experience designing tools for non-technical end-users, standardizing workflows based on the RStudio IDE’s infrastructure, and evangelizing data science methods.
The Proliferation of New Database Technologies and Implications for Data Scie...
In this talk, we’ll describe NoSQL (“not-only SQL”) and document-oriented databases and the value they provide for data science companies like Uptake. We will walk through the unique challenges such datastores pose for data science workflows. To make these challenges and lessons learned concrete, we’ll explore data science workflows through a discussion of the development efforts that led to “uptasticsearch”, an R package released by the Uptake Data Science team to reduce friction in interacting with a document store called Elasticsearch. The talk will conclude with a discussion of recent developments in NoSQL technologies and implications for data scientists.
Racial Bias in Policing: an analysis of Illinois traffic stops data
Since 2004, Illinois has collected demographic information about traffic stops conducted by police in an effort to identify racial bias. This data has been used by groups such as the ACLU and the Stanford Open Policing Project to identify key markers that infer racial bias in policing. We have applied exploratory data analysis to investigate whether systemic racial bias may appear and to what extent. This talk will walk the audience through the insights gleaned from the exploration of this data along with the challenges posed and ongoing questions raised.
Data Quality Analytics: Understanding what is in your data, before using it
Analytics and data science are ever growing fields, as business decision makers continue to use data to drive decisions. The pinnacle of these fields are the models and their accuracy/fit,; what about the data? Is your data clean, and how do you know that? Our discussion will focus on best practices for data preprocessing for analytic uses. Beginning with essential distributional checks of a dataset to a propose method for automated data validation process during ETL for transactional data.
Supporting innovation in insurance with randomized experimentation
Recent technological advances, a dynamic competitive landscape, and an evolving regulatory environment have led to a period of rapid innovation for many insurance providers. Here, we’ll explore how data scientists may use randomized experiments to rigorously assess the causal impact of innovations on business outcomes. Particular emphasis will be placed on experimentation in “offline” channels, with some of the challenges and mitigation strategies highlighted.
Leveraging Data Science in the Automotive Industry
Cars.com Inc. is a decision engine for car buyers and a growth engine for our partners. Data Science is the bread and butter of any decision engine and Cars is no different. In this talk, I will discuss how we quantify various parameters of a car and plan to make use of all the data in hand to put predictive models at various stages of a users’ automobile lifecycle. This talk will also cater to students looking to gain knowledge on how data science is utilized at scale while still following certain processes and leading the way for business and product partners.
Summertime Analytics: Predicting E. coli and West Nile Virus
Lake Michigan and outdoor recreation are enjoyable aspects of summers in Chicago, but it can come with risk of potential E. coli in Lake Michigan or West Nile Virus from mosquitos. This summer, the City of Chicago launched two new predictive analytics projects to forecasts the risks and to proactively limit these risks. Members of the research team, Gene Leynes and Nick Lucius discuss the projects and how they’re being used as part of city operations.
Reproducible Dashboards and other great things to do with Jupyter
Mac Rogers of Domino Data Lab discusses Jupyter
GeoViz: A Canvas for Data Science
Today, more than ever before, maps are being used to bring data to life. In this presentation I will demonstrate how geoviz can make data science more tangible by providing an interactive canvas for spatial data. Gregory Brunner will shows several examples of how maps are being used enhance how we communicate data and how this applies across all scales, including spatial, temporal, and size of data.
Managing Data Science | Lessons from the Field
What you till learn:
GOALS - What is the bar for data science teams
PITFALLS - What are common data science struggles
DIAGNOSES - Why so many of our efforts fail to deliver value
RECOMMENDATIONS - How to address these struggles with best practices
Presented by Mac Steele
Director of Product at Domino Data Lab
Doing your first Kaggle (Python for Big Data sets)
You love python. You love Data Science. But the size of your data set keeps crashing your code. Is it time to bring in big data tools or simply code smarter? Lee is going to show you efficiency hacks, drawn from top Kaggle competitors, to get python to work on large data sets. Skip the hassle of creating a Big Data infrastructure. Let’s find out how far we can push our home laptop first.
Leveraged Analytics at Scale
Most of analytics modeling work today focuses on the production of single-purpose "artisanal" models for predictions. This approach to analytics is fragile with respect to model consistency, reorganization, and resource availability. This talk will argue that instead the focus of analytics modeling should be toward the production of analytics interchangeable parts, which can be combined in creative ways to produce a wide variety of analytics results. This "nuts and bolts" approach allows analytics groups to produce results in an agile way where the time between ask and answer is determined by the right combination of analytics, rather than the modeling.
How I Learned to Stop Worrying and Love Linked Data
In this presentation, Jon Loyens will share:
-Best practices for sharing context and knowledge about your data projects
-How linked data can augment your existing data science workflow and toolchain to accelerate your work
-How a social network can unlock power of Linked Data and data collaboration
-How Linked Data can help you easily combine private and Open Data for fun and profit
Software Engineering for Data Scientists
Although both disciplines are unique in their own ways, Software Engineering and Data Science make heavy use of programing languages to do their respective jobs. Data Science is a relatively new discipline and many of its practitioners have not previously been professional software engineers. There are a few techniques that Data Scientists can leverage from Software Engineering in order to make their tooling and environments, faster to design, more easily debugged and most importantly, clearer to read. This talk will be going over some practical tips that anyone can use to help better understand their code; give clarity around cloud environments, their uses and drawbacks and finally briefly touching on the Software Development Lifecycle.
Making Big Data Smart
Within marketing research, big data is often described as being “census” data for the population that it represents. The devil is in the details and when we take a closer look we can see that this isn’t the case. There are many situations that are not captured within the population that big data purports to be a census of. Big data isn’t even a census of itself since it’s not uncommon for records to be excluded either by accident during the collection process or by design in the cleaning processor. Unfortunately, our industry is so enamored with the size of big data that some users of data are willing to trade off precision for tonnage. Fortunately, if the shortcomings of big data are understood and corrected it can accurately represent the population that it measures in the correct proportion to the universe. We will discuss a method that Nielsen has developed called “Common Homes” that is designed to identify and correct the shortcomings of big data sets that represent media consumption.
Moving Data Science from an Event to A Program: Considerations in Creating Su...
The exponential growth of Big Data and Analytics has outpaced the ability of organizations to govern their data appropriately. The ability to reuse the work done by data scientists work is becoming an economic necessity. The mix of data sources is changing from tradition transactional and ERP systems to include a mix of structured, semi-structured and unstructured data. Data Governance needs to adapt to these changes. This session discusses these data changes and proposed how to adapt current data governance processes. These include, how the concept of a stakeholder has changed and the need for expansion of communications and content management. We look at need to consolidate data from disparate systems and how it governed. Lastly we will investigate how context is emerging as an important factor in governance and how it can be leveraged to provide for accurate, reliable data reuse.
Building Data Analytics pipelines in the cloud using serverless technology
Big Data analytics is well known to uncover hidden insights that gives an organization an edge over the competition. But data does not need to be big in order to be useful. Smaller companies and startups may lack the volume of data that qualifies as big data, yet the variety of data can still yield a trove of insights that helps in driving the business strategies of a company. Startups may also lack the resources to fund an additional, seemingly expensive development project. The key is in simplicity, start small, simple and architect for scalability and performance. But how do you start? In this presentation, we share our experience in building a cost effective, AWS serverless data analytics platform that became an invaluable tool for sales, marketing and operational efficiencies.Serverless architectures simplify development work where servers and software are managed by a third party cloud provider. Developers can focus on just building the data wrangling and data analysis logic where critical aspects like scalability and high availability are guaranteed by the cloud provider. Besides, serverless services offer the pay as you go model, where you pay only based on the amount of resources you use. This turns out to be another attractive aspect where costs can be managed based on the usage. In this presentation we will focus on techniques and best practices to build a big data analytics platform using AWS serverless services like Lambda, DynamoDB, S3, Kinesis, Athena, QuickSight and Amazon ML. We will highlight the strengths of each of these services and what role each plays in the data analytics pipeline. We compare and contrast these services with some of the other popularly used big data technologies like Hadoop, Spark and Kafka. We also demonstrate the usage of these services to build intelligent components that detect anomalies, yield recommendations, simulate chat bots and generate predictive analytics.
Leveraging Open Source Automated Data Science Tools
The data science process seeks to transform and empower organizations by finding and exploiting market inefficiencies and potentially hidden opportunities, but this is often an expensive, tedious process. However, many steps can be automated to provide a streamlined experience for data scientists. Eduardo Arino de la Rubia explores the tools being created by the open source community to free data scientists from tedium, enabling them to work on the high-value aspects of insight creation and impact validation.
The promise of the automated statistician is almost as old as statistics itself. From the creations of vast tables, which saved the labor of calculation, to modern tools which automatically mine datasets for correlations, there has been a considerable amount of advancement in this field. Eduardo compares and contrasts a number of open source tools, including TPOT and auto-sklearn for automated model generation and scikit-feature for feature generation and other aspects of the data science workflow, evaluates their results, and discusses their place in the modern data science workflow.
Along the way, Eduardo outlines the pitfalls of automated data science and applications of the “no free lunch” theorem and dives into alternate approaches, such as end-to-end deep learning, which seek to leverage massive-scale computing and architectures to handle automatic generation of features and advanced models.
Domino and AWS: collaborative analytics and model governance at financial ser...
by Glen Leppert
CTO, DBRS
The Role and Importance of Curiosity in Data Science
by Alfred Lee
Lead Data Scientist, White Ops
Is curiosity useful for more than serendipitous discovery? Can curiosity be taught? How do I foster curiosity in my team? Can someone be too curious? Questions!
Fuzzy Matching to the Rescue
by Jennifer Shin
Senior Principal Data Scientist, Nielsen
With more and more data being collected from consumers, finding a efficient solution to aligning data over time can become increasingly difficult and yet, even more necessary. Whether it's a change in the data collection process or an error in the system, working with big data requires tools that can account for real world complexities.
This talk with introduce the benefits and complexities of implementing a 'fuzzy' solution using the Levenshtein algorithm. Attendees will walk away with a high level understanding of fuzzy matching algorithms and learn how it can be effectively applied to solve real word business problem.
How to Effectively Combine Numerical Features and Categorical Features
by Liangjie Hong
Head of Data Science, Etsy
Latent factor models and decision tree based models are widely used in tasks of prediction, ranking and recommendation. Latent factor models have the advantage of interpreting categorical features by a low-dimensional representation, while such an interpretation does not naturally fit numerical features. In contrast, decision tree based models enjoy the advantage of capturing the nonlinear interactions of numerical features, while their capability of handling categorical features is limited by the cardinality of those features. Since in real-world applications we usually have both abundant numerical features and categorical features with large cardinality (e.g. geolocations, IDs, tags etc.), we design a new model, called GB-CENT, which leverages latent factor embedding and tree components to achieve the merits of both while avoiding their demerits. With two real-world data sets, we demonstrate that GB-CENT can effectively (i.e. fast and accurately) achieve better accuracy than state-of-the-art matrix factorization, decision tree based models and their ensemble
How to Use Data Science to Affect Company Change
Jonathan Roberts
Chief Innovation Officer, Dotdash.com
Companies make many decisions for many different reasons, some of which are sometimes based in data. As a data scientist, your research skills can get you to the correct answer. But the real value of a data science team comes when you can identify the correct question. Jon will talk about his four-year transition from research physicist to senior executive, how data science research helped drive the biggest change in the company’s history, and drove one of the biggest experiments ever undertaken on the internet.
Making Media with Jupyter
Andrew Odewahn
CTO, O'Reilly Media
The Jupyter Notebook is a key part of every data scientist's tool chest, but it's often seen as just an incremental tool in a process, a scratchpad that is discarded once the real work is done. But, what if instead the notebook was itself the final product that communicated the results of a data science team's efforts in a lasting format? In this talk, I'll show how O'Reilly Media is using Jupyter to create a variety of media (books, articles, and new kinds of code+video+text hybrids) and cover some of the lessons learned.
Lean Data Science
by Noelle Sio Saldana
Principal Data Scientist at Pivotal
The success of a Data Science project is not simply the model fit or the accuracy of its predictions; it is whether those models are being leveraged to make smarter business decisions. Over the past few years, Pivotal’s Data Scientists have experimented with software development methods practiced and taught by their Pivotal Labs counterparts in engineering, design and product management. By reframing Data Science as building software and products instead of research, we found that we reaped similar benefits: shorter and more productive iterations, and clients who actually used the models that we built and skills we taught long after we left.
In this talk, we discuss how we have successfully (and maybe not as successfully) borrowed principles from practices like Lean and Agile to Data Science. Topics include:
Minimum Viable Product Models
Build-Measure-Learn instead of a silver bullet
Pair programming
Scrums and retrospectives
Practicing empathy instead of elitism
ThinkFast: Scaling Machine Learning to Modern Demands
by Hristo Spassimirov Paskov
Founder and CEO, ThinkFast Mathematical Intelligence Corporations
Intel Software Innovator for Artificial Intelligence
Machine learning has revolutionized the technological landscape and its success has inspired the collection of vast amounts of data aimed at answering ever deeper questions and solving increasingly harder problems. Continuing this success critically relies on the existence of machine learning paradigms that can perform sophisticated analyses at the data scales required by modern data sets and that reduce development cycle times by improving ease of use. The evolution of machine learning paradigms shows a marked trend toward better addressing these desiderata and a convergence toward paradigms that blend “smooth” modeling techniques classically attributed to statistics with “combinatorial” elements traditionally studied in computer science.
These modern learning paradigms pose a new set of challenges that, when properly addressed, open an unexpected wealth of possibilities. I will discuss how ThinkFast is solving these challenges with fundamental advances in optimization that promote the interpretation of machine learning as a more classical database technology. These advances allow us to scale a variety of techniques to unprecedented data scales using commodity hardware. They also provide surprising insights into how modern techniques learn about data, including a characterization of the limits of what they can learn, and ultimately allow us to devise new, more powerful techniques that do not suffer from these limitations.
Computable content: Notebooks, containers, and data-centric organizational le...
by Paco Nathan
Director, Learning Group at O’Reilly Media
This talk will present:
* the system architecture based on Jupyter as middleware, plus Thebe, Docker, Mesos, Nginx, etc.
* data analytics and project experiences based on delivering _computable content_ at scale
* supporting theory for this pedagogical approach, including Knuth’s _Literate Programming_
* media production techniques that use the video as _subtext_
We will also consider the use of notebooks (Jupyter and others) in an organizational context: how do notebooks help teams share and learn? what impact might notebooks have on developer collaboration that is currently focused on IDEs? The resulting medium provides highly effective tooling for a data-centric organization.
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar
UiPath Test Automation using UiPath Test Suite series, part 4
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
More Related Content
More from Domino Data Lab
GeoViz: A Canvas for Data Science
Today, more than ever before, maps are being used to bring data to life. In this presentation I will demonstrate how geoviz can make data science more tangible by providing an interactive canvas for spatial data. Gregory Brunner will shows several examples of how maps are being used enhance how we communicate data and how this applies across all scales, including spatial, temporal, and size of data.
Managing Data Science | Lessons from the Field
What you till learn:
GOALS - What is the bar for data science teams
PITFALLS - What are common data science struggles
DIAGNOSES - Why so many of our efforts fail to deliver value
RECOMMENDATIONS - How to address these struggles with best practices
Presented by Mac Steele
Director of Product at Domino Data Lab
Doing your first Kaggle (Python for Big Data sets)
You love python. You love Data Science. But the size of your data set keeps crashing your code. Is it time to bring in big data tools or simply code smarter? Lee is going to show you efficiency hacks, drawn from top Kaggle competitors, to get python to work on large data sets. Skip the hassle of creating a Big Data infrastructure. Let’s find out how far we can push our home laptop first.
Leveraged Analytics at Scale
Most of analytics modeling work today focuses on the production of single-purpose "artisanal" models for predictions. This approach to analytics is fragile with respect to model consistency, reorganization, and resource availability. This talk will argue that instead the focus of analytics modeling should be toward the production of analytics interchangeable parts, which can be combined in creative ways to produce a wide variety of analytics results. This "nuts and bolts" approach allows analytics groups to produce results in an agile way where the time between ask and answer is determined by the right combination of analytics, rather than the modeling.
How I Learned to Stop Worrying and Love Linked Data
In this presentation, Jon Loyens will share:
-Best practices for sharing context and knowledge about your data projects
-How linked data can augment your existing data science workflow and toolchain to accelerate your work
-How a social network can unlock power of Linked Data and data collaboration
-How Linked Data can help you easily combine private and Open Data for fun and profit
Software Engineering for Data Scientists
Although both disciplines are unique in their own ways, Software Engineering and Data Science make heavy use of programing languages to do their respective jobs. Data Science is a relatively new discipline and many of its practitioners have not previously been professional software engineers. There are a few techniques that Data Scientists can leverage from Software Engineering in order to make their tooling and environments, faster to design, more easily debugged and most importantly, clearer to read. This talk will be going over some practical tips that anyone can use to help better understand their code; give clarity around cloud environments, their uses and drawbacks and finally briefly touching on the Software Development Lifecycle.
Making Big Data Smart
Within marketing research, big data is often described as being “census” data for the population that it represents. The devil is in the details and when we take a closer look we can see that this isn’t the case. There are many situations that are not captured within the population that big data purports to be a census of. Big data isn’t even a census of itself since it’s not uncommon for records to be excluded either by accident during the collection process or by design in the cleaning processor. Unfortunately, our industry is so enamored with the size of big data that some users of data are willing to trade off precision for tonnage. Fortunately, if the shortcomings of big data are understood and corrected it can accurately represent the population that it measures in the correct proportion to the universe. We will discuss a method that Nielsen has developed called “Common Homes” that is designed to identify and correct the shortcomings of big data sets that represent media consumption.
Moving Data Science from an Event to A Program: Considerations in Creating Su...
The exponential growth of Big Data and Analytics has outpaced the ability of organizations to govern their data appropriately. The ability to reuse the work done by data scientists work is becoming an economic necessity. The mix of data sources is changing from tradition transactional and ERP systems to include a mix of structured, semi-structured and unstructured data. Data Governance needs to adapt to these changes. This session discusses these data changes and proposed how to adapt current data governance processes. These include, how the concept of a stakeholder has changed and the need for expansion of communications and content management. We look at need to consolidate data from disparate systems and how it governed. Lastly we will investigate how context is emerging as an important factor in governance and how it can be leveraged to provide for accurate, reliable data reuse.
Building Data Analytics pipelines in the cloud using serverless technology
Big Data analytics is well known to uncover hidden insights that gives an organization an edge over the competition. But data does not need to be big in order to be useful. Smaller companies and startups may lack the volume of data that qualifies as big data, yet the variety of data can still yield a trove of insights that helps in driving the business strategies of a company. Startups may also lack the resources to fund an additional, seemingly expensive development project. The key is in simplicity, start small, simple and architect for scalability and performance. But how do you start? In this presentation, we share our experience in building a cost effective, AWS serverless data analytics platform that became an invaluable tool for sales, marketing and operational efficiencies.Serverless architectures simplify development work where servers and software are managed by a third party cloud provider. Developers can focus on just building the data wrangling and data analysis logic where critical aspects like scalability and high availability are guaranteed by the cloud provider. Besides, serverless services offer the pay as you go model, where you pay only based on the amount of resources you use. This turns out to be another attractive aspect where costs can be managed based on the usage. In this presentation we will focus on techniques and best practices to build a big data analytics platform using AWS serverless services like Lambda, DynamoDB, S3, Kinesis, Athena, QuickSight and Amazon ML. We will highlight the strengths of each of these services and what role each plays in the data analytics pipeline. We compare and contrast these services with some of the other popularly used big data technologies like Hadoop, Spark and Kafka. We also demonstrate the usage of these services to build intelligent components that detect anomalies, yield recommendations, simulate chat bots and generate predictive analytics.
Leveraging Open Source Automated Data Science Tools
The data science process seeks to transform and empower organizations by finding and exploiting market inefficiencies and potentially hidden opportunities, but this is often an expensive, tedious process. However, many steps can be automated to provide a streamlined experience for data scientists. Eduardo Arino de la Rubia explores the tools being created by the open source community to free data scientists from tedium, enabling them to work on the high-value aspects of insight creation and impact validation.
The promise of the automated statistician is almost as old as statistics itself. From the creations of vast tables, which saved the labor of calculation, to modern tools which automatically mine datasets for correlations, there has been a considerable amount of advancement in this field. Eduardo compares and contrasts a number of open source tools, including TPOT and auto-sklearn for automated model generation and scikit-feature for feature generation and other aspects of the data science workflow, evaluates their results, and discusses their place in the modern data science workflow.
Along the way, Eduardo outlines the pitfalls of automated data science and applications of the “no free lunch” theorem and dives into alternate approaches, such as end-to-end deep learning, which seek to leverage massive-scale computing and architectures to handle automatic generation of features and advanced models.
Domino and AWS: collaborative analytics and model governance at financial ser...
by Glen Leppert
CTO, DBRS
The Role and Importance of Curiosity in Data Science
by Alfred Lee
Lead Data Scientist, White Ops
Is curiosity useful for more than serendipitous discovery? Can curiosity be taught? How do I foster curiosity in my team? Can someone be too curious? Questions!
Fuzzy Matching to the Rescue
by Jennifer Shin
Senior Principal Data Scientist, Nielsen
With more and more data being collected from consumers, finding a efficient solution to aligning data over time can become increasingly difficult and yet, even more necessary. Whether it's a change in the data collection process or an error in the system, working with big data requires tools that can account for real world complexities.
This talk with introduce the benefits and complexities of implementing a 'fuzzy' solution using the Levenshtein algorithm. Attendees will walk away with a high level understanding of fuzzy matching algorithms and learn how it can be effectively applied to solve real word business problem.
How to Effectively Combine Numerical Features and Categorical Features
by Liangjie Hong
Head of Data Science, Etsy
Latent factor models and decision tree based models are widely used in tasks of prediction, ranking and recommendation. Latent factor models have the advantage of interpreting categorical features by a low-dimensional representation, while such an interpretation does not naturally fit numerical features. In contrast, decision tree based models enjoy the advantage of capturing the nonlinear interactions of numerical features, while their capability of handling categorical features is limited by the cardinality of those features. Since in real-world applications we usually have both abundant numerical features and categorical features with large cardinality (e.g. geolocations, IDs, tags etc.), we design a new model, called GB-CENT, which leverages latent factor embedding and tree components to achieve the merits of both while avoiding their demerits. With two real-world data sets, we demonstrate that GB-CENT can effectively (i.e. fast and accurately) achieve better accuracy than state-of-the-art matrix factorization, decision tree based models and their ensemble
How to Use Data Science to Affect Company Change
Jonathan Roberts
Chief Innovation Officer, Dotdash.com
Companies make many decisions for many different reasons, some of which are sometimes based in data. As a data scientist, your research skills can get you to the correct answer. But the real value of a data science team comes when you can identify the correct question. Jon will talk about his four-year transition from research physicist to senior executive, how data science research helped drive the biggest change in the company’s history, and drove one of the biggest experiments ever undertaken on the internet.
Making Media with Jupyter
Andrew Odewahn
CTO, O'Reilly Media
The Jupyter Notebook is a key part of every data scientist's tool chest, but it's often seen as just an incremental tool in a process, a scratchpad that is discarded once the real work is done. But, what if instead the notebook was itself the final product that communicated the results of a data science team's efforts in a lasting format? In this talk, I'll show how O'Reilly Media is using Jupyter to create a variety of media (books, articles, and new kinds of code+video+text hybrids) and cover some of the lessons learned.
Lean Data Science
by Noelle Sio Saldana
Principal Data Scientist at Pivotal
The success of a Data Science project is not simply the model fit or the accuracy of its predictions; it is whether those models are being leveraged to make smarter business decisions. Over the past few years, Pivotal’s Data Scientists have experimented with software development methods practiced and taught by their Pivotal Labs counterparts in engineering, design and product management. By reframing Data Science as building software and products instead of research, we found that we reaped similar benefits: shorter and more productive iterations, and clients who actually used the models that we built and skills we taught long after we left.
In this talk, we discuss how we have successfully (and maybe not as successfully) borrowed principles from practices like Lean and Agile to Data Science. Topics include:
Minimum Viable Product Models
Build-Measure-Learn instead of a silver bullet
Pair programming
Scrums and retrospectives
Practicing empathy instead of elitism
ThinkFast: Scaling Machine Learning to Modern Demands
by Hristo Spassimirov Paskov
Founder and CEO, ThinkFast Mathematical Intelligence Corporations
Intel Software Innovator for Artificial Intelligence
Machine learning has revolutionized the technological landscape and its success has inspired the collection of vast amounts of data aimed at answering ever deeper questions and solving increasingly harder problems. Continuing this success critically relies on the existence of machine learning paradigms that can perform sophisticated analyses at the data scales required by modern data sets and that reduce development cycle times by improving ease of use. The evolution of machine learning paradigms shows a marked trend toward better addressing these desiderata and a convergence toward paradigms that blend “smooth” modeling techniques classically attributed to statistics with “combinatorial” elements traditionally studied in computer science.
These modern learning paradigms pose a new set of challenges that, when properly addressed, open an unexpected wealth of possibilities. I will discuss how ThinkFast is solving these challenges with fundamental advances in optimization that promote the interpretation of machine learning as a more classical database technology. These advances allow us to scale a variety of techniques to unprecedented data scales using commodity hardware. They also provide surprising insights into how modern techniques learn about data, including a characterization of the limits of what they can learn, and ultimately allow us to devise new, more powerful techniques that do not suffer from these limitations.
Computable content: Notebooks, containers, and data-centric organizational le...
by Paco Nathan
Director, Learning Group at O’Reilly Media
This talk will present:
* the system architecture based on Jupyter as middleware, plus Thebe, Docker, Mesos, Nginx, etc.
* data analytics and project experiences based on delivering _computable content_ at scale
* supporting theory for this pedagogical approach, including Knuth’s _Literate Programming_
* media production techniques that use the video as _subtext_
We will also consider the use of notebooks (Jupyter and others) in an organizational context: how do notebooks help teams share and learn? what impact might notebooks have on developer collaboration that is currently focused on IDEs? The resulting medium provides highly effective tooling for a data-centric organization.
More from Domino Data Lab (20)
Doing your first Kaggle (Python for Big Data sets)
Doing your first Kaggle (Python for Big Data sets)
How I Learned to Stop Worrying and Love Linked Data
How I Learned to Stop Worrying and Love Linked Data
Moving Data Science from an Event to A Program: Considerations in Creating Su...
Moving Data Science from an Event to A Program: Considerations in Creating Su...
Building Data Analytics pipelines in the cloud using serverless technology
Building Data Analytics pipelines in the cloud using serverless technology
Leveraging Open Source Automated Data Science Tools
Leveraging Open Source Automated Data Science Tools
Domino and AWS: collaborative analytics and model governance at financial ser...
Domino and AWS: collaborative analytics and model governance at financial ser...
The Role and Importance of Curiosity in Data Science
The Role and Importance of Curiosity in Data Science
How to Effectively Combine Numerical Features and Categorical Features
How to Effectively Combine Numerical Features and Categorical Features
ThinkFast: Scaling Machine Learning to Modern Demands
ThinkFast: Scaling Machine Learning to Modern Demands
Computable content: Notebooks, containers, and data-centric organizational le...
Computable content: Notebooks, containers, and data-centric organizational le...
Recently uploaded
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar
UiPath Test Automation using UiPath Test Suite series, part 4
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
PHP Frameworks: I want to break free (IPC Berlin 2024)
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
By Design, not by Accident - Agile Venture Bolzano 2024
As presented at the Agile Venture Bolzano, 4.06.2024
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Recently uploaded (20)
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024