Data engineering track module 2
•
0 likes•20 views
These are the slides for Module 2 of Data Engineering Track, for University of Toronto, March 2022. The video playlist is available at https://www.youtube.com/playlist?list=PLWoneCyhdP1DWijBQo7zj2uJbuEXaE6E2
Report
Share
Report
Share
Download to read offline
Recommended
Data engineering track module 2
This are the slides for Data Engineering Track Module 2. Prepared for University of Toronto in march 2022. Watch the playlist at https://www.youtube.com/playlist?list=PLWoneCyhdP1DWijBQo7zj2uJbuEXaE6E2
C19013010 the tutorial to build shared ai services session 1
This document provides an agenda and overview for a tutorial on building shared AI services. The tutorial consists of two modules: the first module discusses a case study of AI as a service and challenges of traditional machine learning, and how deep learning can help address these challenges. The second module introduces Keras and options for running Keras on Spark, including a use case, code lab, and prerequisites for running the code lab in Docker containers.
TechEvent 2019: Artificial Intelligence in Dev & Ops; Martin Luckow - Trivadis
This document provides an overview of artificial intelligence trends and applications in development and operations. It discusses how AI is being used for rapid prototyping, intelligent programming assistants, automatic error handling and code refactoring, and strategic decision making. Examples are given of AI tools from Microsoft, Facebook, and Codota. The document also discusses challenges like interpretability of neural networks and outlines a vision of "Software 2.0" where programs are generated automatically to satisfy goals. It emphasizes that AI will transform software development over the next 10 years.
Machine learning
This presentation covers an overview of Analytics and Machine learning. It also covers the Microsoft's contribution in Machine learning space. Azure ML Studio, a SaaS based portal to create, experiment and share Machine Learning Solutions to the external world.
Introducción al Machine Learning Automático
¿Cómo puede llevar el aprendizaje automático a las masas? Los proyectos de Machine Learning con la búsqueda de talento, el tiempo para construir e implementar modelos y confiar en los modelos que se construyen.
¿Cómo puede tener varios equipos en su organización para crear modelos de ML precisos sin ser expertos en ciencia de datos o aprendizaje automático?
¿Se pregunta sobre los diferentes sabores de AutoML?
H2O Driverless AI emplea las técnicas de científicos expertos en datos en una aplicación fácil de usar que ayuda a escalar sus esfuerzos de ciencia de datos. La inteligencia artificial Driverless permite a los científicos de datos trabajar en proyectos más rápido utilizando la automatización y la potencia de computación de vanguardia de las GPU para realizar tareas en minutos que solían tomar meses.
Con H2O Driverless AI, todos, incluyendo expertos y científicos de datos junior, científicos de dominio e ingenieros de datos pueden desarrollar modelos confiables de aprendizaje automático. Esta plataforma de aprendizaje automático de última generación ofrece una funcionalidad única y avanzada para la visualización de datos, la ingeniería de características, la interpretabilidad del modelo y la implementación de baja latencia.
H2O Driverless AI hace:
* Visualización automática de datos
* Ingeniería automática de funciones a nivel de Grandmaster
* Selección automática del modelo
* Ajuste y capacitación automáticos del modelo
* Paralelización automática utilizando múltiples CPU o GPU
* Ensamblaje automático del modelo
*automática del Interpretaciónaprendizaje automático (MLI)
* Generación automática de código de puntuación
¿Quieres probarlo tú mismo? Puede obtener una prueba gratuita aquí: H2O Driverless AI trial.
Venga a esta sesión y descubra cómo comenzar con el Aprendizaje automático automático con AI sin conductor H2O, y cree modelos potentes con solo unos pocos clics.
¡Te veo pronto!
Acerca de H2O.ai
H2O.ai es una empresa visionaria de software de código abierto de Silicon Valley que creó y reimaginó lo que es posible. Somos una empresa de fabricantes que trajeron al mercado nuevas plataformas y tecnologías para impulsar el movimiento de inteligencia artificial. Somos los creadores de, H2O, la principal plataforma de aprendizaje de ciencia de datos de fuente abierta y de aprendizaje automático utilizada por casi la mitad de Fortune 500 y en la que confían más de 14,000 organizaciones y cientos de miles de científicos de datos de todo el mundo.
AI for Software Engineering
Keynote presentation from ECBS conference. The talk is about how to use machine learning and AI in improving software engineering. Experiences from our project in Software Center (www.software-center.se).
Se research update
1) The document summarizes a research update presentation on software engineering and artificial intelligence given by Assistant Professor Nacha Chondamrongkul.
2) It discusses how software engineering research tackles different stages of software production to minimize costs, efforts, and failures. It also examines how AI can be applied to enhance software engineering processes and how software engineering principles are needed to develop AI systems.
3) Key challenges discussed include how to specify requirements for intelligent systems, test AI systems given their unpredictability, and address issues around reliability, fairness, and deployment when integrating machine learning models into complex software.
Deep Learning with CNTK
This document provides an introduction to deep learning with Microsoft's Cognitive Toolkit (CNTK). It discusses key deep learning concepts and how they are implemented in CNTK, including neural networks, backpropagation, loss functions, and common network architectures like convolutional neural networks. It also outlines several of Microsoft's products that use deep learning like Cortana, Bing, and Skype Translator. Examples of training deep learning models with CNTK on datasets like MNIST using logistic regression, multi-layer perceptrons, and CNNs are also presented.
Recommended
Data engineering track module 2
This are the slides for Data Engineering Track Module 2. Prepared for University of Toronto in march 2022. Watch the playlist at https://www.youtube.com/playlist?list=PLWoneCyhdP1DWijBQo7zj2uJbuEXaE6E2
C19013010 the tutorial to build shared ai services session 1
This document provides an agenda and overview for a tutorial on building shared AI services. The tutorial consists of two modules: the first module discusses a case study of AI as a service and challenges of traditional machine learning, and how deep learning can help address these challenges. The second module introduces Keras and options for running Keras on Spark, including a use case, code lab, and prerequisites for running the code lab in Docker containers.
TechEvent 2019: Artificial Intelligence in Dev & Ops; Martin Luckow - Trivadis
This document provides an overview of artificial intelligence trends and applications in development and operations. It discusses how AI is being used for rapid prototyping, intelligent programming assistants, automatic error handling and code refactoring, and strategic decision making. Examples are given of AI tools from Microsoft, Facebook, and Codota. The document also discusses challenges like interpretability of neural networks and outlines a vision of "Software 2.0" where programs are generated automatically to satisfy goals. It emphasizes that AI will transform software development over the next 10 years.
Machine learning
This presentation covers an overview of Analytics and Machine learning. It also covers the Microsoft's contribution in Machine learning space. Azure ML Studio, a SaaS based portal to create, experiment and share Machine Learning Solutions to the external world.
Introducción al Machine Learning Automático
¿Cómo puede llevar el aprendizaje automático a las masas? Los proyectos de Machine Learning con la búsqueda de talento, el tiempo para construir e implementar modelos y confiar en los modelos que se construyen.
¿Cómo puede tener varios equipos en su organización para crear modelos de ML precisos sin ser expertos en ciencia de datos o aprendizaje automático?
¿Se pregunta sobre los diferentes sabores de AutoML?
H2O Driverless AI emplea las técnicas de científicos expertos en datos en una aplicación fácil de usar que ayuda a escalar sus esfuerzos de ciencia de datos. La inteligencia artificial Driverless permite a los científicos de datos trabajar en proyectos más rápido utilizando la automatización y la potencia de computación de vanguardia de las GPU para realizar tareas en minutos que solían tomar meses.
Con H2O Driverless AI, todos, incluyendo expertos y científicos de datos junior, científicos de dominio e ingenieros de datos pueden desarrollar modelos confiables de aprendizaje automático. Esta plataforma de aprendizaje automático de última generación ofrece una funcionalidad única y avanzada para la visualización de datos, la ingeniería de características, la interpretabilidad del modelo y la implementación de baja latencia.
H2O Driverless AI hace:
* Visualización automática de datos
* Ingeniería automática de funciones a nivel de Grandmaster
* Selección automática del modelo
* Ajuste y capacitación automáticos del modelo
* Paralelización automática utilizando múltiples CPU o GPU
* Ensamblaje automático del modelo
*automática del Interpretaciónaprendizaje automático (MLI)
* Generación automática de código de puntuación
¿Quieres probarlo tú mismo? Puede obtener una prueba gratuita aquí: H2O Driverless AI trial.
Venga a esta sesión y descubra cómo comenzar con el Aprendizaje automático automático con AI sin conductor H2O, y cree modelos potentes con solo unos pocos clics.
¡Te veo pronto!
Acerca de H2O.ai
H2O.ai es una empresa visionaria de software de código abierto de Silicon Valley que creó y reimaginó lo que es posible. Somos una empresa de fabricantes que trajeron al mercado nuevas plataformas y tecnologías para impulsar el movimiento de inteligencia artificial. Somos los creadores de, H2O, la principal plataforma de aprendizaje de ciencia de datos de fuente abierta y de aprendizaje automático utilizada por casi la mitad de Fortune 500 y en la que confían más de 14,000 organizaciones y cientos de miles de científicos de datos de todo el mundo.
AI for Software Engineering
Keynote presentation from ECBS conference. The talk is about how to use machine learning and AI in improving software engineering. Experiences from our project in Software Center (www.software-center.se).
Se research update
1) The document summarizes a research update presentation on software engineering and artificial intelligence given by Assistant Professor Nacha Chondamrongkul.
2) It discusses how software engineering research tackles different stages of software production to minimize costs, efforts, and failures. It also examines how AI can be applied to enhance software engineering processes and how software engineering principles are needed to develop AI systems.
3) Key challenges discussed include how to specify requirements for intelligent systems, test AI systems given their unpredictability, and address issues around reliability, fairness, and deployment when integrating machine learning models into complex software.
Deep Learning with CNTK
This document provides an introduction to deep learning with Microsoft's Cognitive Toolkit (CNTK). It discusses key deep learning concepts and how they are implemented in CNTK, including neural networks, backpropagation, loss functions, and common network architectures like convolutional neural networks. It also outlines several of Microsoft's products that use deep learning like Cortana, Bing, and Skype Translator. Examples of training deep learning models with CNTK on datasets like MNIST using logistic regression, multi-layer perceptrons, and CNNs are also presented.
How I became ML Engineer
One of the most popular buzz words nowadays in the technology world is “Machine Learning (ML).” Most economists and business experts foresee Machine Learning changing every aspect of our lives in the next 10 years through automating and optimizing processes. This is leading many organizations to seek experts who can implement Machine Learning into their businesses.
The paper will be written for statistical programmers who want to explore Machine Learning career, add Machine Learning skills to their experiences or enter a Machine Learning fields. The paper will discuss about personal journey to become to a Machine Learning Engineer from a statistical programmer. The paper will share my personal experience on what motivated me to start Machine Learning career, how I started it, and what I have learned and done to be a Machine Learning Engineer. In addition, the paper will also discuss the future of Machine Learning in Pharmaceutical Industry, especially in Biometric department.
MLOps and Data Quality: Deploying Reliable ML Models in Production
Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure.
For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps).
Agenda
- Data Quality and why it matters
- Challenges and solutions of Data Testing
- Challenges and solutions of Model Testing
- MLOps pipelines and why they matter
- How to expand validation pipelines for Data Quality
Bridging the Gap: from Data Science to Production
A recent but quite common observation in industry is that although there is an overall high adoption of data science, many companies struggle to get it into production. Huge teams of well-payed data scientists often present one fancy model after the other to their managers but their proof of concepts never manifest into something business relevant. The frustration grows on both sides, managers and data scientists.
In my talk I elaborate on the many reasons why data science to production is such a hard nut to crack. I start with a taxonomy of data use cases in order to easier assess technical requirements. Based thereon, my focus lies on overcoming the two-language-problem which is Python/R loved by data scientists vs. the enterprise-established Java/Scala. From my project experiences I present three different solutions, namely 1) migrating to a single language, 2) reimplementation and 3) usage of a framework. The advantages and disadvantages of each approach is presented and general advices based on the introduced taxonomy is given.
Additionally, my talk also addresses organisational as well as problems in quality assurance and deployment. Best practices and further references are presented on a high-level in order to cover all facets of data science to production.
With my talk I hope to convey the message that breakdowns on the road from data science to production are rather the rule than the exception, so you are not alone. At the end of my talk, you will have a better understanding of why your team and you are struggling and what to do about it.
201906 02 Introduction to AutoML with ML.NET 1.0
ML.NET 1.0 release is the first major milestone of a great journey that started in May 2018 when we released ML.NET 0.1 as open source. ML.NET is an open-source and cross-platform machine learning framework for .NET developers. Using ML.NET, developers can leverage their existing tools and skillsets to develop and infuse custom AI into their applications by creating custom machine learning models for common scenarios like Sentiment Analysis, Recommendation, Image Classification and more.
“Automated ML” is a collection of new technologies from Microsoft to enhance the data science development process. Still in preview, Auto ML for ML.NET 1.0 will be demonstrated in a Deep Learning Virtual Machine running Windows Server 2016. Code examples are in C# and run in Visual Studio Community 2019.
This presentation is the second of four related to ML.NET and Automated ML. The presentation will be recorded with video posted to this YouTube Channel: http://bit.ly/2ZybKwI
SESE 2021: Where Systems Engineering meets AI/ML
1) The document discusses how systems engineering methods can be integrated with the AI/ML lifecycle to engineer intelligent systems. It identifies 10 major challenges for this integration, including describing AI/ML model needs and capabilities, integrating AI/ML into specification, verification, and other systems engineering processes.
2) The document proposes concepts for tackling each challenge, such as using standards to describe AI/ML model lifecycles and digital twin environments for verification. It also discusses opportunities like reusing existing AI/ML models and the need to educate new professionals.
3) Key points are that research is active in integrating systems engineering and AI/ML to build safer, more cost-effective cyber-physical systems, and
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Presented at #H2OWorld 2017 in Mountain View, CA.
Enjoy the video: https://youtu.be/-rGRHrED94Y.
Learn more about H2O.ai: https://www.h2o.ai/.
Follow @h2oai: https://twitter.com/h2oai.
- - -
Abstract:
Most machine learning systems enable two essential processes: creating a model and applying the model in a repeatable and controlled fashion. These two processes are interrelated and pose technological and organizational challenges as they evolve from research to prototype to production. This presentation outlines common design patterns for tackling such challenges while implementing machine learning in a production environment.
Sergei's Bio:
Dr. Sergei Izrailev is Chief Data Scientist at BeeswaxIO, where he is responsible for data strategy and building AI applications powering the next generation of real-time bidding technology. Before Beeswax, Sergei led data science teams at Integral Ad Science and Collective, where he focused on architecture, development and scaling of data science based advertising technology products. Prior to advertising, Sergei was a quant/trader and developed trading strategies and portfolio optimization methodologies. Previously, he worked as a senior scientist at Johnson & Johnson, where he developed intelligent tools for structure-based drug discovery. Sergei holds a Ph.D. in Physics and Master of Computer Science degrees from the University of Illinois at Urbana-Champaign.
Venkata brundavanam 2020
Padma Brundavanam completed a Master's degree in Electrical and Computer Engineering from Carleton University in Ottawa, Canada, where coursework included internetworking, telemedicine, software engineering, and distributed systems. She holds a Bachelor's degree in Information Technology from GITAM University in Visakhapatnam, India. Skills include Java, C, C++, Python, UML, SQL, networking protocols, cloud computing, and operating systems. Relevant experience includes internships developing web forums using Java and content creation for computer science students. Projects include comparing intrusion detection tools, creating a semantic similarity tool, and developing distributed system and online store prototypes.
Venkata brundavanam 2020
Padma Brundavanam has a Master's degree in Electrical and Computer Engineering from Carleton University in Ottawa, Canada and a Bachelor's degree in Information Technology from GITAM University in Visakhapatnam, India. Their experience includes projects in areas like SDN, network intrusion detection, semantic analysis, and e-commerce. They have skills in languages like Java, C, C++, Python and technologies like UML, SQL, Docker, and networking protocols. Padma Brundavanam is currently learning Python and working towards a CCNA certification.
Managing Machines: The New AI Dev Stack
Machine learning drove massive growth at consumer internet companies over the last decade, and this was enabled by open software, datasets, and AI research. For many problems, machine learning will produce better, faster, and more repeatable decisions at scale. Unfortunately, building and maintaining these systems is still extremely difficult and expensive. As more machine learning software moves to production, many of our traditional tools and best practices in software development will change.
Pete Skomoroch walks you through what you need to know as we shift from a world of deterministic programs to systems that give unpredictable results on ever-changing training data. To navigate this world powered by nondeterministic data-dependent programs, we’ll also need a new development stack to help us write, test, deploy, and monitor machine learning software.
Presented at OSCON Portland July 18, 2019
DevOps for AI Apps
DevOps for AI Applications - How to apply best practices from software engineering to the world of data science and machine learning
Webinar: Machine Learning para Microcontroladores
Neste webinar, serão apresentados conceitos sobre inteligência artificial, assim como ferramentas disponíveis para o desenvolvimento integradas ao MPLAB X e ao Harmony 3 e demonstração de um sistema de detecção de anomalia utilizando um microcontrolador da família ATSAMD21 (ARM Cortex M0+).
Open, Secure & Transparent AI Pipelines
AI algorithms offer great promise in criminal justice, credit scoring, hiring and other domains. However, algorithmic fairness is a legitimate concern. Possible bias and adversarial contamination can come from training data, inappropriate data handling/model selection or incorrect algorithm design. This talk discusses how to build an open, transparent, secure and fair pipeline that fully integrates into the AI lifecycle — leveraging open-source projects such as AI Fairness 360 (AIF360), Adversarial Robustness Toolbox (ART), the Fabric for Deep Learning (FfDL) and the Model Asset eXchange (MAX).
Monitoring AI with AI
Monitoring AI applications with AI
The best performing offline algorithm can lose in production. The most accurate model does not always improve business metrics. Environment misconfiguration or upstream data pipeline inconsistency can silently kill the model performance. Neither prodops, data science or engineering teams are skilled to detect, monitor and debug such types of incidents.
Was it possible for Microsoft to test Tay chatbot in advance and then monitor and adjust it continuously in production to prevent its unexpected behaviour? Real mission critical AI systems require advanced monitoring and testing ecosystem which enables continuous and reliable delivery of machine learning models and data pipelines into production. Common production incidents include:
Data drifts, new data, wrong features
Vulnerability issues, malicious users
Concept drifts
Model Degradation
Biased Training set / training issue
Performance issue
In this demo based talk we discuss a solution, tooling and architecture that allows machine learning engineer to be involved in delivery phase and take ownership over deployment and monitoring of machine learning pipelines.
It allows data scientists to safely deploy early results as end-to-end AI applications in a self serve mode without assistance from engineering and operations teams. It shifts experimentation and even training phases from offline datasets to live production and closes a feedback loop between research and production.
Technical part of the talk will cover the following topics:
Automatic Data Profiling
Anomaly Detection
Clustering of inputs and outputs of the model
A/B Testing
Service Mesh, Envoy Proxy, trafic shadowing
Stateless and stateful models
Monitoring of regression, classification and prediction models
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
In this demo based talk we discuss a solution, tooling and architecture that allows machine learning engineer to be involved in delivery phase and take ownership over deployment and monitoring of machine learning pipelines. It allows data scientists to safely deploy early results as end-to-end AI applications in a self serve mode without assistance from engineering and operations teams. It shifts experimentation and even training phases from offline datasets to live production and closes a feedback loop between research and production.
Microsoft DevOps for AI with GoDataDriven
Artificial Intelligence (AI) and machine learning (ML) technologies extend the capabilities of software applications that are now found throughout our daily life: digital assistants, facial recognition, photo captioning, banking services, and product recommendations. The difficult part about integrating AI or ML into an application is not the technology, or the math, or the science or the algorithms. The challenge is getting the model deployed into a production environment and keeping it operational and supportable. Software development teams know how to deliver business applications and cloud services. AI/ML teams know how to develop models that can transform a business. But when it comes to putting the two together to implement an application pipeline specific to AI/ML — to automate it and wrap it around good deployment practices — the process needs some effort to be successful.
Using Algorithmia to leverage AI and Machine Learning APIs
We are entering a new era of software development. Companies are realizing that AI and machine learning are critical to success in business, both to save cost on repetitive tasks, and to enable to new features and products that would be impossible without machine intelligence. Algorithmia makes these tools available through web APIs that makes tools like computer vision and natural language processing available to companies everywhere. Kenny will talk about how sharing of intelligent APIs can improve your applications.
https://rakutentechnologyconference2016.sched.org/event/8aS5/using-algorithmia-to-leverage-ai-and-machine-learning-apis
Rakuten Technology Conference 2016
http://tech.rakuten.co.jp/
Bring Your Own Recipes Hands-On Session
1. Driverless AI can be used across many industries like banking, healthcare, telecom, and marketing to save time and money through tasks like fraud detection, customer churn prediction, and personalized recommendations.
2. The document highlights new features in Driverless AI 1.7.1 including improved time series recipes, natural language processing features, automatic visualization, and machine learning interpretability tools.
3. Driverless AI provides fully automated machine learning through techniques such as automatic feature engineering, model tuning, standalone scoring pipelines, and massively parallel processing to find optimal solutions.
TechDayPakistan-Slides RAG with Cosmos DB.pptx
This document discusses using retrieval augmented generation (RAG) with Cosmos DB and large language models (LLMs) to power question answering applications. RAG combines information retrieval over stored data with text generation from LLMs to provide customized, up-to-date responses without requiring expensive model retraining. The key components of RAG include data storage, embedding models to index data, a vector database to store embeddings, retrieval of relevant embeddings, and an LLM orchestrator to generate responses using retrieved information as context. Azure Cosmos DB is highlighted as an effective vector database option for RAG applications.
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache Spark has rapidly become a key tool for data scientists to explore, understand and transform massive datasets and to build and train advanced machine learning models. The question then becomes, how do I deploy these model to a production environment? How do I embed what I have learned into customer facing data applications?
In this webinar, we will discuss best practices from Databricks on
how our customers productionize machine learning models
do a deep dive with actual customer case studies,
show live tutorials of a few example architectures and code in Python, Scala, Java and SQL.
Bilal ahmed resume
Bilal Ahmed is a data scientist with experience developing end-to-end deep learning pipelines from scratch at Programmers Force. Some of his projects include COVID-19 safety mask detection using Tensorflow and multi-class classification using Keras. He has certifications in Deep Learning and Python specializations. Bilal obtained a BS in Software Engineering from the University of Engineering and Technology, Taxila, where he completed a final year project on person re-identification using deep learning.
Machine Learning Introduction
Pranav Prakash is a VP of engineering who has worked on projects involving machine learning, computer vision, and recommendations. The document discusses fundamental concepts in artificial intelligence including intelligent search algorithms. It covers categories of machine learning such as supervised, unsupervised, and reinforcement learning. Popular machine learning techniques like classification, clustering, and regression are described. Real-life applications of machine learning like recommender systems, sentiment analysis, and object recognition are also mentioned.
Test document
This short document does not contain enough information to summarize in 3 sentences or less. It only contains the word "test" and does not provide any meaningful context or details that could be extracted to create an informative summary.
More Related Content
Similar to Data engineering track module 2
How I became ML Engineer
One of the most popular buzz words nowadays in the technology world is “Machine Learning (ML).” Most economists and business experts foresee Machine Learning changing every aspect of our lives in the next 10 years through automating and optimizing processes. This is leading many organizations to seek experts who can implement Machine Learning into their businesses.
The paper will be written for statistical programmers who want to explore Machine Learning career, add Machine Learning skills to their experiences or enter a Machine Learning fields. The paper will discuss about personal journey to become to a Machine Learning Engineer from a statistical programmer. The paper will share my personal experience on what motivated me to start Machine Learning career, how I started it, and what I have learned and done to be a Machine Learning Engineer. In addition, the paper will also discuss the future of Machine Learning in Pharmaceutical Industry, especially in Biometric department.
MLOps and Data Quality: Deploying Reliable ML Models in Production
Looking to build a robust machine learning infrastructure to streamline MLOps? Learn from Provectus experts how to ensure the success of your MLOps initiative by implementing Data QA components in your ML infrastructure.
For most organizations, the development of multiple machine learning models, their deployment and maintenance in production are relatively new tasks. Join Provectus as we explain how to build an end-to-end infrastructure for machine learning, with a focus on data quality and metadata management, to standardize and streamline machine learning life cycle management (MLOps).
Agenda
- Data Quality and why it matters
- Challenges and solutions of Data Testing
- Challenges and solutions of Model Testing
- MLOps pipelines and why they matter
- How to expand validation pipelines for Data Quality
Bridging the Gap: from Data Science to Production
A recent but quite common observation in industry is that although there is an overall high adoption of data science, many companies struggle to get it into production. Huge teams of well-payed data scientists often present one fancy model after the other to their managers but their proof of concepts never manifest into something business relevant. The frustration grows on both sides, managers and data scientists.
In my talk I elaborate on the many reasons why data science to production is such a hard nut to crack. I start with a taxonomy of data use cases in order to easier assess technical requirements. Based thereon, my focus lies on overcoming the two-language-problem which is Python/R loved by data scientists vs. the enterprise-established Java/Scala. From my project experiences I present three different solutions, namely 1) migrating to a single language, 2) reimplementation and 3) usage of a framework. The advantages and disadvantages of each approach is presented and general advices based on the introduced taxonomy is given.
Additionally, my talk also addresses organisational as well as problems in quality assurance and deployment. Best practices and further references are presented on a high-level in order to cover all facets of data science to production.
With my talk I hope to convey the message that breakdowns on the road from data science to production are rather the rule than the exception, so you are not alone. At the end of my talk, you will have a better understanding of why your team and you are struggling and what to do about it.
201906 02 Introduction to AutoML with ML.NET 1.0
ML.NET 1.0 release is the first major milestone of a great journey that started in May 2018 when we released ML.NET 0.1 as open source. ML.NET is an open-source and cross-platform machine learning framework for .NET developers. Using ML.NET, developers can leverage their existing tools and skillsets to develop and infuse custom AI into their applications by creating custom machine learning models for common scenarios like Sentiment Analysis, Recommendation, Image Classification and more.
“Automated ML” is a collection of new technologies from Microsoft to enhance the data science development process. Still in preview, Auto ML for ML.NET 1.0 will be demonstrated in a Deep Learning Virtual Machine running Windows Server 2016. Code examples are in C# and run in Visual Studio Community 2019.
This presentation is the second of four related to ML.NET and Automated ML. The presentation will be recorded with video posted to this YouTube Channel: http://bit.ly/2ZybKwI
SESE 2021: Where Systems Engineering meets AI/ML
1) The document discusses how systems engineering methods can be integrated with the AI/ML lifecycle to engineer intelligent systems. It identifies 10 major challenges for this integration, including describing AI/ML model needs and capabilities, integrating AI/ML into specification, verification, and other systems engineering processes.
2) The document proposes concepts for tackling each challenge, such as using standards to describe AI/ML model lifecycles and digital twin environments for verification. It also discusses opportunities like reusing existing AI/ML models and the need to educate new professionals.
3) Key points are that research is active in integrating systems engineering and AI/ML to build safer, more cost-effective cyber-physical systems, and
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Presented at #H2OWorld 2017 in Mountain View, CA.
Enjoy the video: https://youtu.be/-rGRHrED94Y.
Learn more about H2O.ai: https://www.h2o.ai/.
Follow @h2oai: https://twitter.com/h2oai.
- - -
Abstract:
Most machine learning systems enable two essential processes: creating a model and applying the model in a repeatable and controlled fashion. These two processes are interrelated and pose technological and organizational challenges as they evolve from research to prototype to production. This presentation outlines common design patterns for tackling such challenges while implementing machine learning in a production environment.
Sergei's Bio:
Dr. Sergei Izrailev is Chief Data Scientist at BeeswaxIO, where he is responsible for data strategy and building AI applications powering the next generation of real-time bidding technology. Before Beeswax, Sergei led data science teams at Integral Ad Science and Collective, where he focused on architecture, development and scaling of data science based advertising technology products. Prior to advertising, Sergei was a quant/trader and developed trading strategies and portfolio optimization methodologies. Previously, he worked as a senior scientist at Johnson & Johnson, where he developed intelligent tools for structure-based drug discovery. Sergei holds a Ph.D. in Physics and Master of Computer Science degrees from the University of Illinois at Urbana-Champaign.
Venkata brundavanam 2020
Padma Brundavanam completed a Master's degree in Electrical and Computer Engineering from Carleton University in Ottawa, Canada, where coursework included internetworking, telemedicine, software engineering, and distributed systems. She holds a Bachelor's degree in Information Technology from GITAM University in Visakhapatnam, India. Skills include Java, C, C++, Python, UML, SQL, networking protocols, cloud computing, and operating systems. Relevant experience includes internships developing web forums using Java and content creation for computer science students. Projects include comparing intrusion detection tools, creating a semantic similarity tool, and developing distributed system and online store prototypes.
Venkata brundavanam 2020
Padma Brundavanam has a Master's degree in Electrical and Computer Engineering from Carleton University in Ottawa, Canada and a Bachelor's degree in Information Technology from GITAM University in Visakhapatnam, India. Their experience includes projects in areas like SDN, network intrusion detection, semantic analysis, and e-commerce. They have skills in languages like Java, C, C++, Python and technologies like UML, SQL, Docker, and networking protocols. Padma Brundavanam is currently learning Python and working towards a CCNA certification.
Managing Machines: The New AI Dev Stack
Machine learning drove massive growth at consumer internet companies over the last decade, and this was enabled by open software, datasets, and AI research. For many problems, machine learning will produce better, faster, and more repeatable decisions at scale. Unfortunately, building and maintaining these systems is still extremely difficult and expensive. As more machine learning software moves to production, many of our traditional tools and best practices in software development will change.
Pete Skomoroch walks you through what you need to know as we shift from a world of deterministic programs to systems that give unpredictable results on ever-changing training data. To navigate this world powered by nondeterministic data-dependent programs, we’ll also need a new development stack to help us write, test, deploy, and monitor machine learning software.
Presented at OSCON Portland July 18, 2019
DevOps for AI Apps
DevOps for AI Applications - How to apply best practices from software engineering to the world of data science and machine learning
Webinar: Machine Learning para Microcontroladores
Neste webinar, serão apresentados conceitos sobre inteligência artificial, assim como ferramentas disponíveis para o desenvolvimento integradas ao MPLAB X e ao Harmony 3 e demonstração de um sistema de detecção de anomalia utilizando um microcontrolador da família ATSAMD21 (ARM Cortex M0+).
Open, Secure & Transparent AI Pipelines
AI algorithms offer great promise in criminal justice, credit scoring, hiring and other domains. However, algorithmic fairness is a legitimate concern. Possible bias and adversarial contamination can come from training data, inappropriate data handling/model selection or incorrect algorithm design. This talk discusses how to build an open, transparent, secure and fair pipeline that fully integrates into the AI lifecycle — leveraging open-source projects such as AI Fairness 360 (AIF360), Adversarial Robustness Toolbox (ART), the Fabric for Deep Learning (FfDL) and the Model Asset eXchange (MAX).
Monitoring AI with AI
Monitoring AI applications with AI
The best performing offline algorithm can lose in production. The most accurate model does not always improve business metrics. Environment misconfiguration or upstream data pipeline inconsistency can silently kill the model performance. Neither prodops, data science or engineering teams are skilled to detect, monitor and debug such types of incidents.
Was it possible for Microsoft to test Tay chatbot in advance and then monitor and adjust it continuously in production to prevent its unexpected behaviour? Real mission critical AI systems require advanced monitoring and testing ecosystem which enables continuous and reliable delivery of machine learning models and data pipelines into production. Common production incidents include:
Data drifts, new data, wrong features
Vulnerability issues, malicious users
Concept drifts
Model Degradation
Biased Training set / training issue
Performance issue
In this demo based talk we discuss a solution, tooling and architecture that allows machine learning engineer to be involved in delivery phase and take ownership over deployment and monitoring of machine learning pipelines.
It allows data scientists to safely deploy early results as end-to-end AI applications in a self serve mode without assistance from engineering and operations teams. It shifts experimentation and even training phases from offline datasets to live production and closes a feedback loop between research and production.
Technical part of the talk will cover the following topics:
Automatic Data Profiling
Anomaly Detection
Clustering of inputs and outputs of the model
A/B Testing
Service Mesh, Envoy Proxy, trafic shadowing
Stateless and stateful models
Monitoring of regression, classification and prediction models
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
In this demo based talk we discuss a solution, tooling and architecture that allows machine learning engineer to be involved in delivery phase and take ownership over deployment and monitoring of machine learning pipelines. It allows data scientists to safely deploy early results as end-to-end AI applications in a self serve mode without assistance from engineering and operations teams. It shifts experimentation and even training phases from offline datasets to live production and closes a feedback loop between research and production.
Microsoft DevOps for AI with GoDataDriven
Artificial Intelligence (AI) and machine learning (ML) technologies extend the capabilities of software applications that are now found throughout our daily life: digital assistants, facial recognition, photo captioning, banking services, and product recommendations. The difficult part about integrating AI or ML into an application is not the technology, or the math, or the science or the algorithms. The challenge is getting the model deployed into a production environment and keeping it operational and supportable. Software development teams know how to deliver business applications and cloud services. AI/ML teams know how to develop models that can transform a business. But when it comes to putting the two together to implement an application pipeline specific to AI/ML — to automate it and wrap it around good deployment practices — the process needs some effort to be successful.
Using Algorithmia to leverage AI and Machine Learning APIs
We are entering a new era of software development. Companies are realizing that AI and machine learning are critical to success in business, both to save cost on repetitive tasks, and to enable to new features and products that would be impossible without machine intelligence. Algorithmia makes these tools available through web APIs that makes tools like computer vision and natural language processing available to companies everywhere. Kenny will talk about how sharing of intelligent APIs can improve your applications.
https://rakutentechnologyconference2016.sched.org/event/8aS5/using-algorithmia-to-leverage-ai-and-machine-learning-apis
Rakuten Technology Conference 2016
http://tech.rakuten.co.jp/
Bring Your Own Recipes Hands-On Session
1. Driverless AI can be used across many industries like banking, healthcare, telecom, and marketing to save time and money through tasks like fraud detection, customer churn prediction, and personalized recommendations.
2. The document highlights new features in Driverless AI 1.7.1 including improved time series recipes, natural language processing features, automatic visualization, and machine learning interpretability tools.
3. Driverless AI provides fully automated machine learning through techniques such as automatic feature engineering, model tuning, standalone scoring pipelines, and massively parallel processing to find optimal solutions.
TechDayPakistan-Slides RAG with Cosmos DB.pptx
This document discusses using retrieval augmented generation (RAG) with Cosmos DB and large language models (LLMs) to power question answering applications. RAG combines information retrieval over stored data with text generation from LLMs to provide customized, up-to-date responses without requiring expensive model retraining. The key components of RAG include data storage, embedding models to index data, a vector database to store embeddings, retrieval of relevant embeddings, and an LLM orchestrator to generate responses using retrieved information as context. Azure Cosmos DB is highlighted as an effective vector database option for RAG applications.
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache Spark has rapidly become a key tool for data scientists to explore, understand and transform massive datasets and to build and train advanced machine learning models. The question then becomes, how do I deploy these model to a production environment? How do I embed what I have learned into customer facing data applications?
In this webinar, we will discuss best practices from Databricks on
how our customers productionize machine learning models
do a deep dive with actual customer case studies,
show live tutorials of a few example architectures and code in Python, Scala, Java and SQL.
Bilal ahmed resume
Bilal Ahmed is a data scientist with experience developing end-to-end deep learning pipelines from scratch at Programmers Force. Some of his projects include COVID-19 safety mask detection using Tensorflow and multi-class classification using Keras. He has certifications in Deep Learning and Python specializations. Bilal obtained a BS in Software Engineering from the University of Engineering and Technology, Taxila, where he completed a final year project on person re-identification using deep learning.
Similar to Data engineering track module 2 (20)
MLOps and Data Quality: Deploying Reliable ML Models in Production
MLOps and Data Quality: Deploying Reliable ML Models in Production
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Design Patterns for Machine Learning in Production - Sergei Izrailev, Chief D...
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
Using Algorithmia to leverage AI and Machine Learning APIs
Using Algorithmia to leverage AI and Machine Learning APIs
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
Apache ® Spark™ MLlib 2.x: How to Productionize your Machine Learning Models
More from Pranav Prakash
Machine Learning Introduction
Pranav Prakash is a VP of engineering who has worked on projects involving machine learning, computer vision, and recommendations. The document discusses fundamental concepts in artificial intelligence including intelligent search algorithms. It covers categories of machine learning such as supervised, unsupervised, and reinforcement learning. Popular machine learning techniques like classification, clustering, and regression are described. Real-life applications of machine learning like recommender systems, sentiment analysis, and object recognition are also mentioned.
Test document
This short document does not contain enough information to summarize in 3 sentences or less. It only contains the word "test" and does not provide any meaningful context or details that could be extracted to create an informative summary.
Webtech1b
Uniform Resource Locators (URLs) are standardized addresses used to locate resources on the Internet. A URL contains the protocol or scheme being used (such as http or ftp), the domain name or IP address of the server, and the path to the specific file or resource. Well-formed URLs follow a general syntax of <scheme>://<domain>/<path>. They allow both humans and software programs to directly access electronic resources anywhere on the Internet or on private networks.
To Infinity and Beyond - OSDConf2014
The story of how solving one problem the OpenSource way
opened doors to so much more. Talk presented by Pranav Prakash and Hari Prasanna at OSDConf 2014, New Delhi.
#comments
The document discusses using Twitter during a live presentation to engage the audience and get feedback. It notes that everyone has a mobile device and loves to tweet, but there is a problem. It then demonstrates how tweeting can provide relevancy for the audience, allow authors to get feedback and track engagement, and help the audience feel connected through a single platform by leaving comments. The document encourages asking any questions using the hashtag #XCRT.
Ibm haifa.mq.final
This document summarizes experiments comparing the open source search engine Lucene to a custom search engine called Juru on TREC data. The authors investigated differences in search quality between the two engines. They found that Lucene's default scoring was inferior to Juru's. They modified Lucene's scoring function by changing the document length normalization and term frequency normalization. Evaluations showed the modified Lucene performed comparably to Juru and other top systems in the TREC 1-Million Queries track, demonstrating the robustness of the modifications and the new evaluation measures.
Apple banana oranges_peaches
This document lists four types of fruit: apple, banana, oranges, and peaches. It provides a brief listing of common fruits.
Oranges peaches
This very short document lists two types of fruit: oranges and peaches. It does not provide any other details about the fruits or context around them.
Banana peaches
This very short document mentions two types of fruit: banana and peaches. It does not provide any other context or details about the fruits.
Banana oranges
This very short document contains two fruit names but provides no other context or information. It mentions both "banana" and "oranges" but does not explain their relationship or relevance to each other. The intended meaning or purpose is unclear from the limited content provided.
Banana oranges peaches
This very short document lists three types of fruit: banana, oranges, and peaches. It does not provide any other details about the fruits or context around them. The document simply names three different fruits in a list.
Apple peaches
This very short document only contains two words: "apple peaches". It does not provide enough context or details to generate a multi-sentence summary.
Apple oranges
This very short document contains two words: "apple oranges". It does not provide much context or information to summarize in only a few sentences.
Apple oranges peaches
This very short document lists three types of fruit: apples, oranges, and peaches. It does not provide any other details about the fruits or context around them. The document simply names three common fruits in a list.
Apple banana
This very short document contains two words: "apple" and "banana". It lists two types of fruit but does not provide any other context or information.
Apple banana peaches
This very short document contains a list of three fruits: apple, banana, and peaches. It does not provide any additional context or details about the fruits.
More from Pranav Prakash (20)
Recently uploaded
The History of Stoke Newington Street Names
Presented at the Stoke Newington Literary Festival on 9th June 2024
www.StokeNewingtonHistory.com
Walmart Business+ and Spark Good for Nonprofits.pdf
"Learn about all the ways Walmart supports nonprofit organizations.
You will hear from Liz Willett, the Head of Nonprofits, and hear about what Walmart is doing to help nonprofits, including Walmart Business and Spark Good. Walmart Business+ is a new offer for nonprofits that offers discounts and also streamlines nonprofits order and expense tracking, saving time and money.
The webinar may also give some examples on how nonprofits can best leverage Walmart Business+.
The event will cover the following::
Walmart Business + (https://business.walmart.com/plus) is a new shopping experience for nonprofits, schools, and local business customers that connects an exclusive online shopping experience to stores. Benefits include free delivery and shipping, a 'Spend Analytics” feature, special discounts, deals and tax-exempt shopping.
Special TechSoup offer for a free 180 days membership, and up to $150 in discounts on eligible orders.
Spark Good (walmart.com/sparkgood) is a charitable platform that enables nonprofits to receive donations directly from customers and associates.
Answers about how you can do more with Walmart!"
BBR 2024 Summer Sessions Interview Training
Qualitative research interview training by Professor Katrina Pritchard and Dr Helen Williams
Wound healing PPT
This document provides an overview of wound healing, its functions, stages, mechanisms, factors affecting it, and complications.
A wound is a break in the integrity of the skin or tissues, which may be associated with disruption of the structure and function.
Healing is the body’s response to injury in an attempt to restore normal structure and functions.
Healing can occur in two ways: Regeneration and Repair
There are 4 phases of wound healing: hemostasis, inflammation, proliferation, and remodeling. This document also describes the mechanism of wound healing. Factors that affect healing include infection, uncontrolled diabetes, poor nutrition, age, anemia, the presence of foreign bodies, etc.
Complications of wound healing like infection, hyperpigmentation of scar, contractures, and keloid formation.
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Dr. Vinod Kumar Kanvaria
Exploiting Artificial Intelligence for Empowering Researchers and Faculty,
International FDP on Fundamentals of Research in Social Sciences
at Integral University, Lucknow, 06.06.2024
By Dr. Vinod Kumar KanvariaYour Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective UpskillingExcellence Foundation for South Sudan
Strategies for Effective Upskilling is a presentation by Chinwendu Peace in a Your Skill Boost Masterclass organisation by the Excellence Foundation for South Sudan on 08th and 09th June 2024 from 1 PM to 3 PM on each day.How to Make a Field Mandatory in Odoo 17
In Odoo, making a field required can be done through both Python code and XML views. When you set the required attribute to True in Python code, it makes the field required across all views where it's used. Conversely, when you set the required attribute in XML views, it makes the field required only in the context of that particular view.
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama UniversityAkanksha trivedi rama nursing college kanpur.
Natural birth techniques are various type such as/ water birth , alexender method, hypnosis, bradley method, lamaze method etcMain Java[All of the Base Concepts}.docx
This is part 1 of my Java Learning Journey. This Contains Custom methods, classes, constructors, packages, multithreading , try- catch block, finally block and more.
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...Nguyen Thanh Tu Collection
https://app.box.com/s/tacvl9ekroe9hqupdnjruiypvm9rdaneReimagining Your Library Space: How to Increase the Vibes in Your Library No ...
Librarians are leading the way in creating future-ready citizens – now we need to update our spaces to match. In this session, attendees will get inspiration for transforming their library spaces. You’ll learn how to survey students and patrons, create a focus group, and use design thinking to brainstorm ideas for your space. We’ll discuss budget friendly ways to change your space as well as how to find funding. No matter where you’re at, you’ll find ideas for reimagining your space in this session.
How to Fix the Import Error in the Odoo 17
An import error occurs when a program fails to import a module or library, disrupting its execution. In languages like Python, this issue arises when the specified module cannot be found or accessed, hindering the program's functionality. Resolving import errors is crucial for maintaining smooth software operation and uninterrupted development processes.
Hindi varnamala | hindi alphabet PPT.pdf
हिंदी वर्णमाला पीपीटी, hindi alphabet PPT presentation, hindi varnamala PPT, Hindi Varnamala pdf, हिंदी स्वर, हिंदी व्यंजन, sikhiye hindi varnmala, dr. mulla adam ali, hindi language and literature, hindi alphabet with drawing, hindi alphabet pdf, hindi varnamala for childrens, hindi language, hindi varnamala practice for kids, https://www.drmullaadamali.com
Advanced Java[Extra Concepts, Not Difficult].docx
This is part 2 of my Java Learning Journey. This contains Hashing, ArrayList, LinkedList, Date and Time Classes, Calendar Class and more.
The Diamonds of 2023-2024 in the IGRA collection
A review of the growth of the Israel Genealogy Research Association Database Collection for the last 12 months. Our collection is now passed the 3 million mark and still growing. See which archives have contributed the most. See the different types of records we have, and which years have had records added. You can also see what we have for the future.
Recently uploaded (20)
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
Reimagining Your Library Space: How to Increase the Vibes in Your Library No ...
Reimagining Your Library Space: How to Increase the Vibes in Your Library No ...
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
Data engineering track module 2
- 1. Data Engineering Track Pranav Prakash Module 2 for University of Toronto, Mar 2022
- 2. Ensuring your model stays relevant • ML Life cycle • Edge Deployment • Model Feedback • Online Learning • AB Experiment
- 4. Production Grade Tools • MLFlow, DVC • Kube f low
- 5. Model Feedback Loop • Types • Interactive, Reinforcement • Direct, Hidden • Biases in Models (or Ghosts in Machines) • Ethnicity, Gender examples
- 6. Edge Deployments • Mobile • iOS • Android • RPi, Arduino • Industrial AI • Automotive
- 7. Mobile Devices • iOS - CoreML, coremltools python package • Android - TF Lite, PyTorch • ML Kit - On Device ML from Google • Top Considerations • Energy • Resources • Real Time • Internet/Connectivity
- 8. Architecture • Using Messaging Systems • ML Model versioning etc
- 9. AB Testing • Controlled randomised experiment to establish causality • How does a model contribute to “business objective”? • De f ine “Overall Evaluation Criterion”
- 10. AB Testing Architecture • De f ine experiment parameters (sample size, duration) • power (prob of false negative), level (prob of false positive) • Search Session = <User, Query, (Time Window)> • Associate UUID with each session • Split sessions between baseline (85-95%) and experimental models. • Capture feedback
- 12. Lessons learned • More data or better data • Simple models are better than complex, but complex models are sometimes needed • Biases in data • Evaluation approach • ML Engineering & Data Science
- 13. • https://www.wired.com/story/the-toxic-potential-of-youtubes-feedback- loop/ • https://win-vector.com/2014/05/03/a-clear-picture-of-power-and- signi f icance-in-ab-tests/ • https://signalvnoise.com/posts/3004-ab-testing-tech-note-determining- sample-size • https://abtestguide.com/abtestsize/ •