This document discusses ONNX (Open Neural Network Exchange) and its integration with MLflow for model portability and deployment. It provides an overview of ONNX, describing how it allows models to be trained in one framework and deployed in another. It then discusses several companies that support ONNX, including Microsoft's use of ONNX Runtime to accelerate models across various products, AWS and Nvidia's support, and Facebook and Intel's contributions. The document ends by explaining how MLflow recently added support for the ONNX format, allowing models to be exported to and loaded from ONNX.
Seamless End-to-End Production Machine Learning with Seldon and MLflowDatabricks
This document discusses using MLFlow to train machine learning models and Seldon to deploy them in a Kubernetes environment. It provides an example of using a wine quality dataset to train two ElasticNet regression models with MLFlow and deploy them for an A/B test using Seldon. Key steps covered include tracking experiments and hyperparameters with MLFlow, defining the model interface with an MLproject file, and creating the inference graph in Seldon to route traffic between the two models and provide a feedback loop.
A Microservices Framework for Real-Time Model Scoring Using Structured Stream...Databricks
Open-source technologies allow developers to build microservices framework to build myriad real-time applications. One such application is building the real-time model scoring. In this session,
we will showcase how to architect a microservice framework, in particular how to use it to build a low-latency, real-time model scoring system. At the core of the architecture lies Apache Spark’s Structured
Streaming capability to deliver low-latency predictions coupled with Docker and Flask as additional open source tools for model service. In this session, you will walk away with:
* Knowledge of enterprise-grade model as a service
* Streaming architecture design principles enabling real-time machine learning
* Key concepts and building blocks for real-time model scoring
* Real-time and production use cases across industries, such as IIOT, predictive maintenance, fraud detection, sepsis etc.
MLflow: Infrastructure for a Complete Machine Learning Life CycleDatabricks
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but these platforms are limited to each company’s internal infrastructure.
In this talk, we will present MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
MLFlow: Platform for Complete Machine Learning Lifecycle Databricks
Description
Data Science and ML development bring many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work.
MLflow addresses some of these challenges during an ML model development cycle.
Abstract
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure.
In this session, we introduce MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
With a short demo, you see a complete ML model life-cycle example, you will walk away with: MLflow concepts and abstractions for models, experiments, and projects How to get started with MLFlow Using tracking Python APIs during model training Using MLflow UI to visually compare and contrast experimental runs with different tuning parameters and evaluate metrics
Managing the Complete Machine Learning Lifecycle with MLflowDatabricks
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models.
To solve for these challenges, Databricks unveiled last year MLflow, an open source project that aims at simplifying the entire ML lifecycle. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
In the past year, the MLflow community has grown quickly: over 120 contributors from over 40 companies have contributed code to the project, and over 200 companies are using MLflow.
In this tutorial, we will show you how using MLflow can help you:
Keep track of experiments runs and results across frameworks.
Execute projects remotely on to a Databricks cluster, and quickly reproduce your runs.
Quickly productionize models using Databricks production jobs, Docker containers, Azure ML, or Amazon SageMaker.
We will demo the building blocks of MLflow as well as the most recent additions since the 1.0 release.
What you will learn:
Understand the three main components of open source MLflow (MLflow Tracking, MLflow Projects, MLflow Models) and how each help address challenges of the ML lifecycle.
How to use MLflow Tracking to record and query experiments: code, data, config, and results.
How to use MLflow Projects packaging format to reproduce runs on any platform.
How to use MLflow Models general format to send models to diverse deployment tools.
Prerequisites:
A fully-charged laptop (8-16GB memory) with Chrome or Firefox
Python 3 and pip pre-installed
Pre-Register for a Databricks Standard Trial
Basic knowledge of Python programming language
Basic understanding of Machine Learning Concepts
MLflow is aiming to stabilize its API in version 1.0 this spring and add a number of other new features. In this talk, we'll share some of the features we have in mind for the rest of the year. These include ongoing work such as a database store for the tracking server and Docker project packaging, as well as new improvements in multi-step workflows, model management, and production monitoring.
During this presentation, after walking through a few ways to use MLflow on Azure directly, we'll cover how upcoming solutions from our group leverage MLflow for core functionality. BenchML is a new repository that aims to provide consumers of prebuilt ML endpoints visibility into the performance of each public offering for a given dataset as well as comparing results across multiple offerings. Using MLflow, BenchML is able to remain cloud-agnostic and offer a delightful local experience while leveraging the aforementioned integration to provide Azure users with a fully managed experience.
Speaker Bio: Akshaya is an engineer in the AI Platform at Microsoft, having released both GA versions of Azure Machine Learning over the years and the OSS repo MMLSpark. As the recent version of Azure ML pivoted to become more of an open platform rather than a managed product, his focus has shifted outward for open-source platform definitions for cloud-scale implementations and focused on MLflow for the Azure ML managed tracking store.
This talk was presented at the Bay Area MLflow Meetup at Databricks HQs in San Francisco: https://www.meetup.com/Bay-Area-MLflow/events/266614106/
This document discusses ONNX (Open Neural Network Exchange) and its integration with MLflow for model portability and deployment. It provides an overview of ONNX, describing how it allows models to be trained in one framework and deployed in another. It then discusses several companies that support ONNX, including Microsoft's use of ONNX Runtime to accelerate models across various products, AWS and Nvidia's support, and Facebook and Intel's contributions. The document ends by explaining how MLflow recently added support for the ONNX format, allowing models to be exported to and loaded from ONNX.
Seamless End-to-End Production Machine Learning with Seldon and MLflowDatabricks
This document discusses using MLFlow to train machine learning models and Seldon to deploy them in a Kubernetes environment. It provides an example of using a wine quality dataset to train two ElasticNet regression models with MLFlow and deploy them for an A/B test using Seldon. Key steps covered include tracking experiments and hyperparameters with MLFlow, defining the model interface with an MLproject file, and creating the inference graph in Seldon to route traffic between the two models and provide a feedback loop.
A Microservices Framework for Real-Time Model Scoring Using Structured Stream...Databricks
Open-source technologies allow developers to build microservices framework to build myriad real-time applications. One such application is building the real-time model scoring. In this session,
we will showcase how to architect a microservice framework, in particular how to use it to build a low-latency, real-time model scoring system. At the core of the architecture lies Apache Spark’s Structured
Streaming capability to deliver low-latency predictions coupled with Docker and Flask as additional open source tools for model service. In this session, you will walk away with:
* Knowledge of enterprise-grade model as a service
* Streaming architecture design principles enabling real-time machine learning
* Key concepts and building blocks for real-time model scoring
* Real-time and production use cases across industries, such as IIOT, predictive maintenance, fraud detection, sepsis etc.
MLflow: Infrastructure for a Complete Machine Learning Life CycleDatabricks
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but these platforms are limited to each company’s internal infrastructure.
In this talk, we will present MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
MLFlow: Platform for Complete Machine Learning Lifecycle Databricks
Description
Data Science and ML development bring many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work.
MLflow addresses some of these challenges during an ML model development cycle.
Abstract
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure.
In this session, we introduce MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
With a short demo, you see a complete ML model life-cycle example, you will walk away with: MLflow concepts and abstractions for models, experiments, and projects How to get started with MLFlow Using tracking Python APIs during model training Using MLflow UI to visually compare and contrast experimental runs with different tuning parameters and evaluate metrics
Managing the Complete Machine Learning Lifecycle with MLflowDatabricks
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models.
To solve for these challenges, Databricks unveiled last year MLflow, an open source project that aims at simplifying the entire ML lifecycle. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
In the past year, the MLflow community has grown quickly: over 120 contributors from over 40 companies have contributed code to the project, and over 200 companies are using MLflow.
In this tutorial, we will show you how using MLflow can help you:
Keep track of experiments runs and results across frameworks.
Execute projects remotely on to a Databricks cluster, and quickly reproduce your runs.
Quickly productionize models using Databricks production jobs, Docker containers, Azure ML, or Amazon SageMaker.
We will demo the building blocks of MLflow as well as the most recent additions since the 1.0 release.
What you will learn:
Understand the three main components of open source MLflow (MLflow Tracking, MLflow Projects, MLflow Models) and how each help address challenges of the ML lifecycle.
How to use MLflow Tracking to record and query experiments: code, data, config, and results.
How to use MLflow Projects packaging format to reproduce runs on any platform.
How to use MLflow Models general format to send models to diverse deployment tools.
Prerequisites:
A fully-charged laptop (8-16GB memory) with Chrome or Firefox
Python 3 and pip pre-installed
Pre-Register for a Databricks Standard Trial
Basic knowledge of Python programming language
Basic understanding of Machine Learning Concepts
MLflow is aiming to stabilize its API in version 1.0 this spring and add a number of other new features. In this talk, we'll share some of the features we have in mind for the rest of the year. These include ongoing work such as a database store for the tracking server and Docker project packaging, as well as new improvements in multi-step workflows, model management, and production monitoring.
During this presentation, after walking through a few ways to use MLflow on Azure directly, we'll cover how upcoming solutions from our group leverage MLflow for core functionality. BenchML is a new repository that aims to provide consumers of prebuilt ML endpoints visibility into the performance of each public offering for a given dataset as well as comparing results across multiple offerings. Using MLflow, BenchML is able to remain cloud-agnostic and offer a delightful local experience while leveraging the aforementioned integration to provide Azure users with a fully managed experience.
Speaker Bio: Akshaya is an engineer in the AI Platform at Microsoft, having released both GA versions of Azure Machine Learning over the years and the OSS repo MMLSpark. As the recent version of Azure ML pivoted to become more of an open platform rather than a managed product, his focus has shifted outward for open-source platform definitions for cloud-scale implementations and focused on MLflow for the Azure ML managed tracking store.
This talk was presented at the Bay Area MLflow Meetup at Databricks HQs in San Francisco: https://www.meetup.com/Bay-Area-MLflow/events/266614106/
The Fast Path to Building Operational Applications with SparkSingleStore
Nikita Shamgunov gave a presentation about using MemSQL and Spark together. MemSQL is a scalable operational database that can handle petabytes of data with high concurrency. It offers real-time capabilities and compatibility with tools like Spark, Kafka, and ETL/BI tools. The MemSQL Spark Connector allows bidirectional transfer of data between Spark and MemSQL tables for use cases like operationalizing models in Spark, stream/event processing, and live dashboards. Case studies showed customers gaining 10x faster data refresh times and performing entity resolution at scale for fraud detection.
How to Utilize MLflow and Kubernetes to Build an Enterprise ML PlatformDatabricks
This document summarizes a presentation about utilizing MLFlow and Kubernetes to build an enterprise machine learning platform. It discusses challenges that motivated building such a platform, like lack of model management and difficult deployments. The solution presented abstracts data pipelines into modular components to standardize workflows. It also uses MLFlow to package and track models and experiments, and Kubernetes with Kubeflow to deploy models at scale. A demo shows implementing model serving with these tools.
Productionzing ML Model Using MLflow Model ServingDatabricks
Productionzing ML Models are needs to ensure model integrity while it efficiently replicate runtime environments across servers besides it keep track of how each of our models were created. It helps us better trace the root cause of changes and issues over time as we acquire new data and update our model. We have greater accountability over our models and the results they generate.
MLflow Model Serving delivers cost-effective and on-click deployment of model for real-time inferences. Also the Model Version deployed in the Model Serving can also be conveniently managed with MLflow Model Registry. We will going to cover following topics Deployment, Consumption and Monitoring. For deployment, we will demo the different version deployment and validate the deployment. For consumption, we demo connecting power bi and generate prediction report using ML Model deployed in MLflow serving. Lastly will wrap up with managing the MLflow serving like, access rights and monitoring capabilities.
Use MLflow to manage and deploy Machine Learning model on Spark Herman Wu
MLflow is a platform for managing the machine learning lifecycle, including tracking experiments, packaging models into reproducible projects, and deploying models into production. It provides APIs and tools to help with experiment tracking, model deployment, and model monitoring. MLflow supports many frameworks like PyTorch, TensorFlow, Keras, and Spark ML. It integrates with cloud platforms like Azure ML, AWS SageMaker, and Databricks to enable deployment and management of models on those platforms.
Jingshuang Ge is seeking an entry-level software engineer position with relevant education and projects. They have a Master of Science from the University of Florida in Electronic and Computer Engineering with a minor in Computer Science. Their Bachelor's is from Zhejiang University in Electrical Science & Technology Engineering. Notable projects include developing a hierarchical file system with server replication and fault tolerance using Linux, Python, XMLRPC, MongoDB and Memcache. They also created a P2P file transmission network resembling BitTorrent using Java, Eclipse and TCP. Ge has internship experience as a web developer using PHP, Zoos, JSON, MySQL, JIRA and Agile methodology.
Adopting software design practices for better machine learningMLconf
Jeff McGehee discusses Google's Rules of Machine Learning and introduces the concept of Speed of Delivery and Quality of Product (SoDQoP) for evaluating machine learning projects over time. He argues that process is the area with the most room for improvement in machine learning. The document provides recommendations for building lean machine learning systems through fast iteration, leveraging existing machine learning APIs, and making it easy to improve models with new data.
Winning the On-Demand Economy with Spark and Predictive AnalyticsSingleStore
Today’s on-demand economy drives companies to provide fast load times, personalization, and instantaneous service for hungry end-users across all types of applications. Yet most still use dated, legacy systems to process and analyze data. In this session, Ankur Goyal, VP of Engineering at MemSQL will showcase implementing a one-click Lambda Architecture with Apache Spark, Apache Kafka and an operational database, resulting in lightning fast analytics on large, changing datasets.
In the last several months, MLflow has introduced significant platform enhancements that simplify machine learning lifecycle management. Expanded autologging capabilities, including a new integration with scikit-learn, have streamlined the instrumentation and experimentation process in MLflow Tracking. Additionally, schema management functionality has been incorporated into MLflow Models, enabling users to seamlessly inspect and control model inference APIs for batch and real-time scoring. In this session, we will explore these new features. We will share MLflow’s development roadmap, providing an overview of near-term advancements in the platform.
Advanced MLflow: Multi-Step Workflows, Hyperparameter Tuning and Integrating ...Databricks
Because MLflow is an API-first platform, there are many patterns for using it in complex workflows and integrating it with existing tools. In this talk, we’ll demo a few best practices for using MLflow in a more complex workflow. These include:
* Run multi-step workflows on MLflow, such as data preparation steps followed by training, and organizing your projects so you can automatically reuse past work.
* Tune Hyperparameter on MLflow with open source hyperparameter tuning packages.
* Save a model in MLflow (eg, from a new machine learning library) and deploying it to the existing deployment tools.
Sathis Kumar is a fullstack tech lead with over 9 years of experience developing web and mobile applications. Some of his accomplishments include building an anomaly detection framework at Myntra that averted over 40 anomalies with 94% recall, and developing a dynamic segmentation platform that targeted 170 million users and improved performance 4x. He also has experience leading teams and open source development.
Splice Machine's use of Apache Spark and MLflowDatabricks
Splice Machine is an ANSI-SQL Relational Database Management System (RDBMS) on Apache Spark. It has proven low-latency transactional processing (OLTP) as well as analytical processing (OLAP) at petabyte scale. It uses Spark for all analytical computations and leverages HBase for persistence. This talk highlights a new Native Spark Datasource - which enables seamless data movement between Spark Data Frames and Splice Machine tables without serialization and deserialization. This Spark Datasource makes machine learning libraries such as MLlib native to the Splice RDBMS . Splice Machine has now integrated MLflow into its data platform, creating a flexible Data Science Workbench with an RDBMS at its core. The transactional capabilities of Splice Machine integrated with the plethora of DataFrame-compatible libraries and MLflow capabilities manages a complete, real-time workflow of data-to-insights-to-action. In this presentation we will demonstrate Splice Machine's Data Science Workbench and how it leverages Spark and MLflow to create powerful, full-cycle machine learning capabilities on an integrated platform, from transactional updates to data wrangling, experimentation, and deployment, and back again.
The Lyft data platform: Now and in the futuremarkgrover
- Lyft has grown significantly in recent years, providing over 1 billion rides to 30.7 million riders through 1.9 million drivers in 2018 across North America.
- Data is core to Lyft's business decisions, from pricing and driver matching to analyzing performance and informing investments.
- Lyft's data platform supports data scientists, analysts, engineers and others through tools like Apache Superset, change data capture from operational stores, and streaming frameworks.
- Key focuses for the platform include business metric observability, streaming applications, and machine learning while addressing challenges of reliability, integration and scale.
Bighead: Airbnb’s End-to-End Machine Learning Platform with Krishna Puttaswa...Databricks
Bighead is Airbnb's machine learning infrastructure that was created to:
- Standardize and simplify the ML development workflow;
- Reduce the time and effort to build ML models from weeks/months to days/weeks; and
- Enable more teams at Airbnb to utilize ML.
It provides shared services and tools for data management, model training/inference, and model management to make the ML process more efficient and production-ready. This includes services like Zipline for feature storage, Redspot for notebook environments, Deep Thought for online inference, and the Bighead UI for model monitoring.
Facebook leverages machine learning extensively for applications like News Feed, ads, search, and language translation. It designs its own server hardware optimized for different ML workloads and data centers globally. It develops AI frameworks for both production stability and research flexibility. Its ML execution flows involve large-scale data processing, distributed training, and high-volume inference. Supporting ML at Facebook's massive scale presents infrastructure challenges in storage, networking, and computing resources.
Alexandra johnson reducing operational barriers to model trainingMLconf
This document discusses reducing operational barriers to machine learning model training through building machine learning infrastructure. It presents challenges faced by both machine learning experts and infrastructure engineers. It then describes SigOpt's solution of building SigOpt Orchestrate to address these challenges through containerization, Kubernetes for parallel training, and a command line interface for viewing progress and debugging. The final slides invite connecting with SigOpt and note they are hiring.
The document describes the features and capabilities of Visual Studio Tools for AI, an AI developer tool for training models and integrating AI into applications. It can create deep learning projects with frameworks like TensorFlow and CNTK, debug and iterate quickly in Visual Studio. It is integrated with Azure Machine Learning for management of experiments and models, and can scale out training with Azure Batch AI. The tool allows monitoring of training, visualization with TensorBoard, and generation of code from trained models.
Reaktive Programmierung mit den Reactive Extensions (Rx)NETUserGroupBern
This document provides an overview of a presentation on reactive programming. The presentation introduces reactive programming concepts and the Reactive Extensions (RX) framework in .NET. It discusses how RX treats events as observable collections and uses LINQ-like query operators to compose and combine event streams. The presentation includes demos showing how to create observables from events and asynchronous data sources and use RX operators to query and transform the resulting observable sequences.
Sinux is a Linux distribution containing tools for developing Semantic Web applications. It includes libraries, frameworks, editors, IDEs, and other resources for working with microformats, RDF, SPARQL, ontologies, and the Semantic Web. Some featured software includes MonoDevelop, NetBeans IDE, Protege, Jena, and Redland. The distribution aims to provide a hands-on approach to exploring the Semantic Web through its included tools and applications.
The Fast Path to Building Operational Applications with SparkSingleStore
Nikita Shamgunov gave a presentation about using MemSQL and Spark together. MemSQL is a scalable operational database that can handle petabytes of data with high concurrency. It offers real-time capabilities and compatibility with tools like Spark, Kafka, and ETL/BI tools. The MemSQL Spark Connector allows bidirectional transfer of data between Spark and MemSQL tables for use cases like operationalizing models in Spark, stream/event processing, and live dashboards. Case studies showed customers gaining 10x faster data refresh times and performing entity resolution at scale for fraud detection.
How to Utilize MLflow and Kubernetes to Build an Enterprise ML PlatformDatabricks
This document summarizes a presentation about utilizing MLFlow and Kubernetes to build an enterprise machine learning platform. It discusses challenges that motivated building such a platform, like lack of model management and difficult deployments. The solution presented abstracts data pipelines into modular components to standardize workflows. It also uses MLFlow to package and track models and experiments, and Kubernetes with Kubeflow to deploy models at scale. A demo shows implementing model serving with these tools.
Productionzing ML Model Using MLflow Model ServingDatabricks
Productionzing ML Models are needs to ensure model integrity while it efficiently replicate runtime environments across servers besides it keep track of how each of our models were created. It helps us better trace the root cause of changes and issues over time as we acquire new data and update our model. We have greater accountability over our models and the results they generate.
MLflow Model Serving delivers cost-effective and on-click deployment of model for real-time inferences. Also the Model Version deployed in the Model Serving can also be conveniently managed with MLflow Model Registry. We will going to cover following topics Deployment, Consumption and Monitoring. For deployment, we will demo the different version deployment and validate the deployment. For consumption, we demo connecting power bi and generate prediction report using ML Model deployed in MLflow serving. Lastly will wrap up with managing the MLflow serving like, access rights and monitoring capabilities.
Use MLflow to manage and deploy Machine Learning model on Spark Herman Wu
MLflow is a platform for managing the machine learning lifecycle, including tracking experiments, packaging models into reproducible projects, and deploying models into production. It provides APIs and tools to help with experiment tracking, model deployment, and model monitoring. MLflow supports many frameworks like PyTorch, TensorFlow, Keras, and Spark ML. It integrates with cloud platforms like Azure ML, AWS SageMaker, and Databricks to enable deployment and management of models on those platforms.
Jingshuang Ge is seeking an entry-level software engineer position with relevant education and projects. They have a Master of Science from the University of Florida in Electronic and Computer Engineering with a minor in Computer Science. Their Bachelor's is from Zhejiang University in Electrical Science & Technology Engineering. Notable projects include developing a hierarchical file system with server replication and fault tolerance using Linux, Python, XMLRPC, MongoDB and Memcache. They also created a P2P file transmission network resembling BitTorrent using Java, Eclipse and TCP. Ge has internship experience as a web developer using PHP, Zoos, JSON, MySQL, JIRA and Agile methodology.
Adopting software design practices for better machine learningMLconf
Jeff McGehee discusses Google's Rules of Machine Learning and introduces the concept of Speed of Delivery and Quality of Product (SoDQoP) for evaluating machine learning projects over time. He argues that process is the area with the most room for improvement in machine learning. The document provides recommendations for building lean machine learning systems through fast iteration, leveraging existing machine learning APIs, and making it easy to improve models with new data.
Winning the On-Demand Economy with Spark and Predictive AnalyticsSingleStore
Today’s on-demand economy drives companies to provide fast load times, personalization, and instantaneous service for hungry end-users across all types of applications. Yet most still use dated, legacy systems to process and analyze data. In this session, Ankur Goyal, VP of Engineering at MemSQL will showcase implementing a one-click Lambda Architecture with Apache Spark, Apache Kafka and an operational database, resulting in lightning fast analytics on large, changing datasets.
In the last several months, MLflow has introduced significant platform enhancements that simplify machine learning lifecycle management. Expanded autologging capabilities, including a new integration with scikit-learn, have streamlined the instrumentation and experimentation process in MLflow Tracking. Additionally, schema management functionality has been incorporated into MLflow Models, enabling users to seamlessly inspect and control model inference APIs for batch and real-time scoring. In this session, we will explore these new features. We will share MLflow’s development roadmap, providing an overview of near-term advancements in the platform.
Advanced MLflow: Multi-Step Workflows, Hyperparameter Tuning and Integrating ...Databricks
Because MLflow is an API-first platform, there are many patterns for using it in complex workflows and integrating it with existing tools. In this talk, we’ll demo a few best practices for using MLflow in a more complex workflow. These include:
* Run multi-step workflows on MLflow, such as data preparation steps followed by training, and organizing your projects so you can automatically reuse past work.
* Tune Hyperparameter on MLflow with open source hyperparameter tuning packages.
* Save a model in MLflow (eg, from a new machine learning library) and deploying it to the existing deployment tools.
Sathis Kumar is a fullstack tech lead with over 9 years of experience developing web and mobile applications. Some of his accomplishments include building an anomaly detection framework at Myntra that averted over 40 anomalies with 94% recall, and developing a dynamic segmentation platform that targeted 170 million users and improved performance 4x. He also has experience leading teams and open source development.
Splice Machine's use of Apache Spark and MLflowDatabricks
Splice Machine is an ANSI-SQL Relational Database Management System (RDBMS) on Apache Spark. It has proven low-latency transactional processing (OLTP) as well as analytical processing (OLAP) at petabyte scale. It uses Spark for all analytical computations and leverages HBase for persistence. This talk highlights a new Native Spark Datasource - which enables seamless data movement between Spark Data Frames and Splice Machine tables without serialization and deserialization. This Spark Datasource makes machine learning libraries such as MLlib native to the Splice RDBMS . Splice Machine has now integrated MLflow into its data platform, creating a flexible Data Science Workbench with an RDBMS at its core. The transactional capabilities of Splice Machine integrated with the plethora of DataFrame-compatible libraries and MLflow capabilities manages a complete, real-time workflow of data-to-insights-to-action. In this presentation we will demonstrate Splice Machine's Data Science Workbench and how it leverages Spark and MLflow to create powerful, full-cycle machine learning capabilities on an integrated platform, from transactional updates to data wrangling, experimentation, and deployment, and back again.
The Lyft data platform: Now and in the futuremarkgrover
- Lyft has grown significantly in recent years, providing over 1 billion rides to 30.7 million riders through 1.9 million drivers in 2018 across North America.
- Data is core to Lyft's business decisions, from pricing and driver matching to analyzing performance and informing investments.
- Lyft's data platform supports data scientists, analysts, engineers and others through tools like Apache Superset, change data capture from operational stores, and streaming frameworks.
- Key focuses for the platform include business metric observability, streaming applications, and machine learning while addressing challenges of reliability, integration and scale.
Bighead: Airbnb’s End-to-End Machine Learning Platform with Krishna Puttaswa...Databricks
Bighead is Airbnb's machine learning infrastructure that was created to:
- Standardize and simplify the ML development workflow;
- Reduce the time and effort to build ML models from weeks/months to days/weeks; and
- Enable more teams at Airbnb to utilize ML.
It provides shared services and tools for data management, model training/inference, and model management to make the ML process more efficient and production-ready. This includes services like Zipline for feature storage, Redspot for notebook environments, Deep Thought for online inference, and the Bighead UI for model monitoring.
Facebook leverages machine learning extensively for applications like News Feed, ads, search, and language translation. It designs its own server hardware optimized for different ML workloads and data centers globally. It develops AI frameworks for both production stability and research flexibility. Its ML execution flows involve large-scale data processing, distributed training, and high-volume inference. Supporting ML at Facebook's massive scale presents infrastructure challenges in storage, networking, and computing resources.
Alexandra johnson reducing operational barriers to model trainingMLconf
This document discusses reducing operational barriers to machine learning model training through building machine learning infrastructure. It presents challenges faced by both machine learning experts and infrastructure engineers. It then describes SigOpt's solution of building SigOpt Orchestrate to address these challenges through containerization, Kubernetes for parallel training, and a command line interface for viewing progress and debugging. The final slides invite connecting with SigOpt and note they are hiring.
The document describes the features and capabilities of Visual Studio Tools for AI, an AI developer tool for training models and integrating AI into applications. It can create deep learning projects with frameworks like TensorFlow and CNTK, debug and iterate quickly in Visual Studio. It is integrated with Azure Machine Learning for management of experiments and models, and can scale out training with Azure Batch AI. The tool allows monitoring of training, visualization with TensorBoard, and generation of code from trained models.
Reaktive Programmierung mit den Reactive Extensions (Rx)NETUserGroupBern
This document provides an overview of a presentation on reactive programming. The presentation introduces reactive programming concepts and the Reactive Extensions (RX) framework in .NET. It discusses how RX treats events as observable collections and uses LINQ-like query operators to compose and combine event streams. The presentation includes demos showing how to create observables from events and asynchronous data sources and use RX operators to query and transform the resulting observable sequences.
Sinux is a Linux distribution containing tools for developing Semantic Web applications. It includes libraries, frameworks, editors, IDEs, and other resources for working with microformats, RDF, SPARQL, ontologies, and the Semantic Web. Some featured software includes MonoDevelop, NetBeans IDE, Protege, Jena, and Redland. The distribution aims to provide a hands-on approach to exploring the Semantic Web through its included tools and applications.
Come può .NET contribuire alla Data Science? Cosa è .NET Interactive? Cosa c'entrano i notebook? E Apache Spark? E il pythonismo? E Azure? Vediamo in questa sessione di mettere in ordine le idee.
Join the largest JavaScript Conference in Israel Things have changed. Today JS is the language of the web. On the client, on the server, on the mobile, everywhere. But let's face it: JS still has challenges.
Reactive Programming on Android - RxAndroid - RxJavaAli Muzaffar
Introduction to RxJava for reactive programming and how to use RxAndroid to do reactive programming on Android.
There is a sample android app to go with the slides that has all the source shown in the project.
This ppt is all about the MERN stack in JavaScript.
So, will first discuss:
What is Mern,
what is MongoDB, expressJS,reactJS and nodeJS.
Also, I bit more discussed the Installation process.
Getting the MERN stacks will sure increase your web development skills in 2020.
The document discusses Ruby on Rails, a web application framework. It provides an overview of Ruby and Rails, explaining that Ruby is an object-oriented programming language and Rails is a full-stack framework built on Ruby that follows the model-view-controller pattern. It also discusses how Rails emphasizes conventions over configuration and helps developers build applications quickly.
The recent development of the World Wide Web has moved from static HTML pages to more dynamic and intelligent web applications with desktop-like characteristics. AJAX (Asynchronous JavaScript and XML) has been the main driver of this change, allowing for asynchronous updates to web application content from the server without page reloads. AJAX uses a combination of technologies including JavaScript, XML, and HTTP requests to update partial web page content dynamically. This enables web applications to have more interactive interfaces and deliver a more desktop-like experience to users.
The Recent Pronouncement Of The World Wide Web (Www) HadDeborah Gastineau
Here are some key pros and disadvantages of ORM impedance mismatching:
Pros:
- ORMs allow developers to work with objects in code rather than raw SQL, which can be more intuitive and productive. This object-relational mapping handles converting between objects and relational structures.
Disadvantages:
- Impedance mismatch occurs when object models do not map cleanly to the relational model that databases use. This can result in inefficient queries, unnecessary joins, or an inability to represent certain relationships between entities.
- Complex object graphs can be difficult to represent in a relational schema and require denormalization of data. This impacts performance and scalability.
- Queries may need to be constructed programmatically
Alok Ranjan Singh is seeking a position as a Software Developer. He has 2 years and 9 months of experience designing and implementing client-server solutions using technologies like Adobe Flex, Java, MySQL, HTML5 and JavaScript. He is proficient in languages like Java and C/C++ and frameworks like Spring MVC. He currently works at Ericsson India developing network performance tools using technologies like Flex, Java, HTML5 and JavaScript. His responsibilities include requirements analysis, database design, data collection, report generation and unit testing. He holds a B.Tech in Information Technology from NIT Patna.
Node.js is an asynchronous event-driven JavaScript runtime that uses non-blocking I/O to build scalable network applications. It allows for the creation of web servers and networking tools using a event-driven, non-blocking I/O model rather than the traditional threaded model. Node.js is popular because it uses JavaScript and allows code reuse on both the server-side and client-side, offers high performance and scalability for real-time applications, and has a large passionate community supporting its use.
The document lists recent research areas, current topics, and latest topics that can be used for MATLAB-based projects. Some areas mentioned include LTE systems, neural networks, artificial intelligence, robotics, mobile cloud computing, and wireless sensor networks. Current topics include deep convolutional networks, model-based uncertainty reduction, probabilistic frameworks, and minimum uncertainty gap estimation. The five top latest topics listed are unified multiscale frameworks, realigning 2D and 3D object fragments, large-scale image classification, graphical representations for face recognition, and privacy-preserving frequent visual patterns publication. Contact details are provided at the end.
As presented to the Milwaukee Alt.Net group on November 21st, 2011.
UPDATE April 19, 2012: added some domain logic organization slides using Fowler's 4 basic patterns.
Each framework outlined above has characteristics that differentiate it from others. Like Express.JS, It can be used for any enterprise application that requires cross-browser compatibility, and it supports both small and large-scale websites. Meteor.JS is good if you have a development team that is good with JavaScript. It has a smooth learning curve. Hapi can be used if you want the utmost security, and Koa should be used for developing complex, extensive apps with speed and efficiency.
If you are still unsure about which of these Node.JS frameworks would be the best fit for you, do not hesitate to get in touch with our Node.JS developers.
React Js vs Node Js_ Which Framework to Choose for Your Next Web Applicationadityakumar2080
React js is more of a JavaScript library than a framework that is developed and maintained by Facebook. Initially released in May 2013, it is a tool for building UI components. React js has an MVC or Model View Controller architecture. Node js is an open-source Google Chrome v8 JavaScript framework. The main idea of developing applications in Node js is to execute JavaScript on the server-side.
The document discusses Microsoft's Entity Framework ORM technology. It provides an overview of Entity Framework and how it compares to other ORM technologies like LINQ to SQL. It also outlines Microsoft's strategy of promoting Entity Framework as the preferred .NET ORM going forward.
Node.js is an open source JavaScript runtime environment used to build real-time web applications. The document introduces Node.js, the npm package manager, the Express web application framework, and how to connect a Node.js application to MongoDB. Key points covered include Node.js being single-threaded and event-driven, using npm to install and use packages locally or globally, Express providing features for building web apps and APIs, and using modules like Mongoose to connect a Node app to MongoDB.
Similar to Creating a simple doodle (drawing) like application using silverlight and reactive framework (rx) (20)
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
A Comprehensive Guide to DeFi Development Services in 2024Intelisync
DeFi represents a paradigm shift in the financial industry. Instead of relying on traditional, centralized institutions like banks, DeFi leverages blockchain technology to create a decentralized network of financial services. This means that financial transactions can occur directly between parties, without intermediaries, using smart contracts on platforms like Ethereum.
In 2024, we are witnessing an explosion of new DeFi projects and protocols, each pushing the boundaries of what’s possible in finance.
In summary, DeFi in 2024 is not just a trend; it’s a revolution that democratizes finance, enhances security and transparency, and fosters continuous innovation. As we proceed through this presentation, we'll explore the various components and services of DeFi in detail, shedding light on how they are transforming the financial landscape.
At Intelisync, we specialize in providing comprehensive DeFi development services tailored to meet the unique needs of our clients. From smart contract development to dApp creation and security audits, we ensure that your DeFi project is built with innovation, security, and scalability in mind. Trust Intelisync to guide you through the intricate landscape of decentralized finance and unlock the full potential of blockchain technology.
Ready to take your DeFi project to the next level? Partner with Intelisync for expert DeFi development services today!
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Creating a simple doodle (drawing) like application using silverlight and reactive framework (rx)
1. Creating a simple Doodle (drawing) like application using Silverlight and Reactive Framework (RX) Matt Duffield, MVP, MCSD .NET http://mattduffield.wordpress.com @mattduffield
2. What is the Reactive Framework? IEnumerator IObserver IObserver modes Rx keywords Demo code What we are going to cover
3. Rx is a library for composing asynchronous and event-based programs using observable collections. Where can I use Rx? .NET Framework 3.5 SP1, 4 Silverlight 3, 4, 5 Windows Phone 7 XNA 4.0 for Xbox 360 Zune JavaScript What is the Reactive Framework?
4. Asynchronous and event-based – as reflected in the title, the bread and butter of Rx’s mission statement is to simplify those programming models. Composition – combining asynchronous computations today is way too hard. It involves a lot of plumbing code that has little to nothing to do with the problem being solved. Observable collections – by looking at asynchronous computations as data sources, we can leverage the active knowledge of LINQ’s programming model. Three core properties
5.
6. Stock tickers RSS feeds Social media UI events GPS Server management Why does it help?
7.
8.
9.
10.
11.
12.
13.
14.
15.
16. public interface IEnumerator { object Current { get; } boolMoveNext(); void Reset(); } Because we are waiting/”pulling” for a return value, these are blocking. IEnumerator
17. public interface IObserver<T> { void OnNext(T value); void OnError(Exception ex); void OnCompleted(); } We get “pushed” when a new value is available. IObserver
18. Hot – typically values that come in from events or a web service Cold – known list of values from a collection, array, etc. IObserver modes
19. FromEventPattern<T,U> Merge SkipUntil TakeUntil Throttle ObserveOn(Schedular.ThreadPool) Do TimeInterval – allow us to test for something over time Subject<> FromAsnyncPattern<T,U> Rx Keywords
21. Reactive Extensions http://msdn.microsoft.com/en-us/data/gg577609 Erik Meijer and Wes Dyer Reactive Framework (RX) Under the Hood 1 of 2 http://channel9.msdn.com/Shows/Going+Deep/E2E-Erik-Meijer-and-Wes-Dyer-Reactive-Framework-Rx-Under-the-Hood-1-of-2 Erik Meijer and Wes Dyer Reactive Framework (RX) Under the Hood 2 of 2 http://channel9.msdn.com/Shows/Going+Deep/E2E-Erik-Meijer-and-Wes-Dyer-Reactive-Framework-Rx-Under-the-Hood-2-of-2 Bart de Smet http://channel9.msdn.com/Blogs/joshholmes/Bart-de-Smet-in-Dublin-on-the-Rx-Framework Jim Wooley – ThinqLinq http://www.thinqlinq.com/ http://live.visitmix.com/MIX11/Sessions/Speaker/Jim-Wooley References
Asynchronous and event-based – As reflected in the title, the bread and butter of Rx’s mission statement is to simplify those programming models. Everyone knows what stuck user interfaces look like, both on the Windows platform and on the web. And with the cloud around the corner, asynchrony becomes quintessential. Low-level technologies like .NET events, the asynchronous pattern, tasks, AJAX, etc. are often too hard. Composition – Combining asynchronous computations today is way too hard. It involves a lot of plumbing code that has little to nothing to do with the problem being solved. In particular, the data flow of the operations involved in the problem is not clear at all, and code gets spread out throughout event handlers, asynchronous callback procedures, and whatnot. Observable collections – By looking at asynchronous computations as data sources, we can leverage the active knowledge of LINQ’s programming model. That’s right: your mouse is a database of mouse moves and clicks. In the world of Rx, such asynchronous data sources are composed using various combinators in the LINQ sense, allowing things like filters, projections, joins, time-based operations, etc.