This document provides an overview of building a custom machine learning model for image classification on Android. It begins with discussing challenges and ideas, then covers key deep learning concepts like data, tasks, models, loss functions, learning algorithms and evaluation. It explains that a MobileNet model will be retrained for classifying images of artisanal beers. The document also discusses converting the model to TensorFlow Lite and implementing image classification in an Android app using the camera and a TensorFlow Lite interpreter to get classification results.
Creating a custom ML model for your application - DevFest Lima 2019Isabel Palomar
Aprenderás como puede ser creado un modelo de Machine Learning que puedas implementar en tus aplicaciones. Iré mostrando cada uno de los pasos que se tienen que seguir, los tipos de problemas que se pueden resolver, los datos que necesitas para que funcione y por último, las opciones para realizar la implementación de nuestro modelo en nuestras aplicaciones.
Creating a custom Machine Learning Model for your applications - Java Dev Day...Isabel Palomar
Aprenderás como puede ser creado un modelo de Machine Learning que puedas implementar en tu aplicación móvil o Java. Iré mostrando cada uno de los pasos que se tienen que seguir, los tipos de problemas que se pueden resolver, los datos que necesitas para que funcione y por último, las opciones para realizar la implementación de nuestro modelo en nuestras aplicaciones.
Getting Started with OpenCV provides an overview of OpenCV and demonstrates a basic OpenCV program. It discusses OpenCV's structure, loading and saving images, creating windows and trackbars, and using OpenCV with Intel's Integrated Performance Primitives for accelerated computer vision functions. The document contains code samples and explains how to compile, build, and run OpenCV programs on Windows and Linux.
Inteligencia artificial para android como empezarIsabel Palomar
Aprenderás los conceptos basico de deep learning y como crear tu aplicación de Android que puede detectar y etiquetar imágenes utilizando un modelo de Tensorflow Lite
The document provides an overview and agenda for an introduction to running AI workloads on PowerAI. It discusses PowerAI and how it combines popular deep learning frameworks, development tools, and accelerated IBM Power servers. It then demonstrates AI workloads using TensorFlow and PyTorch, including running an MNIST workload to classify handwritten digits using basic linear regression and convolutional neural networks in TensorFlow, and an introduction to PyTorch concepts like tensors, modules, and softmax cross entropy loss.
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Databricks
A long time ago, there was Caffe and Theano, then came Torch and CNTK and Tensorflow, Keras and MXNet and Pytorch and Caffe2….a sea of Deep learning tools but none for Spark developers to dip into. Finally, there was BigDL, a deep learning library for Apache Spark. While BigDL is integrated into Spark and extends its capabilities to address the challenges of Big Data developers, will a library alone be enough to simplify and accelerate the deployment of ML/DL workloads on production clusters? From high level pipeline API support to feature transformers to pre-defined models and reference use cases, a rich repository of easy to use tools are now available with the ‘Analytics Zoo’. We’ll unpack the production challenges and opportunities with ML/DL on Spark and what the Zoo can do
Creating a custom ML model for your application - DevFest Lima 2019Isabel Palomar
Aprenderás como puede ser creado un modelo de Machine Learning que puedas implementar en tus aplicaciones. Iré mostrando cada uno de los pasos que se tienen que seguir, los tipos de problemas que se pueden resolver, los datos que necesitas para que funcione y por último, las opciones para realizar la implementación de nuestro modelo en nuestras aplicaciones.
Creating a custom Machine Learning Model for your applications - Java Dev Day...Isabel Palomar
Aprenderás como puede ser creado un modelo de Machine Learning que puedas implementar en tu aplicación móvil o Java. Iré mostrando cada uno de los pasos que se tienen que seguir, los tipos de problemas que se pueden resolver, los datos que necesitas para que funcione y por último, las opciones para realizar la implementación de nuestro modelo en nuestras aplicaciones.
Getting Started with OpenCV provides an overview of OpenCV and demonstrates a basic OpenCV program. It discusses OpenCV's structure, loading and saving images, creating windows and trackbars, and using OpenCV with Intel's Integrated Performance Primitives for accelerated computer vision functions. The document contains code samples and explains how to compile, build, and run OpenCV programs on Windows and Linux.
Inteligencia artificial para android como empezarIsabel Palomar
Aprenderás los conceptos basico de deep learning y como crear tu aplicación de Android que puede detectar y etiquetar imágenes utilizando un modelo de Tensorflow Lite
The document provides an overview and agenda for an introduction to running AI workloads on PowerAI. It discusses PowerAI and how it combines popular deep learning frameworks, development tools, and accelerated IBM Power servers. It then demonstrates AI workloads using TensorFlow and PyTorch, including running an MNIST workload to classify handwritten digits using basic linear regression and convolutional neural networks in TensorFlow, and an introduction to PyTorch concepts like tensors, modules, and softmax cross entropy loss.
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Databricks
A long time ago, there was Caffe and Theano, then came Torch and CNTK and Tensorflow, Keras and MXNet and Pytorch and Caffe2….a sea of Deep learning tools but none for Spark developers to dip into. Finally, there was BigDL, a deep learning library for Apache Spark. While BigDL is integrated into Spark and extends its capabilities to address the challenges of Big Data developers, will a library alone be enough to simplify and accelerate the deployment of ML/DL workloads on production clusters? From high level pipeline API support to feature transformers to pre-defined models and reference use cases, a rich repository of easy to use tools are now available with the ‘Analytics Zoo’. We’ll unpack the production challenges and opportunities with ML/DL on Spark and what the Zoo can do
Start machine learning in 5 simple stepsRenjith M P
Simple steps to get started with machine learning.
The use case uses python programming. Target audience is expected to have a very basic python knowledge.
The document discusses CNN Lab 256 and various labs involving image classification using ImageNet and MNIST datasets. Lab 2 focuses on image classification using ImageNet, which contains over 14 million images across 20,000 categories. The script classify_image.py is used to classify images using a pre-trained model. Retraining the model on a custom dataset is also discussed. Lab 5 involves classifying handwritten digits from the MNIST dataset using a convolutional neural network model defined in TensorFlow. The model achieves an accuracy of over 99% after training for 15,000 epochs in batches of 100 images.
MLFlow: Platform for Complete Machine Learning Lifecycle Databricks
Description
Data Science and ML development bring many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work.
MLflow addresses some of these challenges during an ML model development cycle.
Abstract
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure.
In this session, we introduce MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
With a short demo, you see a complete ML model life-cycle example, you will walk away with: MLflow concepts and abstractions for models, experiments, and projects How to get started with MLFlow Using tracking Python APIs during model training Using MLflow UI to visually compare and contrast experimental runs with different tuning parameters and evaluate metrics
The document provides an overview of a presentation about Google Cloud developer tools and an easier path to machine learning. It introduces the speaker and their background and experience. It then outlines the agenda which includes introductions to machine learning and Google Cloud, Google APIs, Cloud ML APIs, and other APIs to consider. It provides examples of using various Cloud ML APIs like Vision, Natural Language, and Speech for tasks like image labeling, text analysis, and speech recognition. The goal is to demonstrate how APIs powered by machine learning can help ease the burden of learning machine learning by allowing users to leverage pre-built models if they can call APIs.
The document discusses Google Cloud AI services including Cloud ML Engine for machine learning model training and prediction. It provides examples of using Cloud ML Engine to train models locally and in the cloud, perform distributed training, and hyperparameter tuning. It also covers deploying trained models and making predictions against them.
1. Machine learning is the use and development of computer systems that are able to learn and adapt without explicit instructions by using algorithms and statistical models to analyze patterns in data.
2. The document provides examples of machine learning applications like facial recognition, voice recognition in healthcare, weather forecasting, and more. It also discusses the process of machine learning and popular machine learning algorithms.
3. The document demonstrates machine learning using a decision tree algorithm on music purchase data to predict whether a customer is male or female based on attributes like age and number of songs purchased. It imports relevant Python libraries and splits the data into training and test sets to evaluate the model's performance.
"Deployment for free": removing the need to write model deployment code at St...Stefan Krawczyk
At Stitch Fix we have a dedicated Data Science organization called Algorithms. It has over 130+ Full Stack Data Scientists that build & own a variety of models. These models span from your classic prediction & classification models, through to time-series forecasts, simulations, and optimizations. Rather than hand-off models for productionization to someone else, Data Scientists own and are on-call for that process; we love for our Data Scientists to have autonomy. That said, Data Scientists aren’t without engineering support, as there’s a Data Platform team dedicated to building tooling, services, and abstractions to increase their workflow velocity. One data science task that we have been speeding up is getting models to production and increasing their usability and stability. This is a necessary task that can take a considerable chunk of a Data Scientist’s time, either in terms of developing, or debugging issues; historically everyone largely carved their own path in this endeavor, which meant many different approaches, implementations, and little to leverage across teams.
In this talk I’ll cover how the Model Lifecycle team on Data Platform built a system dubbed the “Model Envelope” to enable “deployment for free”. That is, no code needs to be written by a data scientist to deploy any python model to production, where production means either a micro-service, or a batch python/spark job. With our approach we can remove the need for data scientists to have to worry about python dependencies, or instrumenting model monitoring since we can take care of it for them, in addition to other MLOps concerns.
Specifically the talk will cover:
* Our API interface we provide to data scientists and how it decouples deployment concerns.
* How we approach automatically inferring a type safe API for models of any shape.
* How we handle python dependencies so Data Scientists don’t have to.
* How our relationship & approach enables us to inject & change MLOps approaches without having to coordinate much with Data Scientists.
The document discusses building machine learning solutions with Google Cloud. It describes Nexxworks as a team of data engineers, data scientists, and machine learning engineers who help close the gap between having lots of data and lacking insights by building robust and agile machine learning solutions through Google Cloud's scalable APIs. The document provides examples of use cases like predictive maintenance, logistics optimization, customer service chatbots, and medical image classification. It also discusses techniques like deep learning, word embeddings, convolutional neural networks, and reinforcement learning.
As data science workloads grow, so does their need for infrastructure. But, is it fair to ask data scientists to also become infrastructure experts? If not the data scientists, then, who is responsible for spinning up and managing data science infrastructure? This talk will address the context in which ML infrastructure is emerging, walk through two examples of ML infrastructure tools for launching hyperparameter optimization jobs, and end with some thoughts for building better tools in the future.
Originally given as a talk at the PyData Ann Arbor meetup (https://www.meetup.com/PyData-Ann-Arbor/events/260380989/)
Introduction to Machine Learning for newcomers. It will show you some basic concepts like what is supervised learning, unsupervised learning, classification, regression, under/overfitting, clustering, anomaly detection, and how to have some measures. It will illustrates examples through scikit-learn and tensorflow code
This document provides an introduction to building machine learning models using IBM Data Science Experience. It first discusses data science and machine learning concepts like the CRISP-DM methodology and neural networks. It then introduces IBM Data Science Experience, describing how it allows users to work with big data on the cloud using Python or R on Spark. The document concludes by introducing TensorFlow and providing an overview of key TensorFlow concepts like tensors, data flow graphs, and how neural networks and deep learning are represented.
The document summarizes an ML workshop on fruit detection using image classification. It includes an agenda for introductions on ML/ANNs, a problem statement on fruit salad object detection, hands-on training and testing of a model, and conclusions. Participants need a laptop and download tools. Key learnings included using TensorFlow, implementing a use case, and gaining confidence in ML. Various industries were identified for ML applications. The workshop demonstrated building a classifier using TensorFlow and training it on fruit images to classify images on mobile/Raspberry Pi. Challenges in deployment and optimizations made were also discussed.
The document discusses the key steps in an AI project cycle:
1) Problem scoping involves understanding the problem, stakeholders, location, and reasons for solving it.
2) Data acquisition collects accurate and reliable structured or unstructured data from various sources.
3) Data exploration arranges and visualizes the data to understand trends and patterns using tools like charts and graphs.
4) Modelling creates algorithms and models by training them on large datasets to perform tasks intelligently.
5) Evaluation tests the project by comparing outputs to actual answers to identify areas for improvement.
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Session 1 - Intro to Robotic Process Automation.pdfUiPathCommunity
👉 Check out our full 'Africa Series - Automation Student Developers (EN)' page to register for the full program:
https://bit.ly/Automation_Student_Kickstart
In this session, we shall introduce you to the world of automation, the UiPath Platform, and guide you on how to install and setup UiPath Studio on your Windows PC.
📕 Detailed agenda:
What is RPA? Benefits of RPA?
RPA Applications
The UiPath End-to-End Automation Platform
UiPath Studio CE Installation and Setup
💻 Extra training through UiPath Academy:
Introduction to Automation
UiPath Business Automation Platform
Explore automation development with UiPath Studio
👉 Register here for our upcoming Session 2 on June 20: Introduction to UiPath Studio Fundamentals: https://community.uipath.com/events/details/uipath-lagos-presents-session-2-introduction-to-uipath-studio-fundamentals/
More Related Content
Similar to Building a custom machine learning model on android
Start machine learning in 5 simple stepsRenjith M P
Simple steps to get started with machine learning.
The use case uses python programming. Target audience is expected to have a very basic python knowledge.
The document discusses CNN Lab 256 and various labs involving image classification using ImageNet and MNIST datasets. Lab 2 focuses on image classification using ImageNet, which contains over 14 million images across 20,000 categories. The script classify_image.py is used to classify images using a pre-trained model. Retraining the model on a custom dataset is also discussed. Lab 5 involves classifying handwritten digits from the MNIST dataset using a convolutional neural network model defined in TensorFlow. The model achieves an accuracy of over 99% after training for 15,000 epochs in batches of 100 images.
MLFlow: Platform for Complete Machine Learning Lifecycle Databricks
Description
Data Science and ML development bring many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work.
MLflow addresses some of these challenges during an ML model development cycle.
Abstract
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools, and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure.
In this session, we introduce MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
With a short demo, you see a complete ML model life-cycle example, you will walk away with: MLflow concepts and abstractions for models, experiments, and projects How to get started with MLFlow Using tracking Python APIs during model training Using MLflow UI to visually compare and contrast experimental runs with different tuning parameters and evaluate metrics
The document provides an overview of a presentation about Google Cloud developer tools and an easier path to machine learning. It introduces the speaker and their background and experience. It then outlines the agenda which includes introductions to machine learning and Google Cloud, Google APIs, Cloud ML APIs, and other APIs to consider. It provides examples of using various Cloud ML APIs like Vision, Natural Language, and Speech for tasks like image labeling, text analysis, and speech recognition. The goal is to demonstrate how APIs powered by machine learning can help ease the burden of learning machine learning by allowing users to leverage pre-built models if they can call APIs.
The document discusses Google Cloud AI services including Cloud ML Engine for machine learning model training and prediction. It provides examples of using Cloud ML Engine to train models locally and in the cloud, perform distributed training, and hyperparameter tuning. It also covers deploying trained models and making predictions against them.
1. Machine learning is the use and development of computer systems that are able to learn and adapt without explicit instructions by using algorithms and statistical models to analyze patterns in data.
2. The document provides examples of machine learning applications like facial recognition, voice recognition in healthcare, weather forecasting, and more. It also discusses the process of machine learning and popular machine learning algorithms.
3. The document demonstrates machine learning using a decision tree algorithm on music purchase data to predict whether a customer is male or female based on attributes like age and number of songs purchased. It imports relevant Python libraries and splits the data into training and test sets to evaluate the model's performance.
"Deployment for free": removing the need to write model deployment code at St...Stefan Krawczyk
At Stitch Fix we have a dedicated Data Science organization called Algorithms. It has over 130+ Full Stack Data Scientists that build & own a variety of models. These models span from your classic prediction & classification models, through to time-series forecasts, simulations, and optimizations. Rather than hand-off models for productionization to someone else, Data Scientists own and are on-call for that process; we love for our Data Scientists to have autonomy. That said, Data Scientists aren’t without engineering support, as there’s a Data Platform team dedicated to building tooling, services, and abstractions to increase their workflow velocity. One data science task that we have been speeding up is getting models to production and increasing their usability and stability. This is a necessary task that can take a considerable chunk of a Data Scientist’s time, either in terms of developing, or debugging issues; historically everyone largely carved their own path in this endeavor, which meant many different approaches, implementations, and little to leverage across teams.
In this talk I’ll cover how the Model Lifecycle team on Data Platform built a system dubbed the “Model Envelope” to enable “deployment for free”. That is, no code needs to be written by a data scientist to deploy any python model to production, where production means either a micro-service, or a batch python/spark job. With our approach we can remove the need for data scientists to have to worry about python dependencies, or instrumenting model monitoring since we can take care of it for them, in addition to other MLOps concerns.
Specifically the talk will cover:
* Our API interface we provide to data scientists and how it decouples deployment concerns.
* How we approach automatically inferring a type safe API for models of any shape.
* How we handle python dependencies so Data Scientists don’t have to.
* How our relationship & approach enables us to inject & change MLOps approaches without having to coordinate much with Data Scientists.
The document discusses building machine learning solutions with Google Cloud. It describes Nexxworks as a team of data engineers, data scientists, and machine learning engineers who help close the gap between having lots of data and lacking insights by building robust and agile machine learning solutions through Google Cloud's scalable APIs. The document provides examples of use cases like predictive maintenance, logistics optimization, customer service chatbots, and medical image classification. It also discusses techniques like deep learning, word embeddings, convolutional neural networks, and reinforcement learning.
As data science workloads grow, so does their need for infrastructure. But, is it fair to ask data scientists to also become infrastructure experts? If not the data scientists, then, who is responsible for spinning up and managing data science infrastructure? This talk will address the context in which ML infrastructure is emerging, walk through two examples of ML infrastructure tools for launching hyperparameter optimization jobs, and end with some thoughts for building better tools in the future.
Originally given as a talk at the PyData Ann Arbor meetup (https://www.meetup.com/PyData-Ann-Arbor/events/260380989/)
Introduction to Machine Learning for newcomers. It will show you some basic concepts like what is supervised learning, unsupervised learning, classification, regression, under/overfitting, clustering, anomaly detection, and how to have some measures. It will illustrates examples through scikit-learn and tensorflow code
This document provides an introduction to building machine learning models using IBM Data Science Experience. It first discusses data science and machine learning concepts like the CRISP-DM methodology and neural networks. It then introduces IBM Data Science Experience, describing how it allows users to work with big data on the cloud using Python or R on Spark. The document concludes by introducing TensorFlow and providing an overview of key TensorFlow concepts like tensors, data flow graphs, and how neural networks and deep learning are represented.
The document summarizes an ML workshop on fruit detection using image classification. It includes an agenda for introductions on ML/ANNs, a problem statement on fruit salad object detection, hands-on training and testing of a model, and conclusions. Participants need a laptop and download tools. Key learnings included using TensorFlow, implementing a use case, and gaining confidence in ML. Various industries were identified for ML applications. The workshop demonstrated building a classifier using TensorFlow and training it on fruit images to classify images on mobile/Raspberry Pi. Challenges in deployment and optimizations made were also discussed.
The document discusses the key steps in an AI project cycle:
1) Problem scoping involves understanding the problem, stakeholders, location, and reasons for solving it.
2) Data acquisition collects accurate and reliable structured or unstructured data from various sources.
3) Data exploration arranges and visualizes the data to understand trends and patterns using tools like charts and graphs.
4) Modelling creates algorithms and models by training them on large datasets to perform tasks intelligently.
5) Evaluation tests the project by comparing outputs to actual answers to identify areas for improvement.
Similar to Building a custom machine learning model on android (20)
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsDianaGray10
Join us to learn how UiPath Apps can directly and easily interact with prebuilt connectors via Integration Service--including Salesforce, ServiceNow, Open GenAI, and more.
The best part is you can achieve this without building a custom workflow! Say goodbye to the hassle of using separate automations to call APIs. By seamlessly integrating within App Studio, you can now easily streamline your workflow, while gaining direct access to our Connector Catalog of popular applications.
We’ll discuss and demo the benefits of UiPath Apps and connectors including:
Creating a compelling user experience for any software, without the limitations of APIs.
Accelerating the app creation process, saving time and effort
Enjoying high-performance CRUD (create, read, update, delete) operations, for
seamless data management.
Speakers:
Russell Alfeche, Technology Leader, RPA at qBotic and UiPath MVP
Charlie Greenberg, host
Session 1 - Intro to Robotic Process Automation.pdfUiPathCommunity
👉 Check out our full 'Africa Series - Automation Student Developers (EN)' page to register for the full program:
https://bit.ly/Automation_Student_Kickstart
In this session, we shall introduce you to the world of automation, the UiPath Platform, and guide you on how to install and setup UiPath Studio on your Windows PC.
📕 Detailed agenda:
What is RPA? Benefits of RPA?
RPA Applications
The UiPath End-to-End Automation Platform
UiPath Studio CE Installation and Setup
💻 Extra training through UiPath Academy:
Introduction to Automation
UiPath Business Automation Platform
Explore automation development with UiPath Studio
👉 Register here for our upcoming Session 2 on June 20: Introduction to UiPath Studio Fundamentals: https://community.uipath.com/events/details/uipath-lagos-presents-session-2-introduction-to-uipath-studio-fundamentals/
In our second session, we shall learn all about the main features and fundamentals of UiPath Studio that enable us to use the building blocks for any automation project.
📕 Detailed agenda:
Variables and Datatypes
Workflow Layouts
Arguments
Control Flows and Loops
Conditional Statements
💻 Extra training through UiPath Academy:
Variables, Constants, and Arguments in Studio
Control Flow in Studio
Dandelion Hashtable: beyond billion requests per second on a commodity serverAntonios Katsarakis
This slide deck presents DLHT, a concurrent in-memory hashtable. Despite efforts to optimize hashtables, that go as far as sacrificing core functionality, state-of-the-art designs still incur multiple memory accesses per request and block request processing in three cases. First, most hashtables block while waiting for data to be retrieved from memory. Second, open-addressing designs, which represent the current state-of-the-art, either cannot free index slots on deletes or must block all requests to do so. Third, index resizes block every request until all objects are copied to the new index. Defying folklore wisdom, DLHT forgoes open-addressing and adopts a fully-featured and memory-aware closed-addressing design based on bounded cache-line-chaining. This design offers lock-free index operations and deletes that free slots instantly, (2) completes most requests with a single memory access, (3) utilizes software prefetching to hide memory latencies, and (4) employs a novel non-blocking and parallel resizing. In a commodity server and a memory-resident workload, DLHT surpasses 1.6B requests per second and provides 3.5x (12x) the throughput of the state-of-the-art closed-addressing (open-addressing) resizable hashtable on Gets (Deletes).
QA or the Highway - Component Testing: Bridging the gap between frontend appl...zjhamm304
These are the slides for the presentation, "Component Testing: Bridging the gap between frontend applications" that was presented at QA or the Highway 2024 in Columbus, OH by Zachary Hamm.
The Department of Veteran Affairs (VA) invited Taylor Paschal, Knowledge & Information Management Consultant at Enterprise Knowledge, to speak at a Knowledge Management Lunch and Learn hosted on June 12, 2024. All Office of Administration staff were invited to attend and received professional development credit for participating in the voluntary event.
The objectives of the Lunch and Learn presentation were to:
- Review what KM ‘is’ and ‘isn’t’
- Understand the value of KM and the benefits of engaging
- Define and reflect on your “what’s in it for me?”
- Share actionable ways you can participate in Knowledge - - Capture & Transfer
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...Fwdays
Direct losses from downtime in 1 minute = $5-$10 thousand dollars. Reputation is priceless.
As part of the talk, we will consider the architectural strategies necessary for the development of highly loaded fintech solutions. We will focus on using queues and streaming to efficiently work and manage large amounts of data in real-time and to minimize latency.
We will focus special attention on the architectural patterns used in the design of the fintech system, microservices and event-driven architecture, which ensure scalability, fault tolerance, and consistency of the entire system.
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
In the realm of cybersecurity, offensive security practices act as a critical shield. By simulating real-world attacks in a controlled environment, these techniques expose vulnerabilities before malicious actors can exploit them. This proactive approach allows manufacturers to identify and fix weaknesses, significantly enhancing system security.
This presentation delves into the development of a system designed to mimic Galileo's Open Service signal using software-defined radio (SDR) technology. We'll begin with a foundational overview of both Global Navigation Satellite Systems (GNSS) and the intricacies of digital signal processing.
The presentation culminates in a live demonstration. We'll showcase the manipulation of Galileo's Open Service pilot signal, simulating an attack on various software and hardware systems. This practical demonstration serves to highlight the potential consequences of unaddressed vulnerabilities, emphasizing the importance of offensive security practices in safeguarding critical infrastructure.
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
AppSec PNW: Android and iOS Application Security with MobSFAjin Abraham
Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application.
This talk covers:
Using MobSF for static analysis of mobile applications.
Interactive dynamic security assessment of Android and iOS applications.
Solving Mobile app CTF challenges.
Reverse engineering and runtime analysis of Mobile malware.
How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
ScyllaDB is making a major architecture shift. We’re moving from vNode replication to tablets – fragments of tables that are distributed independently, enabling dynamic data distribution and extreme elasticity. In this keynote, ScyllaDB co-founder and CTO Avi Kivity explains the reason for this shift, provides a look at the implementation and roadmap, and shares how this shift benefits ScyllaDB users.
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation F...AlexanderRichford
QR Secure: A Hybrid Approach Using Machine Learning and Security Validation Functions to Prevent Interaction with Malicious QR Codes.
Aim of the Study: The goal of this research was to develop a robust hybrid approach for identifying malicious and insecure URLs derived from QR codes, ensuring safe interactions.
This is achieved through:
Machine Learning Model: Predicts the likelihood of a URL being malicious.
Security Validation Functions: Ensures the derived URL has a valid certificate and proper URL format.
This innovative blend of technology aims to enhance cybersecurity measures and protect users from potential threats hidden within QR codes 🖥 🔒
This study was my first introduction to using ML which has shown me the immense potential of ML in creating more secure digital environments!
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillLizaNolte
HERE IS YOUR WEBINAR CONTENT! 'Mastering Customer Journey Management with Dr. Graham Hill'. We hope you find the webinar recording both insightful and enjoyable.
In this webinar, we explored essential aspects of Customer Journey Management and personalization. Here’s a summary of the key insights and topics discussed:
Key Takeaways:
Understanding the Customer Journey: Dr. Hill emphasized the importance of mapping and understanding the complete customer journey to identify touchpoints and opportunities for improvement.
Personalization Strategies: We discussed how to leverage data and insights to create personalized experiences that resonate with customers.
Technology Integration: Insights were shared on how inQuba’s advanced technology can streamline customer interactions and drive operational efficiency.
8. 8
🤯 After the class…..
The key outcome of this lesson is that we'll have trained an
image classifier which can recognize pet breeds at state of
the art accuracy. The key to this success is the use of
transfer learning, which will be a key platform for much of
this course.
We also discuss how to set the most important
hyper-parameter when training neural networks:
the learning rate, using Leslie Smith's fantastic learning rate
finder method. Finally, we'll look at the important but rarely
discussed topic of labeling, and learn about some of the
features that fastai provides for allowing you to easily add
labels to your images. https://course.fast.ai/videos/?lesson=1
9. challenges….
‐ Many courses, even basic, assume that
you already know the subject.
‐ Reaching the final result without
learning the basics is not good.
9
10. “When you are starting to learn about
Deep Learning it seems that there
are thousands of concepts,
mathematical functions and
scientific articles that you have to
read.
10
myths
13. Let’s take a look of the
implementation
We are going to build an app to
classify the artisanal beers of
Cervecería Colima
Place your screenshot here
13
14. 1.- dATA
Data is distinct pieces of information which
acts as a fuel
14
17. How? Where do we get data from?
Data curation is the organization and integration
of data collected from various sources.
17
Techniques
You can use techniques like Questionnaires and surveys,
conducting interviews, using data scraping and data
crawling techniques.
18. Public datasets
● Google AI
● UCI ML Repository
● Data.gov.in
● Kaggle
Where do we get data from?
Crowdsourcing
Marketplaces
● Amazon Mechanical
Turk
● Dataturks
● Figure-eight
18
19. BACK TO OUR EXAMPLE...
● Google Images
● https://github.com/hardikvasa/google-images-download
● https://forums.fast.ai/t/tips-for-building-large-image-datasets/26688
19
22. TASK FOR OUR EXAMPLE
22
Classify Images of
Artisanal Beers
23. Image classification
A common use of machine learning is to identify
what an image represents.
The task of predicting what an image
represents is called image classification.
23
25. models
25
There are many models that are created over
the years.
Each model has its own advantages and
disadvantages based on the type of data on
which we are creating a model.
26. IMAGE CLASSIFICATION MODEL
An image classification model is trained to recognize various
classes of images.
26
When we subsequently
provide a new image as input
to the model, it will output the
probabilities of the image
representing each of the
types it was trained on.
27. An example output might be as follows:
Beer type Probability
Cayaco 0.02
Colimita 0.96
Piedra Lisa 0.01
Ticus 0.00
Paramo 0.01
27
Based on the output, we
can see that the
classification model has
predicted that the image
has a high probability of
representing a Colimita
Beer.
28. In this example, we will retrain a
MobileNet. MobileNet is a a small efficient
convolutional neural network.
https://ai.googleblog.com/2017/06/mobilenets-open-source-models-for.html
Model for our example
28
29. Retraining the mobileNet model
29
We use MobileNet model and retrain it.
python3 -m scripts.retrain
--bottleneck_dir=tf_files/bottlenecks
--model_dir=tf_files/models/"${ARCHITECTURE}"
--summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}"
--output_graph=tf_files/retrained_graph.pb
--output_labels=tf_files/retrained_labels.txt
--architecture="${ARCHITECTURE}"
--image_dir=tf_files/beer_photos
IMAGE_SIZE=224
ARCHITECTURE="mobilenet_0.50_${IMAGE_SIZE}"
tHE RESULT...
30. USING THE RETRAINED MODEL
3030
Evaluation time (1-image): 0.250s
ticus (score=0.99956)
paramo (score=0.00043)
cayaco (score=0.00000)
piedra lisa (score=0.00000)
colimita (score=0.00000)
python3 -m scripts.label_image
--graph=tf_files/retrained_graph.pb
--image=tf_files/beer_photos/ticus/"3. ticus.jpg"
31. 4.- loss function
How do we know which model is better?
Loss function (also known as the error)
answers this question.
31
32. Models and loss function
How good a
prediction model
does in terms of
being able to predict
the expected
outcome.
32
33. Classification losses:
● Mean Square Error/L2 Loss
● Mean Absolute Error/L1 Loss
Regression losses:
● Hinge Loss/Multi-class SVM Loss
● Cross Entropy
● Loss/Negative Log Likelihood
LOSS FUNCTIONS
To know which model
is good for our data,
we compute the loss
function by
comparing the
predicted outputs to
actual output.
33
34. 5.- learning algorithm
The Learning Algorithms also known as
Optimization algorithms helps us to minimize
Error
34
35. First Order Optimization
Algorithms
● Gradient Descent
Types of learning algorithms
Second Order Optimization
Algorithms
● Hessian
https://towardsdatascience.com/types-of-optimization-algorithms-used-in-neural-networks-and-
ways-to-optimize-gradient-95ae5d39529f
35
36. Is something you do everyday...
You are optimizing
variables and basing your
personal decisions all day
long, most of the time
without even recognizing
the process consciously
https://mitsloan.mit.edu/ideas-made-to-matter/how-to-use
-algorithms-to-solve-everyday-problems
36
42. MACHINE LEARNING IN YOUR APPS
● ML Kit For Firebase
● Core ML (Apple)
● TensorFlow Lite
● Cloud-based web services
● Your own service
Place your screenshot here
42
44. USING THE RETRAINED MODEL
4444
Evaluation time (1-image): 0.250s
ticus (score=0.99956)
paramo (score=0.00043)
cayaco (score=0.00000)
piedra lisa (score=0.00000)
colimita (score=0.00000)
python3 -m scripts.label_image
--graph=tf_files/retrained_graph.pb
--image=tf_files/beer_photos/ticus/"3. ticus.jpg"
45. TENSORFLOW LITE
45
TensorFlow Lite is a set of tools to
help developers run TensorFlow
models on mobile, embedded, and
IoT devices.
● TensorFlow Lite converter
● TensorFlow Lite interpreter
TensorFlow Lite converter
Converts TensorFlow models into
an efficient form for use by the
interpreter
46. Command line: tflite_convert
Starting from TensorFlow
1.9, the command-line tool
tflite_convert is installed as
part of the Python package.
46
pip install --upgrade "tensorflow==1.9.*"
50. repositories {
maven {
url 'https://google.bintray.com/tensorflow'
}
}
dependencies {
// ...
compile 'org.tensorflow:tensorflow-lite:+'
}
TensorFlow Lite interpreter
50
android {
aaptOptions {
noCompress "tflite"
noCompress "lite"
}
}
The TensorFlow Lite
interpreter is designed to be
lean and fast. The interpreter
uses a static graph ordering
and a custom (less-dynamic)
memory allocator to ensure
minimal load, initialization, and
execution latency.
dependencies
settings
51. Load model and create interpreter
protected Classifier… {
tfliteOptions.setNumThreads(numThreads);
tflite = new Interpreter(tfliteModel, tfliteOptions);
labels = loadLabelList(activity);
...
}
51
// Name of the model file stored in Assets.
private static final String MODEL_PATH = "graph.lite";
// Name of the label file stored in Assets.
private static final String LABEL_PATH = "labels.txt";
52. cAMERA, Read the labels…..
52
https://developer.android.com/training/camerax
// Convert the image to bytes
convertBitmapToByteBuffer(bitmap);
// An array to hold inference results, to be feed
into Tensorflow Lite as outputs.
PriorityQueue<Map.Entry<String, Float>> sortedLabels =
new PriorityQueue<>(
RESULTS_TO_SHOW,
(element1, element2) ->
(element1.getValue()).compareTo(element2.getValue()));
53. Show the results
53
// Get the results
textToShow = String.format("n%s: %4.2f", label.getKey(),
label.getValue())
// Label (In this case PARAMO)
label.getKey()
// Value (In this case 1.0)
label.getValue()
ticus (score=0.00000)
paramo (score=1.00000)
cayaco (score=0.00000)
piedra lisa (score=0.00000)
colimita (score=0.00000)