Presentation to the consortium of the ECSEL Fitoptivis project of the Single-Source methodology under development in the ECSEL MegaMart project towards the modeling&design of complex, distributed and heterogeneous systems
This document provides an introduction to Java concurrency and the ExecutorService API. It discusses parallelism and threads in Java, shows how to create and manage thread pools using ExecutorService, and emphasizes that ExecutorService is now preferred over directly starting and managing threads. The goal is to give a simple overview of Java concurrency basics and how ExecutorService can help manage threads.
This document discusses concurrency in C++. It notes that while C++ is commonly used, it lacks built-in concurrency support. To take advantage of parallel hardware, concurrency must be explicitly specified by programmers. The document proposes augmenting C++ with a unified concurrency mechanism called μC++ that tightly integrates advanced control flow using threads represented as objects. It would use mutexes for synchronization and focus on method calls for communication to match C++'s design. The outline describes how coroutines, threads, locking, errors, monitors, and tasks would be addressed.
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsEugenio Villar
Slides of the lecture given at the Summer School on Cyber-Physical Systems and Internet-of-Things - CPS&IoT’2019, held in Budva, Montenegro, June 10-14, 2019
JDD2015: Java Everywhere Again—with DukeScript - Jaroslav TulachPROIDEA
JAVA EVERYWHERE AGAIN—WITH DUKESCRIPT
For a long time, Java was perfect for creating cross-platform applications, but the advent of iPhone, iPad, and Android devices changed everything, resulting in a totally fragmented world. Catering to all these platform is troublesome and expensive. That’s why DukeScript was created: to make it easy to create cross-platform Java applications again. The goal of this hands-on lab is to create a cross-platform application from scratch that will run on iOS, Android, desktop, browser, and embedded devices such as the Raspberry Pi. You’ll learn about the Model-View-ViewModel (MVVM) architecture, which enables you to write and test business code totally independently of the view, and, finally, you’ll see it combined with a view to complete a working application.
IMPORTANT
Before conference, please follow the steps to prepare for the session:
- perform the Maven repository initialization by creating the archetype and building it as
described at DukeScript website
- also download NetBeans IDE (either latest beta or at least 8.0.2)
- Installing Android SDK rev. 19 or bringing own Mac Book with XCode installed can be also found beneficial
What is a Service Mesh and what can it do for your MicroservicesMatt Turner
e’ll explore what a service mesh is and what it can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we’ll find out!
This session will also briefly cover how a service mesh works, giving us a mental model with which to explore and evaluate after the talk. Matt will show a simple installation and demo, giving us all the knowledge to go home and try for ourself.
This document discusses digital system verification techniques. It reviews the conventional design and verification flow including simulation at different levels of abstraction. Key verification techniques are discussed including simulation, formal verification, and static timing analysis. An emerging verification paradigm is described that uses cycle-based simulation and formal verification for functional verification and static timing analysis for timing verification.
Open Standards for ADAS: Andrew Richards, Codeplay, at AutoSens 2016Andrew Richards
This presentation discusses the open standards and software tools needed to enable vision processing for autonomous vehicles. It focuses on the hardware and software platforms that can deliver results, the tools to build solutions for those platforms, and open standards that allow solutions to interoperate. The presentation advocates for using graph programming and C++-based approaches like SYCL, along with open standards like SPIR-V and HSA, to develop software today that can run on future platforms and achieve full autonomy.
Deploying large-scale, serverless and asynchronous systems - without integrat...DiUS
Modern distributed architectures are increasingly composed of large numbers of decoupled, asynchronous components. In AWS, these components are plumbed together via services like SQS, SNS, Kinesis and S3 often integrated via small and frequent numbers of microservices or lambdas. But how do you test these architectures if they are cloud native?
It’s 2018, and we can do better than deploying the entire stack and running a battery of E2E tests against them.
In his talk, Matt will demonstrate how you can scale development of large-scale systems across teams, technology and process, and unlock the agility of your cloud-native architecture.
WARNING: there will be code (https://github.com/mefellows/serverless-testing-example)
This document provides an introduction to Java concurrency and the ExecutorService API. It discusses parallelism and threads in Java, shows how to create and manage thread pools using ExecutorService, and emphasizes that ExecutorService is now preferred over directly starting and managing threads. The goal is to give a simple overview of Java concurrency basics and how ExecutorService can help manage threads.
This document discusses concurrency in C++. It notes that while C++ is commonly used, it lacks built-in concurrency support. To take advantage of parallel hardware, concurrency must be explicitly specified by programmers. The document proposes augmenting C++ with a unified concurrency mechanism called μC++ that tightly integrates advanced control flow using threads represented as objects. It would use mutexes for synchronization and focus on method calls for communication to match C++'s design. The outline describes how coroutines, threads, locking, errors, monitors, and tasks would be addressed.
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsEugenio Villar
Slides of the lecture given at the Summer School on Cyber-Physical Systems and Internet-of-Things - CPS&IoT’2019, held in Budva, Montenegro, June 10-14, 2019
JDD2015: Java Everywhere Again—with DukeScript - Jaroslav TulachPROIDEA
JAVA EVERYWHERE AGAIN—WITH DUKESCRIPT
For a long time, Java was perfect for creating cross-platform applications, but the advent of iPhone, iPad, and Android devices changed everything, resulting in a totally fragmented world. Catering to all these platform is troublesome and expensive. That’s why DukeScript was created: to make it easy to create cross-platform Java applications again. The goal of this hands-on lab is to create a cross-platform application from scratch that will run on iOS, Android, desktop, browser, and embedded devices such as the Raspberry Pi. You’ll learn about the Model-View-ViewModel (MVVM) architecture, which enables you to write and test business code totally independently of the view, and, finally, you’ll see it combined with a view to complete a working application.
IMPORTANT
Before conference, please follow the steps to prepare for the session:
- perform the Maven repository initialization by creating the archetype and building it as
described at DukeScript website
- also download NetBeans IDE (either latest beta or at least 8.0.2)
- Installing Android SDK rev. 19 or bringing own Mac Book with XCode installed can be also found beneficial
What is a Service Mesh and what can it do for your MicroservicesMatt Turner
e’ll explore what a service mesh is and what it can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we’ll find out!
This session will also briefly cover how a service mesh works, giving us a mental model with which to explore and evaluate after the talk. Matt will show a simple installation and demo, giving us all the knowledge to go home and try for ourself.
This document discusses digital system verification techniques. It reviews the conventional design and verification flow including simulation at different levels of abstraction. Key verification techniques are discussed including simulation, formal verification, and static timing analysis. An emerging verification paradigm is described that uses cycle-based simulation and formal verification for functional verification and static timing analysis for timing verification.
Open Standards for ADAS: Andrew Richards, Codeplay, at AutoSens 2016Andrew Richards
This presentation discusses the open standards and software tools needed to enable vision processing for autonomous vehicles. It focuses on the hardware and software platforms that can deliver results, the tools to build solutions for those platforms, and open standards that allow solutions to interoperate. The presentation advocates for using graph programming and C++-based approaches like SYCL, along with open standards like SPIR-V and HSA, to develop software today that can run on future platforms and achieve full autonomy.
Deploying large-scale, serverless and asynchronous systems - without integrat...DiUS
Modern distributed architectures are increasingly composed of large numbers of decoupled, asynchronous components. In AWS, these components are plumbed together via services like SQS, SNS, Kinesis and S3 often integrated via small and frequent numbers of microservices or lambdas. But how do you test these architectures if they are cloud native?
It’s 2018, and we can do better than deploying the entire stack and running a battery of E2E tests against them.
In his talk, Matt will demonstrate how you can scale development of large-scale systems across teams, technology and process, and unlock the agility of your cloud-native architecture.
WARNING: there will be code (https://github.com/mefellows/serverless-testing-example)
Functional verification is one of the key bottlenecks in the rapid design of integrated circuits. It is estimated that verification in its entirety accounts for up to 60% of design resources, including duration, computer resources and total personnel. The three primary tools used in logic and functional verification of commercial integrated circuits are simulation (at various levels), emulation at the chip level, and formal verification.
Low Latency Polyglot Model Scoring using Apache ApexApache Apex
This document discusses challenges in building low-latency machine learning applications and how Apache Apex can help address them. It introduces Apache Apex as a distributed streaming engine and describes how it allows embedding models from frameworks like R, Python, H2O through custom operators. It provides various data and model scoring patterns in Apex like dynamic resource allocation, checkpointing, exactly-once processing to meet SLAs. The document also demonstrates techniques like canary deployment, dormant models, model ensembles through logical overlays on the Apex DAG.
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...Chris Fregly
Pipeline.AI is a platform for deploying and optimizing machine learning models at scale. It allows users to package models with their runtime dependencies, perform load testing and optimizations, deploy models to production safely using techniques like canary deployments, and monitor models both offline and online. The platform aims to enable live, continuous model training directly in production environments.
Optimizing, Profiling, and Deploying High Performance Spark ML and TensorFlow AIData Con LA
Abstract:-
Using the latest advancements from TensorFlow including the Accelerated Linear Algebra (XLA) Framework, JIT/AOT Compiler, and Graph Transform Tool , I’ll demonstrate how to optimize, profile, and deploy TensorFlow Models - and the TensorFlow Runtime - in GPU-based production environment.
This talk is 100% demo based with open source tools and completely reproducible through Docker on your own GPU cluster.
Bio:-
Chris Fregly is Founder and Research Engineer at PipelineAI, a Streaming Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production."
Pipeline.AI was also the recent winner of the O'Reilly Media AI Startup Showcase at the AI conference.
Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.
In this presentation, latest results of our research activity in the ESCEL Comp4Drones project is presented. The application of S3D to the modeling and performance analysis of drone-based services is described
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo MazzaferroPyData
Those of us who use TensorFlow often focus on building the model that's most predictive, not the one that's most deployable. So how to put that hard work to work? In this talk, we'll walk through a strategy for taking your machine learning models from Jupyter Notebook into production and beyond.
This document summarizes a presentation about scaling FreeSWITCH performance. It discusses how FreeSWITCH uses an insanely threaded model with session threads for each call leg. It also discusses some performance tweaks like reducing logging levels and moving the SQLite database to tmpfs memory to avoid I/O bottlenecks. Migrating to a database like PostgreSQL or MySQL may eventually be needed to move the database workload elsewhere for better performance.
1. Roslyn is the .NET compiler platform that allows compiling code on-demand as a service.
2. It started in 2008 and is now included in Visual Studio 2015, allowing developers to interact with code in new ways like through a REPL-like experience and syntax/semantic APIs.
3. Roslyn provides capabilities for analyzing code through diagnostics and code fixes, and also enables scripting scenarios.
Do You Need a Service Mesh? @ London Devops, January 2019Matt Turner
Service meshes are cool, but are they useful? We'll explore what a service mesh is and what they can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we'll find out!
What we learnt at carousell tw for golang gathering #31Ronald Hsu
The document discusses the architecture and design of a payment and shipping system built using Golang. It covers topics like code structure, dependency injection, microservices communication using gRPC, gRPC status codes, and some gRPC tricks. The system includes features like payment methods, address preferences, order requests, delivery tracking, and wallet/bank integration. It also discusses responsibilities of the payment system, potential issues to watch out for like import cycles and decoupling services, and concludes with recommendations on service design and performance.
The document discusses component interface design. It defines an interface and contrasts an API approach with a protocol approach. It provides examples of common bad interface practices like deceptive APIs and DSLs as APIs. It also discusses issues with distributed systems like idempotency keys and the coupling spectrum. The document emphasizes designing error messages for the caller and distinguishing component purpose from implementation.
GC Tuning Confessions Of A Performance EngineerMonica Beckwith
This document provides an overview of garbage collection (GC) tuning and various GC algorithms used in OpenJDK HotSpot. It discusses key concepts like throughput collectors, latency-sensitive collectors, and the tradeoffs between throughput, latency and footprint. Specifically, it summarizes the Parallel Old, CMS and G1 garbage collectors - their goals, techniques used and failure scenarios. It also covers GC tuning concepts like heap configuration, GC logs and metrics.
H2O Design and Infrastructure with Matt DowleSri Ambati
This document provides an overview of H2O, an open source machine learning platform that allows for distributed, in-memory analytics of large datasets. It discusses how H2O works, including how it uses a map-reduce style to parallelize machine learning algorithms across multiple nodes. The document demonstrates starting an 8-node H2O cluster on Amazon EC2 and importing a 23GB dataset in under a minute, significantly faster than with other tools. It also summarizes how H2O's distributed fork-join framework executes tasks across nodes and shares data through its distributed data structures.
SELF - Becoming a Rails Developer - The Rest of the StoryNathanial McConnell
This document provides guidance on becoming a Rails developer beyond just learning the Rails framework. It discusses key competencies needed like understanding Ruby and object-oriented programming. It also recommends learning processes like mentoring, code reviews, and pair programming. Standard learning paths are outlined covering Ruby, Rails, SQL, testing, and more. Advice is given on freelancing, finding clients, and maintaining work-life balance.
TechUG Glasgow talk 22/Feb/17 Configuration Management Best PracticesDag Sonstebo
The document discusses configuration management best practices for cloud environments. It describes configuration management as establishing consistency of a product's attributes throughout its lifecycle. The document recommends starting configuration management gradually with small automated tasks before implementing larger projects. It also emphasizes the importance of version control, choosing the right tools like Ansible, and gaining organizational buy-in for successful configuration management.
Trends and development practices in Serverless architecturesDiUS
AWS ISV Event - Unlocking Business Agility with the AWS Serverless Application Model
Matt Fellows, Principal Consultant from DiUS will talk about the evolution to serverless architectures, and discuss key development and testing practices for these modern distributed systems
This document outlines different steps in scaling Node.js applications from 2012 to 2019. It begins with using Node.js in cluster mode with Nginx as a reverse proxy. It progresses to using CDNs for static files, in-memory databases, and eventually custom protocols for real-time data synchronization across servers and clients. Key aspects discussed include data synchronization, offline capabilities, interactivity, scalability, and high connectivity. Alternative approaches and bad practices are also addressed.
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Storyvanphp
The document describes Hootsuite's scaling journey from using Apache and PHP on one MySQL server to a microservices architecture using multiple technologies like Nginx, PHP-FPM, Memcached, MongoDB, Gearman, and Scala/Akka services communicating via ZeroMQ. Key steps included caching with Memcached to reduce MySQL load, using Gearman for asynchronous tasks, and MongoDB for large datasets. Monitoring with Statsd, Logstash and Elasticsearch was added for visibility. They moved to a service-oriented architecture with independent services to keep scaling their large codebase and engineering team.
How to Productionize Your Machine Learning Models Using Apache Spark MLlib 2....Databricks
Richard Garris presented on ways to productionize machine learning models built with Apache Spark MLlib. He discussed serializing models using MLlib 2.X to save models for production use without reimplementation. This allows data scientists to build models in Python/R and deploy them directly for scoring. He also reviewed model scoring architectures and highlighted Databricks' private beta solution for deploying serialized Spark MLlib models for low latency scoring outside of Spark.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Functional verification is one of the key bottlenecks in the rapid design of integrated circuits. It is estimated that verification in its entirety accounts for up to 60% of design resources, including duration, computer resources and total personnel. The three primary tools used in logic and functional verification of commercial integrated circuits are simulation (at various levels), emulation at the chip level, and formal verification.
Low Latency Polyglot Model Scoring using Apache ApexApache Apex
This document discusses challenges in building low-latency machine learning applications and how Apache Apex can help address them. It introduces Apache Apex as a distributed streaming engine and describes how it allows embedding models from frameworks like R, Python, H2O through custom operators. It provides various data and model scoring patterns in Apex like dynamic resource allocation, checkpointing, exactly-once processing to meet SLAs. The document also demonstrates techniques like canary deployment, dormant models, model ensembles through logical overlays on the Apex DAG.
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...Chris Fregly
Pipeline.AI is a platform for deploying and optimizing machine learning models at scale. It allows users to package models with their runtime dependencies, perform load testing and optimizations, deploy models to production safely using techniques like canary deployments, and monitor models both offline and online. The platform aims to enable live, continuous model training directly in production environments.
Optimizing, Profiling, and Deploying High Performance Spark ML and TensorFlow AIData Con LA
Abstract:-
Using the latest advancements from TensorFlow including the Accelerated Linear Algebra (XLA) Framework, JIT/AOT Compiler, and Graph Transform Tool , I’ll demonstrate how to optimize, profile, and deploy TensorFlow Models - and the TensorFlow Runtime - in GPU-based production environment.
This talk is 100% demo based with open source tools and completely reproducible through Docker on your own GPU cluster.
Bio:-
Chris Fregly is Founder and Research Engineer at PipelineAI, a Streaming Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production."
Pipeline.AI was also the recent winner of the O'Reilly Media AI Startup Showcase at the AI conference.
Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.
In this presentation, latest results of our research activity in the ESCEL Comp4Drones project is presented. The application of S3D to the modeling and performance analysis of drone-based services is described
RESTful Machine Learning with Flask and TensorFlow Serving - Carlo MazzaferroPyData
Those of us who use TensorFlow often focus on building the model that's most predictive, not the one that's most deployable. So how to put that hard work to work? In this talk, we'll walk through a strategy for taking your machine learning models from Jupyter Notebook into production and beyond.
This document summarizes a presentation about scaling FreeSWITCH performance. It discusses how FreeSWITCH uses an insanely threaded model with session threads for each call leg. It also discusses some performance tweaks like reducing logging levels and moving the SQLite database to tmpfs memory to avoid I/O bottlenecks. Migrating to a database like PostgreSQL or MySQL may eventually be needed to move the database workload elsewhere for better performance.
1. Roslyn is the .NET compiler platform that allows compiling code on-demand as a service.
2. It started in 2008 and is now included in Visual Studio 2015, allowing developers to interact with code in new ways like through a REPL-like experience and syntax/semantic APIs.
3. Roslyn provides capabilities for analyzing code through diagnostics and code fixes, and also enables scripting scenarios.
Do You Need a Service Mesh? @ London Devops, January 2019Matt Turner
Service meshes are cool, but are they useful? We'll explore what a service mesh is and what they can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we'll find out!
What we learnt at carousell tw for golang gathering #31Ronald Hsu
The document discusses the architecture and design of a payment and shipping system built using Golang. It covers topics like code structure, dependency injection, microservices communication using gRPC, gRPC status codes, and some gRPC tricks. The system includes features like payment methods, address preferences, order requests, delivery tracking, and wallet/bank integration. It also discusses responsibilities of the payment system, potential issues to watch out for like import cycles and decoupling services, and concludes with recommendations on service design and performance.
The document discusses component interface design. It defines an interface and contrasts an API approach with a protocol approach. It provides examples of common bad interface practices like deceptive APIs and DSLs as APIs. It also discusses issues with distributed systems like idempotency keys and the coupling spectrum. The document emphasizes designing error messages for the caller and distinguishing component purpose from implementation.
GC Tuning Confessions Of A Performance EngineerMonica Beckwith
This document provides an overview of garbage collection (GC) tuning and various GC algorithms used in OpenJDK HotSpot. It discusses key concepts like throughput collectors, latency-sensitive collectors, and the tradeoffs between throughput, latency and footprint. Specifically, it summarizes the Parallel Old, CMS and G1 garbage collectors - their goals, techniques used and failure scenarios. It also covers GC tuning concepts like heap configuration, GC logs and metrics.
H2O Design and Infrastructure with Matt DowleSri Ambati
This document provides an overview of H2O, an open source machine learning platform that allows for distributed, in-memory analytics of large datasets. It discusses how H2O works, including how it uses a map-reduce style to parallelize machine learning algorithms across multiple nodes. The document demonstrates starting an 8-node H2O cluster on Amazon EC2 and importing a 23GB dataset in under a minute, significantly faster than with other tools. It also summarizes how H2O's distributed fork-join framework executes tasks across nodes and shares data through its distributed data structures.
SELF - Becoming a Rails Developer - The Rest of the StoryNathanial McConnell
This document provides guidance on becoming a Rails developer beyond just learning the Rails framework. It discusses key competencies needed like understanding Ruby and object-oriented programming. It also recommends learning processes like mentoring, code reviews, and pair programming. Standard learning paths are outlined covering Ruby, Rails, SQL, testing, and more. Advice is given on freelancing, finding clients, and maintaining work-life balance.
TechUG Glasgow talk 22/Feb/17 Configuration Management Best PracticesDag Sonstebo
The document discusses configuration management best practices for cloud environments. It describes configuration management as establishing consistency of a product's attributes throughout its lifecycle. The document recommends starting configuration management gradually with small automated tasks before implementing larger projects. It also emphasizes the importance of version control, choosing the right tools like Ansible, and gaining organizational buy-in for successful configuration management.
Trends and development practices in Serverless architecturesDiUS
AWS ISV Event - Unlocking Business Agility with the AWS Serverless Application Model
Matt Fellows, Principal Consultant from DiUS will talk about the evolution to serverless architectures, and discuss key development and testing practices for these modern distributed systems
This document outlines different steps in scaling Node.js applications from 2012 to 2019. It begins with using Node.js in cluster mode with Nginx as a reverse proxy. It progresses to using CDNs for static files, in-memory databases, and eventually custom protocols for real-time data synchronization across servers and clients. Key aspects discussed include data synchronization, offline capabilities, interactivity, scalability, and high connectivity. Alternative approaches and bad practices are also addressed.
PHP At 5000 Requests Per Second: Hootsuite’s Scaling Storyvanphp
The document describes Hootsuite's scaling journey from using Apache and PHP on one MySQL server to a microservices architecture using multiple technologies like Nginx, PHP-FPM, Memcached, MongoDB, Gearman, and Scala/Akka services communicating via ZeroMQ. Key steps included caching with Memcached to reduce MySQL load, using Gearman for asynchronous tasks, and MongoDB for large datasets. Monitoring with Statsd, Logstash and Elasticsearch was added for visibility. They moved to a service-oriented architecture with independent services to keep scaling their large codebase and engineering team.
How to Productionize Your Machine Learning Models Using Apache Spark MLlib 2....Databricks
Richard Garris presented on ways to productionize machine learning models built with Apache Spark MLlib. He discussed serializing models using MLlib 2.X to save models for production use without reimplementation. This allows data scientists to build models in Python/R and deploy them directly for scoring. He also reviewed model scoring architectures and highlighted Databricks' private beta solution for deploying serialized Spark MLlib models for low latency scoring outside of Spark.
Similar to Model-Driven Analysis&Design of Distributed, Heterogeneous Systems (20)
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Zilliz
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
3. Introduction
▪ Model-Driven Design (MDD)
▪ High-abstraction level
▪ Mature SW engineering methodology
▪ State-of-the-Art
▪ Matlab-Simulink
▪ Proprietary, only one MoC, M language
▪ CoFluent
▪ Proprietary, a few MoCs, C/C++ language
▪ Ptolemy II
▪ Academic, any MoC, C/C++ inside a Java block
▪ …
4. Introduction
▪ UML
▪ Standard, any (user-defined) MoC, any language
▪ Natural way to capture system architecture
N2 BN1
M A
▪ Semantic lacks
▪ Domain-specific profiles
▪ MetaMorph
▪ Commercial, any (user-defined) MoC, language agnostic
▪ CHESS
▪ Open Source, any (user-defined) MoC, language agnostic
7. Model-Driven Analysis of IoE Services
▪ Programming the Internet of Everything
▪ Services provided on computing platforms of many kind
8. Model-Driven Analysis of IoE Services
▪ Programming the Internet of Everything
▪ Services provided on computing platforms of many kind
C/C++,…
Autosar RTE
ForTran,…
Python,…
MPI,…
Java,…
MPI,…
C/C++,…
MCAPI,…
HTML5,…
Java,…
Dalvik VM,…
UML
DC DSLs
UML
ES DSLs
UML
SmartPhone
DSLs
Service
UML
ES DSLs
UML
CPSoS DSL
9. Model-Driven Analysis of IoE Services
▪ Programming the Internet of Everything
▪ Services provided on computing platforms of many kind
HW
Synthesis
SW
Synthesis
(RT)OS,
Runtime,…
Corba, TCP/IP,
MPI, MCAPI,…
ForTran,
Python, Java,
HTML5,
C/C++,…
Concerns
Performance,
safety, cost,
security, size,
…
Trust
Simulation
Verification
Performance
AnalysisPrivacy
Optimization
Service
UML
CPSoS DSL
Security
Safety
10. Model-Driven Analysis of IoE Services
▪ UML/MARTE System Modeling Methodology
▪ Platform-Independent
▪ Component-Based
▪ Supporting
▪ Object-Orientation
▪ Actor-Orientation
required
interface
Component 1
Component 3
Component 2
Port 3.1
Port 3.2 Port 2.1Port 2.2
Port 1
provided
interface
required
interface
provided
interface
required
interface
OO
CB
AO
12. Model-Driven Analysis of IoE Services
▪ Properties of the Provided Port
▪ NotAttendedService
▪ Retry
▪ Properties of the Interface Methods
▪ concurrency
▪ exekind
▪ syncKind
▪ Properties of the Required Port
▪ queueSize
▪ FullPoolPolicy
13. Model-Driven Analysis of IoE Services
▪ Function Call/RPC/RMI
▪ Rendezvous
Required Port RtService Provided Port
MoCNotAttendedService retry concurrency exekind syncKind queueSize FullPoolPolicy
infiniteWait none G or C rem.Im. sync. none none exactly once
infiniteWait none G or C rem.Im. async. none none at most once
dynamic none G or C rem.Im. sync. none none exactly once
dynamic none G or C rem.Im. async. none none at most once
timedWait 0 G or C rem.Im. sync. none none exactly once
timedWait 0 G or C rem.Im. async. none none at most once
timedWait > 0 G or C rem.Im. sync. none none at least once
timedWait > 0 G or C rem.Im. async. none none maybe once
Required Port RtService Provided Port
MoCNotAttendedService retry concurrency exekind syncKind queueSize FullPoolPolicy
infiniteWait none G or C rem.Im. rendezvous none none CSP
timedWait 0 G or C rem.Im. rendezvous none none RV
timedWait > 0 G or C rem.Im. rendezvous none none RV
14. Model-Driven Analysis of IoE Services
▪ Data-Flow
▪ Discrete-Event/Time-Triggered/Timed Data-Flow
Required Port RtService Provided Port
MoCNotAttendedService retry concurrency exekind syncKind queueSize FullPoolPolicy
infiniteWait none G or C deferred async. > 0 block KPN/SDF
infiniteWait none G or C deferred async. > 0 (any other) DF
dynamic none G or C deferred async. > 0 any DF
timedWait 0 G or C deferred async. > 0 any DF
timedWait > 0 G or C deferred async. > 0 any DF
Required Port RtService Provided Port
MoCNotAttendedService retry concurrency exekind syncKind queueSize FullPoolPolicy
dynamic none G or C rem.Im. async. none none DE/TT/TDF
15. Conclusions
▪ The IoE demands new CPSoS design methods and tools
▪ Model-Driven system design is a powerful candidate
▪ A CPSoS system modeling language is required
▪ Supporting Mega-Modeling
▪ Analysis & design of the whole IoE service
▪ Single-Source Approach