Autonomic Resource Provisioning for Cloud-Based Software

•

1 like•753 views

This document summarizes Pooyan Jamshidi's research on autonomic resource provisioning for cloud-based software. The research was conducted in collaboration with Aakash Ahmad at the Irish Centre for Cloud Computing and Commerce at Dublin City University under the supervision of Dr. Claus Pahl. The research aims to develop techniques to dynamically provision cloud resources in response to changing demand in order to improve resource utilization and meet service level agreements.

Technology Business

Autonomic Resource Provisioning for
Cloud-Based Software
Pooyan Jamshidi
Supervisor: Dr. Claus Pahl
In collaboration with: Aakash Ahmad
IC4- Irish Centre for Cloud Computing and Commerce
School of Computing, Dublin City University

Department of Computer Engineering
Shahid Beheshti University, Tehran, Iran
30th Dec, 2013

Measured
Service

Resource
Pooling

Elasticity

Ubiquitous
Network
Access

Self
Service

Capacity we can provision
with RobusT2Scale

scale-up =
bigger VMs
scale-out = more
VMs

SUT

Criteria

Big spike Dual phase

Large
variations

Quickly
varying

Slowly
varying

Steep tri
phase

with
RobusT2Scale

𝑟𝑡95%

973ms

537ms

509ms

451ms

423ms

498ms

𝑣𝑚

3.2

3.8

5.1

5.3

3.7

3.9

with
overprovisioning

𝑟𝑡95%

354ms

411ms

395ms

446ms

371ms

491ms

𝑣𝑚

6

6

6

6

6

6

with under
provisioning

𝑟𝑡95%

1465ms

1832ms

1789ms

1594ms

1898ms

2194ms

𝑣𝑚

2

2

2

2

2

2

SLA with 𝒓𝒕 𝟗𝟓 = 𝟔𝟎𝟎𝒎𝒔

0.1

0.08

0.06

0.04

0.02

0
alpha=0.1

alpha=0.5

alpha=0.9

alpha=1.0

Benefits

Limitations,
open issues
& future directions

~75% = wasted hardware
Actual
traffic

0.1

0.08

0.06

0.04

0.02

0
alpha=0.1

alpha=0.5

alpha=0.9

alpha=1.0

What's hot

Task Scheduling methodology in cloud computing

Qutub-ud- Din

Job sequence scheduling for cloud computing

Samruddhi Gaikwad

QoS-Aware Data Replication for Data-Intensive Applications in Cloud Computing...

Papitha Velumani

During the last years, except from the traditional CPU based hardware servers, hardware accelerators are widely used in various HPC application areas. More specifically, Graphics Processing Units (GPUs), Many Integrated Cores (MICs) and Field-Programmable Gate Arrays (FPGAs) have shown a great potential in HPC and have been widely mobilised in supercomputing and in HPC-Clouds. This presentation focuses on the development of a cloud simulation framework that supports hardware accelerators. The design and implementation of the framework are also discussed. This presentation was given by Dr. Konstantinos Giannoutakis (CERTH) at the CloudLightning Conference on 11th April 2017.

Simulation of Heterogeneous Cloud Infrastructures

CloudLightning

Energy-aware Task Scheduling using Ant-colony Optimization in cloud

Linda J

Cloud computing is the requirement based on clients that this computing which provides software, infrastructure and platform as a service as per pay for use norm. The scheduling main goal is to achieve the accuracy and correctness on task completion. The scheduling in cloud environment which enables the various cloud services to help framework implementation. Thus the far reaching way of different type of scheduling algorithms in cloud computing environment surveyed which includes the workflow scheduling and grid scheduling. The survey gives an elaborate idea about grid, cloud, workflow scheduling to minimize the energy cost, efficiency and throughput of the system.

A Review on Scheduling in Cloud Computing

ijujournal

Scheduling in cloud

Dr.Manjunath Kotari

UnaCloud is an opportunistic based cloud infrastructure (IaaS) that allows to access on-demand computing capabilities using commodity desktops. Although UnaCloud tried to maximize the use of idle resources to deploy virtual machines on them, it does not use energy-efficient resource allocation algorithms. In this paper, we design and implement different energy-aware techniques to operate in an energyefficient way and at the same time guarantee the performance to the users. Performance tests with different algorithms and scenarios using real trace workloads from UnaCloud, show how different policies can change the energy consumption patterns and reduce the energy consumption in opportunistic cloud infrastructures. The results show that some algorithms can reduce the energy-consumption power up to 30% over the percentage earned by opportunistic environment.

Energy-aware VM Allocation on An Opportunistic Cloud Infrastructure

Mario Jose Villamizar Cano

Scheduling in CCE

Mayuri Saxena

Fault tolerance on cloud computing

www.pixelsolutionbd.com

In datacenter performance analysis of a tensor processing unit

Jinwon Lee

Cloud computing has taken an importantposition in the field of research as well as in thegovernment organisations. Cloud computing uses virtualnetwork technology to provide computer resources tothe end users as well as to the customer’s. Due tocomplex computing environment the use of high logicsand task scheduler algorithms are increase which resultsin costly operation of cloud network. Researchers areattempting to build such kind of job scheduling algorithms that are compatible and applicable in cloud computing environment.In this paper, we review research work which is recently proposed by researchers on the base of energy saving scheduling techniques. We also studying various scheduling algorithms and issues related to them in cloud computing.

Cloud computing Review over various scheduling algorithms

IJEEE

Cloud Computing is the latest networking technology and also popular archetype for hosting the application and delivering of services over the network. The foremost technology of the cloud computing is virtualization which enables of building the applications, dynamically sharing of resources and providing diverse services to the cloud users. With virtualization, a service provider can guarantee Quality of Service to the user at the same time as achieving higher server consumption and energy competence. One of the most important challenges in the cloud computing environment is the VM placemnt and task scheduling problem. This paper focus on Metaheuristic Swarm Optimisation Algorithms(MSOA) deals with the problem of VM placement and Task scheduling in cloud environment. The MSOA is a simple parallel algorithm that can be applied in different ways to resolve the task scheduling problems. The proposed algorithm is considered an amalgamation of the SO algorithm and the Cuckoo search (CS) algorithm; called MSOACS. The proposed algorithm is evaluated using Cloudsim Simulator. The results proves the reduction of the makespan and increase the utilization ratio of the proposed MSOACS algorithm compared with SOA algorithms and Randomised Allocation Allocation (RA).

TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...

Journal For Research

In addition to software delivered as services over the Internet, hardware and software systems that make the delivery possible are referred as cloud computing. In cloud computing, resources such as CPU, memory, bandwidth and storage are treated as utilities that can scale up and down on demand. It also allows per-usage metering and billing of these resources. By means of cloud computing, cloud users can handle unexpected high demands without over-provisioning and they do not need to invest for abundant hardware resources initially. Cloud providers, on the other hand, have the opportunity of reallocating idle resources for other cloud users. One general research challenge in cloud computing is the efficient allocation of cloud resources to users since cloud providers should satisfy quality of service (QoS) objectives while minimizing their operational cost. Up to 85 percent of computing capacity remains idle in distributed computing environments and this wastage is mainly due to poor optimization of job placement, parallelization and scheduling. We aim to model resource allocation problem in cloud systems, analyze and optimize it using graphs and formal behavioral models (e.g. finite automata). The problem that we are currently interested, is to distribute virtual machines (VMs) to the datacenters (DCs) with geographical locations in such a way that network latency and infrastructure cost is minimized while WAN bandwidth and DC capacity limits are respected. We model latency as a function of DC load, inter-DC communication and proximity to user. Both VM requests and cloud infrastructure are represented by graphs where vertices correspond to machines (either physical or virtual) and edges correspond to network connections between them. Our approach employs weighted graph similarity and subgraph matching to suggest an efficient placement or the list of migrations to reach an efficient placement. In the first six months period of the thesis project, we collected evaluation data, developed the simulation environment and defined the experimental setup which contains metrics and baseline methods. In addition, we developed the preliminary version of the resource selection algorithm. While, in the second period, we completed the algorithm design and tuning, carried out detailed, progressive evaluation on the suggested algorithm and documented our work.

Modeling and Optimization of Resource Allocation in Cloud [PhD Thesis Progres...

AtakanAral

Cloud Computing and PSo

surya kumar palla

Drops division and replication of data in cloud for optimal performance and s...

Pvrtechnologies Nellore

Multicloud Deployment of Computing Clusters for Loosely Coupled Multi Task C...

IOSR Journals

Cloudsim & greencloud

nedamaleki87

Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection). The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by ~3% AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.5% AP with a MaskR-CNN detector and achieves 52.1% AP with a RetinaNet detector on COCO for a single model without test-time augmentation, significantly outperforms prior art of detectors. SpineNet can transfer to classification tasks, achieving 5% top-1 accuracy improvement on a challenging iNaturalist fine-grained dataset. Code is at: this https URL.

Spine net learning scale permuted backbone for recognition and localization

Devansh16

Outsourcing information to an outsider authoritative control, as is done in distributed computing, offers ascend to security concerns. The information trade off may happen because of assaults by different clients and hubs inside of the cloud. Hence, high efforts to establish safety are required to secure information inside of the cloud. On the other hand, the utilized security procedure should likewise consider the advancement of the information recovery time. In this paper, we propose Division and Replication of Data in the Cloud for Optimal Performance and Security (DROPS) that all in all methodologies the security and execution issues. In the DROPS procedure, we partition a record into sections, and reproduce the divided information over the cloud hubs. Each of the hubs stores just a itary part of a specific information record that guarantees that even in the event of a fruitful assault, no important data is uncovered to the assailant. Additionally, the hubs putting away the sections are isolated with certain separation by method for diagram T-shading to restrict an assailant of speculating the areas of the sections. Moreover, the DROPS procedure does not depend on the customary cryptographic procedures for the information security; in this way alleviating the arrangement of computationally costly approaches. We demonstrate that the likelihood to find and bargain the greater part of the hubs putting away the sections of a solitary record is to a great degree low. We likewise analyze the execution of the DROPS system with ten different plans. The more elevated amount of security with slight execution overhead was watched.

Survey on Division and Replication of Data in Cloud for Optimal Performance a...

IJSRD

What's hot (20)

Task Scheduling methodology in cloud computing

Job sequence scheduling for cloud computing

QoS-Aware Data Replication for Data-Intensive Applications in Cloud Computing...

Simulation of Heterogeneous Cloud Infrastructures

Energy-aware Task Scheduling using Ant-colony Optimization in cloud

A Review on Scheduling in Cloud Computing

Scheduling in cloud

Energy-aware VM Allocation on An Opportunistic Cloud Infrastructure

Scheduling in CCE

Fault tolerance on cloud computing

In datacenter performance analysis of a tensor processing unit

Cloud computing Review over various scheduling algorithms

TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...

Modeling and Optimization of Resource Allocation in Cloud [PhD Thesis Progres...

Cloud Computing and PSo

Drops division and replication of data in cloud for optimal performance and s...

Multicloud Deployment of Computing Clusters for Loosely Coupled Multi Task C...

Cloudsim & greencloud

Spine net learning scale permuted backbone for recognition and localization

Survey on Division and Replication of Data in Cloud for Optimal Performance a...

Similar to Autonomic Resource Provisioning for Cloud-Based Software

Different types of clients over the globe uses Cloud services because cloud computing involves various features and advantages such as building cost effectives solutions for business, scale resources up and down according to the current demand and many more. But from the cloud provider point of view, there are many challenges that need to be faced in order to ensure a hassle free service delivery to the clients. One such problem is to maintain high availability of services. This project aims at presenting a high available HA solution for business continuity and disaster recovery through configuration of various other services such as load balancing, elasticity and replication. Miss Pratiksha Bhagawati | Mrs. Priya N "A Study on Replication and Failover Cluster to Maximize System Uptime" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd41249.pdf Paper URL: https://www.ijtsrd.comcomputer-science/other/41249/a-study-on-replication-and-failover-cluster-to-maximize-system-uptime/miss-pratiksha-bhagawati

A Study on Replication and Failover Cluster to Maximize System Uptime

YogeshIJTSRD

Parallel and distributed system projects for java and dot net

redpel dot com

FT Architecture For Cloud Service Computing

destruck

In machine learning, datastream prediction is a challenging issue, particularly when dealing with enormous amounts of continuous data. The dynamic nature of data makes it difficult for traditional models to handle and sustain real-time prediction accuracy. This research uses a multi-processor long short-term memory (MPLSTM) architecture to present a unique framework for datastream regression. By employing several central processing units (CPUs) to divide the datastream into multiple parallel chunks, the MPLSTM framework illustrates the intrinsic parallelism of long short-term memory (LSTM) networks. The MPLSTM framework ensures accurate predictions by skillfully learning and adapting to changing data distributions. Extensive experimental assessments on real-world datasets have demonstrated the clear superiority of the MPLSTM architecture over previous methods. This study uses the transformer, the most recent deep learning breakthrough technology, to demonstrate how well it can handle challenging tasks and emphasizes its critical role as a cutting-edge approach to raising the bar for machine learning.

Enhanced transformer long short-term memory framework for datastream prediction

IJECEIAES

Networking project list for java and dotnet

redpel dot com

Oracle Coherence

Liran Zelkha

ieee projects 2012 for cse in mobile computing, ieee projects mobile computing 2012, bulk ieee projects, bulk ieee projects 2012, bulk ieee projects 2012 for cse, bulk ieee projects 2012 for cse with abstract, bulk ieee projects 2012 for cse in mobile computing, bulk ieee projects 2012 mobile computing, ieee projects 2012 mobile computing, ieee 2012 projects, ieee 2012 projects for cse, ieee projects for cse 2012 free download, ieee projects for cse 2012 with abstract, ieee 2012 papers, ieee 2012 projects on mobile computing, ieee 2012 projects in mobile computing, ieee 2012 projects for it, ieee projects 2012 for cse in java, 2012 ieee projects, 2012 ieee projects for cse, 2012 ieee projects list, latest ieee projects 2012 for cse, latest ieee projects 2012 for it

Mobile computing java ieee projects 2012 Seabirds ( Chennai, Pondicherry, Vel...

SBGC

Java & dotnet

tema_solution

2013 NEW PROJECT TITLES LIST FOR JAVA AND DOTNET

tema_solution

NON IEEE PROJECT TITLES FOR JAVA & DOTNET

tema_solution

Java & dotnet

tema_solution

NON IEEE REAL TIME PROJECT FOR JAVA,DOTNET,ANDROID

tema_solution

The transmission of a traffic flows with a certain bandwidth demand over a single network path is either not possible or not cost-effective. In these cases, it is veritably periodic usable to improve focus the network's bandwidth appliance by breaking the traffic flow upon multiple qualified paths. Using multiple paths for the equivalent traffic flow increases the certainty of the network, it absorbs deluxe forwarding resources from the network nodes and also it overcomes link failure provide security. In this paper, we illustrate several problems related to splitting a traffic flow over multiple paths while minimizing the absorption of forwarding resources mitigates failures and implementing security.

Mitigating Link Failures & Implementing Security Mechanism in Multipath Flows...

Eswar Publications

Mobile computing projects list for java and dot net

redpel dot com

Black-box and Gray-box Strategies for Virtual Machine Migration

elliando dias

Cs6703 grid and cloud computing book

kaleeswaranme

Paper id 41201624

IJRAT

ieee projects download, base paper for ieee projects, ieee projects list, ieee projects titles, ieee projects for cse, ieee projects on networking,ieee projects 2012, ieee projects 2013, final year project, computer science final year projects, final year projects for information technology, ieee final year projects, final year students projects, students projects in java, students projects download, students projects in java with source code, students projects architecture, free ieee papers

Ieee projects 2012 2013 - Mobile Computing

K Sundaresh Ka

IEEE Parallel and distributed system 2016 Title and Abstract

tsysglobalsolutions

Cloud Spotlight Series 2020 - Building Real-time Intelligent & Responsive Eve...

Wee Meng Chew

Similar to Autonomic Resource Provisioning for Cloud-Based Software (20)

A Study on Replication and Failover Cluster to Maximize System Uptime

Parallel and distributed system projects for java and dot net

FT Architecture For Cloud Service Computing

Enhanced transformer long short-term memory framework for datastream prediction

Networking project list for java and dotnet

Oracle Coherence

Mobile computing java ieee projects 2012 Seabirds ( Chennai, Pondicherry, Vel...

Java & dotnet

2013 NEW PROJECT TITLES LIST FOR JAVA AND DOTNET

NON IEEE PROJECT TITLES FOR JAVA & DOTNET

Java & dotnet

NON IEEE REAL TIME PROJECT FOR JAVA,DOTNET,ANDROID

Mitigating Link Failures & Implementing Security Mechanism in Multipath Flows...

Mobile computing projects list for java and dot net

Black-box and Gray-box Strategies for Virtual Machine Migration

Cs6703 grid and cloud computing book

Paper id 41201624

Ieee projects 2012 2013 - Mobile Computing

IEEE Parallel and distributed system 2016 Title and Abstract

Cloud Spotlight Series 2020 - Building Real-time Intelligent & Responsive Eve...

More from Pooyan Jamshidi

LWF Chain graphs were introduced by Lauritzen, Wermuth, and Frydenberg as a generalization of graphical models based on undirected graphs and DAGs. From the causality point of view, in an LWF CG: Directed edges represent direct causal effects. Undirected edges represent causal effects due to interference, which occurs when an individual’s outcome is influenced by their social interaction with other population members, e.g., in situations that involve contagious agents, educational programs, or social networks. The construction of chain graph models is a challenging task that would be greatly facilitated by automation. Markov blanket discovery has an important role in structure learning of Bayesian network. It is surprising, however, how little attention it has attracted in the context of learning LWF chain graphs. In this work, we provide a graphical characterization of Markov blankets in chain graphs. The characterization is different from the well-known one for Bayesian networks and generalizes it. We provide a novel scalable and sound algorithm for Markov blanket discovery in LWF chain graphs. We also provide a sound and scalable constraint-based framework for learning the structure of LWF CGs from faithful causally sufficient data. With the use of our algorithm, the problem of structure learning is reduced to finding an efficient algorithm for Markov blanket discovery in LWF chain graphs. This greatly simplifies the structure-learning task and makes a wide range of inference/learning problems computationally tractable because our approach exploits locality.

Learning LWF Chain Graphs: A Markov Blanket Discovery Approach

Pooyan Jamshidi

We enable reliable and dependable self‐adaptations of component connectors in unreliable environments with imperfect monitoring facilities and conflicting user opinions about adaptation policies by developing a framework which comprises: (a) mechanisms for robust model evolution, (b) a method for adaptation reasoning, and (c) tool support that allows an end‐to‐end application of the developed techniques in real‐world domains.

A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...

Pooyan Jamshidi

Modern cyber-physical systems (e.g., robotics systems) are typically composed of physical and software components, the characteristics of which are likely to change over time. Assumptions about parts of the system made at design time may not hold at run time, especially when a system is deployed for long periods (e.g., over decades). Self-adaptation is designed to find reconfigurations of systems to handle such run-time inconsistencies. Planners can be used to find and enact optimal reconfigurations in such an evolving context. However, for systems that are highly configurable, such planning becomes intractable due to the size of the adaptation space. To overcome this challenge, in this paper we explore an approach that (a) uses machine learning to find Pareto-optimal configurations without needing to explore every configuration and (b) restricts the search space to such configurations to make planning tractable. We explore this in the context of robot missions that need to consider task timeliness and energy consumption. An independent evaluation shows that our approach results in high-quality adaptation plans in uncertain and adversarial environments. Paper: https://arxiv.org/abs/1903.03920

Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Aut...

Pooyan Jamshidi

Despite achieving state-of-the-art performance across many domains, machine learning systems are highly vulnerable to subtle adversarial perturbations. Although defense approaches have been proposed in recent years, many have been bypassed by even weak adversarial attacks. Previous studies showed that ensembles created by combining multiple weak defenses (i.e., input data transformations) are still weak. In this talk, I will show that it is indeed possible to construct effective ensembles using weak defenses to block adversarial attacks. However, to do so requires a diverse set of such weak defenses. Based on this motivation, I will present Athena, an extensible framework for building effective defenses to adversarial attacks against machine learning systems. I will talk about the effectiveness of ensemble strategies with a diverse set of many weak defenses that comprise transforming the inputs (e.g., rotation, shifting, noising, denoising, and many more) before feeding them to target deep neural network classifiers. I will also discuss the effectiveness of the ensembles with adversarial examples generated by various adversaries in different threat models. In the second half of the talk, I will explain why building defenses based on the idea of many diverse weak defenses works, when it is most effective, and what its inherent limitations and overhead are.

Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...

Pooyan Jamshidi

Transfer Learning for Performance Analysis of Machine Learning Systems

Pooyan Jamshidi

Modern systems (e.g., deep neural networks, big data analytics, and compilers) are highly configurable, which means they expose different performance behavior under different configurations. The fundamental challenge is that one cannot simply measure all configurations due to the sheer size of the configuration space. Transfer learning has been used to reduce the measurement efforts by transferring knowledge about performance behavior of systems across environments. Previously, research has shown that statistical models are indeed transferable across environments. In this work, we investigate identifiability and transportability of causal effects and statistical relations in highly-configurable systems. Our causal analysis agrees with previous exploratory analysis~\cite{Jamshidi17} and confirms that the causal effects of configuration options can be carried over across environments with high confidence. We expect that the ability to carry over causal relations will enable effective performance analysis of highly-configurable systems.

Transfer Learning for Performance Analysis of Configurable Systems:A Causal ...

Pooyan Jamshidi

Machine Learning meets DevOps

Pooyan Jamshidi

Learning to Sample

Pooyan Jamshidi

Machine learn models efficiently under budget constraints to adapt to perturbations such as environmental changes or changes in the internal resources. Modern software-intensive systems are composed of components that are likely to change their behaviour over time (e.g., adding/removing components). For software to continue to operate under such changes, the assumptions about parts of the system made at design time may not hold at runtime due to uncertainty. Mechanisms must be put in place that can dynamically learn new models of these assumptions and use them to make decisions about missions, configurations, etc.

Integrated Model Discovery and Self-Adaptation of Robots

Pooyan Jamshidi

A wide range of modern software-intensive systems (e.g., autonomous systems, big data analytics, robotics, deep neural architectures) are built configurable. These systems offer a rich space for adaptation to different domains and tasks. Developers and users often need to reason about the performance of such systems, making tradeoffs to change specific quality attributes or detecting performance anomalies. For instance, developers of image recognition mobile apps are not only interested in learning which deep neural architectures are accurate enough to classify their images correctly, but also which architectures consume the least power on the mobile devices on which they are deployed. Recent research has focused on models built from performance measurements obtained by instrumenting the system. However, the fundamental problem is that the learning techniques for building a reliable performance model do not scale well, simply because the configuration space is exponentially large that is impossible to exhaustively explore. For example, it will take over 60 years to explore the whole configuration space of a system with 25 binary options. In this talk, I will start motivating the configuration space explosion problem based on my previous experience with large-scale big data systems in industry. I will then present my transfer learning solution to tackle the scalability challenge: instead of taking the measurements from the real system, we learn the performance model using samples from cheap sources, such as simulators that approximate the performance of the real system, with a fair fidelity and at a low cost. Results show that despite the high cost of measurement on the real system, learning performance models can become surprisingly cheap as long as certain properties are reused across environments. In the second half of the talk, I will present empirical evidence, which lays a foundation for a theory explaining why and when transfer learning works by showing the similarities of performance behavior across environments. I will present observations of environmental changes‘ impacts (such as changes to hardware, workload, and software versions) for a selected set of configurable systems from different domains to identify the key elements that can be exploited for transfer learning. These observations demonstrate a promising path for building efficient, reliable, and dependable software systems. Finally, I will share my research vision for the next five years and outline my immediate plans to further explore the opportunities of transfer learning. Related Papers: https://arxiv.org/pdf/1709.02280 https://arxiv.org/pdf/1704.00234 https://arxiv.org/pdf/1606.06543

Transfer Learning for Performance Analysis of Highly-Configurable Software

Pooyan Jamshidi

In classical software development, developers write explicit instructions in a programming language to hardcode the explicit behavior of software systems. By writing each line of code, the programmer instructs the software to have the desirable behavior by exploring a specific point in program space. Recently, however, software systems are adding learning components that, instead of hardcoding an explicit behavior, learn a behavior through data. The learning-intensive software systems are written in terms of models and their parameters that need to be adjusted based on data. In learning-enabled systems, we specify some constraints on the behavior of a desirable program (e.g., a data set of input–output pairs of examples) and use the computational resources to search through the program space to find a program that satisfies the constraints. In neural networks, we restrict the search to a continuous subset of the program space. This talk provides experimental evidence of making tradeoffs for deep neural network models, using the Deep Neural Network Architecture system as a case study. Concrete experimental results are presented; also featured are additional case studies in big data (Storm, Cassandra), data analytics (configurable boosting algorithms), and robotics applications.

Architectural Tradeoff in Learning-Based Software

Pooyan Jamshidi

Production-Ready Machine Learning for the Software Architect

Pooyan Jamshidi

Transfer Learning for Software Performance Analysis: An Exploratory Analysis

Pooyan Jamshidi

Architecting for Scale

Pooyan Jamshidi

Sensitivity Analysis for Building Adaptive Robotic Software

Pooyan Jamshidi

Modern software systems are now being built to be used in dynamic environments utilizing configuration capabilities to adapt to changes and external uncertainties. In a self-adaptation context, we are often interested in reasoning about the performance of the systems under different configurations. Usually, we learn a black-box model based on real measurements to predict the performance of the system given a specific configuration. However, as modern systems become more complex, there are many configuration parameters that may interact and, therefore, we end up learning an exponentially large configuration space. Naturally, this does not scale when relying on real measurements in the actual changing environment. We propose a different solution: Instead of taking the measurements from the real system, we learn the model using samples from other sources, such as simulators that approximate performance of the real system at low cost.

Transfer Learning for Improving Model Predictions in Highly Configurable Soft...

Pooyan Jamshidi

Transfer Learning for Improving Model Predictions in Robotic Systems

Pooyan Jamshidi

Machine Learning meets DevOps

Pooyan Jamshidi

https://arxiv.org/abs/1606.06543 Finding optimal configurations for Stream Processing Systems (SPS) is a challenging problem due to the large number of parameters that can influence their performance and the lack of analytical models to anticipate the effect of a change. To tackle this issue, we consider tuning methods where an experimenter is given a limited budget of experiments and needs to carefully allocate this budget to find optimal configurations. We propose in this setting Bayesian Optimization for Configuration Optimization (BO4CO), an auto-tuning algorithm that leverages Gaussian Processes (GPs) to iteratively capture posterior distributions of the configuration spaces and sequentially drive the experimentation. Validation based on Apache Storm demonstrates that our approach locates optimal configurations within a limited experimental budget, with an improvement of SPS performance typically of at least an order of magnitude compared to existing configuration algorithms.

An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing S...

Pooyan Jamshidi

Configuration Optimization Tool

Pooyan Jamshidi

More from Pooyan Jamshidi (20)

Learning LWF Chain Graphs: A Markov Blanket Discovery Approach

A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...

Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Aut...

Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...

Transfer Learning for Performance Analysis of Machine Learning Systems

Transfer Learning for Performance Analysis of Configurable Systems:A Causal ...

Machine Learning meets DevOps

Learning to Sample

Integrated Model Discovery and Self-Adaptation of Robots

Transfer Learning for Performance Analysis of Highly-Configurable Software

Architectural Tradeoff in Learning-Based Software

Production-Ready Machine Learning for the Software Architect

Transfer Learning for Software Performance Analysis: An Exploratory Analysis

Architecting for Scale

Sensitivity Analysis for Building Adaptive Robotic Software

Transfer Learning for Improving Model Predictions in Highly Configurable Soft...

Transfer Learning for Improving Model Predictions in Robotic Systems

Machine Learning meets DevOps

An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing S...

Configuration Optimization Tool

Recently uploaded

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place. Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects. Here’s what you’ll gain: - Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows. - Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy. - Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency. - Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity. We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic. Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.

Essentials of Automations: Optimizing FME Workflows with Parameters

Safe Software

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

ODC, Data Fabric and Architecture User Group

CatarinaPereira64715

I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.

"Impact of front-end architecture on development cost", Viktor Turskyi

Fwdays

How world-class product teams are winning in the AI era by CEO and Founder, P...

Product School

Key Trends Shaping the Future of Infrastructure.pdf

Cheryl Hung

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to: Create a campaign using Mailchimp with merge tags/fields Send an interactive Slack channel message (using buttons) Have the message received by managers and peers along with a test email for review But there’s more: In a second workflow supporting the same use case, you’ll see: Your campaign sent to target colleagues for approval If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team But—if the “Reject” button is pushed, colleagues will be alerted via Slack message Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors. And... Speakers: Akshay Agnihotri, Product Manager Charlie Greenberg, Host

Connector Corner: Automate dynamic content and events by pushing a button

DianaGray10

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

UiPath Test Automation using UiPath Test Suite series, part 3

DianaGray10

The Future of Platform Engineering

Jemma Hussein Allen

IoT Analytics Company Presentation May 2024

IoTAnalytics

Join us as we dive into the latest updates to the UiPath Orchestrator API, including new limits and features for 2024. Discover how these changes can enhance your automation projects and streamline your workflows. 📚 Overview of UiPath Orchestrator API 🔧 Recent changes to API limits 🛠️ How to adapt to new limits 📋 Best practices for using the Orchestrator API efficiently ❓ Q&A session

Exploring UiPath Orchestrator API: updates and limits in 2024 🚀

DianaGray10

Mission to Decommission: Importance of Decommissioning Products to Increase E...

Product School

I'm excited to share my latest predictions on how AI, robotics, and other technological advancements will reshape industries in the coming years. The slides explore the exponential growth of computational power, the future of AI and robotics, and their profound impact on various sectors. Why this matters: The success of new products and investments hinges on precise timing and foresight into emerging categories. This deck equips founders, VCs, and industry leaders with insights to align future products with upcoming tech developments. These insights enhance the ability to forecast industry trends, improve market timing, and predict competitor actions. Highlights: ▪ Exponential Growth in Compute: How $1000 will soon buy the computational power of a human brain ▪ Scaling of AI Models: The journey towards beyond human-scale models and intelligent edge computing ▪ Transformative Technologies: From advanced robotics and brain interfaces to automated healthcare and beyond ▪ Future of Work: How automation will redefine jobs and economic structures by 2040 With so many predictions presented here, some will inevitably be wrong or mistimed, especially with potential external disruptions. For instance, a conflict in Taiwan could severely impact global semiconductor production, affecting compute costs and related advancements. Nonetheless, these slides are intended to guide intuition on future technological trends.

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl

Peter Udo Diehl

Recently uploaded (20)

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

Essentials of Automations: Optimizing FME Workflows with Parameters

When stars align: studies in data quality, knowledge graphs, and machine lear...

ODC, Data Fabric and Architecture User Group

"Impact of front-end architecture on development cost", Viktor Turskyi

How world-class product teams are winning in the AI era by CEO and Founder, P...

Key Trends Shaping the Future of Infrastructure.pdf

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

Connector Corner: Automate dynamic content and events by pushing a button

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

Neuro-symbolic is not enough, we need neuro-*semantic*

UiPath Test Automation using UiPath Test Suite series, part 3

The Future of Platform Engineering

IoT Analytics Company Presentation May 2024

Exploring UiPath Orchestrator API: updates and limits in 2024 🚀

Mission to Decommission: Importance of Decommissioning Products to Increase E...

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl

Autonomic Resource Provisioning for Cloud-Based Software

1. Autonomic Resource Provisioning for Cloud-Based Software Pooyan Jamshidi Supervisor: Dr. Claus Pahl In collaboration with: Aakash Ahmad IC4- Irish Centre for Cloud Computing and Commerce School of Computing, Dublin City University Department of Computer Engineering Shahid Beheshti University, Tehran, Iran 30th Dec, 2013

2. Measured Service Resource Pooling Elasticity Ubiquitous Network Access Self Service

3. Actual traffic ~50% = wasted hardware

4. ~75% = wasted hardware Actual traffic

5. Really like this??

6. Capacity we can provision with RobusT2Scale

8. scale-up = bigger VMs scale-out = more VMs

10.

11.

12.

13.

14.

15.

16.

17. SUT Criteria Big spike Dual phase Large variations Quickly varying Slowly varying Steep tri phase with RobusT2Scale 𝑟𝑡95% 973ms 537ms 509ms 451ms 423ms 498ms 𝑣𝑚 3.2 3.8 5.1 5.3 3.7 3.9 with overprovisioning 𝑟𝑡95% 354ms 411ms 395ms 446ms 371ms 491ms 𝑣𝑚 6 6 6 6 6 6 with under provisioning 𝑟𝑡95% 1465ms 1832ms 1789ms 1594ms 1898ms 2194ms 𝑣𝑚 2 2 2 2 2 2 SLA with 𝒓𝒕 𝟗𝟓 = 𝟔𝟎𝟎𝒎𝒔

18. 0.1 0.08 0.06 0.04 0.02 0 alpha=0.1 alpha=0.5 alpha=0.9 alpha=1.0

19. Benefits Limitations, open issues & future directions

20. ~75% = wasted hardware Actual traffic 0.1 0.08 0.06 0.04 0.02 0 alpha=0.1 alpha=0.5 alpha=0.9 alpha=1.0

Autonomic Resource Provisioning for Cloud-Based Software

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Autonomic Resource Provisioning for Cloud-Based Software

Similar to Autonomic Resource Provisioning for Cloud-Based Software (20)

More from Pooyan Jamshidi

More from Pooyan Jamshidi (20)

Recently uploaded

Recently uploaded (20)

Autonomic Resource Provisioning for Cloud-Based Software