The progressive industrial adoption of Model-Driven Engineering (MDE) is fostering the development of large tool ecosystems like the Eclipse Modeling project. These tools are built on top of a set of base technologies that have been primarily designed for small-scale scenarios, where models are manually developed. In particular, efficient runtime manipulation for large-scale models is an under-studied problem and this is hampering the application of MDE to several industrial scenarios.
In this paper we introduce and evaluate a map-based persistence model for MDE tools. We use this model to build a transparent persistence layer for modeling tools, on top of a map-based database engine. The layer can be plugged into the Eclipse Modeling Framework, lowering execution times and memory consumption levels of other existing approaches. Empirical tests are performed based on a typical industrial scenario, model-driven reverse engineering, where very large software models originate from the analysis of massive code bases. The layer is freely distributed and can be immediately used for enhancing the scalability of any existing Eclipse Modeling tool.
http://www.emn.fr/z-info/atlanmod/index.php/NeoEMF/Map
UMLtoGraphDB: Mapping Conceptual Schemas to Graph DatabasesGwendal Daniel
UMLtoGraphDB presentation at ER2016. Related article available online at https://hal.archives-ouvertes.fr/hal-01344015/document
Related post on modeling-languages.com: http://modeling-languages.com/uml-to-nosql-graph-database/
PrefetchML: a Framework for Prefetching and Caching modelsJordi Cabot
Speed up the access and queries on large models thanks to our language (and execution environment) to define access plans for specific scenarios.
This talk was awarded the best paper award at the Models 2016 conference
Read more at http://modeling-languages.com/prefetchml-dsl-prefetching-caching-emf-models/
Close encounters in MDD: when Models meet Codelbergmans
Model-Driven Development (MDD) promises a number of advantages, which include the ability to work at higher abstraction levels, static reasoning about models, and generation of platform-specific code. To achieve this, generally a transformation-based approach is adopted, which generates code from models. In this presentation we discuss –in addition to the potential advantages– a number of possible misunderstandings and risks of MDD.
In particular, we address the risks of transformation-based software development, such as:
• It is rarely possible to generate the full functionality of a (sub-)system from models; as a result, it is necessary to either do additional ‘manual coding’ –a challenge to integrate with the generated code– or annotate the model with small or larger fragments of executable code, which has several restrictions and practical consequences: for instance it mingles abstraction levels, and reduces maintainability of code and models.
• MDD is particularly effective when various different models can be used, each optimized for a specific domain. However, when using transformation techniques, de combination of multiple models in an integrated application is far from trivial.
In this talk we propose –as a low-threshold approach–, ‘bottom-up’ model-driven development. This means that the focus on domain-specific abstractions remains, as well as the separation of platform-specific and platform-independent software. This approach, which is related to Domain-Driven Design and domain-specific languages (DSLs), aims to exploit the advantages of modeling in terms of abstractions, while at the same time reducing the gap between models and code. This can be achieved by specifying the models in code, while separating platform-specific code from the model code. An important issue is the capability to combine several different models, without getting into technical difficulties: we discuss existing as well as a novel approach, entitled Co-op, which aim to address this problem.
Close Encounters in MDD: when models meet codelbergmans
“Close encounters in MDD: when Models meet Code”
Model-Driven Development (MDD) promises a number of advantages, which include the ability to work at higher abstraction levels, static reasoning about models, and generation of platform-specific code. To achieve this, generally a transformation-based approach is adopted, which generates code from models. In this presentation we discuss –in addition to the potential advantages– a number of possible misunderstandings and risks of MDD.
In particular, we address the risks of transformation-based software development, such as:
• It is rarely possible to generate the full functionality of a (sub-)system from models; as a result, it is necessary to either do additional ‘manual coding’ –a challenge to integrate with the generated code– or annotate the model with small or larger fragments of executable code, which has several restrictions and practical consequences: for instance it mingles abstraction levels, and reduces maintainability of code and models.
• MDD is particularly effective when various different models can be used, each optimized for a specific domain. However, when using transformation techniques, de combination of multiple models in an integrated application is far from trivial.
In this talk we propose –as a low-threshold approach–, ‘bottom-up’ model-driven development. This means that the focus on domain-specific abstractions remains, as well as the separation of platform-specific and platform-independent software. This approach, which is related to Domain-Driven Design and domain-specific languages (DSLs), aims to exploit the advantages of modeling in terms of abstractions, while at the same time reducing the gap between models and code. This can be achieved by specifying the models in code, while separating platform-specific code from the model code. An important issue is the capability to combine several different models, without getting into technical difficulties: we discuss existing as well as a novel approach, entitled Co-op, which aim to address this problem.
Finally, we discuss how the presented approach fits with the ‘scalable design’ approach for developing software that is scalable with respect to evolving requirements.
Accelerating Deep Learning Inference on Mobile SystemsDarian Frajberg
International Conference on AI and Mobile Services
Services Conference Federation (SCF)
San Diego, CA, USA
June 2019
Artificial Intelligence on the edge is a matter of great importance towards the enhancement of smart devices that rely on operations with real-time constraints. Despite the rapid growth of computational power in embedded systems, such as smartphones, wearable devices, drones and FPGAs, the deployment of highly complex and considerably big models remains challenging. Optimized execution requires managing memory allocation efficiently, to avoid overloading, and exploiting the available hardware resources for acceleration, which is not trivial given the non standardized access to such resources. We present PolimiDL, an open source framework for the acceleration of Deep Learning inference on mobile and embedded systems with limited resources and heterogeneous architectures. Experimental results show competitive results w.r.t. TensorFlow Lite for the execution of small models.
Directive-based approach to Heterogeneous ComputingRuymán Reyes
The document discusses a directive-based approach to heterogeneous computing. It describes how applications used in HPC centers commonly use MPI and OpenMP programming models. It also discusses how complexity arises from mixing different Fortran dialects and the need for faster ways to migrate code to new architectures like accelerators without rewriting the code. The document proposes using directives to enhance legacy code for heterogeneous systems in a portable way.
UMLtoGraphDB: Mapping Conceptual Schemas to Graph DatabasesGwendal Daniel
UMLtoGraphDB presentation at ER2016. Related article available online at https://hal.archives-ouvertes.fr/hal-01344015/document
Related post on modeling-languages.com: http://modeling-languages.com/uml-to-nosql-graph-database/
PrefetchML: a Framework for Prefetching and Caching modelsJordi Cabot
Speed up the access and queries on large models thanks to our language (and execution environment) to define access plans for specific scenarios.
This talk was awarded the best paper award at the Models 2016 conference
Read more at http://modeling-languages.com/prefetchml-dsl-prefetching-caching-emf-models/
Close encounters in MDD: when Models meet Codelbergmans
Model-Driven Development (MDD) promises a number of advantages, which include the ability to work at higher abstraction levels, static reasoning about models, and generation of platform-specific code. To achieve this, generally a transformation-based approach is adopted, which generates code from models. In this presentation we discuss –in addition to the potential advantages– a number of possible misunderstandings and risks of MDD.
In particular, we address the risks of transformation-based software development, such as:
• It is rarely possible to generate the full functionality of a (sub-)system from models; as a result, it is necessary to either do additional ‘manual coding’ –a challenge to integrate with the generated code– or annotate the model with small or larger fragments of executable code, which has several restrictions and practical consequences: for instance it mingles abstraction levels, and reduces maintainability of code and models.
• MDD is particularly effective when various different models can be used, each optimized for a specific domain. However, when using transformation techniques, de combination of multiple models in an integrated application is far from trivial.
In this talk we propose –as a low-threshold approach–, ‘bottom-up’ model-driven development. This means that the focus on domain-specific abstractions remains, as well as the separation of platform-specific and platform-independent software. This approach, which is related to Domain-Driven Design and domain-specific languages (DSLs), aims to exploit the advantages of modeling in terms of abstractions, while at the same time reducing the gap between models and code. This can be achieved by specifying the models in code, while separating platform-specific code from the model code. An important issue is the capability to combine several different models, without getting into technical difficulties: we discuss existing as well as a novel approach, entitled Co-op, which aim to address this problem.
Close Encounters in MDD: when models meet codelbergmans
“Close encounters in MDD: when Models meet Code”
Model-Driven Development (MDD) promises a number of advantages, which include the ability to work at higher abstraction levels, static reasoning about models, and generation of platform-specific code. To achieve this, generally a transformation-based approach is adopted, which generates code from models. In this presentation we discuss –in addition to the potential advantages– a number of possible misunderstandings and risks of MDD.
In particular, we address the risks of transformation-based software development, such as:
• It is rarely possible to generate the full functionality of a (sub-)system from models; as a result, it is necessary to either do additional ‘manual coding’ –a challenge to integrate with the generated code– or annotate the model with small or larger fragments of executable code, which has several restrictions and practical consequences: for instance it mingles abstraction levels, and reduces maintainability of code and models.
• MDD is particularly effective when various different models can be used, each optimized for a specific domain. However, when using transformation techniques, de combination of multiple models in an integrated application is far from trivial.
In this talk we propose –as a low-threshold approach–, ‘bottom-up’ model-driven development. This means that the focus on domain-specific abstractions remains, as well as the separation of platform-specific and platform-independent software. This approach, which is related to Domain-Driven Design and domain-specific languages (DSLs), aims to exploit the advantages of modeling in terms of abstractions, while at the same time reducing the gap between models and code. This can be achieved by specifying the models in code, while separating platform-specific code from the model code. An important issue is the capability to combine several different models, without getting into technical difficulties: we discuss existing as well as a novel approach, entitled Co-op, which aim to address this problem.
Finally, we discuss how the presented approach fits with the ‘scalable design’ approach for developing software that is scalable with respect to evolving requirements.
Accelerating Deep Learning Inference on Mobile SystemsDarian Frajberg
International Conference on AI and Mobile Services
Services Conference Federation (SCF)
San Diego, CA, USA
June 2019
Artificial Intelligence on the edge is a matter of great importance towards the enhancement of smart devices that rely on operations with real-time constraints. Despite the rapid growth of computational power in embedded systems, such as smartphones, wearable devices, drones and FPGAs, the deployment of highly complex and considerably big models remains challenging. Optimized execution requires managing memory allocation efficiently, to avoid overloading, and exploiting the available hardware resources for acceleration, which is not trivial given the non standardized access to such resources. We present PolimiDL, an open source framework for the acceleration of Deep Learning inference on mobile and embedded systems with limited resources and heterogeneous architectures. Experimental results show competitive results w.r.t. TensorFlow Lite for the execution of small models.
Directive-based approach to Heterogeneous ComputingRuymán Reyes
The document discusses a directive-based approach to heterogeneous computing. It describes how applications used in HPC centers commonly use MPI and OpenMP programming models. It also discusses how complexity arises from mixing different Fortran dialects and the need for faster ways to migrate code to new architectures like accelerators without rewriting the code. The document proposes using directives to enhance legacy code for heterogeneous systems in a portable way.
The document discusses Mirage, an operating system constructed using OCaml and designed to run on the cloud. It aims for end-to-end static type safety using OCaml and domain-specific languages. It describes Mirage's approach of using a simple single-threaded core with the hypervisor dividing cores. It also summarizes Mirage's use of LWT for concurrency without magic, typed memory allocators, I/O implementation in pure OCaml, and goal of being portable across environments like POSIX, Javascript, and Google AppEngine.
C++ Data-flow Parallelism sounds great! But how practical is it? Let’s see ho...Jason Hearne-McGuiness
The document summarizes a presentation on using data-parallelism in C++ to parallelize the SPEC2006 benchmark suite. It discusses the design of a data-flow library called Parallel Pixie Dust (PPD) and how it was used to parallelize some STL algorithms. Analysis of two SPEC2006 benchmarks found that only a small number of STL algorithm usages could be easily parallelized due to the functional decomposition of the code into small blocks and potential side effects. Larger functions and avoidance of side effects may enable more opportunities for parallelization.
Accelerating Spark MLlib and DataFrame with Vector Processor “SX-Aurora TSUBASA”Databricks
NEC has developed a new vector processor called SX-Aurora TSUBASA to accelerate machine learning and data analytics workloads. They developed a middleware framework called Frovedis that provides Spark-like functionality and is optimized for SX-Aurora TSUBASA. Frovedis achieved 10-100x speedups on machine learning algorithms and SQL-like queries compared to Spark on CPUs. NEC has also opened a lab called VEDAC for external users to access SX-Aurora TSUBASA systems running Frovedis.
The document discusses Model Driven Architecture (MDA), which uses models to automate application construction. It defines a model as a simplified representation of a system or phenomenon. Code is considered a type of model that represents logical structure and hides implementation details through different levels of abstraction. Models can represent applications, services, and be executed in the same way code can through tools that support capabilities like model checking, transformation, testing, and debugging. Successful adoption of MDA depends on factors like development process, tool integration, and model lifecycle management.
NEO4EMF, a Neo4j-based model repository and persistence framework allowing on-demand loading, storage, and unloading of large-scale EMF models.
Check us at : https://neo4emf.com
Fork us at : https://github.com/neo4emf/Neo4EMF
Spark Summit EU talk by Francois Garillot and Mohamed KafsiSpark Summit
Mobility Insights at Swisscom analyzes collective mobility patterns in Switzerland using anonymized mobile network data. The presentation discusses:
1) Swisscom's big data architecture for processing mobility data using Spark, including techniques for preserving privacy and ensuring data flows comply with regulations.
2) Methods for classifying transportation modes using bursty cell patterns and machine learning to determine the proportion of trips associated with trains.
3) Streaming analytics applications including monitoring road conditions and selecting anonymous users moving along paths of interest.
4) Challenges of ensuring high quality ground truth data for modeling and validation purposes.
Mobility insights at Swisscom - Understanding collective mobility in SwitzerlandFrançois Garillot
Swisscom is the leading mobile-service provider in Switzerland, with a market share high enough to enable us to model and understand the collective mobility in every area of the country. To accomplish that, we built an urban planning tool that helps cities better manage their infrastructure based on data-based insights, produced with Apache Spark, YARN, Kafka and a good dose of machine learning. In this talk, we will explain how building such a tool involves mining a massive amount of raw data (1.5E9 records/day) to extract fine-grained mobility features from raw network traces. These features are obtained using different machine learning algorithms. For example, we built an algorithm that segments a trajectory into mobile and static periods and trained classifiers that enable us to distinguish between different means of transport. As we sketch the different algorithmic components, we will present our approach to continuously run and test them, which involves complex pipelines managed with Oozie and fuelled with ground truth data. Finally, we will delve into the streaming part of our analytics and see how network events allow Swisscom to understand the characteristics of the flow of people on roads and paths of interest. This requires making a link between network coverage information and geographical positioning in the space of milliseconds and using Spark streaming with libraries that were originally designed for batch processing. We will conclude on the advantages and pitfalls of Spark involved in running this kind of pipeline on a multi-tenant cluster. Audiences should come back from this talk with an overall picture of the use of Apache Spark and related components of its ecosystem in the field of trajectory mining.
Fast Insights to Optimized Vectorization and Memory Using Cache-aware Rooflin...Intel® Software
Integrated into Intel® Advisor, Cache-aware Roofline Modeling (CARM) provides insight into how an application behaves by helping to determine a) how optimally it works on a given hardware, b) the main factors that limit performance, c) if the workload is memory or compute-bound, and d) the right strategy to improve application performance.
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy Jeffrey Holden
This document discusses deploying cloud native red team infrastructure using Kubernetes, Istio and Envoy. It provides introductions to Larry Suto and Jeff Holden and their backgrounds. It then covers goals of being automated, portable and scriptable. Key points covered include using Kubernetes for its infrastructure as code capabilities. It discusses concepts like Docker, Kubernetes, Kops, External DNS, SSL Cert Manager and recipes for containerizing tools like Cobalt Strike, Merlin and configuring deployments.
We build AI and HPC solutions. Expertise: highly optimized AI Engines and HPC Apps.
• HPC: accelerating time to results and adapting complex algorithms to GPU, FPGA, many-CPU architectures.
Leverage byteLAKE expertise in complex algorithms adaptation and optimization for NVIDIA GPUs, Xilinx Alveo FPGAs, Intel, AMD and ARM solutions. From single nodes to clusters.
More: www.byteLAKE.com/en/Alveo
The document provides an overview and introduction to crash dump analysis on Nexenta systems. It discusses core dumps, crash dumps, the panic process, and basic crash dump analysis using mdb. Key topics include process and thread terminology, interrupts and traps, hangs vs crashes vs panics, forensic data sources like console logs and crash dumps, and C language basics relevant to crash analysis like data types and functions. Examples of panic strings, stack traces, and thread lists from crash dumps are also provided, as well as guidance on determining if an issue is hardware, firmware, or software-related.
ParaForming - Patterns and Refactoring for Parallel Programmingkhstandrews
Despite Moore's "law", uniprocessor clock speeds have now stalled. Rather than single processors running at ever higher clock speeds, it is
common to find dual-, quad- or even hexa-core processors, even in consumer laptops and desktops.
Future hardware will not be slightly parallel, however, as in today's multicore systems, but will be
massively parallel, with manycore and perhaps even megacore systems
becoming mainstream.
This means that programmers need to start thinking parallel. To achieve this they must move away
from traditional programming models where parallelism is a
bolted-on afterthought. Rather, programmers must use languages where parallelism is deeply embedded into the programming model
from the outset.
By providing a high level model of computation, without explicit ordering of computations,
declarative languages in general, and functional languages in particular, offer many advantages for parallel
programming.
One of the most fundamental advantages of the functional paradigm is purity.
In a purely functional language, as exemplified by Haskell, there are simply no side effects: it is therefore impossible for parallel computations to conflict with each
other in ways that are not well understood.
ParaForming aims to radically improve the process
of parallelising purely functional programs through a comprehensive set of high-level parallel refactoring patterns for Parallel Haskell,
supported by advanced refactoring tools.
By matching parallel design patterns with appropriate algorithmic skeletons
using advanced software refactoring techniques and novel cost information, we will bridge the gap between fully automatic
and fully explicit approaches to parallelisation, helping programmers "think parallel" in a systematic,
guided way. This talk introduces the ParaForming approach, gives some examples and shows how
effective parallel programs can be developed using advanced refactoring technology.
The document summarizes a technical workshop on wireless sensor networks. It provides an overview of the hardware and software used, including the Tmote Sky and EE sensor nodes, the iNode embedded PCs, and the TinyOS software platform. It also describes the Job scheduling system and iPlatform that are used to define and run experiments on the testbed.
A DSL-Based Approach for Cloud-Based Systems Elasticity Testing Michel Albonico
This document presents a DSL-based approach for elasticity testing of cloud systems. The approach aims to reduce tester effort by providing a DSL for specifying elasticity setup, driving cloud systems through elastic states, and executing elasticity-based tests. Preliminary results show that using the DSL requires less writing effort compared to native cloud provider CLIs, and allows adapting tests to different cloud providers with only small code increments. Future work includes adding automatic resource discovery and more validation of the approach.
So you've been deploying Java in the cloud and are wondering how to handle the new world of containers, microservices, and memory constraints. Cold starts got you down? Come to this session to learn about how the OpenJ9 and the JVM in general can help you on your Cloud Native journey.
This presentation is from the PBS User Group 2013, as presented by Greg Clifford.
"Cray scalable solutions are optimized for complex simulations, where parallel and massively concurrent applications need access to data, fast. This presentation will cover recent updates from Cray, showing how Cray continues to deliver the largest production supercomputers leveraging Altair PBS Works and HyperWorks software solutions."
Learn more: http://www.altairatc.com/page.aspx?region=na&name=agenda
Watch the presentation video: http://insidehpc.com/2013/10/04/cray-hpc-environments-leading-edge-simulation/
Container Attached Storage (CAS) with OpenEBS - SDC 2018OpenEBS
The document discusses container attached storage (CAS), which aims to provide storage for containers in a container-native way. CAS is designed to run in containers for containers in user space, using the Kubernetes substrate. It addresses challenges like small working sets, ephemeral storage, and cloud lock-in by keeping data local to workloads and allowing per-workload optimization and migration. The document outlines the CAS design and implementation, including using an input/output container to handle storage IO in user space and leveraging technologies like SPDK, virtio, and Kubernetes custom resources.
This document introduces a course on concurrency concepts, models, and programming. It discusses how concurrency arises from multiple threads of control and how it is error-prone. It emphasizes using abstract models like finite state machines to design, analyze, and verify concurrent systems before implementation. Java is used for programming examples to demonstrate concepts and apply modeling techniques. The course aims to provide a sound understanding of concurrency principles, modeling, and programming practice.
A model based approach for developing event-driven architectures with AsyncAPIabgolla
In this Internet of Things (IoT) era, our everyday objects have evolved into the so-called cyber-physical systems (CPS).
The use and deployment of CPS has especially penetrated the industry, giving rise to the Industry 4.0 or Industrial IoT (IIoT).
Typically, architectures in IIoT environments are distributed and asynchronous, communication being guided by events such as the publication of (and corresponding subscription to) messages.
While these architectures have some clear advantages (such as scalability and flexibility), they also raise interoperability challenges among the agents in the network. Indeed, the knowledge about the message content and its categorization (topics) gets diluted, leading to consistency problems, potential losses of information and complex processing requirements on the subscriber side to try to understand the received messages.
In this paper, we present our proposal relying on \emph{AsyncAPI} to automate the design and implementation of these architectures using model-based techniques for the generation of (part of) event-driven infrastructures.
The prototype that implements this proposal as an open-source project is available at https://github.com/SOM-Research/asyncapi-toolkit
A Modeling Editor and Code Generator for AsyncAPIabgolla
Talk presented at the 1st AsyncAPI conference - https://www.asyncapiconf.com/
Talk summary:
In the new Internet of Things (IoT) era, our everyday objects have evolved in the so-called cyber-physical systems (CPS). The use and deployment of CPS has especially penetrated the industry, giving rise to the Industry 4.0 or Industrial IoT (IIoT). Typically, architectures in IIot environments are distributed and asynchronous, communication being guided by events such as the publication of (and corresponding subscription to) messages.
In this talk, we present AsyncAPI toolkit, our proposal relying on AsyncAPI to automate the design and implementation of these architectures using model-based techniques. AsyncAPI toolkit provides a set of editors and Eclipse-based tools which allow defining JSON-based specifications of message-driven APIs using AsyncAPI. From these specifications, the prototype is able to generate the Java code supporting the creation and serialization of JSON-based message payloads according to the modeled AsyncAPI, including nested JSON objects, as well as the necessary code to publish and subscribe to different topics. The initial prototype that implements this proposal as an open-source project is available at https://github.com/SOM-Research/asyncapi-toolkit
More Related Content
Similar to Fase 2015 - Map-based Transparent Persistence for Very Large Models
The document discusses Mirage, an operating system constructed using OCaml and designed to run on the cloud. It aims for end-to-end static type safety using OCaml and domain-specific languages. It describes Mirage's approach of using a simple single-threaded core with the hypervisor dividing cores. It also summarizes Mirage's use of LWT for concurrency without magic, typed memory allocators, I/O implementation in pure OCaml, and goal of being portable across environments like POSIX, Javascript, and Google AppEngine.
C++ Data-flow Parallelism sounds great! But how practical is it? Let’s see ho...Jason Hearne-McGuiness
The document summarizes a presentation on using data-parallelism in C++ to parallelize the SPEC2006 benchmark suite. It discusses the design of a data-flow library called Parallel Pixie Dust (PPD) and how it was used to parallelize some STL algorithms. Analysis of two SPEC2006 benchmarks found that only a small number of STL algorithm usages could be easily parallelized due to the functional decomposition of the code into small blocks and potential side effects. Larger functions and avoidance of side effects may enable more opportunities for parallelization.
Accelerating Spark MLlib and DataFrame with Vector Processor “SX-Aurora TSUBASA”Databricks
NEC has developed a new vector processor called SX-Aurora TSUBASA to accelerate machine learning and data analytics workloads. They developed a middleware framework called Frovedis that provides Spark-like functionality and is optimized for SX-Aurora TSUBASA. Frovedis achieved 10-100x speedups on machine learning algorithms and SQL-like queries compared to Spark on CPUs. NEC has also opened a lab called VEDAC for external users to access SX-Aurora TSUBASA systems running Frovedis.
The document discusses Model Driven Architecture (MDA), which uses models to automate application construction. It defines a model as a simplified representation of a system or phenomenon. Code is considered a type of model that represents logical structure and hides implementation details through different levels of abstraction. Models can represent applications, services, and be executed in the same way code can through tools that support capabilities like model checking, transformation, testing, and debugging. Successful adoption of MDA depends on factors like development process, tool integration, and model lifecycle management.
NEO4EMF, a Neo4j-based model repository and persistence framework allowing on-demand loading, storage, and unloading of large-scale EMF models.
Check us at : https://neo4emf.com
Fork us at : https://github.com/neo4emf/Neo4EMF
Spark Summit EU talk by Francois Garillot and Mohamed KafsiSpark Summit
Mobility Insights at Swisscom analyzes collective mobility patterns in Switzerland using anonymized mobile network data. The presentation discusses:
1) Swisscom's big data architecture for processing mobility data using Spark, including techniques for preserving privacy and ensuring data flows comply with regulations.
2) Methods for classifying transportation modes using bursty cell patterns and machine learning to determine the proportion of trips associated with trains.
3) Streaming analytics applications including monitoring road conditions and selecting anonymous users moving along paths of interest.
4) Challenges of ensuring high quality ground truth data for modeling and validation purposes.
Mobility insights at Swisscom - Understanding collective mobility in SwitzerlandFrançois Garillot
Swisscom is the leading mobile-service provider in Switzerland, with a market share high enough to enable us to model and understand the collective mobility in every area of the country. To accomplish that, we built an urban planning tool that helps cities better manage their infrastructure based on data-based insights, produced with Apache Spark, YARN, Kafka and a good dose of machine learning. In this talk, we will explain how building such a tool involves mining a massive amount of raw data (1.5E9 records/day) to extract fine-grained mobility features from raw network traces. These features are obtained using different machine learning algorithms. For example, we built an algorithm that segments a trajectory into mobile and static periods and trained classifiers that enable us to distinguish between different means of transport. As we sketch the different algorithmic components, we will present our approach to continuously run and test them, which involves complex pipelines managed with Oozie and fuelled with ground truth data. Finally, we will delve into the streaming part of our analytics and see how network events allow Swisscom to understand the characteristics of the flow of people on roads and paths of interest. This requires making a link between network coverage information and geographical positioning in the space of milliseconds and using Spark streaming with libraries that were originally designed for batch processing. We will conclude on the advantages and pitfalls of Spark involved in running this kind of pipeline on a multi-tenant cluster. Audiences should come back from this talk with an overall picture of the use of Apache Spark and related components of its ecosystem in the field of trajectory mining.
Fast Insights to Optimized Vectorization and Memory Using Cache-aware Rooflin...Intel® Software
Integrated into Intel® Advisor, Cache-aware Roofline Modeling (CARM) provides insight into how an application behaves by helping to determine a) how optimally it works on a given hardware, b) the main factors that limit performance, c) if the workload is memory or compute-bound, and d) the right strategy to improve application performance.
Deploying Cloud Native Red Team Infrastructure with Kubernetes, Istio and Envoy Jeffrey Holden
This document discusses deploying cloud native red team infrastructure using Kubernetes, Istio and Envoy. It provides introductions to Larry Suto and Jeff Holden and their backgrounds. It then covers goals of being automated, portable and scriptable. Key points covered include using Kubernetes for its infrastructure as code capabilities. It discusses concepts like Docker, Kubernetes, Kops, External DNS, SSL Cert Manager and recipes for containerizing tools like Cobalt Strike, Merlin and configuring deployments.
We build AI and HPC solutions. Expertise: highly optimized AI Engines and HPC Apps.
• HPC: accelerating time to results and adapting complex algorithms to GPU, FPGA, many-CPU architectures.
Leverage byteLAKE expertise in complex algorithms adaptation and optimization for NVIDIA GPUs, Xilinx Alveo FPGAs, Intel, AMD and ARM solutions. From single nodes to clusters.
More: www.byteLAKE.com/en/Alveo
The document provides an overview and introduction to crash dump analysis on Nexenta systems. It discusses core dumps, crash dumps, the panic process, and basic crash dump analysis using mdb. Key topics include process and thread terminology, interrupts and traps, hangs vs crashes vs panics, forensic data sources like console logs and crash dumps, and C language basics relevant to crash analysis like data types and functions. Examples of panic strings, stack traces, and thread lists from crash dumps are also provided, as well as guidance on determining if an issue is hardware, firmware, or software-related.
ParaForming - Patterns and Refactoring for Parallel Programmingkhstandrews
Despite Moore's "law", uniprocessor clock speeds have now stalled. Rather than single processors running at ever higher clock speeds, it is
common to find dual-, quad- or even hexa-core processors, even in consumer laptops and desktops.
Future hardware will not be slightly parallel, however, as in today's multicore systems, but will be
massively parallel, with manycore and perhaps even megacore systems
becoming mainstream.
This means that programmers need to start thinking parallel. To achieve this they must move away
from traditional programming models where parallelism is a
bolted-on afterthought. Rather, programmers must use languages where parallelism is deeply embedded into the programming model
from the outset.
By providing a high level model of computation, without explicit ordering of computations,
declarative languages in general, and functional languages in particular, offer many advantages for parallel
programming.
One of the most fundamental advantages of the functional paradigm is purity.
In a purely functional language, as exemplified by Haskell, there are simply no side effects: it is therefore impossible for parallel computations to conflict with each
other in ways that are not well understood.
ParaForming aims to radically improve the process
of parallelising purely functional programs through a comprehensive set of high-level parallel refactoring patterns for Parallel Haskell,
supported by advanced refactoring tools.
By matching parallel design patterns with appropriate algorithmic skeletons
using advanced software refactoring techniques and novel cost information, we will bridge the gap between fully automatic
and fully explicit approaches to parallelisation, helping programmers "think parallel" in a systematic,
guided way. This talk introduces the ParaForming approach, gives some examples and shows how
effective parallel programs can be developed using advanced refactoring technology.
The document summarizes a technical workshop on wireless sensor networks. It provides an overview of the hardware and software used, including the Tmote Sky and EE sensor nodes, the iNode embedded PCs, and the TinyOS software platform. It also describes the Job scheduling system and iPlatform that are used to define and run experiments on the testbed.
A DSL-Based Approach for Cloud-Based Systems Elasticity Testing Michel Albonico
This document presents a DSL-based approach for elasticity testing of cloud systems. The approach aims to reduce tester effort by providing a DSL for specifying elasticity setup, driving cloud systems through elastic states, and executing elasticity-based tests. Preliminary results show that using the DSL requires less writing effort compared to native cloud provider CLIs, and allows adapting tests to different cloud providers with only small code increments. Future work includes adding automatic resource discovery and more validation of the approach.
So you've been deploying Java in the cloud and are wondering how to handle the new world of containers, microservices, and memory constraints. Cold starts got you down? Come to this session to learn about how the OpenJ9 and the JVM in general can help you on your Cloud Native journey.
This presentation is from the PBS User Group 2013, as presented by Greg Clifford.
"Cray scalable solutions are optimized for complex simulations, where parallel and massively concurrent applications need access to data, fast. This presentation will cover recent updates from Cray, showing how Cray continues to deliver the largest production supercomputers leveraging Altair PBS Works and HyperWorks software solutions."
Learn more: http://www.altairatc.com/page.aspx?region=na&name=agenda
Watch the presentation video: http://insidehpc.com/2013/10/04/cray-hpc-environments-leading-edge-simulation/
Container Attached Storage (CAS) with OpenEBS - SDC 2018OpenEBS
The document discusses container attached storage (CAS), which aims to provide storage for containers in a container-native way. CAS is designed to run in containers for containers in user space, using the Kubernetes substrate. It addresses challenges like small working sets, ephemeral storage, and cloud lock-in by keeping data local to workloads and allowing per-workload optimization and migration. The document outlines the CAS design and implementation, including using an input/output container to handle storage IO in user space and leveraging technologies like SPDK, virtio, and Kubernetes custom resources.
This document introduces a course on concurrency concepts, models, and programming. It discusses how concurrency arises from multiple threads of control and how it is error-prone. It emphasizes using abstract models like finite state machines to design, analyze, and verify concurrent systems before implementation. Java is used for programming examples to demonstrate concepts and apply modeling techniques. The course aims to provide a sound understanding of concurrency principles, modeling, and programming practice.
Similar to Fase 2015 - Map-based Transparent Persistence for Very Large Models (20)
A model based approach for developing event-driven architectures with AsyncAPIabgolla
In this Internet of Things (IoT) era, our everyday objects have evolved into the so-called cyber-physical systems (CPS).
The use and deployment of CPS has especially penetrated the industry, giving rise to the Industry 4.0 or Industrial IoT (IIoT).
Typically, architectures in IIoT environments are distributed and asynchronous, communication being guided by events such as the publication of (and corresponding subscription to) messages.
While these architectures have some clear advantages (such as scalability and flexibility), they also raise interoperability challenges among the agents in the network. Indeed, the knowledge about the message content and its categorization (topics) gets diluted, leading to consistency problems, potential losses of information and complex processing requirements on the subscriber side to try to understand the received messages.
In this paper, we present our proposal relying on \emph{AsyncAPI} to automate the design and implementation of these architectures using model-based techniques for the generation of (part of) event-driven infrastructures.
The prototype that implements this proposal as an open-source project is available at https://github.com/SOM-Research/asyncapi-toolkit
A Modeling Editor and Code Generator for AsyncAPIabgolla
Talk presented at the 1st AsyncAPI conference - https://www.asyncapiconf.com/
Talk summary:
In the new Internet of Things (IoT) era, our everyday objects have evolved in the so-called cyber-physical systems (CPS). The use and deployment of CPS has especially penetrated the industry, giving rise to the Industry 4.0 or Industrial IoT (IIoT). Typically, architectures in IIot environments are distributed and asynchronous, communication being guided by events such as the publication of (and corresponding subscription to) messages.
In this talk, we present AsyncAPI toolkit, our proposal relying on AsyncAPI to automate the design and implementation of these architectures using model-based techniques. AsyncAPI toolkit provides a set of editors and Eclipse-based tools which allow defining JSON-based specifications of message-driven APIs using AsyncAPI. From these specifications, the prototype is able to generate the Java code supporting the creation and serialization of JSON-based message payloads according to the modeled AsyncAPI, including nested JSON objects, as well as the necessary code to publish and subscribe to different topics. The initial prototype that implements this proposal as an open-source project is available at https://github.com/SOM-Research/asyncapi-toolkit
Enabling Performance Modeling for the Masses: Initial Experiencesabgolla
Performance problems such as sluggish response time or low throughput are especially annoying, frustrating and noticeable to users. Fixing performance problems after they occur results in unplanned expenses and time. Our vision is an MDE-intensive software development paradigm for complex systems in which software designers can evaluate performance early in development, when the analysis can have the greatest impact. We seek to empower designers to do the analysis themselves by automating the creation of performance models out of standard design models. Such performance models can be automatically solved, providing results meaningful to them. In our vision, this automation can be enabled by using model-to-model transformations: First, designers create UML design models embellished with the Modeling and Analysis of Real Time and Embedded systems (MARTE) design specifications; and secondly, such models are transformed to automatically solvable performance models by using QVT. This work reports on our first experiences when implementing these two initial activities.
See full articla at: https://dx.doi.org/10.1007/978-3-030-01042-3_7
TemporalEMF: A Temporal Metamodeling Frameworkabgolla
Existing modeling tools provide direct access to the most current version of a model but very limited support to inspect the model state in the past. This typically requires looking for a model version (usually stored in some kind of external versioning system like Git) roughly corresponding to the desired period and using it to manually retrieve the required data. This approximate answer is not enough in scenarios that require a more precise and immediate response to temporal queries like complex collaborative co-engineering processes or runtime models.
In this paper, we reuse well-known concepts from temporal languages to propose a temporal metamodeling framework, called TemporalEMF, that adds native temporal support for models. In our framework, models are automatically treated as temporal models and can be subjected to temporal queries to retrieve the model contents at different points in time. We have built our framework on top of the Eclipse Modeling Framework (EMF). Behind the scenes, the history of a model is transparently stored in a NoSQL database. We evaluate the resulting TemporalEMF framework with an Industry 4.0 case study about a production system simulator. The results show good scalability for storing and accessing temporal models without requiring changes to the syntax and semantics of the simulator.
On the Opportunities of Scalable Modeling Technologies: An Experience Report ...abgolla
Paper "On the Opportunities of Scalable Modeling Technologies: An Experience Report on Wind Turbines Control Applications Development" presented at ECMFA 2017, part of STAF, @ Marburg.
Una herramienta para evaluar el rendimiento de aplicaciones intensivas en datosabgolla
Las aplicaciones intensivas en datos (AID) que usan tecnologías de \emph{Big Data} se están convirtiendo en una parte importante del mercado de desarrollo de software. Sin embargo, las técnicas ---y su automatización--- para el asesoramiento de la calidad para este tipo de aplicaciones es claramente insuficiente.
El proyecto DICE H2020 tiene como objetivo definir metodologías y crear herramientas para desarrollar y monitorizar AID mediante técnicas de ingeniería dirigida por modelos. En este artículo presentamos un componente clave del proyecto DICE: su herramienta de simulación. Esta herramienta es capaz de evaluar el rendimiento de AID simulando su comportamiento mediante modelos de redes de Petri. Como complemento, existe a disposición un vídeo mostrando la herramienta en http://tiny.cc/z1qzay.
Comparing Evolved Extractive Text Summary Scores of Bidirectional Encoder Rep...University of Maribor
Slides from:
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Track: Artificial Intelligence
https://www.etran.rs/2024/en/home-english/
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...Sérgio Sacani
Context. With a mass exceeding several 104 M⊙ and a rich and dense population of massive stars, supermassive young star clusters
represent the most massive star-forming environment that is dominated by the feedback from massive stars and gravitational interactions
among stars.
Aims. In this paper we present the Extended Westerlund 1 and 2 Open Clusters Survey (EWOCS) project, which aims to investigate
the influence of the starburst environment on the formation of stars and planets, and on the evolution of both low and high mass stars.
The primary targets of this project are Westerlund 1 and 2, the closest supermassive star clusters to the Sun.
Methods. The project is based primarily on recent observations conducted with the Chandra and JWST observatories. Specifically,
the Chandra survey of Westerlund 1 consists of 36 new ACIS-I observations, nearly co-pointed, for a total exposure time of 1 Msec.
Additionally, we included 8 archival Chandra/ACIS-S observations. This paper presents the resulting catalog of X-ray sources within
and around Westerlund 1. Sources were detected by combining various existing methods, and photon extraction and source validation
were carried out using the ACIS-Extract software.
Results. The EWOCS X-ray catalog comprises 5963 validated sources out of the 9420 initially provided to ACIS-Extract, reaching a
photon flux threshold of approximately 2 × 10−8 photons cm−2
s
−1
. The X-ray sources exhibit a highly concentrated spatial distribution,
with 1075 sources located within the central 1 arcmin. We have successfully detected X-ray emissions from 126 out of the 166 known
massive stars of the cluster, and we have collected over 71 000 photons from the magnetar CXO J164710.20-455217.
Nucleophilic Addition of carbonyl compounds.pptxSSR02
Nucleophilic addition is the most important reaction of carbonyls. Not just aldehydes and ketones, but also carboxylic acid derivatives in general.
Carbonyls undergo addition reactions with a large range of nucleophiles.
Comparing the relative basicity of the nucleophile and the product is extremely helpful in determining how reversible the addition reaction is. Reactions with Grignards and hydrides are irreversible. Reactions with weak bases like halides and carboxylates generally don’t happen.
Electronic effects (inductive effects, electron donation) have a large impact on reactivity.
Large groups adjacent to the carbonyl will slow the rate of reaction.
Neutral nucleophiles can also add to carbonyls, although their additions are generally slower and more reversible. Acid catalysis is sometimes employed to increase the rate of addition.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Ana Luísa Pinho
Functional Magnetic Resonance Imaging (fMRI) provides means to characterize brain activations in response to behavior. However, cognitive neuroscience has been limited to group-level effects referring to the performance of specific tasks. To obtain the functional profile of elementary cognitive mechanisms, the combination of brain responses to many tasks is required. Yet, to date, both structural atlases and parcellation-based activations do not fully account for cognitive function and still present several limitations. Further, they do not adapt overall to individual characteristics. In this talk, I will give an account of deep-behavioral phenotyping strategies, namely data-driven methods in large task-fMRI datasets, to optimize functional brain-data collection and improve inference of effects-of-interest related to mental processes. Key to this approach is the employment of fast multi-functional paradigms rich on features that can be well parametrized and, consequently, facilitate the creation of psycho-physiological constructs to be modelled with imaging data. Particular emphasis will be given to music stimuli when studying high-order cognitive mechanisms, due to their ecological nature and quality to enable complex behavior compounded by discrete entities. I will also discuss how deep-behavioral phenotyping and individualized models applied to neuroimaging data can better account for the subject-specific organization of domain-general cognitive systems in the human brain. Finally, the accumulation of functional brain signatures brings the possibility to clarify relationships among tasks and create a univocal link between brain systems and mental functions through: (1) the development of ontologies proposing an organization of cognitive processes; and (2) brain-network taxonomies describing functional specialization. To this end, tools to improve commensurability in cognitive science are necessary, such as public repositories, ontology-based platforms and automated meta-analysis tools. I will thus discuss some brain-atlasing resources currently under development, and their applicability in cognitive as well as clinical neuroscience.
The ability to recreate computational results with minimal effort and actionable metrics provides a solid foundation for scientific research and software development. When people can replicate an analysis at the touch of a button using open-source software, open data, and methods to assess and compare proposals, it significantly eases verification of results, engagement with a diverse range of contributors, and progress. However, we have yet to fully achieve this; there are still many sociotechnical frictions.
Inspired by David Donoho's vision, this talk aims to revisit the three crucial pillars of frictionless reproducibility (data sharing, code sharing, and competitive challenges) with the perspective of deep software variability.
Our observation is that multiple layers — hardware, operating systems, third-party libraries, software versions, input data, compile-time options, and parameters — are subject to variability that exacerbates frictions but is also essential for achieving robust, generalizable results and fostering innovation. I will first review the literature, providing evidence of how the complex variability interactions across these layers affect qualitative and quantitative software properties, thereby complicating the reproduction and replication of scientific studies in various fields.
I will then present some software engineering and AI techniques that can support the strategic exploration of variability spaces. These include the use of abstractions and models (e.g., feature models), sampling strategies (e.g., uniform, random), cost-effective measurements (e.g., incremental build of software configurations), and dimensionality reduction methods (e.g., transfer learning, feature selection, software debloating).
I will finally argue that deep variability is both the problem and solution of frictionless reproducibility, calling the software science community to develop new methods and tools to manage variability and foster reproducibility in software systems.
Exposé invité Journées Nationales du GDR GPL 2024
hematic appreciation test is a psychological assessment tool used to measure an individual's appreciation and understanding of specific themes or topics. This test helps to evaluate an individual's ability to connect different ideas and concepts within a given theme, as well as their overall comprehension and interpretation skills. The results of the test can provide valuable insights into an individual's cognitive abilities, creativity, and critical thinking skills
Current Ms word generated power point presentation covers major details about the micronuclei test. It's significance and assays to conduct it. It is used to detect the micronuclei formation inside the cells of nearly every multicellular organism. It's formation takes place during chromosomal sepration at metaphase.
Unlocking the mysteries of reproduction: Exploring fecundity and gonadosomati...AbdullaAlAsif1
The pygmy halfbeak Dermogenys colletei, is known for its viviparous nature, this presents an intriguing case of relatively low fecundity, raising questions about potential compensatory reproductive strategies employed by this species. Our study delves into the examination of fecundity and the Gonadosomatic Index (GSI) in the Pygmy Halfbeak, D. colletei (Meisner, 2001), an intriguing viviparous fish indigenous to Sarawak, Borneo. We hypothesize that the Pygmy halfbeak, D. colletei, may exhibit unique reproductive adaptations to offset its low fecundity, thus enhancing its survival and fitness. To address this, we conducted a comprehensive study utilizing 28 mature female specimens of D. colletei, carefully measuring fecundity and GSI to shed light on the reproductive adaptations of this species. Our findings reveal that D. colletei indeed exhibits low fecundity, with a mean of 16.76 ± 2.01, and a mean GSI of 12.83 ± 1.27, providing crucial insights into the reproductive mechanisms at play in this species. These results underscore the existence of unique reproductive strategies in D. colletei, enabling its adaptation and persistence in Borneo's diverse aquatic ecosystems, and call for further ecological research to elucidate these mechanisms. This study lends to a better understanding of viviparous fish in Borneo and contributes to the broader field of aquatic ecology, enhancing our knowledge of species adaptations to unique ecological challenges.