Covers a brief history of Open Source Math, Science and Engineering Software on Linux. A look at the software tools currently available for mathematical analysis and plotting for math science and engineering. Presented at 2011 Ohio LinuxFest.
For the full video of this presentation, please visit:
https://www.edge-ai-vision.com/2021/02/introduction-to-the-tvm-open-source-deep-learning-compiler-stack-a-presentation-from-octoml/
Luis Ceze, Co-founder and CEO of OctoML, a Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, and Venture Partner at Madrona Venture Group, presents the “Introduction to the TVM Open Source Deep Learning Compiler Stack” tutorial at the September 2020 Embedded Vision Summit.
There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narrow range of server-class GPUs. Deploying workloads to new platforms — such as mobile phones, embedded devices, and accelerators — requires significant manual effort.
In this talk, Ceze presents his work on the TVM stack, which exposes graph- and operator-level optimizations to provide performance portability for deep learning workloads across diverse hardware back-ends. TVM solves optimization challenges specific to deep learning, such as high-level operator fusion, mapping to arbitrary hardware primitives and memory latency hiding. It also automates optimization of low-level programs to hardware characteristics by employing a novel, learning-based cost modeling method for rapid exploration of optimizations.
This slide deck is used as an introduction to the MapReduce programming model, trying hard to be Hadoop-agnostic, as part of the Distributed Systems and Cloud Computing course I hold at Eurecom.
Course website:
http://michiard.github.io/DISC-CLOUD-COURSE/
Sources available here:
https://github.com/michiard/DISC-CLOUD-COURSE
Adapting to a Cambrian AI/SW/HW explosion with open co-design competitions an...Grigori Fursin
Slides from ARM's Research Summit'17 about "Community-Driven and Knowledge-Guided Optimization of AI Applications Across the Whole SW/HW Stack" (http://cKnowledge.org/repo , http://cKnowledge.org/ai , http://tinyurl.com/zlbxvmw , https://developer.arm.com/research/summit )
Co-designing the whole AI/SW/HW stack in terms of speed, accuracy, energy consumption, size, costs and other metrics has become extremely complex, long and costly. With no rigorous methodology for analyzing performance and accumulating optimisation knowledge, we are simply destined to drown in the ever growing number of design choices, system
features and conflicting optimisation goals.
We present our novel community-driven approach to solve the above problems. Originating from natural sciences, this approach is embodied in Collective Knowledge (CK), our open-source cross-platform workflow framework and repository for automatic, collaborative and reproducible experimentation. CK helps organize, unify and share representative workloads, data sets, AI frameworks, libraries, compilers, scripts, models and other artifacts as customizable and reusable components with a common JSON API.
CK helps bring academia, industry and end-users together to
gradually expose optimisation choices at all levels (e.g. from parameterized models and algorithmic skeletons to compiler
flags and hardware configurations) and autotune them across diverse inputs and platforms. Optimization knowledge gets continuously aggregated in public or private repositories such as cKnowledge.org/repo in a reproducible way, and can be then mined and extrapolated to predict better AI algorithm choices, compiler transformations and hardware designs.
We also demonstrate how we use this approach in practice together with ARM and other companies to adapt to a Cambrian AI/SW/HW explosion by creating an open repository of reusable AI artifacts, and then collaboratively optimising and co-designing the whole deep learning stack (software, hardware and models).
Backend Cloud Storage Access in Video StreamingRufael Mekuria
Presentation about optimizing access of backend storage in cloud video streaming deployments, given at Packet Video 2018 in Amsterdam the Netherlands, joint work with Christina Kylili TU Delft
A lecture given for Stats 285 at Stanford on October 30, 2017. I discuss how OSS technology developed at Anaconda, Inc. has helped to scale Python to GPUs and Clusters.
For the full video of this presentation, please visit:
https://www.edge-ai-vision.com/2021/02/introduction-to-the-tvm-open-source-deep-learning-compiler-stack-a-presentation-from-octoml/
Luis Ceze, Co-founder and CEO of OctoML, a Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington, and Venture Partner at Madrona Venture Group, presents the “Introduction to the TVM Open Source Deep Learning Compiler Stack” tutorial at the September 2020 Embedded Vision Summit.
There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narrow range of server-class GPUs. Deploying workloads to new platforms — such as mobile phones, embedded devices, and accelerators — requires significant manual effort.
In this talk, Ceze presents his work on the TVM stack, which exposes graph- and operator-level optimizations to provide performance portability for deep learning workloads across diverse hardware back-ends. TVM solves optimization challenges specific to deep learning, such as high-level operator fusion, mapping to arbitrary hardware primitives and memory latency hiding. It also automates optimization of low-level programs to hardware characteristics by employing a novel, learning-based cost modeling method for rapid exploration of optimizations.
This slide deck is used as an introduction to the MapReduce programming model, trying hard to be Hadoop-agnostic, as part of the Distributed Systems and Cloud Computing course I hold at Eurecom.
Course website:
http://michiard.github.io/DISC-CLOUD-COURSE/
Sources available here:
https://github.com/michiard/DISC-CLOUD-COURSE
Adapting to a Cambrian AI/SW/HW explosion with open co-design competitions an...Grigori Fursin
Slides from ARM's Research Summit'17 about "Community-Driven and Knowledge-Guided Optimization of AI Applications Across the Whole SW/HW Stack" (http://cKnowledge.org/repo , http://cKnowledge.org/ai , http://tinyurl.com/zlbxvmw , https://developer.arm.com/research/summit )
Co-designing the whole AI/SW/HW stack in terms of speed, accuracy, energy consumption, size, costs and other metrics has become extremely complex, long and costly. With no rigorous methodology for analyzing performance and accumulating optimisation knowledge, we are simply destined to drown in the ever growing number of design choices, system
features and conflicting optimisation goals.
We present our novel community-driven approach to solve the above problems. Originating from natural sciences, this approach is embodied in Collective Knowledge (CK), our open-source cross-platform workflow framework and repository for automatic, collaborative and reproducible experimentation. CK helps organize, unify and share representative workloads, data sets, AI frameworks, libraries, compilers, scripts, models and other artifacts as customizable and reusable components with a common JSON API.
CK helps bring academia, industry and end-users together to
gradually expose optimisation choices at all levels (e.g. from parameterized models and algorithmic skeletons to compiler
flags and hardware configurations) and autotune them across diverse inputs and platforms. Optimization knowledge gets continuously aggregated in public or private repositories such as cKnowledge.org/repo in a reproducible way, and can be then mined and extrapolated to predict better AI algorithm choices, compiler transformations and hardware designs.
We also demonstrate how we use this approach in practice together with ARM and other companies to adapt to a Cambrian AI/SW/HW explosion by creating an open repository of reusable AI artifacts, and then collaboratively optimising and co-designing the whole deep learning stack (software, hardware and models).
Backend Cloud Storage Access in Video StreamingRufael Mekuria
Presentation about optimizing access of backend storage in cloud video streaming deployments, given at Packet Video 2018 in Amsterdam the Netherlands, joint work with Christina Kylili TU Delft
A lecture given for Stats 285 at Stanford on October 30, 2017. I discuss how OSS technology developed at Anaconda, Inc. has helped to scale Python to GPUs and Clusters.
EuroMPI 2016 Keynote: How Can MPI Fit Into Today's Big ComputingJonathan Dursi
HTML slides and longer abstract can be found at https://github.com/ljdursi/EuroMPI2016.
For years, the academic science and engineering community was almost alone in pursuing very large-scale numerical computing, and MPI was the lingua franca for such work. But starting in the mid-2000s, we were no longer alone. First internet-scale companies like Google and Yahoo! started performing fairly basic analytics tasks at enormous scale, and since then others have begun tackling increasingly complex and data-heavy machine-learning computations, which involve very familiar scientific computing primitives such as linear algebra, unstructured mesh decomposition, and numerical optimization. These new communities have created programming environments which emphasize what we’ve learned about computer science and programmability since 1994 – with greater levels of abstraction and encapsulation, separating high-level computation from the low-level implementation details.
At about the same time, new academic research communities began using computing at scale to attack their problems - but in many cases, an ideal distributed-memory application for them begins to look more like the new concurrent distributed databases than a large CFD simulation, with data structures like dynamic hash tables and Bloom trees playing more important roles than rectangular arrays or unstructured meshes. These new academic communities are among the first to adopt emerging big-data technologies over traditional HPC options; but as big-data technologies improve their tightly-coupled number-crunching capabilities, they are unlikely to be the last.
In this talk, I sketch out the landscape of distributed technical computing frameworks and environments, and look to see where MPI and the MPI community fits in to this new ecosystem.
This lecture covers the principles and the architectures of modern cluster schedulers, including Apache Mesos, Apache Yarn, Google Borg and K8s, and some notes on Omega
In this deck from the Perth HPC Conference, Rob Farber from TechEnablement presents: AI is Impacting HPC Everywhere.
"The convergence of AI and HPC has created a fertile venue that is ripe for imaginative researchers — versed in AI technology — to make a big impact in a variety of scientific fields. From new hardware to new computational approaches, the true impact of deep- and machine learning on HPC is, in a word, “everywhere”. Just as technology changes in the personal computer market brought about a revolution in the design and implementation of the systems and algorithms used in high performance computing (HPC), so are recent technology changes in machine learning bringing about an AI revolution in the HPC community. Expect new HPC analytic techniques including the use of GANs (Generative Adversarial Networks) in physics-based modeling and simulation, as well as reduced precision math libraries such as NLAFET and HiCMA to revolutionize many fields of research. Other benefits of the convergence of AI and HPC include the physical instantiation of data flow architectures in FPGAs and ASICs, plus the development of powerful data analytic services."
Learn more: http://www.techenablement.com/
and
http://hpcadvisorycouncil.com/events/2019/australia-conference/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Use C++ and Intel® Threading Building Blocks (Intel® TBB) for Hardware Progra...Intel® Software
In this presentation, we focus on an alternative approach that uses nodes that contain Intel® Xeon® processors and Intel® Xeon Phi™ coprocessors. Programming models and the development tools are identical for these resources, greatly simplifying development. We discuss how the same models for vectorization and threading can be used across these compute resources to create software that performs well on them. We further propose an extension to the Intel® Threading Building Blocks (Intel® TBB) flow graph interface that enables intra-node distributed memory programming, simplifying communication, and load balancing between the processors and coprocessors. Finally, we validate this approach by presenting a benchmark of a risk analysis implementation that achieves record-setting performance.
This was a brief 1-hour introduction to R programming, presented at the 1st Inter-experimental Machine Learning (IML) Working Group Workshop at CERN, 20-22 March 2017.
We describe ocl, a Python library built on top of pyOpenCL and numpy. It allows programming
GPU devices using Python. Python functions which are marked up using the provided
decorator, are converted into C99/OpenCL and compiled using the JIT at runtime. This
approach lowers the barrier to entry to programming GPU devices since it requires only
Python syntax and no external compilation or linking steps. The resulting Python program runs
even if a GPU is not available. As an example of application, we solve the problem of
computing the covariance matrix for historical stock prices and determining the optimal
portfolio according to Modern Portfolio Theory
NERSC is the production high-performance computing (HPC) center for the United States Department of Energy (DOE) Office of Science. The center supports over 6,000 users in 600 projects, using a variety of applications in materials science, chemistry, biology, astrophysics, high energy physics, climate science, fusion science, and more.
NERSC deployed the Cori system on over 9,000 Intel® Xeon Phi™ processors. This session describes the optimization strategy for porting codes that target traditional manycore architectures to the processors. We also discuss highlights and lessons learned from the optimization process on 20 applications associated with the NERSC Exascale Science Application Program (NESAP).
pythonOCC presentation at the 11th NASA-ESA Workshop on Product Data Exchange (http://step.nasa.gov/pde2009/).
Abstract : The combination of open-source software and open standards are known to be considered as a key success factor of further technology development. The interest around the STEP standard is growing these last years, since the release of Application Protocols that allow to foresee new software solutions able to ensure a consistent product data description covering the whole lifecycle. This presentation focuses on the features of a free 3D modeling/data exchange library in the context of a research work dealing with the STEP AP239 standard for PDM/ERP interoperability. The goal is to merge these two approaches and describe the product from design to manufacturing, with a granularity that allow to keep the control over product configuration.
Accelerating open science and AI with automated, portable, customizable and r...Grigori Fursin
Validating experimental results from articles has finally become a norm at many systems and ML conferences. Nowadays, more than half of accepted papers pass artifact evaluation and share related code and data. Unfortunately, lack of a common experimental framework, common research methodology and common formats places an increasing burden on evaluators to validate a growing number of ad-hoc artifacts. Furthermore, having too many ad-hoc artifacts and Docker snapshots is almost as bad as not having any (!), since they cannot be easily reused, customized and built upon.
While overviewing more than 100 papers during artifact evaluation at PPoPP, CGO, PACT, Supercomputing and other conferences, we noticed that many of them use similar experimental setups, benchmarks, models, data sets, environments and platforms. This motivated us to develop Collective Knowledge (CK), an open workflow framework with a unified Python API to automate common researchers’ tasks such as detecting software and hardware dependencies, installing missing packages, downloading data sets and models, compiling and running programs, performing autotuning and co-design, crowdsourcing time-consuming experiments across computing resources provided by volunteers similar to SETI@home, applying statistical analysis and machine learning, validating results and plotting them on a common scoreboard for open and fair comparison, automatically generating interactive articles, and so on: http://cKnowledge.org.
In this presentation we will introduce CK concepts and present several real world use cases from General Motors and Arm
on collaborative benchmarking, autotuning and co-design of efficient software/hardware stacks for deep learning. We also present results and reusable CK components from the 1st ACM ReQuEST optimization tournament: http://cKnowledge.org/request. Finally, we introduce our latest initiative to create
an open repository of reusable research components and workflows to reboot and accelerate open science, quantum computing and AI!
EuroMPI 2016 Keynote: How Can MPI Fit Into Today's Big ComputingJonathan Dursi
HTML slides and longer abstract can be found at https://github.com/ljdursi/EuroMPI2016.
For years, the academic science and engineering community was almost alone in pursuing very large-scale numerical computing, and MPI was the lingua franca for such work. But starting in the mid-2000s, we were no longer alone. First internet-scale companies like Google and Yahoo! started performing fairly basic analytics tasks at enormous scale, and since then others have begun tackling increasingly complex and data-heavy machine-learning computations, which involve very familiar scientific computing primitives such as linear algebra, unstructured mesh decomposition, and numerical optimization. These new communities have created programming environments which emphasize what we’ve learned about computer science and programmability since 1994 – with greater levels of abstraction and encapsulation, separating high-level computation from the low-level implementation details.
At about the same time, new academic research communities began using computing at scale to attack their problems - but in many cases, an ideal distributed-memory application for them begins to look more like the new concurrent distributed databases than a large CFD simulation, with data structures like dynamic hash tables and Bloom trees playing more important roles than rectangular arrays or unstructured meshes. These new academic communities are among the first to adopt emerging big-data technologies over traditional HPC options; but as big-data technologies improve their tightly-coupled number-crunching capabilities, they are unlikely to be the last.
In this talk, I sketch out the landscape of distributed technical computing frameworks and environments, and look to see where MPI and the MPI community fits in to this new ecosystem.
This lecture covers the principles and the architectures of modern cluster schedulers, including Apache Mesos, Apache Yarn, Google Borg and K8s, and some notes on Omega
In this deck from the Perth HPC Conference, Rob Farber from TechEnablement presents: AI is Impacting HPC Everywhere.
"The convergence of AI and HPC has created a fertile venue that is ripe for imaginative researchers — versed in AI technology — to make a big impact in a variety of scientific fields. From new hardware to new computational approaches, the true impact of deep- and machine learning on HPC is, in a word, “everywhere”. Just as technology changes in the personal computer market brought about a revolution in the design and implementation of the systems and algorithms used in high performance computing (HPC), so are recent technology changes in machine learning bringing about an AI revolution in the HPC community. Expect new HPC analytic techniques including the use of GANs (Generative Adversarial Networks) in physics-based modeling and simulation, as well as reduced precision math libraries such as NLAFET and HiCMA to revolutionize many fields of research. Other benefits of the convergence of AI and HPC include the physical instantiation of data flow architectures in FPGAs and ASICs, plus the development of powerful data analytic services."
Learn more: http://www.techenablement.com/
and
http://hpcadvisorycouncil.com/events/2019/australia-conference/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Use C++ and Intel® Threading Building Blocks (Intel® TBB) for Hardware Progra...Intel® Software
In this presentation, we focus on an alternative approach that uses nodes that contain Intel® Xeon® processors and Intel® Xeon Phi™ coprocessors. Programming models and the development tools are identical for these resources, greatly simplifying development. We discuss how the same models for vectorization and threading can be used across these compute resources to create software that performs well on them. We further propose an extension to the Intel® Threading Building Blocks (Intel® TBB) flow graph interface that enables intra-node distributed memory programming, simplifying communication, and load balancing between the processors and coprocessors. Finally, we validate this approach by presenting a benchmark of a risk analysis implementation that achieves record-setting performance.
This was a brief 1-hour introduction to R programming, presented at the 1st Inter-experimental Machine Learning (IML) Working Group Workshop at CERN, 20-22 March 2017.
We describe ocl, a Python library built on top of pyOpenCL and numpy. It allows programming
GPU devices using Python. Python functions which are marked up using the provided
decorator, are converted into C99/OpenCL and compiled using the JIT at runtime. This
approach lowers the barrier to entry to programming GPU devices since it requires only
Python syntax and no external compilation or linking steps. The resulting Python program runs
even if a GPU is not available. As an example of application, we solve the problem of
computing the covariance matrix for historical stock prices and determining the optimal
portfolio according to Modern Portfolio Theory
NERSC is the production high-performance computing (HPC) center for the United States Department of Energy (DOE) Office of Science. The center supports over 6,000 users in 600 projects, using a variety of applications in materials science, chemistry, biology, astrophysics, high energy physics, climate science, fusion science, and more.
NERSC deployed the Cori system on over 9,000 Intel® Xeon Phi™ processors. This session describes the optimization strategy for porting codes that target traditional manycore architectures to the processors. We also discuss highlights and lessons learned from the optimization process on 20 applications associated with the NERSC Exascale Science Application Program (NESAP).
pythonOCC presentation at the 11th NASA-ESA Workshop on Product Data Exchange (http://step.nasa.gov/pde2009/).
Abstract : The combination of open-source software and open standards are known to be considered as a key success factor of further technology development. The interest around the STEP standard is growing these last years, since the release of Application Protocols that allow to foresee new software solutions able to ensure a consistent product data description covering the whole lifecycle. This presentation focuses on the features of a free 3D modeling/data exchange library in the context of a research work dealing with the STEP AP239 standard for PDM/ERP interoperability. The goal is to merge these two approaches and describe the product from design to manufacturing, with a granularity that allow to keep the control over product configuration.
Accelerating open science and AI with automated, portable, customizable and r...Grigori Fursin
Validating experimental results from articles has finally become a norm at many systems and ML conferences. Nowadays, more than half of accepted papers pass artifact evaluation and share related code and data. Unfortunately, lack of a common experimental framework, common research methodology and common formats places an increasing burden on evaluators to validate a growing number of ad-hoc artifacts. Furthermore, having too many ad-hoc artifacts and Docker snapshots is almost as bad as not having any (!), since they cannot be easily reused, customized and built upon.
While overviewing more than 100 papers during artifact evaluation at PPoPP, CGO, PACT, Supercomputing and other conferences, we noticed that many of them use similar experimental setups, benchmarks, models, data sets, environments and platforms. This motivated us to develop Collective Knowledge (CK), an open workflow framework with a unified Python API to automate common researchers’ tasks such as detecting software and hardware dependencies, installing missing packages, downloading data sets and models, compiling and running programs, performing autotuning and co-design, crowdsourcing time-consuming experiments across computing resources provided by volunteers similar to SETI@home, applying statistical analysis and machine learning, validating results and plotting them on a common scoreboard for open and fair comparison, automatically generating interactive articles, and so on: http://cKnowledge.org.
In this presentation we will introduce CK concepts and present several real world use cases from General Motors and Arm
on collaborative benchmarking, autotuning and co-design of efficient software/hardware stacks for deep learning. We also present results and reusable CK components from the 1st ACM ReQuEST optimization tournament: http://cKnowledge.org/request. Finally, we introduce our latest initiative to create
an open repository of reusable research components and workflows to reboot and accelerate open science, quantum computing and AI!
Reactive Microservices with Spring 5: WebFlux Trayan Iliev
On November 27 Trayan Iliev from IPT presented “Reactive microservices with Spring 5: WebFlux” @Dev.bg in Betahaus Sofia. IPT – Intellectual Products & Technologies has been organizing Java & JavaScript trainings since 2003.
Spring 5 introduces a new model for end-to-end functional and reactive web service programming with Spring 5 WebFlow, Spring Data & Spring Boot. The main topics include:
– Introduction to reactive programming, Reactive Streams specification, and project Reactor (as WebFlux infrastructure)
– REST services with WebFlux – comparison between annotation-based and functional reactive programming approaches for building.
– Router, handler and filter functions
– Using reactive repositories and reactive database access with Spring Data. Building end-to-end non-blocking reactive web services using Netty-based web runtime
– Reactive WebClients and integration testing. Reactive WebSocket support
– Realtime event streaming to WebClients using JSON Streams, and to JS client using SSE.
Eclipse Con Europe 2014 How to use DAWN Science ProjectMatthew Gerring
This is a talk given at Eclipse Con Europe 2014 on how to use the open source project DAWN, Data Analysis Workbench. This project has two papers with more than three hundred citations of using the software.
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
The Future of Computing is Distributed
Professor Ion Stoica, UC Berkeley RISELab
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
Node-RED and Minecraft - CamJam September 2015Boris Adryan
This workshop uses the Node-RED framework as development tool for JavaScript. Building on functionality available for generic programming challenges, we’re going to use the communication standard TCP (Transmission Control Protocol) to interact with the Minecraft API (Application Programming Interface). The material is aimed at people who have had first experience with the Minecraft API on a Raspberry Pi (say, using Python), who now want to understand what's going on behind the scenes and what TCP, API and all those other acronyms mean. It also introduces flow-based programming concepts.
20 years of technology leadership through highly challenging projects: microelectronics, embedded systems, telecommunications, railways, mobile and web applications, IT systems for business management.
For additional information:
https://www.pmprofessional.ch
Natural Language Processing with CNTK and Apache Spark with Ali ZaidiDatabricks
Apache Spark provides an elegant API for developing machine learning pipelines that can be deployed seamlessly in production. However, one of the most intriguing and performant family of algorithms – deep learning – remains difficult for many groups to deploy in production, both because of the need for tremendous compute resources and also because of the inherent difficulty in tuning and configuring.
In this session, you’ll discover how to deploy the Microsoft Cognitive Toolkit (CNTK) inside of Spark clusters on the Azure cloud platform. Learn about the key considerations for administering GPU-enabled Spark clusters, configuring such workloads for maximum performance, and techniques for distributed hyperparameter optimization. You’ll also see a real-world example of training distributed deep learning learning algorithms for speech recognition and natural language processing.Microsoft Cognitive Toolkit (CNTK) inside of Spark clusters on the Azure cloud platform. We’ll discuss the key considerations for administering GPU-enabled Spark clusters, configuring such workloads for maximum performance, and techniques for distributed hyperparameter optimization. We’ll illustrate a real-world example of training distributed deep learning learning algorithms for speech recognition and natural language processing.
At the technology meeting of the Association of Independent Research Centers (http://airi.org): An overview of recent Scientific Computing activities at Fred Hutch, Seattle
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...Rafael Ferreira da Silva
Scientific workflows are used routinely in numerous scientific domains, and Workflow Management Systems (WMSs) have been developed to orchestrate and optimize workflow executions on distributed platforms. WMSs are complex software systems that interact with complex software infrastructures. Most WMS research and development activities rely on empirical experiments conducted with full-fledged software stacks on actual hardware platforms. Such experiments, however, are limited to hardware and software infrastructures at hand and can be labor- and/or time-intensive. As a result, relying solely on real- world experiments impedes WMS research and development. An alternative is to conduct experiments in simulation.
In this work we present WRENCH, a WMS simulation framework, whose objectives are (i) accurate and scalable simula- tions; and (ii) easy simulation software development. WRENCH achieves its first objective by building on the SimGrid framework. While SimGrid is recognized for the accuracy and scalability of its simulation models, it only provides low-level simulation abstractions and thus large software development efforts are required when implementing simulators of complex systems. WRENCH thus achieves its second objective by providing high- level and directly re-usable simulation abstractions on top of SimGrid. After describing and giving rationales for WRENCH’s software architecture and APIs, we present a case study in which we apply WRENCH to simulate the Pegasus production WMS. We report on ease of implementation, simulation accuracy, and simulation scalability so as to determine to which extent WRENCH achieves its two above objectives. We also draw both qualitative and quantitative comparisons with a previously proposed workflow simulator.
A performance analysis of OpenStack Cloud vs Real System on Hadoop ClustersKumari Surabhi
It introduces the performance analysis of OpenStack Cloud with the commodity computers in the big data environments. It concludes that the data storage and analysis in hadoop cluster in cloud is more flexible and easily scalable than the real system cluster. It also concludes the cluster in commodities computers are faster than the cloud clusters.
Similar to Linux and Open Source in Math, Science and Engineering (20)
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking.
📕 Curious on our agenda? Wait no more!
10:00 Welcome note - UiPath Community in Dubai
Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank
10:20 A UiPath cross-region MEA overview
Ashraf El Zarka, VP and Managing Director MEA, UiPath
10:35: Customer Success Journey
Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank
11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more
Boris Krumrey, Global VP, Automation Innovation, UiPath
12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services.
Brendan Lingam, Director of Sales and Business Development, Marc Ellis
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
Linux and Open Source in Math, Science and Engineering
1. Linux and Open Source in Math,
Science and Engineering
Dr. Doug Davis (3D)
pde1d.blogspot.com
2. Agenda
A bit of history
Where we are
Visualization
Math
Engineering
Science
Watt's steam engine at the lobby of the Higher
Technical School of Industrial Engineering of Madrid
Wikimedia Commons
3. A bit of History
Computers originally for Computing
Unix developed for telephone system
Human computers
in the NACA High
Speed Flight
Station "Computer
Room", Dryden
Flight Research
Center Facilities
WikiMedia Commons
4. 1960s and 70s
Mainframe era
IBM, CDC, Cray, Burroughs Corporation, DEC,
NCR, General Electric, Honeywell, RCA, and
UNIVAC
Fortran Programs
Hollerith Cards
Beginnings of interactive programing
Thomas J. Watson, Sr. was fired several times by John
Henry Patterson the owner of NCR in Dayton, Ohio. So
he went to work for Computing-Tabulating-Recording
Company (CTR) which became IBM under his
leadership
5. 1980 and 90s
Early PC and MAC
2D AutoCAD
Graphic Engineering Workstations
3D graphics
High price
High performance
Unix variants
6. 2000s
This is a picture of the TinyHPC Beowulf cluster
Beowulf cluster
MPI – Message
Passing Interface
PVM – Parallel
Virtual Machine
COW – Cluster of
Workstations
LOBOS – Lots Of
Boxes On Shelves
The name Beowulf originally
referred to a specific computer
built in 1994 by Thomas
Sterling and Donald Becker at
NASA.
Photo by Mukarram Ahmad
http://en.wikipedia.org/wiki/File:Tinyhpc.jpg
Instructions for building a Low cost Beowulf
(cir. 2000) http://www.clustercompute.com/
7. Today
Top500.org
91% of 500
Fastest
Computers in
the World use
Linux
1.2% Use MS
Windows
The Network is
the Computer
Sun's John Gage – CNET June 9,
2008
10. One Killer Feature of Linux
for S&E in My Opinion Anyway
GPM - gpm provides mouse cut-and-paste
services
Highlight, then mouse middle click, to paste
selected text
Fast and simple way to transfer numbers
between programs
Command line file names,
ls → <command> → double click→ middle click
With OO Calc can transfer whole text tables
eMail and web addresses
Works with vi, kwrite, gedit, OO writer, etc.
11. Plotting
“The purpose of computing is insight, not numbers.”
R.W.Hamming, Numerical Methods for Scientiust and Engineers, Dover Publications
Gnuplot
Fit function
Plot data or functions
Fast script running
Paraview
3D
Filter Tree View
Many Commercial Options
Tecplot, Fieldview,
EnSight
12. gnuplot
plot and splot main commands
> help plot
> help set
set terminal and set output for plot file output
text data in columns
default white-space separators
> set datafile separator ','
quotes for file name in plot command
no titles in file
two blank lines separate index number
user functions
> sinc(x) = sin(pi*x)/(pi*x)
multiple error bars
16. Sage
SAGE Math
www.sagemath.org
Author Video on Why Open Source
"I am a
cool
multiedge
graph with
loops"
http://www.sagemath.org/tour-graphics.html
17. Sagemath.org
Python
>>> notebook()
@interact
environment
PARI/GP
Maxima
scipy, matplolib
R
singular
Everything Open
Source Math
Better than MATLAB – Arbitrary Precision, Faster, Object Oriented
Programming, Symbolic Algebra, OPEN SOURCE
18. Maxima
Symbolic Math
Maxima is a
descendent of
Macsyma
Mathmatica and
Maple are
cousins
Written in lisp
wxMaxima –
wxwidgets gui
version
19. Octave
GNU Octave is a high-level interpreted
language, primarily intended for numerical
computations.
Most compatible with Matlab
Can run many .m files
Octave Forge
Examples and tools
http://octave.sourceforge.net/
http://www.gnu.org/software/octave/index.html
20. Programing
A Much Bigger Learning Curve
May be needed for better understanding
Efficiency – Faster computations
Many Libraries
Fortran or C/C++
22. Math Libraries
Linear Algebra
Fast Fourier Transform
GSL – Gnu Scientific Library
C functions, with many wrappers
info gsl
Complex Numbers
Special Functions
Permutations
BLAS Support
Eigensystems
Quadrature
Quasi-Random Sequences
Discrete Wavelet Transforms
Simulated Annealing
Chebyshev Approximation
Discrete Hankel Transforms
Roots of Polynomials
Vectors and Matrices
Sorting
Linear Algebra
Fast Fourier Transforms
Random Numbers
Random Distributions
Histograms
Monte Carlo Integration
Differential Equations
Numerical Differentiation
Basis splines
IEEE Floating-Point
Physical Constants
Minimization
Least-Squares Fitting
Root-Finding
Interpolation
N-Tuples
Series Acceleration
Statistics
23. Other Math Libraries
AMD Core Math Library (ACML)
Fortran
Supports 6 compilers on Linux and 2 on
Windows
AMD Accelerated Parallel Processing Math
Libraries (APPML)
AMD
OpenCL GPUs
Intel® Math Kernel Library (Intel® MKL)
Fortran and C
24. High Performance Libraries
PETSC – PDE library
Dakota - Optimization
Trilinos - object-oriented software framework
Paraview – 3-D Visualization Tool
VTK – 3-D Visualization Library
HDF5 – Hierarchical Data Facility
NetCDF – Network Common Data Form
25. Other Languages
R
Statistics Language Package
This means data analysis
Good plotting facilities
Powerful Non-Linear Regression
Java
JScience – http://jscience.org/
Java Scientific Library
-http://www.ee.ucl.ac.uk/~mflanaga/java/
26. Engineering Tools
Linux weakness is lack of popular CAD
Most geometry engines work under Linux
ACIS, Granite, OpenCascade
Some Linux Cad Packages
QCAD – 2D – Free , GPL
VariCAD - ~$700 – free trial - varicad.com
BricsCAD - $$ - http://www.bricsys.com
Graphite One CAD - ~$2000 for 3-D
freeCAD – sourceforge.net/projects/free-cad/
Some Light 3D for Grid Generators
CUBIT, Salomè
27. CAELinux
http://www.caelinux.com
Linux Distribution Focused on Computer Aided Engineering
CAELinux is an installable LiveDVD Linux distribution based on Ubuntu 10.04 LTS 64 bit
Pre-post processors & CAD: Salome_Meca 2010, Salome 5.1.3, GMSH 2.4, Netgen ,
Tetgen, enGrid, Discretizer / Discretizer::Setup, Paraview, OpenDX, CGX, QCad,
FreeCAD, SagCAD, ElmerGUI / ElmeFront / ElmerGrid
Finite Element solvers: Salome_Meca 2010, Code_Aster STA10.1, Elmer v5.5 , Calculix
2.1, Impact 0.76, Dynela, Fenics
Computationnal Fluid Dynamics: OpenFOAM v1.7, Gerris flow solver and GFSView,
Code-Saturne 2.0 with CFD analysis wizard and tutorial , Elmer Solver, air foil and airplane
analysis tools (Xfoil, Javafoil, AVL, Datcom+)
Multibody dynamics : MBDyn with its Blender interface and EasyAnim
3D bio-medical image processing: ITK-Snap, Image J, Voxel_Mesher
Mathematics: GNU Octave +QtOctave, Scilab, wxMaxima, R & Rkward, Gnuplot, Latex,
Labplot, g3data, Scipy.
Is provided with interactive flash tutorials, videos as well as examples to start learning how
to use the included open-source softwares for realistic simulation
32. OpenFOAM
http://www.openfoam.com/
Arbitrary Polyhedral Cell Field Solver
Separate Solvers for Different Problems
Really a framework
OpenCFD Ltd. acquired by SGI
An extensive set of OpenFOAM solvers has evolved
(and is forever growing) that are available to users.
OpenFOAM is used mainly for CFD but has found use
in other areas such as stress analysis,
electromagnetics and finance because it is
fundamentally a tool for solving partial differential
equations rather than a CFD package in the traditional
sense.
33. Catalogs of Programs
SAL – Scientific Applications for Linux
http://www.sai.msu.su/sal/sal2.shtml
Search-able reference to many programs
Very Old – Last update May 2001
Netlib - Netlib is a collection of mathematical
software, papers, and databases
http://www.netlib.org/
160+ packages and catagories
34. Other
Latest Linux Kernel adding
more Real Time functions
Process Control
Requirement
Mathscript for Android
(SymPy)
TeX Live -
Kile, Texmaker and Lyx
KDE Cantor
GNU TexMacs
https://market.android.com/details?
id=com.funmath.mathscript