Docker allows building and running applications inside lightweight containers. Some key benefits of Docker include:
- Portability - Dockerized applications are completely portable and can run on any infrastructure from development machines to production servers.
- Consistency - Docker ensures that application dependencies and environments are always the same, regardless of where the application is run.
- Efficiency - Docker containers are lightweight since they don't need virtualization layers like VMs. This allows for higher density and more efficient use of resources.
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me to download the slides
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me to download the slides
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.
Swarm in a nutshell
• Exposes several Docker Engines as a single virtual Engine
• Serves the standard Docker API
• Extremely easy to get started
• Batteries included but swappable
Stephen Nguyen a Developer Evangelist for ClusterHQ reviews how volumes work and overviews the benefits of allowing Flocker to orchestrate your Volumes. (video coming soon)
SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud InfrastructureMatt Ray
Chef is an open source configuration management and automation framework used to configure, deploy and manage infrastructure of every type. Deploying to the cloud has made it easy to run large numbers of
servers and Chef makes it even easier to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. This talk will provide a quick introduction to Chef and is intended for sysadmins and developers familiar with the concepts behind managing applications and infrastructure in the cloud, without diving too deeply into technical specifics.
You have amazing content and you want to get it to your users as fast as possible. In today’s industry, milliseconds matter and slow websites will never keep up. You can use a CDN but they are expensive, make you dependent on a third party to deliver your content, and can be notoriously inflexible. Enter Varnish, a powerful, open-source caching reverse proxy that lives in your network and lets you take control of how your content is managed and delivered. We’ll discuss how to install and configure Varnish in front of a typical web application, how to handle sessions and security, and how you can customize Varnish to your unique needs. This session will teach you how Varnish can help you give your users a better experience while saving your company and clients money at the same time.
The age of orchestration: from Docker basics to cluster managementNicola Paolucci
The container abstraction hit the collective developer mind with great force and created a space of innovation for the distribution, configuration and deployment of cloud based applications. Now that this new model has established itself work is moving towards orchestration and coordination of loosely coupled network services. There is an explosion of tools in this arena at different degrees of stability but the momentum is huge.
On the above premise this session we'll delve into a selection of the following topics:
- Two minute Docker intro refresher
- Overview of the orchestration landscape (Kubernetes, Mesos, Helios and Docker tools)
- Introduction to Docker own ecosystem orchestration tools (machine, swarm and compose)
- Live demo of cluster management using a sample application.
A basic understanding of Docker is suggested to fully enjoy the talk.
Swarm in a nutshell
• Exposes several Docker Engines as a single virtual Engine
• Serves the standard Docker API
• Extremely easy to get started
• Batteries included but swappable
Stephen Nguyen a Developer Evangelist for ClusterHQ reviews how volumes work and overviews the benefits of allowing Flocker to orchestrate your Volumes. (video coming soon)
SCALE12X Build a Cloud Day: Chef: The Swiss Army Knife of Cloud InfrastructureMatt Ray
Chef is an open source configuration management and automation framework used to configure, deploy and manage infrastructure of every type. Deploying to the cloud has made it easy to run large numbers of
servers and Chef makes it even easier to deploy to nearly every public and private cloud platform as well as virtualized and physical servers. This talk will provide a quick introduction to Chef and is intended for sysadmins and developers familiar with the concepts behind managing applications and infrastructure in the cloud, without diving too deeply into technical specifics.
You have amazing content and you want to get it to your users as fast as possible. In today’s industry, milliseconds matter and slow websites will never keep up. You can use a CDN but they are expensive, make you dependent on a third party to deliver your content, and can be notoriously inflexible. Enter Varnish, a powerful, open-source caching reverse proxy that lives in your network and lets you take control of how your content is managed and delivered. We’ll discuss how to install and configure Varnish in front of a typical web application, how to handle sessions and security, and how you can customize Varnish to your unique needs. This session will teach you how Varnish can help you give your users a better experience while saving your company and clients money at the same time.
The age of orchestration: from Docker basics to cluster managementNicola Paolucci
The container abstraction hit the collective developer mind with great force and created a space of innovation for the distribution, configuration and deployment of cloud based applications. Now that this new model has established itself work is moving towards orchestration and coordination of loosely coupled network services. There is an explosion of tools in this arena at different degrees of stability but the momentum is huge.
On the above premise this session we'll delve into a selection of the following topics:
- Two minute Docker intro refresher
- Overview of the orchestration landscape (Kubernetes, Mesos, Helios and Docker tools)
- Introduction to Docker own ecosystem orchestration tools (machine, swarm and compose)
- Live demo of cluster management using a sample application.
A basic understanding of Docker is suggested to fully enjoy the talk.
Docker - Demo on PHP Application deployment Arun prasath
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
In this demo, I will show how to build a Apache image from a Dockerfile and deploy a PHP application which is present in an external folder using custom configuration files.
Docker is the world's leading software containerization platform.
This is a comprehensive introduction to Docker, suitable for delivering in introductory meetups to an audience who does not know about docker.
In case you want to deliver this presentation somewhere, kindly drop me a mail at aditya.konarde@gmail.com
You can contact me at:
Connect with me onLinkedIN: https://www.linkedin.com/in/adityakonarde
Add me on Facebook: https://www.facebook.com/Aditya.Konarde
Tweet to me @aditya_konarde
Docker is a tool designed to make it easier to create, deploy, and run applications
by using containers. Containers allow a developer to package up
an application with all of the parts it needs, such as libraries and other dependencies,
and ship it all out as one package. By doing so, thanks to the
container, the developer can rest assured that the application will run on
any other Linux machine regardless of any customized settings that machine
might have that could differ from the machine used for writing and testing
the code.
In a way, Docker is a bit like a virtual machine. But unlike a virtual
machine, rather than creating a whole virtual operating system, Docker allows
applications to use the same Linux kernel as the system that they’re
running on and only requires applications be shipped with things not already
running on the host computer. This gives a significant performance boost
and reduces the size of the application.
In these slides is given an overview of the different parts of Apache Spark.
We analyze spark shell both in scala and python. Then we consider Spark SQL with an introduction to Data Frame API. Finally we describe Spark Streaming and we make some code examples.
Topics:spark-shell, pyspark, HDFS, how to copy file to HDFS, spark transformations, spark actions, Spark SQL (Shark),
spark streaming, streaming transformation stateless vs stateful, sliding windows, examples
In these slides we analyze why the aggregate data models change the way data is stored and manipulated. We introduce MapReduce and its open source implementation Hadoop. We consider how MapReduce jobs are written and executed by Hadoop.
Finally we introduce spark using a docker image and we show how to use anonymous function in spark.
The topics of the next slides will be
- Spark Shell (Scala, Python)
- Shark Shell
- Data Frames
- Spark Streaming
- Code Examples: Data Processing and Machine Learning
In this lecture we analyze graph oriented databases. In particular, we consider TtitanDB as graph database. We analyze how to query using gremlin and how to create edges and vertex.
Finally, we presents how to use rexster to visualize the storeg graph/
In this lecture we analyze document oriented databases. In particular we consider why there are the first approach to nosql and what are the main features. Then, we analyze as example MongoDB. We consider the data model, CRUD operations, write concerns, scaling (replication and sharding).
Finally we presents other document oriented database and when to use or not document oriented databases.
In this lecture we analyze document oriented databases. In particular we consider why there are the first approach to nosql and what are the main features. Then, we analyze as example MongoDB. We consider the data model, CRUD operations, write concerns, scaling (replication and sharding).
Finally we presents other document oriented database and when to use or not document oriented databases.
In these slides we introduce Column-Oriented Stores. We deeply analyze Google BigTable. We discuss about features, data model, architecture, components and its implementation. In the second part we discuss all the major open source implementation for column-oriented databases.
In this lecture we analyze key-values databases. At first we introduce key-value characteristics, advantages and disadvantages.
Then we analyze the major Key-Value data stores and finally we discuss about Dynamo DB.
In particular we consider how Dynamo DB: How is implemented
1. Motivation Background
2. Partitioning: Consistent Hashing
3. High Availability for writes: Vector Clocks
4. Handling temporary failures: Sloppy Quorum
5. Recovering from failures: Merkle Trees
6. Membership and failure detection: Gossip Protocol
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me for other informations and to download
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me for other informations and to download the slides
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
Contact me to download the slides
1. Introduction to the Course "Designing Data Bases with Advanced Data Models...Fabio Fumarola
The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data.
Course Website http://pbdmng.datatoknowledge.it/
A Parallel Algorithm for Approximate Frequent Itemset Mining using MapReduce Fabio Fumarola
Recently, several algorithms based on the MapRe- duce framework have been proposed for frequent pattern mining in Big Data. However, the proposed solutions come with their own technical challenges, such as inter-communication costs, in- process synchronizations, balanced data distribution and input parameters tuning, which negatively affect the computation time. In this paper we present MrAdam, a novel parallel, distributed algorithm which addresses these problems. The key principle underlying the design of MrAdam is that one can make reasonable decisions in the absence of perfect answers. Indeed, given the classical threshold for minimum support and a user- specified error bound, MrAdam exploits the Chernoff bound to mine “approximate” frequent itemsets with statistical error guarantees on their actual supports. These itemsets are generated in parallel and independently from subsets of the input dataset, by exploiting the MapReduce parallel computation framework. The result collections of frequent itemsets from each subset are aggregated and filtered by using a novel technique to provide a single collection in output. MrAdam can scale well on gigabytes of data and tens of machines, as experimentally proven on real datasets. In the experiments we also show that the proposed algorithm returns a good statistically bounded approximation of the exact results.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Hierarchical Digital Twin of a Naval Power SystemKerry Sado
A hierarchical digital twin of a Naval DC power system has been developed and experimentally verified. Similar to other state-of-the-art digital twins, this technology creates a digital replica of the physical system executed in real-time or faster, which can modify hardware controls. However, its advantage stems from distributing computational efforts by utilizing a hierarchical structure composed of lower-level digital twin blocks and a higher-level system digital twin. Each digital twin block is associated with a physical subsystem of the hardware and communicates with a singular system digital twin, which creates a system-level response. By extracting information from each level of the hierarchy, power system controls of the hardware were reconfigured autonomously. This hierarchical digital twin development offers several advantages over other digital twins, particularly in the field of naval power systems. The hierarchical structure allows for greater computational efficiency and scalability while the ability to autonomously reconfigure hardware controls offers increased flexibility and responsiveness. The hierarchical decomposition and models utilized were well aligned with the physical twin, as indicated by the maximum deviations between the developed digital twin hierarchy and the hardware.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxR&R Consult
CFD analysis is incredibly effective at solving mysteries and improving the performance of complex systems!
Here's a great example: At a large natural gas-fired power plant, where they use waste heat to generate steam and energy, they were puzzled that their boiler wasn't producing as much steam as expected.
R&R and Tetra Engineering Group Inc. were asked to solve the issue with reduced steam production.
An inspection had shown that a significant amount of hot flue gas was bypassing the boiler tubes, where the heat was supposed to be transferred.
R&R Consult conducted a CFD analysis, which revealed that 6.3% of the flue gas was bypassing the boiler tubes without transferring heat. The analysis also showed that the flue gas was instead being directed along the sides of the boiler and between the modules that were supposed to capture the heat. This was the cause of the reduced performance.
Based on our results, Tetra Engineering installed covering plates to reduce the bypass flow. This improved the boiler's performance and increased electricity production.
It is always satisfying when we can help solve complex challenges like this. Do your systems also need a check-up or optimization? Give us a call!
Work done in cooperation with James Malloy and David Moelling from Tetra Engineering.
More examples of our work https://www.r-r-consult.dk/en/cases-en/
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
2. Contents
• The Evolution of IT
• The Solutions: Virtual Machines vs Vagrant vs Docker
• Differences
• Examples
– Vagrant
– Boot2Docker
– Docker
– Docker Hub
• CoreOS
2
3. From 1995 to 2015
3
Client-Server
App
Well-defined stack:
- O/S
- Runtime
- Middleware
Monolithic
Physical
Infrastructure
Thin app on mobile,
tablet
Assembled by
developers using
best available
services
Running on any
available set of
physical resources
(public/private/
virtualized)
4. Static website
Web frontend
User DB
Queue Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap
2
postgresql + pgv8 + v8
hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv
+ nodejs + phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-
client
Development VM
QA server
Public Cloud
Disaster recovery
Contributor’s laptop
Production Servers
2015 in Detail
Production Cluster
Customer Data Center
4
5. Challenges
• How to ensure that services interact consistently?
• How to avoid to setup N different configurations and
dependencies for each service?
• How to migrate and scale quickly ensuring
compatibility?
• How to replicate my VM and services quickly?
5
6. How to deal with different confs?
6
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development
VM
QA Server
Single Prod
Server
Onsite Cluster Public Cloud
Contributor’s
laptop
Customer
Servers
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
8. Virtual Machines
• Run on top of an Hypervisor
Pros
– fully virtualized OS
– Totally isolated
Cons
– Needs to take a snapshot of
the entire VM to replicate
– Uses a lot of space
– Slow to move around
8
App
A
Hypervisor
Host OS
Server
Guest
OS
Bins/
Libs
App
A’
Guest
OS
Bins/
Libs
App
B
Guest
OS
Bins/
Libs
Guest
OS
Guest
OS
VM
9. Hypervisors Trend
2011
– XEN: Default choice given Rackspace and Amazon use
– KVM: Bleeding edge users
2012
– KVM: Emerges as the lead
– XEN: Loses momentum
9
10. Hipervisors Trend
2013
– KVM: Maintains lead (around 90%+ for Mirantis)
– Vmware: Emerges as a surprising second choice
– Containers (LXC, Parallels, Docker): Web Hosting and SAS
focused
– Xen and HyperV: Infrequent requests (XenServer.org)
2014 – 2015
– ???
10
12. Vagrant
• Open source VM manager released in 2010
• It allows you to script and package VMs config and
the provisioning setup via a VagrantFile
• It is designed to run on top of almost any VM tool:
VirtualBox, VMVare, AWS, OpenStack1
• It can be used together with provisioning tools such
as shell scripts, Chef and Puppet.
12
1. https://github.com/cloudbau/vagrant-openstack-plugin
13. Vagrant: idea
Use a VagrantFile to install
1.an operating system
2.Required libraries and
software
and finally run programs and
processes of your final
application
13
15. Vagrant: Demo
• It allows us to interact with Vagrant
• It offers the following commands: box, connect,
destroy, halt, init, login, package a vm, rdp, …
https://docs.vagrantup.com/v2/cli/index.html
15
16. Vagrant Example
1. Download and install VirtualBox and Vagrant
1. This will place a VagrantFile in the directory
2. Install a Box
3. Using a Box -> https://vagrantcloud.com/
16
$ mkdir vagrant_first_vm && cd vagrant_first_vm
$ vagrant init
$ vagrant box add ubuntu/trusty64
Vagrant.configure("2") do |config|
config.vm.box = "ubuntu/trusty64"
end
17. Vagran: Start
1. Start the box
2. Login into the vm
3. You can destroy the vm by
17
$ vagrant up
$ vagrant ssh
$ vagrant destroy
18. Vagrant: Synced Folders
• By default, it shares your project directory to the /vagrant
directory on the guest machine.
• If you create a file on your gues os the file will be on the
vagrant vm.
18
$ vagrant up
$ vagrant ssh
$ ls /vagrant
--Vagrantfile
$ touch pippo.txt
$vagrant ssh
$ls /vagrant/
19. Vagrant: Provisioning
• Let’s install Apache via a boostrap.sh file
• If you create a file on your gues os the file will be on the
vagrant vm. (vagrant reload --provision)
19
#!/usr/bin/env bash
apt-get update
apt-get install -y apache2
rm -rf /var/www
ln -fs /vagrant /var/www
Vagrant.configure("2") do |config|
config.vm.box = "hashicorp/precise32"
config.vm.provision :shell, path: "bootstrap.sh"
end
20. Vagrant: Networking
• Port Forwarding: llows you to specify ports on the guest
machine to share via a port on the host machine
• By running vagrant reload or vagrant up we can see on
http://127.0.0.1:4567 our apache
• It supports also bridge configurations and other
configurations (https://docs.vagrantup.com/v2/networking/)
20
Vagrant.configure("2") do |config|
config.vm.box = "hashicorp/precise32"
config.vm.provision :shell, path: "bootstrap.sh"
config.vm.network :forwarded_port, host: 4567, guest: 80
end
21. Vagrant: Share and Provider
• It is possible to share Vagrant box via vagrant cloud (but?)
Providers
• By default Vagrant is configured with VirtualBox but you can
change the provider
• How?
21
$ vagrant up --provider=vmware_fusion
$ vagrant up --provider=aws
$ vagrant plugin install vagrant-aws
22. Vagrant: AWS Vagrantfile
22
Vagrant.configure("2") do |config|
# config.vm.box = "sean"
config.vm.provider :aws do |aws, override|
aws.access_key_id = "AAAAIIIIYYYY4444AAAA”
aws.secret_access_key =
"c344441LooLLU322223526IabcdeQL12E34At3mm”
aws.keypair_name = "iheavy"
aws.ami = "ami-7747d01e"
override.ssh.username = "ubuntu"
override.ssh.private_key_path = "/var/root/iheavy_aws/pk-
XHHHHHMMMAABPEDEFGHOAOJH1QBH5324.pem"
end
end
24. Quick Survey
• How many people have heard of Docker before this
Seminar?
• How many people have tried Docker ?
• How many people are using Docker in production ?
24
25. What is Docker?
"With Docker, developers can build any app in any
language using any toolchain. “Dockerized” apps are
completely portable and can run anywhere -
colleagues’ OS X and Windows laptops, QA servers
running Ubuntu in the cloud, and production data
center VMs running Red Hat.”
Docker.io
25
26. Docker in simple words
• It is a technology that allow you running applications
inside containers (not VM)
• This assures that libraries and package needed by the
application you run are always the same.
• This means you can make a container for Memcache
and another for Redis and they will work the same in
any OS (also in Vagrant).
26
27. Why Docker?
• Fast delivery of your applications
• Deploy and scale more easily
• Get higher density and run more workload
• Faster deployment makes for easier management
27
28. How does docker work?
• LinuX Containers (LXC)
• Control Groups & Namespaces (CGroups)
• AUFS
• Client – Server with an HTTP API
28
29. LXC- Linux Containers
• It is a user-space interface for the Linux kernel containment
features
• Through a powerful API and simple tools, it lets Linux users easily
create and manage system or application containers.
• Currently LXC can apply the following kernel features to contain
processes:
– Kernel namespaces (ipc, uts, mount, pid, network and user)
– Apparmor and SELinux profiles
– Seccomp policies
– Chroots (using pivot_root)
– Kernel capabilities & Control groups (cgroups)
29
30. Cgroups
• Control groups is a Linux kernel feature to limit, account and
isolate resource usage (CPU, memory, disk I/O, etc) of process
groups.
• Features:
– Resource limitation: limit CPU, memory…
– Prioritization: assign more CPU etc to some groups.
– Accounting: to measure the resource usage.
– Control: freezing groups or check-pointing and restarting.
30
31. LCX based Containers
• It allows us to run a Linux system within another Linux system.
• A container is a group of processes on a Linux box, put together
is an isolated environment.
31
AppA’
Docker Engine
Host OS
Server
Bins/Libs
AppA
Bins/Libs
AppB
AppB’
AppB’
AppB’
AppB’
Container
• From the inside it looks like a VM
• From the outside, it looks like normal
processes
32. Docker Features
• VE (Virtual Environments) based on LXC
• Portable deployment across machines
• Versioning: docker include git-like capabilities for tracking
versions of a container
• Component reuse: it allows building or stacking already
created packages. You can create ‘base images’ and then
running more machine based on the image.
• Shared libraries: there is a public repository with several
images (https://registry.hub.docker.com/)
32
33. Why are Docker Containers lightweight?
33
Bins
/
Libs
App
A
Original App
(No OS to take
up space, resources,
or require restart)
AppΔ
Bins/
App
A
Bins/
Libs
App
A’
Gues
t
OS
Bins/
Libs
Modified App
Union file system allows
us to only save the diffs
Between container A
and container A’
VMs
App
A
Gues
t
OS
Bins/
Libs
Copy of
App
No OS. Can
Share bins/libs
App
A
Gues
t
OS
Gues
t
OS
Containers
34. Prerequisites
• I use Oh My Zsh1
with the Docker plugin2
for autocompletion
of docker commands
• Linux at least with kernel 3.8 but 3.10.x is recommended
– $ uname –r
• MacOS or Windows via Boot2Docker3
or via Vagrant
34
1. https://github.com/robbyrussell/oh-my-zsh
2. https://github.com/robbyrussell/oh-my-zsh/wiki/Plugins#docker
3. http://boot2docker.io/
36. Docker install Vagrant
• Create the folders
$ mkdir ~/boot2docker
$ cd ~/boot2docker
• Init the vagrant box
$ vagrant init yungsang/boot2docker
$ vagrant up; export DOCKER_HOST=tcp://localhost:2375
• Check docker
$ docker version
* NOTE: the YungSang boot2docker opens up port forwarding
to the network, so is not safe on public wifi.
36
37. Docker Installation Vagrant
• Clone the docker repository
$ git clone https://github.com/dotcloud/docker.git
• Startup the vagrant image
$ vagrant up
• SSH into the image
$ vagrant ssh
• Docker client works normally
37
40. Docker: hello world
• Get one base image from https://registry.hub.docker.com
$ sudo docker pull centos
• List images on your system
$ sudo docker images
• Check the images
–$ sudo docker images
• Run your first container
–$ sudo docker run centos:latest echo “hello world”
40
41. An Interactive Container
• Run bash in your container
– $ sudo docker run -t -i centos /bin/bash
• The -t flag assigns a pseudo-tty or terminal inside our new
container
• The -i flag allows us to make an interactive connection by
grabbing the standard in (STDIN) of the container
• We also specified a command for the container
41
42. A Daemonized Hello world
• Run a sh script
– sudo docker run -d centos:6 /bin/sh –c ‘while true; do echo hello
world; sleep 1; done’
• The -d flag tells Docker to run the container and put it in the
background, to daemonize it.
• To list the docker containers running
– $ docker ps
• To get the logs of the container
– $ sudo docker logs container_id
• To stop the container:
– $ sudo docker stop container_id
42
43. A web container with docker
• To run a Python Flask application
– $ sudo docker run -d -P training/webapp python app.py
• The -P flag is new and tells Docker to map any required
network ports inside our container to our host.
• To view our application with the port mapping
– $ sudo docker ps –l
• We can see that the default flask port 5000 is exposed to
49155
– $ sudo docker run -d -p 5000:5000 training/webapp python app.py
• Check the url to continue the guide
– https://docs.docker.com/userguide/usingdocker/
43
44. Working with docker images
• To find images go to
– https://hub.docker.com/
• To pull an image
– $ sudo docker pull training/sinatra
• Updating and committing an image
– $ sudo docker run -t -i training/sinatra /bin/bash
– # gem install json
– $ sudo docker commit -m="Added json gem" -a="Kate Smith"
0b2616b0e5a8 ouruser/sinatra:v2
- $ sudo docker images
44
45. Create an image from a Dockerfile
FROM library/centos:centos6
MAINTAINER fabio fumarola fabiofumarola@gmail.com
RUN yum install -y curl which tar sudo openssh-server openssh-clients rsync
# passwordless ssh
RUN ssh-keygen -q -N "" -t dsa -f /etc/ssh/ssh_host_dsa_key
RUN ssh-keygen -q -N "" -t rsa -f /etc/ssh/ssh_host_rsa_key
RUN ssh-keygen -q -N "" -t rsa -f /root/.ssh/id_rsa
RUN cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
45
46. Build and run an image
• $docker build –t fabio/centos:ssh .
• $docker run –i –t fabio/centos:ssh /bin/bash
• Or
• $docker run –d fabio/centos:ssh /bin/bash
• Check the following commands:
– $ docker top
– $ docker logs
– $ docker inspect
46
47. Other Commands
• Docker cp: copy a file from container to host
• Docker diff: print container changes
• Docker top: display running processes in a container
• Docker rm /rmi: delete container/image
• Docker wait: wait until container stop and print exit code
More on: http://docs.docker.io/en/latest/commandline/cli
47
49. Steps
1. Build the docker image via the docker file
– $ docker build –t ouruser/biginsight:v ./
1. Run the container
– docker run -d –p --name biginsight –v
/abs/install/path:/opt/ibm ouruser/biginsight:v /bin/sh
path/to/install.sh
This will mount the host directory, /abs/install/path,
into the container at /opt/ibm
49
51. Steps
• There are several dockerfile that we can use
– http://blog.sequenceiq.com/blog/2014/12/02/hadoop-2-
6-0-docker/
– https://github.com/sequenceiq?page=2&query=docker
– https://registry.hub.docker.com/u/oddpoet/hbase-cdh5/
51
52. Docker vs Vagrant?
• Less memory for Dockers w.r.t VMs
• With a VM you get more isolation, but is much heavier.
Indeed you can run 1000 of Dockers in a machine but not
thousand of VMs with Xen.
• A VM requires minutes to start a Docker seconds
There are pros and cons for each type.
• If you want full isolation with guaranteed resources a full VM
is the way to go.
• If you want hundred of isolate processes into a reasonably
sized host then Docker might be the best solution
52
54. CoreOS
• A minimal operating system
• Painless updating: utilizes active/passive scheme to update
the OS as single unit instead of package by package.
• Docker container
• Clustered by default
• Distributed System tools: etcd key-value store
• Service discovery: easily locate where service are running in
the cluster
• High availability and automatic fail-over
54
56. Docker with CoreOS
Features
•Automatically runs on each CoreOS
machine
•Updated with regular automatic OS
updates
•Integrates with etcd
•Networking automatically configured
Example Akka cluster + Docker + CoreOS
https://github.com/dennybritz/akka-
cluster-deploy
56