Very few trends in IT have generated as much buzz as cloud computing. This session will cut through the hype and quickly clarify the ontology for cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complimentary open source management tools that can be combined to automate the management of cloud computing environments.
The session will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software.
OSCON 2014 - Crash Course in Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
Presentation on the current state of cloud computing and the role that open source, containers and microservices are playing in the cloud.
Presented to Florida Linux Users Exchange on April 9th, 2015
CloudOpen 2014 - Mixing Your Open Source Cloud CocktailMark Hinkle
Add two parts virtualization, one part orchestration add a little networking shake and pour. Unfortunately cloud computing isn’t that easy but then again not all clouds are the same and tastes may vary. This talk will discuss how the varying open source technologies like OpenStack, Docker, LXC and others can be mixed together to make something that appeals to the needs of a wide variety of users. There’s also no problem in abstaining from building your own cloud but still benefiting from the open source tooling to maximize the benefits of the public cloud.
Crash Course in Open Source Cloud Computing Mark Hinkle
Introduction on open source technologies that can be used to deploy and manage cloud computing environments. Especially geared toward Infrastructure-as-a-service environments. Updated for presentation at Indiana Linuxfest (3/26/2011).
Updates:
- Open source cloud storage (CEPH, Swift, Gluster)
- Orchestration - MCollective
- Cloud Infrastructure Diagrams
Cloud 2.0 - How Containers, Microservices and Open Source Software are Redefi...Mark Hinkle
Led by the rocket like success of Amazon Web Services cloud computing is a paradigm shift in the way we host and deploy infrastructure. Organizations are consuming cloud infrastructure across multiple cloud providers both inside their data center and the data centers of others. The advent of highly portable workloads via containers (e.g. Docker) and discrete units of computing delivered by microservices are enabling organizations (like Netflix) to deploy complex multi-layered products and services at breakneck speeds.
This talk will give an overview of the major cloud services and the open source software (e.g. OpenStack, Apache CloudStack) that can be used to deliver and manage cloud computing infrastructure(e.g. Puppet, Chef, Ansible). The discussion will cover the evolution of cloud computing and how that sets the stage for realizing the agility, flexibility and power of cloud computing.
Attendees should expect to learn about the leading technologies in cloud computing, strategies for using open source software to create/manage cloud computing services and to gain an understanding how current developments are providing a way to create a single cloud fabric that best serves their individual needs.
RICON 2014 - Build a Cloud Day - Crash Course Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
Cloud 2.0: Containers, Microservices and Cloud HybridizationMark Hinkle
In a very short time cloud computing has become a major factor in the way we deliver infrastructure and services. Though we’ve quickly breezed through the ideas of hosted cloud and orchestration. This talk will focus on the next evolution of cloud and how the evolution of technologies like container (like Docker), microservices the way Netflix runs their cloud) and how hybridization (applications running on Mesos across Kubernetes clusters in both private and public clouds).
Interop - Crash Course In Open Source Cloud ComputingMark Hinkle
This will be an overview of the open source software that can be used to deploy and manage a cloud computing environment. The session will include information on storage, networking(e.g. OpenDaylight) and compute virtualization (Xen, KVM, LXC) and the orchestration(Apache CloudStack, OpenStack) of the three to build their own cloud services.
OSCON 2014 - Crash Course in Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
Presentation on the current state of cloud computing and the role that open source, containers and microservices are playing in the cloud.
Presented to Florida Linux Users Exchange on April 9th, 2015
CloudOpen 2014 - Mixing Your Open Source Cloud CocktailMark Hinkle
Add two parts virtualization, one part orchestration add a little networking shake and pour. Unfortunately cloud computing isn’t that easy but then again not all clouds are the same and tastes may vary. This talk will discuss how the varying open source technologies like OpenStack, Docker, LXC and others can be mixed together to make something that appeals to the needs of a wide variety of users. There’s also no problem in abstaining from building your own cloud but still benefiting from the open source tooling to maximize the benefits of the public cloud.
Crash Course in Open Source Cloud Computing Mark Hinkle
Introduction on open source technologies that can be used to deploy and manage cloud computing environments. Especially geared toward Infrastructure-as-a-service environments. Updated for presentation at Indiana Linuxfest (3/26/2011).
Updates:
- Open source cloud storage (CEPH, Swift, Gluster)
- Orchestration - MCollective
- Cloud Infrastructure Diagrams
Cloud 2.0 - How Containers, Microservices and Open Source Software are Redefi...Mark Hinkle
Led by the rocket like success of Amazon Web Services cloud computing is a paradigm shift in the way we host and deploy infrastructure. Organizations are consuming cloud infrastructure across multiple cloud providers both inside their data center and the data centers of others. The advent of highly portable workloads via containers (e.g. Docker) and discrete units of computing delivered by microservices are enabling organizations (like Netflix) to deploy complex multi-layered products and services at breakneck speeds.
This talk will give an overview of the major cloud services and the open source software (e.g. OpenStack, Apache CloudStack) that can be used to deliver and manage cloud computing infrastructure(e.g. Puppet, Chef, Ansible). The discussion will cover the evolution of cloud computing and how that sets the stage for realizing the agility, flexibility and power of cloud computing.
Attendees should expect to learn about the leading technologies in cloud computing, strategies for using open source software to create/manage cloud computing services and to gain an understanding how current developments are providing a way to create a single cloud fabric that best serves their individual needs.
RICON 2014 - Build a Cloud Day - Crash Course Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
Cloud 2.0: Containers, Microservices and Cloud HybridizationMark Hinkle
In a very short time cloud computing has become a major factor in the way we deliver infrastructure and services. Though we’ve quickly breezed through the ideas of hosted cloud and orchestration. This talk will focus on the next evolution of cloud and how the evolution of technologies like container (like Docker), microservices the way Netflix runs their cloud) and how hybridization (applications running on Mesos across Kubernetes clusters in both private and public clouds).
Interop - Crash Course In Open Source Cloud ComputingMark Hinkle
This will be an overview of the open source software that can be used to deploy and manage a cloud computing environment. The session will include information on storage, networking(e.g. OpenDaylight) and compute virtualization (Xen, KVM, LXC) and the orchestration(Apache CloudStack, OpenStack) of the three to build their own cloud services.
Great Wide Open: Crash Course Open Source Cloud Computing - 2014Mark Hinkle
Very few trends in IT have generated as much buzz as cloud computing. This session will cut through the hype and quickly clarify the ontology for cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complimentary open source management tools that can be combined to automate the management of cloud computing environments.
The session will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software.
InteropNY/CloudConnect 2014 - Quick Crash Course in Open Source Cloud ComputingMark Hinkle
This will be an overview of the open source software that can be used to deploy and manage a cloud computing environment. The session will include information on storage, networking(e.g. OpenDaylight) and compute virtualization (Xen, KVM, LXC) and the orchestration(Apache CloudStack, OpenStack) of the three to build their own cloud services.
CNCF general introduction to beginners at openstack meetup Pune & Bangalore February 2018. Covers broadly the activities and structure of the Cloud Native Computing Foundation.
Open source and cloud computing are two terms that everyone seems to be talking about. Powerhouses on their own, when paired together open source and cloud computing can create a developer’s dream scenario.
In this session, Bret Piatt, technical alliances at Rackspace Hosting will discuss the history of open source software development and the spread of open source across the internet. Cloud computing providers are now incorporating open source into their business models through open APIs and contributions to various open source projects such as Cassandra and Drizzle, and Bret will discuss these developments while taking a close look at the intersection of cloud computing and open source to cover:
How cloud computing is changing open source
How cloud computing can benefit from open source
How open source will lead the interoperability push
How the success of cloud is tied to mass adoption that requires interoperability
Open Source Tool Chains for Cloud ComputingMark Hinkle
This presentation was given at LinuxCon 2010.
The proliferation of cloud computing is inevitable, hosted apps, software-as-as-service and now dynamic on-demand utility computing is becoming the norm. The session will be a “fire-side” chat style discussion of the types of challenges presented by IT management operations personnel and how they can manage cloud infrastructure using open source tools. The talk will discuss options for deploying and integrating tools that provision, configure, orchestrate and monitor cloud (and physical)infrastructure. The session will appeal to those IT professionals (syadmins, net-ops, developers) who develop and manage infrastructure that resides in hosted environments like Amazon EC2 without disregarding traditionally hosted internal infrastructure.
Fossetcon: Crash Course on Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
[Updated with new Docker projects]
Taking the Next Hot Mobile Game Live with Docker and IBM SoftLayerDaniel Krook
Presentation at the IBM InterConnect Conference in Las Vegas, Nevada on February 24, 2016.
Mobile games are the fastest-growing sector of the $70 billion video game industry, far outpacing traditional consoles. But companies that aspire to create the next hot title have to account for more than just the app downloaded to a user device. They must prepare for huge spikes in game play with scalable backends to handle massive data and transactions behind socially linked user profiles and global leaderboards. This talk looks at how IBM successfully partnered with Firemonkeys, a major studio that had hit their vertical scaling limit, to design and deploy a new Docker-based architecture on SoftLayer. This scale-out architecture is able to handle an order of magnitude more customers for their next major release.
Linuxcon Europe 2011: Overview - Building Cloud Computing EnvironmentsMark Hinkle
Cloud Computing has been touted as an almost magical solution for changing the way enterprise IT infrastructure is deployed. Despite all the “cloudwashing” there is no magic, cloud computing still requires the same rigor in planning and design as in legacy IT architecture. The difference is that thanks to inexpensive hardware and exceptional free and open source software state-of-the-art technology is now this evolution of technology is accessible to any organization. This levels the IT playing field allowing users them to be competitive by deploying systems that are agile, scalable and adaptable to their needs. This presentation will cover the open source software that can be combined to build cloud computing environments for a variety of different uses as well as informing potential cloud users on how to choose technologies to best address the computing needs of their organization.
https://events.linuxfoundation.org/events/linuxcon-europe/hinkle
State of the Stack v4 - OpenStack in All It's GloryRandy Bias
The almost annual State of the Stack, version 4, an end-to-end view of OpenStack. This edition focuses on what the challenges are within the community and how they can be addressed.
v1 of SOTS has over 90,000 views and is one of the highest viewed OpenStack presentations ever.
DevNexus 2015
Docker: containerizing a monolithic app into a microservice-based PaaS
Convert a monolithic application into a microservice-based PaaS using Docker and related, containerization technologies. This will be the third presentation of a series of presentations that began greater than one year ago to evangelize the benefits of Docker. The scope of content spans from a development environment to a hybrid PaaS, and how Containerization is an enabler of architectural choice, innovation, scalability, and polyglot solutions.
The basics of Docker will be examined including repositories, brief discussion about managing and monitoring Docker containers, service discovery, and security. New and emerging technologies will be a constant theme, particularly about microservices, in addition to the ongoing evolution of the market and what the future may bring. Common organizational issues (and tactical solutions) that may impede successful decomposition and migration of legacy monoliths will be discussed, including security, DevOps and refactoring.
Hypothetical architectures will be described for building progressively more robust and complex applications and deployment models. The goal is to highlight the power, flexibility and scalability that containers enable.
Examples will start simple, from a local development environment, that is a simple two container setup that encapsulate a database and application tier. Subsequent discussion will involve progressively more complex and robust deployments that include features such as service discovery, automatic load balancing, and abstractions to simplify linking of containers including service gateways. With the stopping point of a hybrid PaaS.
Serverless architectures are one of the hottest trends in cloud computing this year, and for good reason. There are several technical capabilities and business factors coming together to make this approach compelling from both an application development and deployment cost perspective. The new OpenWhisk project provides an open source platform to enable these cloud-native, event-driven applications.
This talk will lay out the technical and business drivers behind the rise of serverless architectures, provide an introduction to the OpenWhisk open source project (and describe how it differs from other services like AWS Lambda), and give a demonstration showing how to start developing with this new cloud computing model using the OpenWhisk implementation available on IBM Bluemix.
Presented on October 12, 2016 at the NYC Bluemix meetup
LinuxFest Northwest: Crash Course in Open Source Cloud Computing Mark Hinkle
Few IT trends have generated as much buzz as cloud computing. This talk will cut through the hype and clarify cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complementary open source management tools that can be combined to automate the management of cloud computing environments. The discussion will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options for building and managing their own cloud computing environments using free and open source software.
IBM Bluemix OpenWhisk: Interconnect 2016, Las Vegas: CCD-1088: The Future of ...OpenWhisk
Learn more about the IBM Bluemix OpenWhisk, a serverless event-driven compute platform, which quickly executes application logic in response to events or direct invocations from web/mobile apps or other endpoints.
Great Wide Open: Crash Course Open Source Cloud Computing - 2014Mark Hinkle
Very few trends in IT have generated as much buzz as cloud computing. This session will cut through the hype and quickly clarify the ontology for cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complimentary open source management tools that can be combined to automate the management of cloud computing environments.
The session will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software.
InteropNY/CloudConnect 2014 - Quick Crash Course in Open Source Cloud ComputingMark Hinkle
This will be an overview of the open source software that can be used to deploy and manage a cloud computing environment. The session will include information on storage, networking(e.g. OpenDaylight) and compute virtualization (Xen, KVM, LXC) and the orchestration(Apache CloudStack, OpenStack) of the three to build their own cloud services.
CNCF general introduction to beginners at openstack meetup Pune & Bangalore February 2018. Covers broadly the activities and structure of the Cloud Native Computing Foundation.
Open source and cloud computing are two terms that everyone seems to be talking about. Powerhouses on their own, when paired together open source and cloud computing can create a developer’s dream scenario.
In this session, Bret Piatt, technical alliances at Rackspace Hosting will discuss the history of open source software development and the spread of open source across the internet. Cloud computing providers are now incorporating open source into their business models through open APIs and contributions to various open source projects such as Cassandra and Drizzle, and Bret will discuss these developments while taking a close look at the intersection of cloud computing and open source to cover:
How cloud computing is changing open source
How cloud computing can benefit from open source
How open source will lead the interoperability push
How the success of cloud is tied to mass adoption that requires interoperability
Open Source Tool Chains for Cloud ComputingMark Hinkle
This presentation was given at LinuxCon 2010.
The proliferation of cloud computing is inevitable, hosted apps, software-as-as-service and now dynamic on-demand utility computing is becoming the norm. The session will be a “fire-side” chat style discussion of the types of challenges presented by IT management operations personnel and how they can manage cloud infrastructure using open source tools. The talk will discuss options for deploying and integrating tools that provision, configure, orchestrate and monitor cloud (and physical)infrastructure. The session will appeal to those IT professionals (syadmins, net-ops, developers) who develop and manage infrastructure that resides in hosted environments like Amazon EC2 without disregarding traditionally hosted internal infrastructure.
Fossetcon: Crash Course on Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
[Updated with new Docker projects]
Taking the Next Hot Mobile Game Live with Docker and IBM SoftLayerDaniel Krook
Presentation at the IBM InterConnect Conference in Las Vegas, Nevada on February 24, 2016.
Mobile games are the fastest-growing sector of the $70 billion video game industry, far outpacing traditional consoles. But companies that aspire to create the next hot title have to account for more than just the app downloaded to a user device. They must prepare for huge spikes in game play with scalable backends to handle massive data and transactions behind socially linked user profiles and global leaderboards. This talk looks at how IBM successfully partnered with Firemonkeys, a major studio that had hit their vertical scaling limit, to design and deploy a new Docker-based architecture on SoftLayer. This scale-out architecture is able to handle an order of magnitude more customers for their next major release.
Linuxcon Europe 2011: Overview - Building Cloud Computing EnvironmentsMark Hinkle
Cloud Computing has been touted as an almost magical solution for changing the way enterprise IT infrastructure is deployed. Despite all the “cloudwashing” there is no magic, cloud computing still requires the same rigor in planning and design as in legacy IT architecture. The difference is that thanks to inexpensive hardware and exceptional free and open source software state-of-the-art technology is now this evolution of technology is accessible to any organization. This levels the IT playing field allowing users them to be competitive by deploying systems that are agile, scalable and adaptable to their needs. This presentation will cover the open source software that can be combined to build cloud computing environments for a variety of different uses as well as informing potential cloud users on how to choose technologies to best address the computing needs of their organization.
https://events.linuxfoundation.org/events/linuxcon-europe/hinkle
State of the Stack v4 - OpenStack in All It's GloryRandy Bias
The almost annual State of the Stack, version 4, an end-to-end view of OpenStack. This edition focuses on what the challenges are within the community and how they can be addressed.
v1 of SOTS has over 90,000 views and is one of the highest viewed OpenStack presentations ever.
DevNexus 2015
Docker: containerizing a monolithic app into a microservice-based PaaS
Convert a monolithic application into a microservice-based PaaS using Docker and related, containerization technologies. This will be the third presentation of a series of presentations that began greater than one year ago to evangelize the benefits of Docker. The scope of content spans from a development environment to a hybrid PaaS, and how Containerization is an enabler of architectural choice, innovation, scalability, and polyglot solutions.
The basics of Docker will be examined including repositories, brief discussion about managing and monitoring Docker containers, service discovery, and security. New and emerging technologies will be a constant theme, particularly about microservices, in addition to the ongoing evolution of the market and what the future may bring. Common organizational issues (and tactical solutions) that may impede successful decomposition and migration of legacy monoliths will be discussed, including security, DevOps and refactoring.
Hypothetical architectures will be described for building progressively more robust and complex applications and deployment models. The goal is to highlight the power, flexibility and scalability that containers enable.
Examples will start simple, from a local development environment, that is a simple two container setup that encapsulate a database and application tier. Subsequent discussion will involve progressively more complex and robust deployments that include features such as service discovery, automatic load balancing, and abstractions to simplify linking of containers including service gateways. With the stopping point of a hybrid PaaS.
Serverless architectures are one of the hottest trends in cloud computing this year, and for good reason. There are several technical capabilities and business factors coming together to make this approach compelling from both an application development and deployment cost perspective. The new OpenWhisk project provides an open source platform to enable these cloud-native, event-driven applications.
This talk will lay out the technical and business drivers behind the rise of serverless architectures, provide an introduction to the OpenWhisk open source project (and describe how it differs from other services like AWS Lambda), and give a demonstration showing how to start developing with this new cloud computing model using the OpenWhisk implementation available on IBM Bluemix.
Presented on October 12, 2016 at the NYC Bluemix meetup
LinuxFest Northwest: Crash Course in Open Source Cloud Computing Mark Hinkle
Few IT trends have generated as much buzz as cloud computing. This talk will cut through the hype and clarify cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complementary open source management tools that can be combined to automate the management of cloud computing environments. The discussion will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options for building and managing their own cloud computing environments using free and open source software.
IBM Bluemix OpenWhisk: Interconnect 2016, Las Vegas: CCD-1088: The Future of ...OpenWhisk
Learn more about the IBM Bluemix OpenWhisk, a serverless event-driven compute platform, which quickly executes application logic in response to events or direct invocations from web/mobile apps or other endpoints.
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud ComputingMark Hinkle
Presented on April 27th, 2013 at LinuxFest NW
Imagine it’s eight o’clock on a Thursday morning and you awake to see a bulldozer out your window ready to plow over your data center. Normally you may wish to consult the Encyclopedia Galáctica to discern the best course of action but your copy is likely out of date. And while the Hitchhiker’s Guide to the Galaxy (HHGTTG) is a wholly remarkable book it doesn’t cover the nuances of cloud computing. That’s why you need the Hitchhiker’s Guide to Cloud Computing (HHGTCC) or at least to attend this talk understand the state of open source cloud computing. Specifically this talk will cover infrastructure-as-a-service, platform-as-a-service and developments in big data and how to more effectively take advantage of these technologies using open source software. Technologies that will be covered in this talk include Apache CloudStack, Chef, CloudFoundry, NoSQL, OpenStack, Puppet and many more.
Specific topics for discussion will include:
Infrastructure-as-a-Service - The Systems Cloud - Get a comparision of the open source cloud platforms including OpenStack, Apache CloudStack, Eucalyptus, OpenNebula
Platform-as-a-Service - The Developers Cloud - Find out what tools are availble to build portable auto-scaling applications including CloudFoundry, OpenShift, Stackato and more.
Data-as-a-Service - The Analytics Cloud - Want to figure out the who, what , where , when and why of big data ? You get an overview of open source NoSQL databases and technologies like MapReduce to help crunch massive data sets in the cloud.
Finally you'll get a overview of the tools that can help you really take advantage of the cloud? Want to auto-scale virtual machiens to serve millions of web pages or want to automate the configuration of cloud computing environments. You'll learn how to combine these tools to provide continous deployment systems that will help you earn DevOps cred in any data center.
[Finally, for those of you that are Douglas Adams fans please accept the deepest apologies for bad analogies to the HHGTTG.]
Linux Foundation Collaboration Summit: Hitchhiker's Guide to the CloudMark Hinkle
Imagine it's eight o'clock on a Thursday morning and you awake to see a bulldozer out your window ready to plow over your data center. Normally you may wish to consult the Encyclopedia Galáctica to discern the best course of action but your copy is likely out of date. And while the Hitchhiker's Guide to the Galaxy (HHGTTG) is a wholly remarkable book it doesn't cover the nuances of cloud computing. That's why you need the Hitchhiker's Guide to Cloud Computing (HHGTCC) or at least to attend this talk understand the state of open source cloud computing. Specifically this talk will cover infrastructure-as-a-service, platform-as-a-service and developments in big data and how to more effectively take advantage of these technologies using open source software. Technologies that will be covered in this talk include Apache CloudStack, Chef, CloudFoundry, NoSQL, OpenStack, Puppet and many more.
OSCON 2013 - The Hitchiker’s Guide to Open Source Cloud ComputingMark Hinkle
And while the Hitchhiker’s Guide to the Galaxy (HHGTTG) is a wholly remarkable book it doesn’t cover the nuances of cloud computing. Whether you want to build a public, private or hybrid cloud there are free and open source tools that can help provide you a complete solution or help augment your existing Amazon or other hosted cloud solution. That’s why you need the Hitchhiker’s Guide to (Open Source) Cloud Computing (HHGTCC) or at least to attend this talk understand the current state of open source cloud computing. This talk will cover infrastructure-as-a-service, platform-as-a-service and developments in big data and how to more effectively deploy and manage open source flavors of these technologies. Specific the guide will cover:
Infrastructure-as-a-Service – The Systems Cloud – Get a comparison of the open source cloud platforms including OpenStack, Apache CloudStack, Eucalyptus and OpenNebula
Platform-as-a-Service – The Developers Cloud – Learn about the tools that abstract the complexity for developers and used to build portable auto-scaling applications ton CloudFoundry, OpenShift, Stackato and more.
Data-as-a-Service – The Analytics Cloud – Want to figure out the who, what, where, when and why of big data? You’ll get an overview of open source NoSQL databases and technologies like MapReduce to help parallelize data mining tasks and crunch massive data sets in the cloud.
Network-as-a-Service – The Network Cloud – The final pillar for truly fungible network infrastructure is network virtualization. We will give an overview of software-defined networking including OpenStack Quantum, Nicira, open Vswitch and others.
Finally this talk will provide an overview of the tools that can help you really take advantage of the cloud. Do you want to auto-scale to serve millions of web pages and scale back down as demand fluctuates. Are you interested in automating the total lifecycle of cloud computing environments You’ll learn how to combine these tools into tool chains to provide continuous deployment systems that will help you become agile and spend more time improving your IT rather than simply maintaining it.
[Finally, for those of you that are Douglas Adams fans please accept the deepest apologies for bad analogies to the HHGTTG.]
Rackspace::Solve NYC - The Future of Applications with Ken Cochrane, Engineer...Rackspace
What does intermodal shipping have to do with managing your app’s components in different environments? Ken Cochrane, Engineering Manager at Docker, explains in this presentation from Rackspace::Solve NYC. For more information about Rackspace::Solve, visit http://www.rackspacesolve.com.
Rackspace (NYSE: RAX) is the #1 managed cloud company. Our technical expertise and Fanatical Support® allow companies to tap the power of the cloud without the pain of hiring experts in dozens of complex technologies. Rackspace is also the leader in hybrid cloud, giving each customer the best fit for its unique needs — whether on single- or multi-tenant servers, or a combination of those platforms. Rackspace is the founder of OpenStack®, the open-source operating system for the cloud. Headquartered in San Antonio, we serve more than 200,000 business customers from data centers on four continents. We rank 29th on Fortune’s list of 100 Best Companies to Work For. For more information, visit www.rackspace.com.
Getting Started with Docker - Nick StinematesAtlassian
Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere. In this session, you will learn how to get started building your first Docker container, and how to use Docker containers to simplify your CI process.
Oscon 2017: Build your own container-based system with the Moby projectPatrick Chanezon
Build your own container-based system
with the Moby project
Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.
Introduction to dockers and kubernetes. Learn how this helps you to build scalable and portable applications with cloud. It introduces the basic concepts of dockers, its differences with virtualization, then explain the need for orchestration and do some hands-on experiments with dockers
An introduction to the Moby Project and LinuxKit. The demo essentially walked through the LinuxKit examples available on Github at https://github.com/linuxkit/linuxkit paying specific attention to the linuxkit.yml nginx example in the home directory, and the redis-os example in the examples directory.
Similar to All Things Open : Crash Course in Open Source Cloud Computing (20)
"Is serverless another passing technology fad or the new standard for application deployment in cloud computing?” It’s a good question and the topic of this presentation. We will discuss the current state of serverless computing and the many considerations before investing time and resources in serverless infrastructure.
For many, data center priorities have shifted from absolute uptime and performance to ”move fast and break things” as espoused by Silicon Valley, a great mantra for those with limited legacy systems and a greenfield of new products. Though the question for many enterprises though is "How does serverless integrate into their existing data center strategy?"
The discussion will not only explain the state of today’s growing serverless landscape but how you can integrate your existing data center with a cloud-native serverless architecture.
Triangle Kubernetes Meet-Up - Serverless is FaaS-tasticMark Hinkle
Talk Delivered 3/19/2019 - Serverless can be misleading as a descriptor. Serverless infrastructure actually runs on servers. However, the “server-less” reference comes from the fact that serverless abstracts the complexity of running servers away from the software developer which enables them to develop software without having to worry about the scaling, redundancy and overall infrastructure design. This is called Function-as-a-Service or Faas for short.
For the purposes of this talk, we’ll discuss serverless technologies where someone else is providing serverless infrastructure. Popular serverless platforms include Amazon Web Services Lambda, Google Cloud Functions, and Microsoft Azure Functions.
The presentation will also discuss the software that can be used to deliver Functions-as-a-Service (FaaS) that enables serverless, including serverless frameworks like Knative, Kubeless, OpenFaaS, and Oracle’s fn.
Finally, we’ll cover what a cloud-native application might look like including the use cases and design patterns that serverless is geared towards providing.
Serverless is FaaS-tastic - Columbia Open Source Meet-Up Mark Hinkle
Serverless can be misleading as a descriptor. Serverless infrastructure actually runs on servers. However, the “server-less” reference comes from the fact that serverless abstracts the complexity of running servers away from the software developer which enables them to develop software without having to worry about the scaling, redundancy and overall infrastructure design. This is called Function-as-a-Service or Faas for short.
For the purposes of this talk, we’ll discuss serverless technologies where someone else is providing serverless infrastructure. Popular serverless platforms include Amazon Web Services Lambda, Google Cloud Functions, and Microsoft Azure Functions.
The presentation will also discuss the software that can be used to deliver Functions-as-a-Service (FaaS) that enables serverless, including serverless frameworks like Knative, Kubeless, OpenFaaS, and Oracle’s fn.
Finally, we’ll cover what a cloud-native application might look like including the use cases and design patterns that serverless is geared towards providing.
Serverless is FaaS-tastic - All Things Open Meet-upMark Hinkle
Serverless can be misleading as a descriptor. Serverless infrastructure actually runs on servers. However, the “server-less” reference comes from the fact that serverless abstracts the complexity of running servers away from the software developer which enables them to develop software without having to worry about the scaling, redundancy and overall infrastructure design. This is called Function-as-a-Service or Faas for short.
For the purposes of this talk, we’ll discuss serverless technologies where someone else is providing serverless infrastructure. Popular serverless platforms include Amazon Web Services Lambda, Google Cloud Functions, and Microsoft Azure Functions.
The presentation will also discuss the software that can be used to deliver Functions-as-a-Service (FaaS) that enables serverless, including serverless frameworks like Knative, Kubeless, OpenFaaS, and Oracle’s fn.
Finally, we’ll cover what a cloud-native application might look like including the use cases and design patterns that serverless is geared towards providing.
Keynote - Open Source 101 - How JavaScript Became a Legitimate Open Source En...Mark Hinkle
JavaScript has been a primary language of the browser for many years but at the same time become a first-class enterprise application platform as well. Driven by a need for applications that can scale to handle extreme workloads that are exchanging data and a vibrant open source community developing best-of-breed software for web, mobile, and IoT JavaScript is currently the most widely developed programming language on the planet.
Keynote All Things Open - Open Source: The Punk Rock of the 21st CenturyMark Hinkle
It's easy to draw a comparison between open source software. Many bands self-produced recordings (like software developers) and distributed them through informal channels (like open source projects)….technical accessibility and a DIY spirit are prized in punk rock(as we see in open source)…….Punk rock is meant to be our freedom(as in free software). We're meant to be able to do what we want to do…. The issue of authenticity is important in the punk subculture—the pejorative term "poseur" is applied to those who associate with punk and adopt its stylistic attributes but are deemed not to share or understand the underlying values and philosophy…. At the end of the 20th century, punk rock had been adopted by the mainstream, as pop punk and punk rock bands such as Green Day, the Offspring and Blink-182 brought the genre to widespread popularity. Open source is enjoying that same popularity in the 21st century.
Bay Area Open Source Meet-Up: Things I Learned about Open Source The Hard Way Mark Hinkle
Mark Hinkle runs the Citrix Open Source Business Office and has spent 20 years working with open source communities and delivering open source software. Topics covered in this presentation will include the benefit of his mistakes and successes both in evaluating open source ad an end-user and in delivering enterprise solutions based on open source software.
Keynote Devops Days Amsterdam - Hacking IT, Culture over Code Bringing Devops...Mark Hinkle
The term DevOps has crossover over from a culture movement around improved IT delivery to a buzzword co-opted by headline minded journalists and companies who want to reinvent their antiquated practices by acquiring new talent. This presentation will talk about DevOps the movement, desired outcomes from DevOps practices and how to bring those practices to your organization especially those with entrenched practices that lack the agility, automation and other benefits of DevOps.
ApacheCon 2014; Let Me Help You. Don’t Fear the Man with the Free T-ShirtsMark Hinkle
The Apache Way™ is an incredible process for developing software as good or better than any other software development methodology. While we do a great job producing software that powers the Internet we often don’t do everything we can do to promote that technology, encourage new users and get more awareness of the work we do. This talk will outline considerations for how to promote a project and track progress and drive adoption to help insure the viability of the project and sell your boss on how to allow him to invest more of your time and company resources to help develop your Apache project.
Linuxcon Europe 2013 | Keynote: We Won What's NextMark Hinkle
It’s been over twenty years since Linux birth and it grown up to become the most successful collaborative endeavor of all time. Linus’ little project now cumulatively powers more servers, mobile phones and other embedded systems than any other operating system. Linux runs our economy and touches the lives of literally every single human being on the planet in one way, shape or form. Time Magazine named Linux Torvalds the 17th most influential man of the century 20th century. No longer do we have to defend the viability of Linux it’s been proven beyond a shadow of a doubt. So where do we go from here? We’ll explore how the Linux and open source community can build upon their success for the betterment of technology and the world around them.
Cloud Expo Silicon Valley 2013 | Why Lease When You Can Buy Your CloudMark Hinkle
Perhaps one of the perplexing things about cloud computing is the choice around renting time in someone else’s cloud (Amazon, Google, Rackspace or a myriad of others) or building your own. It’s not unlike the age-old car buyer’s dilemma, take the lower payments and lower total miles lease or buy the car and drive it for the long haul. Cloud computing users are often faced with the same conundrum. This presentation will focus on how to buy and build a cloud that can be fulfill the needs of most users including strategies for making use of the open source private cloud or managing workloads in both the private and public cloud using open source software.
LinuxCon North America 2013: Why Lease When You Can Buy Your CloudMark Hinkle
Perhaps one of the perplexing things about cloud computing is the choice around renting time in someone else’s cloud (Amazon, Google, Rackspace or a myriad of others) or building your own. It’s not unlike the age-old car buyer’s dilemma, take the lower payments and lower total miles lease or buy the car and drive it for the long haul. Cloud computing users are often faced with the same conundrum. This presentation will focus on how to buy and build a cloud that can be fulfill the needs of most users including strategies for making use of the open source private cloud or managing workloads in both the private and public cloud using open source software.
OSCON 2013 - Keynote - Creating Communities of InclusionMark Hinkle
Free and open source software is equal parts technology and humanity. Beyond the coding standards, development environments and essential parts of delivering free software the ideals that drive this movement are powerful. This is a reflection on the lessons gleaned from successful F/LOSS communities and a call to action to spread their ideals to other endeavors such as medicine and government.
Cloud Expo East 2013: Essential Open Source Software for Building the Open CloudMark Hinkle
Cloud computing is more than a buzz-phrase it’s a transformative IT paradigm shift. The emphasis in the cloud is on elasticity, scalability, agility and open. Not just open standards but open APIs and open source. The delivery of software is also going through a paradigm shift. Open source software was often a commoditization of a market leader; Unix to Linux or Oracle to MySQL what’s changing is that the iterative nature, user context and the motto of releasing early and often are driving real innovation in open source.
This session will cover those essential open source technologies for delivering cloud computing in the enterprise.
Speaker Bio:
Mark Hinkle is the Senior Director, Open Source Solutions at Citrix Systems Inc. He joined Citrix as a result of their July 2011 acquisition of Cloud.com where he was their Vice President of Community. He is currently responsible for Citrix open source efforts around the open source cloud computing platform, Apache CloudStack and the Xen Hypervisor. Previously he was the VP of Community at Zenoss Inc., a producer of the open source application, server, and network management software, where he grew the Zenoss Core project to over 100,000 users and 20,000 organizations on all seven continents. He also is a longtime open source expert and author having served as Editor-in-Chief for both LinuxWorld Magazine and Enterprise Open Source Magazine. His blog on open source, technology, and new media can be found at http://www.socializedsoftware.com.
Users who have publicly stated they are using CloudStack for private or public cloud computing including those who are using products based on Apache CloudStack-based products.
Hitchhiker's Guide to Open Source Cloud ComputingMark Hinkle
Imagine it’s eight o’clock on a Thursday morning and you awake to see a bulldozer out your window ready to plow over your data center. Normally you may wish to consult the Encyclopedia Galáctica to discern the best course of action but your copy is likely out of date. And while the Hitchhiker’s Guide to the Galaxy (HHGTTG) is a wholly remarkable book it doesn’t cover the nuances of cloud computing. That’s why you need the Hitchhiker’s Guide to Cloud Computing (HHGTCC) or at least to attend this talk understand the state of open source cloud computing. Specifically this talk will cover infrastructure-as-a-service, platform-as-a-service and developments in big data and how to more effectively take advantage of these technologies using open source software. Technologies that will be covered in this talk include Apache CloudStack, Chef, CloudFoundry, NoSQL, OpenStack, Puppet and many more.
Specific topics for discussion will include:
Infrastructure-as-a-Service - The Systems Cloud - Get a comparision of the open source cloud platforms including OpenStack, Apache CloudStack, Eucalyptus, OpenNebula
Platform-as-a-Service - The Developers Cloud - Find out what tools are availble to build portable auto-scaling applications including CloudFoundry, OpenShift, Stackato and more.
Data-as-a-Service - The Analytics Cloud - Want to figure out the who, what , where , when and why of big data ? You get an overview of open source NoSQL databases and technologies like MapReduce to help crunch massive data sets in the cloud.
Finally you'll get a overview of the tools that can help you really take advantage of the cloud? Want to auto-scale virtual machiens to serve millions of web pages or want to automate the configuration of cloud computing environments. You'll learn how to combine these tools to provide continous deployment systems that will help you earn DevOps cred in any data center.
[Finally, for those of you that are Douglas Adams fans please accept the deepest apologies for bad analogies to the HHGTTG.]
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
All Things Open : Crash Course in Open Source Cloud Computing
1. All Things Open 2014
Crash Course in
Open Source Cloud Computing
Mark Hinkle
Senior Director, Open Source Solutions
Citrix Inc.
mark.hinkle@citrix.com
mrhinkle@gmail.com
@mrhinkle
2. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
ABOUT ME
I Help Build Open Source Ecosystems
Open Source Experience
• Manage Citrix Open Source Business Office
• Apache CloudStack Committer and PMC Member
• Advisory boards Gluster and Xen Project
• Joined Citrix via Cloud.com acquisition July 2011
• Zenoss Core open source project to 100,000 users,
1.5 million downloads
• Former LinuxWorld Magazine Editor-in-Chief
• Open Management Consortium organizer
• Author - “Windows to Linux Business Desktop
Migration” – Thomson
• NetDirector Project - Open Source Configuration
Management
All Things Open 2014 - Open Source Cloud Computing
3. http://www.slideshare.net/socializedsoftware
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes
were made. You may do so in any reasonable manner, but not in any way that suggests the licensor
endorses you or your use.
ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions
under the same license as the original.
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
Slides Available on Slideshare:
Creative Commons Attributions-ShareAlike 4.0 International
Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material
for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms.
All Things Open 2014 - Open Source Cloud Computing
4. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
AGENDA
• Vetting Open Source Cloud Projects
• Virtualization
• Infrastructure-as-a-Service
• Platform-as-a-Service
• SDN
• Open Source for Amazon Web Services
All Things Open 2014 - Open Source Cloud Computing
5. VETTING OPEN SOURCE PROJECTS
How can you tell if they’re Legit
• Code Velocity
• Committers
• Committer Reputation
• User-driven or Vendor-Driven
Innovation
• User Activity
• Corporate Support*
• Reputation of Foundation*
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
All Things Open 2014 - Open Source Cloud Computing
6. …the future of technological innovation is not stealing limited
resources away from one another, but creating new resources
— and new opportunities to create new resources — together in
a rich ecosystem.”
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPEN SOURCE ISN’T
A ZERO-SUM GAME
Allison Randal
Open Source Hacker
Former OSCON Program Chair
@allisonrandal
All Things Open 2014 - Open Source Cloud Computing
8. DevOps
Toolchain
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPEN SOURCE CLOUD STACK
Platform-as-a-Service (PaaS)
? ?
Infrastructure-as-a-Service (IaaS)
Orchestration
?
Compute Storage Networking
(Networking-as-a-Service)
All Things Open 2014 - Open Source Cloud Computing
Orchestration
Configuration
Management
Monitoring
9. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
VIRTUALIZATION
Carving up compute resources
OPEN SOURCE
• Xen Project
• Citrix XenServer
• KVM
• VirtualBox
• OpenVZ
• LXC
• libcontainer
PROPRIETARY
• VMware
• Microsoft Hyper-V
• OracleVM (Based on Xen Project)
All Things Open 2014 - Open Source Cloud Computing
10. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
HYPERVISORS AND CONTAINERS
Differences in virtualization
Type 1 Hypervisors
VMware, Xen Project, Hyper-V
Type 2 Hypervisors
KVM, VirtualBox
All Things Open 2014 - Open Source Cloud Computing
Containers
LXC, libcontainer
11. • Different file formats for virtual machines
(VMware uses vmdk file format, Xen and
Hyper-V use VHD, KVM uses Raw or QCOW2)
• Guest images may be “processor architecture”
• VMware and Xen can manage SCSI devices, but
• KVM and Xen can use virtio drivers but not
• VMware uses a proprietary agent inside the
guest OS (VMware tools) which does not work
with Xen or KVM
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
THE PORTABILITY PROBLEM
Containers compared to Hardware Virtualization
bound
KVM cannot
VMware
• Yada, Yada, Yada
All Things Open 2014 - Open Source Cloud Computing
12. • Lets your run a Linux system within
• A container is a group of processes on a
Linux box, put together the provide an
isolated environment
• From the inside, it looks like a VM
• Externally it looks like normal processes
• “chroot on steroids”
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
LINUX CONTAINERS
“Lightweight” Linux Virtualization
another Linux system
All Things Open 2014 - Open Source Cloud Computing
13. • Code – Application is stored
• Build – Code is built (Jenkins)
• Test – Unit tests are
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
CONTINUOUS INTEGRATION
Rebuild Applications on any Cloud and/or Virtualized Infrastructure
in a repository
(Subversion,Git)
automated (Jenkins)
• Deploy – Deploy code to
server various ways
Code
Build
Test
Deploy
Thoughtworks Go – Open Source
Continuous Deliver System
All Things Open 2014 - Open Source Cloud Computing
14. Docker is an open-source project to easily
create lightweight, portable, self-sufficient
containers from any application. The same
container that a developer builds and tests
on a laptop can run at scale, in production,
on VMs, bare metal, public clouds and
more.
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
DOCKER CONTAINER PACKAGING
Open source LXC Packaging Engine
To learn more please visit:
www.docker.io
All Things Open 2014 - Open Source Cloud Computing
15. • Compliment to LXC not a replacement
• Managed daemonized processes on Linux
• Create ability to re-use and manage similar
• Content agnostic
• Hardware agnostic
• Easy to automate
• Integrated with other tools: Chef, OpenShift,
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
WHAT IS DOCKER
System for Managing and Deploying LXC Containers
using LXC libcontainer
applications
Puppet, VMware, etc.
All Things Open 2014 - Open Source Cloud Computing
16. DOCKER’S GROWING
ECOSYSTEM
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
All Things Open 2014 - Open Source Cloud Computing
17. Kubernetes builds on top of Docker to
construct a clustered container scheduling
service. Kubernetes enables users to ask
a cluster to run a set of containers. The
system will automatically pick worker
nodes to run those containers on, which
we think of more as "scheduling" than
"orchestration”
To learn more please visit:
https://github.com/GoogleCloudPlatform/kubernetes Greek for Shipmaster
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
KUBERNETES
Container Cluster Management – Scheduler
All Things Open 2014 - Open Source Cloud Computing
18. DOCKER RELATED PROJECTS
• Fig -Fast, isolated development environments
• Flynn - Next-generation application platform
• Panamax – Drag-and-Drop Docker Containerization
• Project Atomic – JEOS designed to run Docker
containers
• SocketPlane – Docker Networking (coming soon)
• Weave – Docker Networking
• 13,000+ Docker-related repos on Github
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
All Things Open 2014 - Open Source Cloud Computing
19. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
$141 Billion Market Cap
$363 Billion Market Cap
All Things Open 2014 - Open Source Cloud Computing
$356 Billion Market Cap
PUBLIC CLOUD
20. Project Year Started License Virtualization
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
MINIMUM VIABLE CLOUD
Infrastructure-as-a-Service | IaaS |Compute Orchestration
All Things Open 2014 - Open Source Cloud Computing
Technologies
Apache
CloudStack
2008 Apache (Bare Metal), Xenserver,
KVM, LXC VMware Hyper-
V
Eucalyptus 2006 GPL Xen, KVM, VMware
(commercial version)
OpenNebula 2005 Apache Xen, KVM, VMware
OpenStack 2010 (Developed by
NASA by Anso Labs
previously)
Apache VMware ESX and ESXi, ,
Xen, XenServer, KVM,
LXC, QEMU and Virtual
Box
21. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPENSTACK
The Boy Band of the Open Source Cloud
All Things Open 2014 - Open Source Cloud Computing
22. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPENSTACK SHARED SERVICES
Span Compute, Storage and Networking
IDENTITY
SERVICE
IMAGE
SERVICE
All Things Open 2014 - Open Source Cloud Computing
TELEMETRY
SERVICE
ORCHESTRATION
SERVICE
23. EVEN MORE OPENSTACK PROJECTS
Span Compute, Storage and Networking
• Trove
Database Service
• Ironic
Bare Metal (Ironic)
• Marconi
Queue Service
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
• Cinder
Block Storage Service
• Ceilometer
Metering/Monitoring
• Heat
Orchestration
All Things Open 2014 - Open Source Cloud Computing
24. OPENSTACK SOLUTION PROVIDERS
If you can’t do it yourself
“OpenStack is not a product. If you are building a large infrastructure, it’s
more like a tool kit. It gives you a lot of technologies that do take a lot of
effort to integrate.”
Chris Kemp, OpenStack Board Member and Co-Founder
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
All Things Open 2014 - Open Source Cloud Computing
CEO of Piston Computing
25. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
CLOUD APIS
Everything (should) have an API in the Cloud
All Things Open 2014 - Open Source Cloud Computing
• Deltacloud(ruby)
• Daisein(java)
• Jclouds(java)
• Libcloud(python)
• Fog(ruby)
26. Project Description
Ceph Distributed file storage system developed by DreamHost ->
GlusterFS Scale Out NAS system aggregating storage over Ethernet or
Riak CS Riak CS is open source software designed to provide simple,
available, distributed cloud storage at any scale. Riak CS is S3-
API compatible and supports per-tenant reporting for billing and
metering use cases. (object)
Sheepdog Distributed storage for KVM hypervisors, distributed iSCSI
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
CLOUD STORAGE
Virtualized, Distributed usually on Commodity Hardware
InkTank -> Red Hat (block, object, file)
Infiniband (file)
OpenStack
Storage
Long-term object storage system (object)
All Things Open 2014 - Open Source Cloud Computing
27. CLOUD AUTOMATION TOOLS
One to many tools for managing large numbers of devices
Ansible Ansible's SSH-key based access allows contributors to the Fedora Project to assist in
automating infrastructure while having access limited appropriately. (Originally authored Func)
Capistrano Utility and framework for executing commands in parallel on multiple remote machines, via SSH.
It uses a simple DSL that allows you to define tasks, which may be applied to machines in
certain roles
RunDeck Rundeck is an open-source process automation and command orchestration tool with a web
Func Func provides a two-way authenticated system for generically executing tasks, integrations with
MCollective The Marionette Collective AKA MCollective is a framework to build server orchestration or
Salt Execute arbitrary shell commands or choose from dozens of pre-built modules of common (or
Scalr Provide scaling across multiple cloud computing platforms, integrates with Chef.
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
Project Description
console.
puppet and cobbler.
parallel job execution systems.
complex) commands.
All Things Open 2014 - Open Source Cloud Computing
28. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
All Things Open 2014 - Open Source Cloud Computing
29. Project Sponsors Languages/Frameworks
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
PLATFORM-AS-A-SERVICE
Abstracted Cloud-Scale Run-Time Environments
CloudFoundry VMware -> Pivotal -> CloudFoundry
Foundation
All Things Open 2014 - Open Source Cloud Computing
Spring for Java, Ruby for Rails and
Sinatra, node.js, Grails, Scala on
Lift and more via partners (e.g.
Python, PHP)
Cloudify Gigaspaces [Groovy for deployment recipes]
OpenShift Origin Red Hat Java, Ruby, PHP, Perl and Python
Apache Stratos WSO2 - >Apache Stratus PHP, Tomcat, MySQL “cartridges”
30. Apache Mesos is a cluster manager that simplifies the
complexity of running applications on a shared pool of
servers. Largely supported by Twitter, used by LinkedIn,
AirBNB too.
Features
• Fault-tolerant replicated master using ZooKeeper
• Scalability to 10,000s of nodes
• Isolation between tasks with Linux Containers
• Multi-resource scheduling (memory and CPU aware)
• Java, Python and C++ APIs for developing new
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
APACHE MESOS
One to many tools for managing large numbers of devices
parallel applications
• Web UI for viewing cluster state
To learn more please visit:
http://mesos.apache.org/
All Things Open 2014 - Open Source Cloud Computing
31. Decoupling of the control and data planes of the network to
improve efficiency. Communication from a SDN controller via a
protocol to network devices both physical and virtual.
Abstractions allow for programmable networks.
Network can be changed quickly via a controller
Network offerings can match virtualization offerings for finer
grained security in a highly volatile compute landscape.
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
SOFTWARE DEFINED
VNirtuEalizTatiWon mOeetRs thKe neItwNorkG(SDN)
Automation
Dynamic Networks
Security
Heterogeneous Management
Single control point for various devices.
All Things Open 2014 - Open Source Cloud Computing
32. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
Business Applications
SDN OVERVIEW
All Things Open 2014 - Open Source Cloud Computing
Network Services
SDN
Control
Software
API API
Network Devices Network Devices Network Devices
Network Devices Network Devices Network Devices
Application
Layer
Control
Layer
Infrastructure
Layer
Control Data Plane Interface (e.g. OpenFlow)
33. BENEFITS OF SDN
Network Virtualization is the final frontier of Software Defined Datacenter
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
• Dynamically update networks
• Automate network
functionality
• “Program” security into the
network
• Centrally apply policies to
network and services
• Optimize networks
All Things Open 2014 - Open Source Cloud Computing
34. OpenFlow enables networks to
evolve, by giving a remote
controller the power to modify
the behavior of network
devices, through a well-defined
"forwarding instruction set".
The growing OpenFlow
ecosystem now includes
routers, switches, virtual
switches, and access points
from a range of vendors.
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPENFLOW
Virtualization meets the network
All Things Open 2014 - Open Source Cloud Computing
35. OPEN SOURCE SDN
Software Defined Network Controllers and more
Floodlight The Floodlight Open SDN Controller is an enterprise-class, Apache-licensed, Java-based OpenFlow
Controller. It is supported by a community of developers including a number of engineers from Big Switch
Networks. - See more at: http://www.projectfloodlight.org/floodlight/#sthash.9IhA1Ih5.dpuf
Indigo Indigo is an open source project aimed at enabling support for OpenFlow on physical and hypervisor
switches. Big Switch has helped numerous companies OpenFlow enable their equipment, and we
provide firmware for a number of popular switches. Indigo is the basis of Switch Light by Big Switch
Networks. - See more at: http://www.projectfloodlight.org/indigo/#sthash.K7LiHcqc.dpuf
Lincx LINCX is a pure OpenFlow software switch written in Erlang. It runs within a separate domain under Xen
Nox NOX is the original OpenFlow controller, and facilitates development of fast C++ controllers on Linux.
Open Daylight Linux Foundation Collaborative Project based on Cisco One Controller and plugins from numerous
Open vSwitch Open vSwitch is a open source (ASL 2.0), multilayer virtual switch designed to enable massive network
automation through programmatic extension, while still supporting standard management interfaces and
protocols (e.g. NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag).
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
Project Description
hypervisor using LING (erlangonxen.org).
vendors in development. E.g IBM DOVE
All Things Open 2014 - Open Source Cloud Computing
36. Open vSwitch is a production quality,
multilayer virtual switch licensed under the
open source Apache 2.0 license. It is
designed to enable massive network
automation through programmatic extension,
while still supporting standard management
interfaces and protocols (e.g. NetFlow, sFlow,
SPAN, RSPAN, CLI, LACP, 802.1ag).
To learn more please visit our website:
http://openvswitch.org/
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPEN VSWITCH
All Things Open 2014 - Open Source Cloud Computing
37. DevOps
Toolchain
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
OPEN SOURCE CLOUD STACK
CloudFoundry, OpenShift, Gigaspaces
Docker
Platform-as-a-Service
Mesos Kubernetes
Infrastructure-as-a-Service | IaaS | Orchestration
(OpenStack, Apache CloudStack, Eucalyptus)
Compute
(Containers,
KVM, Xen)
Storage
(Ceph, Gluster)
Networking
(OpenDaylight,
All Things Open 2014 - Open Source Cloud Computing
Contrail)
Orchestration
-
Ansible/SaltStack/Scalr*
Configuration
Management
(CFengine/Chef/Puppet)
Monitoring
(logstash,graphite,)
38. EUREKA PRIAM SIMIAN ARMY
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
ASGARD ASTYANAX EDDA
All Things Open 2014 - Open Source Cloud Computing
38
http://netflix.github.com
NETFLIX AWS TOOLBAG
Tools developed by a super Amazon Web Services Power User
39. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
CONTACT ME
Happy to Chat about Open Source, Cloud or Pittsburgh Sports
Professional: mark.hinkle@citrix.com
Personal: mrhinkle@gmail.com
Phone: 919.228.8049
Professional: http://open.citrix.com
Personal: http://www.socializedsoftware.com
Twitter: @mrhinkle
All Things Open 2014 - Open Source Cloud Computing
40. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
APPENDIX A
Additional Links to related stuff
All Things Open 2014 - Open Source Cloud Computing
41. ADDITIONAL LINKS
• Devops Toolchains Group
• Software Defined Networking: The New Norm for Networks
(Whitepaper)
• DevOps Wikipedia Page
• NoSQL-Database.org – Ultimate Guide to the Non-Relational Universe
• Open Cloud Initiative
• NIST Cloud Computing Platform
• Open Virtualization Format Specs
• Clouderati Twitter Account
• Planet DevOps
• Nicira Whitepaper – It’s Time to Virtualize the Network
• Why Open vSwitch FAQ
• Stanford Seminar - Software-Defined Networking at the Crossroads
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
All Things Open 2014 - Open Source Cloud Computing
42. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
ADDITIONAL LINKS (CONT’D)
• SDN, NFV, and open source: The Operator’s View
• Puppet Labs: Build a Toolbox for Continuous Delivery
All Things Open 2014 - Open Source Cloud Computing
43. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
APPENDIX B
Stuff I’d liked to have talked
about but didn’t have time
All Things Open 2014 - Open Source Cloud Computing
44. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
60 SECOND CLOUD DEFINITION
Just because Software Marketing Guys Think it’s the Internet
5 CHARACTERISTICS OF CLOUD
1. On-Demand Self-Service
2. Broad Network Access
3. Resource Pooling
4. Rapid Elasticity
5. Measured Service
User Cloud a.k.a.
SOFTWARE-AS-A-SERVICE
Developer Cloud a.k.a.
PLATFORM-AS-A-SERVICE
Systems Cloud a.k.a.
INFRASTRUCTURE-AS-A-SERVICE
All Things Open 2014 - Open Source Cloud Computing
45. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
SCALE-UP SCALE OUT
Elasticity and the cloud
Vertical Scaling (Scale-Up)
Allocate additional resources to
VMs, requires a reboot, no need for
distributed app logic, single-point of
OS failure
Horizontal Scaling (Scale-Out)
Application needs logic to work in
distributed fashion (e.g. HA-Proxy
and Apache Hadoop)
All Things Open 2014 - Open Source Cloud Computing
46. Bitnami BitNami provides free, ready to run environments for your favorite open source
web applications and frameworks, including Drupal, Joomla!, Wordpress, PHP,
Rails, Django and many more.
Boxgrinder BoxGrinder is a set of projects that help you grind out appliances for multiple
Oz Command-line tool that has the ability to create images for common Linux
SUSE Studio SUSE Studio supports building and deploying directly to cloud services such as
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
SOURCING CLOUD APPLIANCES
Packaging Engines for VMs
Tool/Project What you can do with them
virtualization and Cloud providers
distributions to run on KVM
Amazon EC2.
All Things Open 2014 - Open Source Cloud Computing
47. PACKER MULTIPLATFORM VM CREATION
Packer is easy to use and automates the
creation of any type of machine image. It
embraces modern configuration
management by encouraging you to use
automated scripts to install and configure
the software within your Packer-made
images.
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
To learn more please visit:
www.packer.io
Open source Automation for VMs
All Things Open 2014 - Open Source Cloud Computing
48. CONFIGURATION MANAGEMENT
TOOLS Tools with features for configuring cloud infrastructure
Project Year Started Language License Client/Server
Chef 2009 Ruby Apache Chef Solo – No
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
CFengine 1993 C Apache Yes
All Things Open 2014 - Open Source Cloud Computing
Chef Server - Yes
Puppet 2004 Ruby GPL Yes & standalone
Salt 2011 Python Apache yes
Hitchhiker’s Guide to the
Open Cloud by @mrhinkle
48
49. CLOUD MONITORING TOOLS
Tools with features for monitoring cloud infrastructure
Project Type of Monitoring Collection Methods
Cacti / RRDTool Performance SNMP, syslog
Nagios Availability SNMP,TCP, ICMP, IPMI,
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
Graphite Performance Agent
All Things Open 2014 - Open Source Cloud Computing
syslog
Sensu Availability Agent
Zabbix Availability/ Performance and more SNMP, TCP/ICMP, IPMI,
Synthetic Transactions
Zenoss Availability, Performance, Event
Management
SNMP, ICMP, SSH, syslog,
WMI
Hitchhiker’s Guide to the
Open Cloud by @mrhinkle
49
50. CLOUD PROVISIONING TOOLS
Packaging Engines for VMs
Can provision 10s to 1000s of machines on various clouds.
Cobbler Distributed virtual infrastructure using koan (kickstart of a network to PXE
boot VMs) for Red Hat, OpenSUSE Fedora, Debian, Ubuntu VMs
Salt Cloud Tool to provision “salted” VMs that can then be updated by a central server
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
Project Installation Targets
Apache Provisionr
(incubating)
Crowbar (Bare metal provisioning)
JuJu Public Clouds - Amazon Web Services HP Cloud,
Private OpenStack clouds, Bare Metal via MAAS.
via ZeroMQ
All Things Open 2014 - Open Source Cloud Computing
Hitchhiker’s Guide to the
Open Cloud by @mrhinkle
50
51. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
BIG DATA
All Things Open 2014 - Open Source Cloud Computing
52. API: many » Query Method: MapReduce, Replicaton: , Written in: Java, Concurrency: eventually
consistent , Misc: like "Big-Table on Amazon Dynamo alike", initiated by Facebook
CouchDB Document Store API: Memcached API+protocol (binary and ASCII) , most languages, Protocol: Memcached REST interface
for cluster conf + management, Written in: C/C++ + Erlang (clustering), Replication: Peer to Peer, fully
consistent, Misc: Transparent topology changes during operation, provides memcached-compatible
caching buckets
API: Java / any writer, Protocol: any write call, Query Method: MapReduce Java / any exec, Replication:
HDFS Replication, Written in: Java
PI: Thrift (Java, PHP, Perl, Python, Ruby, etc.), Protocol: Thrift, Query Method: HQL, native Thrift API,
Replication: HDFS Replication, Concurrency: MVCC, Consistency Model: Fully consistent Misc: High
performance C++ implementation of Google's Bigtable.
MongoDB Document Store API: BSON, Protocol: C, Query Method: dynamic object-based language & MapReduce, Replication:
Redis Key Value/ Tuple Store API: Tons of languages, Written in: C, Concurrency: in memory and saves asynchronous disk after a
defined time. Append only mode available. Different kinds of fsync policies. Replication: Master / Slave,
Misc: also lists, sets, sorted sets, hashes, queues.
Riak Key Value / Tuple Store API: JSON, Protocol: REST, Query Method: MapReduce term matching , Scaling: Multiple Masters; Written
in: Erlang, Concurrency: eventually consistent (stronger then MVCC via Vector Clocks)
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
NOSQL DATABASES
Horizontally scalable unstructured data retrieval
Name Type Description
Apache
Wide Column
Cassandra
Store/Families
HBase Wide Column
Store/Families
Hypertable Wide Column
Store/Families
Master Slave & Auto-Sharding, Written in: C++,Concurrency
All Things Open 2014 - Open Source Cloud Computing
53. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
MAP REDUCE
Algorithm for Parallelized Data Set Processing
Problem
Data
Master
Node
All Things Open 2014 - Open Source Cloud Computing
Worker
Node 1
Worker
Node 2
Worker
Node 3
Solution
Data
Map
Reduce
54. By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
APACHE HADOOP
Apache Project for Parallelized Data Set Processing
Overview
• Handles large amounts of
data
• Stores data in native format
• Delivers linear scalability at
low cost
• Resilient in case of
infrastructure failures
• Transparent application
scalability
All Things Open 2014 - Open Source Cloud Computing
Features
• Handles large amounts of
data
• Stores data in native format
• Delivers linear scalability at
low cost
• Resilient in case of
infrastructure failures
• Transparent application
scalability
55. Machine Learning
By Mark R. Hinkle
@mrhinkle
mrhinkle@gmail.com
APACHE HADOOP ECOSYSTEM
Non-Relational DB
Hadoop Hadoop Common
HDFS
Distributes & replicates data
across machines
All Things Open 2014 - Open Source Cloud Computing
MapReduce
Distributes & monitors tasks
Hive
Data warehouse that
provides SQL interface.
Ad hoc projection of
data structure to
unstructured
MapReduce
• Parallel programming
• Handles large data blocks
HBase
Column-oriented
schema-less distributed
DB modeled after
Google’s BigTable
Random real time
read/write.
Scripting
Pig
Platform for
manipulating and
analyzing large data sets.
Scripting language for
analysts.
Mahout
Machine learning
libraries for
recommendations ,
clustering, classifications
and item sets.
Chuckwa Zookeeper
Editor's Notes
Dashboard of Performance
Openhub has a good graphical representation of code velocity and listing of developers – www.openhub.com
Bitgeria
Bitgeria does number of dashboards.
Top choices for Cloud Computing are Xen and KVM.
OpenVZ, container virtualization for Linux, is an interesting option as it has a very minimal overhead to scale application space similar to containers like BSD Jails. Advantage is that memory allocation is soft and unutilized memory can be used by other applications.
Type 1 Hypervisor – VMware, Xen Project, XenServer, Hyper-V
Type 1 (or native, bare metal) hypervisors run directly on the host's hardware to control the hardware and to manage guest operating systems. A guest operating-system thus runs on another level above the hypervisor.
Type 2 Hypervisor – VM
Type 2 (or hosted) hypervisors run within a conventional operating-system environment. With the hypervisor layer as a distinct second software level, guest operating-systems run at the third level above the hardware. VMware Workstation and VirtualBox exemplify Type 2 hypervisors.
Image portability across hypervisorshttps://www.ibm.com/developerworks/community/blogs/9e696bfa-94af-4f5a-ab50-c955cca76fd0/entry/image_portability_across_hypervisors1?lang=en
Martin Fowler - Continuous Integration
http://www.martinfowler.com/articles/continuousIntegration.html
What is Docker: http://www.docker.com/whatisdocker/
Common use cases for Docker include:
Automating the packaging and deployment of applications
Creation of lightweight, private PAAS environments
Automated testing and continuous integration/deployment
Deploying and scaling web apps, databases and backend services
OpenStack Shared Services - https://www.openstack.org/software/openstack-shared-services/
Identity Service
OpenStack Identity provides a central directory of users mapped to the OpenStack services they can access. It acts as a common authentication system across the cloud operating system and can integrate with existing backend directory services like LDAP. It supports multiple forms of authentication including standard username and password credentials, token-based systems and AWS-style logins.
Image Service
The OpenStack Image Service provides discovery, registration and delivery services for disk and server images. The ability to copy or snapshot a server image and immediately store it away is a powerful capability of the OpenStack cloud operating system. Stored images can be used as a template to get new servers up and running quickly and more consistently if you are provisioning multiple servers than installing a server operating system and individually configuring additional services. It can also be used to store and catalog an unlimited number of backups.
Telemetry Service
The OpenStack Telemetry service aggregates usage and performance data across the services deployed in an OpenStack cloud. This powerful capability provides visibility and insight into the usage of the cloud across dozens of data points and allows cloud operators to view metrics globally or by individual deployed resources.
Orchestration Service
OpenStack Orchestration is a template-driven engine that allows application developers to describe and automate the deployment of infrastructure. The flexible template language can specify compute, storage and networking configurations as well as detailed post-deployment activity to automate the full provisioning of infrastructure as well as services and applications. Through integration with the Telemetry service, the Orchestration engine can also perform auto-scaling of certain infrastructure elements.
OpenStack Shared Services
https://www.openstack.org/software/openstack-shared-services/
Identity Service
OpenStack Identity provides a central directory of users mapped to the OpenStack services they can access. It acts as a common authentication system across the cloud operating system and can integrate with existing backend directory services like LDAP. It supports multiple forms of authentication including standard username and password credentials, token-based systems and AWS-style logins.
Image Service
The OpenStack Image Service provides discovery, registration and delivery services for disk and server images. The ability to copy or snapshot a server image and immediately store it away is a powerful capability of the OpenStack cloud operating system. Stored images can be used as a template to get new servers up and running quickly and more consistently if you are provisioning multiple servers than installing a server operating system and individually configuring additional services. It can also be used to store and catalog an unlimited number of backups.
Telemetry Service
The OpenStack Telemetry service aggregates usage and performance data across the services deployed in an OpenStack cloud. This powerful capability provides visibility and insight into the usage of the cloud across dozens of data points and allows cloud operators to view metrics globally or by individual deployed resources.
Orchestration Service
OpenStack Orchestration is a template-driven engine that allows application developers to describe and automate the deployment of infrastructure. The flexible template language can specify compute, storage and networking configurations as well as detailed post-deployment activity to automate the full provisioning of infrastructure as well as services and applications. Through integration with the Telemetry service, the Orchestration engine can also perform auto-scaling of certain infrastructure elements.
Debate: How Many Open Source Platforms Are Enough?
http://www.enterprisetech.com/2014/06/23/debate-many-open-source-platforms-enough/
OpenStack came in for the most criticism for issues such cost of deployment and maintenance. “OpenStack is not a product,” Kemp responded. “If you are building a large infrastructure, it’s more like a tool kit. It gives you a lot of technologies that do take a lot of effort to integrate.” The tool kit is used to create a product, Kemp stressed.
OpenStack Vendors
Canonical Ubuntu OpenStack - http://www.ubuntu.com/cloud/tools/openstack
CloudScaling – Elastic Cloud Infrastructure - http://www.cloudscaling.com/
Elastic Cloud Infrastructure – built on OpenStack – enables any IT group to deploy cloud services comparable to the capabilities of the world’s largest and most successful public clouds. Cloudscaling solutions allow your organization to rapidly scale resources, achieve new levels of agility and improve market responsiveness. All with full control and governance in the privacy of your on-premise data center.
HP Cloud OS - http://www8.hp.com/us/en/business-solutions/solution.html?compURI=1421776#.UzoD3K1dVDo
Based on OpenStack technology, HP Cloud OS provides the foundation for the HP Cloud common architecture across private, public, and hybrid cloud delivery.
Piston Cloud Computing - http://www.pistoncloud.com/openstack-cloud-software/
Piston OpenStack is a software product that uses advanced systems intelligence to orchestrate an entire private cloud environment using commodity hardware. Starting with an extremely lightweight custom Linux OS called Iocane Micro-OS™, and using an advanced high-availability system called Moxie Runtime Environment™, Piston keeps your cloud running no matter what – through hardware failure, operator error, upgrades, and power outages.
Red Hat Distribution of OpenStack - http://openstack.redhat.com/Main_Page
RDO is a community of people using and deploying OpenStack on Red Hat Enterprise Linux, Fedora and distributions derived from these (such as CentOS, Scientific Linux and others). We have documentation to help get started, forums where you can connect with other users, and community-supported packages of the most up-to-date OpenStack releases available for download.
Rackspace Private Cloud powered by OpenStack - http://www.rackspace.com/cloud/private/
Types of Tasks Accomplished by an API
Provisioning (creating, re-creating, moving, or deleting components e.g. virtual machines, vlans)
Configuration (assigning or changing attributes of the architecture such as security and network settings)
Cloud Providers
Daisein -
Jclouds – java API Abstraction
Libcloud – started by CloudKick (now Rackspace) to abstract clouds, Apache incubator project
Deltacloud – started by Red Hat to abstract clouds, Apache incubator project
Fog - provider and abstraction level API across compute and storage, written in Ruby
Ansible
Ansible's SSH-key based access allows contributors to the Fedora Project to assist in automating infrastructure while having access limited appropriately. Ansible is also used to roll out and manage clusters of machines and ISV software, such as Basho's flagship key-value store Riak.
Capistrano
Capistrano is a developer tool for deploying web applications. It is typically installed on a workstation, and used to deploy code from your source code management (SCM) to one, or more servers.
Capistrano recently added classes capabilities that match cobbler.
RunDeck
RunDeck is cross-platform open source software that helps you automate ad-hoc and routine procedures in data center or cloud environments. RunDeck allows you to run tasks on any number of nodes from a web-based or command-line interface. RunDeck also includes other features that make it easy to scale up your scripting efforts including: access control, workflow building, scheduling, logging, and integration with external sources for node and option data.
Func
Func allows for running commands on remote systems in a secure way, like SSH, but offers several improvements.
Func allows you to manage an arbitrary group of machines all at once.
Func automatically distributes certificates to all "slave" machines. There's almost nothing to configure.
Func comes with a command line for sending remote commands and gathering data.
There are lots of modules already provided for common tasks.
Anyone can write their own modules using the simple Python module API.
Everything that can be done with the command line can be done with the Python client API. The hack potential is unlimited.
You'll never have to use "expect" or other ugly hacks to automate your workflow.
It's really simple under the covers. Func works over XMLRPC and SSL.
Since func uses certmaster, any program can use func certificates, latch on to them, and take advantage of secure master-to-slave communication.
There are no databases or crazy stuff to install and configure. Again, certificate distribution is automatic too.
Mcollective
The Marionette Collective AKA mcollective is a framework to build server orchestration or parallel job execution systems.
Mcollective is used as a means of programmatic execution of Systems Administration actions on clusters of servers.
MCollective use modern tools like Publish Subscribe Middleware and modern philosophies like real time discovery of network resources using meta data and not hostnames. Delivering a very scalable and very fast parallel execution environment.
Scalr
Scalr is a pretty darn good open source cloud management tool. It provides both an automation framework (do Foo when Bar) and a web interface (where is this volume mounted) for managing infrastructure on the cloud, like EC2.
FEATURES
* Integrated into Opscode Chef, for configuration management.
* Pre-automated software, such as nginx, mysql, redis, mongo, and rabbitmq
* Blazing fast UI
* Multi-cloud
* More at http://scalr.net/features/
ROADMAP
* http://wiki.scalr.net/Roadmap
Software Defined Networking (SDN) is an emerging network architecture where network control is decoupled from forwarding and is directly programmable. This migration of control, formerly tightly bound in individual network devices, into accessible computing devices enables the underlying infrastructure to be abstracted for applications and network services, which can treat the network as a logical or virtual entity.
This figure depicts a logical view of the SDN architecture. Network intelligence is (logically) centralized in software-based SDN controllers, which maintain a global view of the network. As a result, the network appears to the applications and policy engines as a single, logical switch. With SDN, enterprises and carriers gain vendor-independent control over the entire network from a single logical point, which greatly simplifies the network design and operation. SDN also greatly simplifies the network devices themselves, since they no longer need to understand and process thousands of protocol standards but merely accept instructions from the SDN controllers.
Open Flow
OpenFlow is an open standard that enables researchers to run experimental protocols in the campus networks we use every day. OpenFlow is added as a feature to commercial Ethernet switches, routers and wireless access points – and provides a standardized hook to allow researchers to run experiments, without requiring vendors to expose the internal workings of their network devices. OpenFlow is currently being implemented by major vendors, with OpenFlow-enabled switches now commercially available.
In a classical router or switch, the fast packet forwarding (data path) and the high level routing decisions (control path) occur on the same device. An OpenFlow Switch separates these two functions. The data path portion still resides on the switch, while high-level routing decisions are moved to a separate controller, typically a standard server. The OpenFlow Switch and Controller communicate via the OpenFlow protocol, which defines messages, such as packet-received, send-packet-out, modify-forwarding-table, and get-stats.
The data path of an OpenFlow Switch presents a clean flow table abstraction; each flow table entry contains a set of packet fields to match, and an action (such as send-out-port, modify-field, or drop). When an OpenFlow Switch receives a packet it has never seen before, for which it has no matching flow entries, it sends this packet to the controller. The controller then makes a decision on how to handle this packet. It can drop the packet, or it can add a flow entry directing the switch on how to forward similar packets in the future.
OpenFlow is the first standard communications interface defined betweenthe control and forwarding layers of an SDN architecture. OpenFlow allows direct access to and manipulation of the forwarding plane of network devices such as switches and routers, both physical and virtual (hypervisor-based). It is the absence of an open interface to the forwarding plane that has led to the characterization of today’s networking devices as monolithic, closed, and mainframe-like. No other standard protocol does what OpenFlow does, and a protocol like OpenFlow is needed to move network control out of the networking switches to logically centralized control software
Floodlight - http://www.projectfloodlight.org/floodlight/
- OpenFlow – works with physical- and virtual- switches that speak the OpenFlow protocol
- Apache-licensed – lets you use Floodlight for almost any purpose Open community
Floodlight is developed by an open community of developers. We welcome code contributions from active participants and we’ll openly share information on project status, roadmap, bugs, etc.
Easy to Use- Floodlight is drop dead simple to build and run. Read through the Documentation (link)
Tested and Supported – Floodlight is the core of a commercial controller product from Big Switch Networks (link) and is actively tested and improved by a community of professional developers.
Indigo - http://www.projectfloodlight.org/indigo/
Indigo is an open source project aimed at enabling support for OpenFlow on physical and hypervisor switches. Big Switch has helped numerous companies OpenFlow enable their equipment, and we provide firmware for a number of popular switches.
Indigo is the basis of Switch Light by Big Switch Networks.
- See more at: http://www.projectfloodlight.org/indigo/#sthash.K7LiHcqc.dpuf
Lincx - https://github.com/FlowForwarding/lincx
LINCX is a pure OpenFlow software switch written in Erlang. It runs within a separate domain under Xen hypervisor using LING (erlangonxen.org).
LINCX is a new faster version of LINC-Switch.
Open Daylight – http://www.opendaylight.com
The adoption of new technologies and pursuit of programmable networks has the potential to significantly improve levels of functionality, flexibility and adaptability of mainstream datacenter architectures. To leverage this abstraction to its fullest requires the network to adapt and evolve to a Software-Defined architecture. One of the architectural elements required to achieve this goal is a Software-Defined-Networking (SDN) platform that enables network control and programmability.
Open vSwitch
Open vSwitch is a production quality, multilayer virtual switch licensed under the open source Apache 2.0 license. It is designed to enable massive network automation through programmatic extension, while still supporting standard management interfaces and protocols (e.g. NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag). In addition, it is designed to support distribution across multiple physical servers similar to VMware's vNetwork distributed vswitch or Cisco's Nexus 1000V. See the full feature list here
Why Open vSwitch - http://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=WHY-OVS;hb=HEAD
Hypervisors need the ability to bridge traffic between VMs and with theoutside world. On Linux-based hypervisors, this used to mean using thebuilt-in L2 switch (the Linux bridge), which is fast and reliable. So,
it is reasonable to ask why Open vSwitch is used.
The answer is that Open vSwitch is targeted at multi-server virtualization deployments, a landscape for which the previous stack is not well suited. These environments are often characterized by highly
dynamic end-points, the maintenance of logical abstractions, and (sometimes) integration with or offloading to special purpose switching hardware.
NetFlix AWS Toolbag – http://netflix.github.com
Over 25 projects developed by NetFlix to manager their cloud deployments.
Asgard
Asgard is a web-based tool for managing cloud-based applications and infrastructure.
Astyanaz
Astyanax is a high level Java client for Apache Cassandra. Apache Cassandra is a highly available column oriented database.
Edda
Edda is a Service to track changes in your cloud deployments.
Eureka
Eureka is a REST (Representational State Transfer) based service that is primarily used in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers.
At Netflix, Eureka is used for the following purposes apart from playing a critical part in mid-tier load balancing.
For aiding Netflix Asgard - an open source service which makes cloud deployments easier, in
Fast rollback of versions in case of problems avoiding the re-launch of 100's of instances which could take a long time.
In rolling pushes, for avoiding propagation of a new version to all instances in case of problems.
For our cassandra deployments to take instances out of traffic for maintenance.
For our memcached caching services to identify the list of nodes in the ring.
Priam
Priam is a process/tool that runs alongside Apache Cassandra to automate the following:
- Backup and recovery (Complete and incremental)
- Token management
- Seed discovery
- Configuration
Support AWS environment
Simian Army
The Simian Army is a suite of tools for keeping your cloud operating in top form. Chaos Monkey, the first member, is a resiliency tool that helps ensure that your applications can tolerate random instance failures
Salt - https://github.com/saltstack/salt
Cacti
Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices.
RRDTool
RRDtool is the OpenSource industry standard, high performance data logging and graphing system for time series data. RRDtool can be easily integrated in shell scripts, perl, python, ruby, lua or tcl applications.
Graphite
Graphite is a highly scalable real-time graphing system. As a user, you write an application that collects numeric time-series data that you are interested in graphing, and send it to Graphite's processing backend, carbon, which stores the data in Graphite's specialized database. The data can then be visualized through graphite's web interfaces.
These tools are all appropriate for Linux guest operating systems, Windows operating system provisioning is not well addressed in OSS.
Axembler Provisonr
Provisionr solves the problem of cloud portability by hiding completely the APIs and only focusing on building a cluster that matches the same set of assumptions on all clouds, assumptions like: a specific OS, pre-installed packages and binaries, sane dns settings, ssh & vpn access etc. - think a solid foundation for configuration.
As a secondary goal Provisionr will also provide primitives for building automatic or semi-automatic workflows for configuring and monitoring services, workflows that assume that all the machines share a common set of characteristics as described above.
Cobbler
Cobbler is a Linux installation server that allows for rapid setup of network installation environments. It glues together and automates many associated Linux tasks so you do not have to hop between lots of various commands and applications when rolling out new systems, and, in some cases, changing existing ones.
With a simple series of commands, network installs can be configured for PXE, reinstallations, media-based net-installs, and virtualized installs (supporting Xen, qemu, KVM, and some variants of VMware). Cobbler uses a helper program called 'koan' (which interacts with Cobbler) for reinstallation and virtualization support.
Crowbar
Bare metal provisioning for CloudStack developed by Dell using Opscode Chef.
Juju
Metal as a Service (MAAS)
MAAS offers a nice UI to provision your Ubuntu servers. Each physical server (“node”) will be commissioned automatically on first boot. During the commissioning process administrators are able to configure hardware settings manually before an automated smoke test and burn-in test are done. Once commissioned, a node can be deployed on demand by name, or allocated to a queue for dynamic allocation to services being deployed on this MAAS.
Salt Cloud
Salt Cloud is a tool for provisioning salted minions across various cloud providers. Currently supported providers are:
- Amazon EC2
- GoGrid
- HP Cloud (using OpenStack)
- Joyent
- Linode
- OpenStack
- Rackspace (using OpenStack)
The salt-cloud command can be used to query configured providers, create VMs on them, deploy salt-minion on those VMs and destroy them when no longer needed.
Salt Cloud requires Salt to be installed, but does not require any Salt daemons to be running. However, if used in a salted environment, it is best to run Salt Cloud on the salt-master, so that it can properly lay down salt keys when it deploys machines, and then properly remove them later. If Salt Cloud is run in this manner, minions will automatically be approved by the master; no need to manually authenticate them later.
Deprecated
Spacewalk
Spacewalk manages software content updates for Red Hat derived distributions such as Fedora, CentOS, and Scientific Linux, within your firewall. You can stage software content through different environments, managing the deployment of updates to systems and allowing you to view at which update level any given system is at across your deployment. A clean central web interface allows viewing of systems and their software update status, and initiating update actions.
Big data the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, curation, storage, search, sharing, transfer, analysis[4] and visualization. The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data, as compared to separate smaller sets with the same total amount of data, allowing correlations to be found to "spot business trends, determine quality of research, prevent diseases, link legal citations, combat crime, and determine real-time roadway traffic conditions.
NoSQL
In computing, NoSQL (commonly interpreted as "not only SQL"[1]) is a broad class of database management systems identified by non-adherence to the widely used relational database management system model. NoSQL databases are not built primarily on tables, and generally do not use SQL for data manipulation.
NoSQL database systems are often highly optimized for retrieval and appending operations and often offer little functionality beyond record storage (e.g. key–value stores). The reduced run-time flexibility compared to full SQL systems is compensated by marked gains in scalability and performance for certain data models.
Apache Cassandra
The Apache Cassandra database is the right choice when you need scalability and high availability without compromising performance. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data. Cassandra's support for replicating across multiple datacenters is best-in-class, providing lower latency for your users and the peace of mind of knowing that you can survive regional outages.
Cassandra's ColumnFamily data model offers the convenience of column indexes with the performance of log-structured updates, strong support for materialized views, and powerful built-in caching.
Cassandra is in use at Netflix, Twitter, Urban Airship, Constant Contact, Reddit, Cisco, OpenX, Digg, CloudKick, Ooyala, and more companies that have large, active data sets. The largest known Cassandra cluster has over 300 TB of data in over 400 machines.
Hypertable
Hypertable is based on a design developed by Googl(e.g. BigTable clone) to meet their scalability requirements and solves the scale problem better than any of the other NoSQL solutions out there.
Mongo DB
MongoDB (from "humongous") is a cross-platform document-oriented database system.
Redis
Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.
Riak
Riak is known for its ability to distribute data across nodes using consistent hashing in a simple key/value scheme in namespaces called buckets.
MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.
A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies). The "MapReduce System" (also called "infrastructure" or "framework") orchestrates by marshalling the distributed servers, running the various tasks in parallel, managing all communications and data transfers between the various parts of the system, and providing for redundancy and fault tolerance.