Sebastien Goasguen is a developer who works on Apache CloudStack and other open source projects related to cloud computing, containers, and big data. He gave a talk covering CloudStack, Docker, Kubernetes, CoreOS, and how various technologies can work together for managing distributed applications and infrastructure. He also discussed the evolving landscape of cloud computing and how big data solutions fit within that landscape.
Thoughts about how to use CloudStack to develop a Big Data Solution in your data center.
Cloud as a virtual machine infrastructure and Big Data are converging as two key technical evolution in the data center. Virtualization enables multi-tenancy, heterogenous Operating systems and added security via isolation. Clouds like AWS EC2, Rackspace, or google GCE are good examples. Big Data tackles the challenge of the increase scale (amount) and complexity (type) of data faced in the enterprise. While Compute cloud show a departure from traditional hardware provisioning and configuration management via virtualization, big data is a departure from traditional relational databases and file systems. These two technical evolutions have been triggered by the new workloads of the internet (search, streaming) and the scale needed to server millions of users and millions/billions of objects to store or serve.
In this talk we show how CloudStack and its support for bare-metal provisioning is compatible with a public cloud. CloudStack being a data center orchestrator that can tackle both traditional enterprise workloads and internet scale/type workloads. Multiple zones can be created for compute cloud or big data. Big data can used as backend store to the compute cloud or as zone type to enabled big data workload on the bare metal hardware.
This hybrid mode of operation is seen as the next evolution of clouds and positions a data center orchestrator has more than a VM management system and a solution to big data management as well.
Sebastien Goasguen. With VM seemingly taking a back seat with containers coming back in fame, what is the role of CloudStack or OpenStack. In this talk Sebastien will briefly review the state of the art and bring some context around container orchestrators and how they relate to CloudStack. He will then discuss how containers orchestration can be easily integrated in CloudStack.
Guaranteeing Storage Performance by Mike Tutkowskibuildacloud
This session will introduce the basics of primary storage in CloudStack. Additionally, I discuss the challenges of guaranteeing storage performance in a cloud and how by leveraging the latest enhancements to CloudStack, storage administrators can deliver consistent, repeatable performance to 10s, 100s or 1,000s of application workloads in parallel. I'll review the CloudStack enhancements in detail, outline the management benefits they provide and discuss common go-to-market approaches.
About Mike Tutkowski
Mike Tutkowski, a member of the CloudStack PMC, develops software for the Apache Software Foundation's CloudStack project to help drive improvements in its storage component and to integrate SolidFire more deeply into the product.
Jenkins, jclouds, CloudStack, and CentOS by David Nalleybuildacloud
Setting up continuous integration for a single project can be a pretty daunting task. Doing that for hundreds of projects becomes a challenge of a different magnitude. Not only are their capacity problems, but some tests are destructive to the testing environment, some have esoteric environment demands. See how this is solved in the real world using Jenkins, jclouds, CloudStack to build an on-demand build infrastructure.
About David Nalley
David Nalley is the Vice President, Infrastructure at the Apache Software Foundation and a CloudStack PMC member.
Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CSJohn Burwell
Software is eating infrastructure. By pulling reliability and scalability responsibilities up the stack from hardware into software, object stores such as Basho's Riak CS and cloud orchestration platforms such as Apache CloudStack increase the utilization of compute and storage resources by dynamically shifting workloads based on demand. Together, those platforms can saturate compute and storage of 1000s of hosts with strong operational visibility and end-user self-service.
This talk will cover the following topics to explore private cloud design principles and best practices:
* Why Private Cloud?
* Anatomy of a Private Cloud
* Building a Apache CloudStack Compute Offering
* Large Object Storage using Riak CS
Adam Dagnall: Advanced S3 compatible storage integration in CloudStackShapeBlue
Adam's slides from his talk at the CloudStack European User group meetup, March 13, London. To provide tighter integration between the S3 compatible object store and CloudStack, Cloudian has developed a connector to allow users and their applications to utilize the object store directly from within the CloudStack platform in a single sign-on manner with self-service provisioning. Additionally, CloudStack templates and snapshots are centrally stored within the object store and managed through the CloudStack service. The object store offers protection of these templates and snapshots across data centres using replication or erasure coding.
Thoughts about how to use CloudStack to develop a Big Data Solution in your data center.
Cloud as a virtual machine infrastructure and Big Data are converging as two key technical evolution in the data center. Virtualization enables multi-tenancy, heterogenous Operating systems and added security via isolation. Clouds like AWS EC2, Rackspace, or google GCE are good examples. Big Data tackles the challenge of the increase scale (amount) and complexity (type) of data faced in the enterprise. While Compute cloud show a departure from traditional hardware provisioning and configuration management via virtualization, big data is a departure from traditional relational databases and file systems. These two technical evolutions have been triggered by the new workloads of the internet (search, streaming) and the scale needed to server millions of users and millions/billions of objects to store or serve.
In this talk we show how CloudStack and its support for bare-metal provisioning is compatible with a public cloud. CloudStack being a data center orchestrator that can tackle both traditional enterprise workloads and internet scale/type workloads. Multiple zones can be created for compute cloud or big data. Big data can used as backend store to the compute cloud or as zone type to enabled big data workload on the bare metal hardware.
This hybrid mode of operation is seen as the next evolution of clouds and positions a data center orchestrator has more than a VM management system and a solution to big data management as well.
Sebastien Goasguen. With VM seemingly taking a back seat with containers coming back in fame, what is the role of CloudStack or OpenStack. In this talk Sebastien will briefly review the state of the art and bring some context around container orchestrators and how they relate to CloudStack. He will then discuss how containers orchestration can be easily integrated in CloudStack.
Guaranteeing Storage Performance by Mike Tutkowskibuildacloud
This session will introduce the basics of primary storage in CloudStack. Additionally, I discuss the challenges of guaranteeing storage performance in a cloud and how by leveraging the latest enhancements to CloudStack, storage administrators can deliver consistent, repeatable performance to 10s, 100s or 1,000s of application workloads in parallel. I'll review the CloudStack enhancements in detail, outline the management benefits they provide and discuss common go-to-market approaches.
About Mike Tutkowski
Mike Tutkowski, a member of the CloudStack PMC, develops software for the Apache Software Foundation's CloudStack project to help drive improvements in its storage component and to integrate SolidFire more deeply into the product.
Jenkins, jclouds, CloudStack, and CentOS by David Nalleybuildacloud
Setting up continuous integration for a single project can be a pretty daunting task. Doing that for hundreds of projects becomes a challenge of a different magnitude. Not only are their capacity problems, but some tests are destructive to the testing environment, some have esoteric environment demands. See how this is solved in the real world using Jenkins, jclouds, CloudStack to build an on-demand build infrastructure.
About David Nalley
David Nalley is the Vice President, Infrastructure at the Apache Software Foundation and a CloudStack PMC member.
Better, Faster, Cheaper Infrastructure: Apache CloudStack and Riak CSJohn Burwell
Software is eating infrastructure. By pulling reliability and scalability responsibilities up the stack from hardware into software, object stores such as Basho's Riak CS and cloud orchestration platforms such as Apache CloudStack increase the utilization of compute and storage resources by dynamically shifting workloads based on demand. Together, those platforms can saturate compute and storage of 1000s of hosts with strong operational visibility and end-user self-service.
This talk will cover the following topics to explore private cloud design principles and best practices:
* Why Private Cloud?
* Anatomy of a Private Cloud
* Building a Apache CloudStack Compute Offering
* Large Object Storage using Riak CS
Adam Dagnall: Advanced S3 compatible storage integration in CloudStackShapeBlue
Adam's slides from his talk at the CloudStack European User group meetup, March 13, London. To provide tighter integration between the S3 compatible object store and CloudStack, Cloudian has developed a connector to allow users and their applications to utilize the object store directly from within the CloudStack platform in a single sign-on manner with self-service provisioning. Additionally, CloudStack templates and snapshots are centrally stored within the object store and managed through the CloudStack service. The object store offers protection of these templates and snapshots across data centres using replication or erasure coding.
Paul Angus - CloudStack Container ServiceShapeBlue
A walkthrough of the recently released update to ShapeBlue’s CloudStack Container Service (CCS). This update brings CCS bang up-to-date by running the latest version of Kubernetes (v1.11.3) on the latest version of Container Linux. CCS also now makes use of CloudStack’s new CA framework to automatically secure the Kubernetes environments it creates.
This session will examine the many options the data scientist has for running Spark clusters in public and private clouds. We will discuss various environments employing AWS, Mesos, containers, docker, and BlueData EPIC technologies and the benefits and challenges of each.
Speakers:
Tom Phelan, Co-founder and Chief Architect - BlueData Inc. Tom has spent the last 25 years as a senior architect, developer, and team lead in the computer software industry in Silicon Valley. Prior to co-founding BlueData, Tom spent 10 years at VMware as a senior architect and team lead in the core R&D Storage and Availability group. Most recently, Tom led one of the key projects – vFlash, focusing on integration of server-based Flash into the vSphere core hypervisor. Prior to VMware, Tom was part of the early team at Silicon Graphics that developed XFS, one of the most successful open source file systems. Earlier in his career, he was a key member of the Stratus team that ported the Unix operating system to their highly available computing platform. Tom received his Computer Science degree from the University of California, Berkeley.
The CloudStack usage service is used to track consumption of resources in Apache CloudStack for reporting and billing purposes. This talk will give an overview of the service before diving deeper into how data is processed from the CloudStack database into the different usage types before being aggregated into billable units or time slices in the usage database.
For this upcoming meetup, we welcome Patrick Eaton PhD, Systems Architect at Stackdriver, and Joey Imbasciano, Cloud Platform Engineer at Stackdriver.
What You'll Learn At This Meetup:
• Why Stackdriver chose Cassandra over other DB offerings
• Stackdriver's data pipeline that runs into Cassandra
• Operating Cassandra Running on AWS
• Stackdriver's approach to disaster recovery
Patrick and Joey will be presenting their use of Apache Cassandra at Stackdriver, some lesson's learned, technical tips and a Q&A to end the evening.
The Future of SDN in CloudStack by Chiradeep Vittalbuildacloud
The core of CloudStack networking has always been software-defined. As the networking industry evolves to a software-defined future, CloudStack will have to evolve with it.
The presentation will examine the present state of SDN in CloudStack, look at some industry directions and attempt to predict the evolution of CloudStack with those trends.
Bio
Chiradeep Vittal is a Distinguished Engineer in the Converged Infrastructure Group at Citrix where he has technology leadership responsibilities around Citrix Cloud Platform, Citrix Lifecycle Manager and Citrix Workspace Pod. He is also a Project Management Committee member of the Apache CloudStack Project. At cloud.com (acquired by Citrix), he was a founding engineer, often tasked with the thorny details of virtualized networking and storage. Prior to cloud.com, he worked at several Silicon Valley startups in various architectural roles.
Chiradeep has a B.Tech in Computer Science from IIT, Bombay and a M.Sc from the University of Alberta. He has spoken / presented at several conferences, including CloudStack Collab, LISA, OSCON, ONS, SDN Summit and LinuxCon. His twitter handle is @chiradeep and occasionally blogs at http://cloudierthanthou.wordpress.com
Paul Angus - CloudStack Container ServiceShapeBlue
A walkthrough of the recently released update to ShapeBlue’s CloudStack Container Service (CCS). This update brings CCS bang up-to-date by running the latest version of Kubernetes (v1.11.3) on the latest version of Container Linux. CCS also now makes use of CloudStack’s new CA framework to automatically secure the Kubernetes environments it creates.
This session will examine the many options the data scientist has for running Spark clusters in public and private clouds. We will discuss various environments employing AWS, Mesos, containers, docker, and BlueData EPIC technologies and the benefits and challenges of each.
Speakers:
Tom Phelan, Co-founder and Chief Architect - BlueData Inc. Tom has spent the last 25 years as a senior architect, developer, and team lead in the computer software industry in Silicon Valley. Prior to co-founding BlueData, Tom spent 10 years at VMware as a senior architect and team lead in the core R&D Storage and Availability group. Most recently, Tom led one of the key projects – vFlash, focusing on integration of server-based Flash into the vSphere core hypervisor. Prior to VMware, Tom was part of the early team at Silicon Graphics that developed XFS, one of the most successful open source file systems. Earlier in his career, he was a key member of the Stratus team that ported the Unix operating system to their highly available computing platform. Tom received his Computer Science degree from the University of California, Berkeley.
The CloudStack usage service is used to track consumption of resources in Apache CloudStack for reporting and billing purposes. This talk will give an overview of the service before diving deeper into how data is processed from the CloudStack database into the different usage types before being aggregated into billable units or time slices in the usage database.
For this upcoming meetup, we welcome Patrick Eaton PhD, Systems Architect at Stackdriver, and Joey Imbasciano, Cloud Platform Engineer at Stackdriver.
What You'll Learn At This Meetup:
• Why Stackdriver chose Cassandra over other DB offerings
• Stackdriver's data pipeline that runs into Cassandra
• Operating Cassandra Running on AWS
• Stackdriver's approach to disaster recovery
Patrick and Joey will be presenting their use of Apache Cassandra at Stackdriver, some lesson's learned, technical tips and a Q&A to end the evening.
The Future of SDN in CloudStack by Chiradeep Vittalbuildacloud
The core of CloudStack networking has always been software-defined. As the networking industry evolves to a software-defined future, CloudStack will have to evolve with it.
The presentation will examine the present state of SDN in CloudStack, look at some industry directions and attempt to predict the evolution of CloudStack with those trends.
Bio
Chiradeep Vittal is a Distinguished Engineer in the Converged Infrastructure Group at Citrix where he has technology leadership responsibilities around Citrix Cloud Platform, Citrix Lifecycle Manager and Citrix Workspace Pod. He is also a Project Management Committee member of the Apache CloudStack Project. At cloud.com (acquired by Citrix), he was a founding engineer, often tasked with the thorny details of virtualized networking and storage. Prior to cloud.com, he worked at several Silicon Valley startups in various architectural roles.
Chiradeep has a B.Tech in Computer Science from IIT, Bombay and a M.Sc from the University of Alberta. He has spoken / presented at several conferences, including CloudStack Collab, LISA, OSCON, ONS, SDN Summit and LinuxCon. His twitter handle is @chiradeep and occasionally blogs at http://cloudierthanthou.wordpress.com
Compare Docker deployment options in the public cloudSreenivas Makam
Compare Docker public cloud deployment options using Docker machine, Docker Cloud, Docker datacenter, Docker for AWS, Azure and Google cloud, AWS ECS, Google Container engine, Azure Container service.
Kubernetes – An open platform for container orchestrationinovex GmbH
Datum: 30.08.2017
Event: GridKA School 2017
Speaker: Johannes M. Scheuermann
Mehr Tech-Vorträge: https://www.inovex.de/de/content-pool/vortraege/
Mehr Tech-Artikel: https://www.inovex.de/blog/
Intro to coreOS linux distributions and how it can be used to run docker based workloads in the cloud.
coreOS instances can be started in a cloudstack cloud, it makes use of cloud-init basics to
Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet
A session in the DevNet Zone at Cisco Live, Berlin. Confused by all the open source PaaS options out there? What criteria should you use to evaluate them? We seek to answer these questions in a systematic manner and will explore top technologies such as Mesos, Apprenda, Cloud Foundry and Kubernetes along with Cisco's Project Shipped and open source Mantl. The aim of this session will be to shed light on which platforms add value to your needs, applications and workloads.
Adopting Docker for production applications and services used to be hard. You had to hand-roll a lot of the underlying infrastructure and write lots of custom code for service discovery, load balancing, orchestration, desired state, etc. Today, with the rise of open source container orchestration platforms and cloud-native offerings, it's a lot easier to get up and running.
Github repo for demo: https://github.com/elabor8/dockertalk
Watch this presentation and learn about Kubernetes Networking:
How to build applications without knowing subnets & IP addresses and build modern cloud-friendly applications in an agile fashion.
OpenNebula Conf 2014 | Cloud Automation for OpenNebula by Kishorekumar Neelam...NETWAYS
Kishore works with the engineering team in building the open source product with a future focussed cloud technical strategy for “Megam – Cloud Automation Platform “http://gomegam.com”. In his prior incarnation Kishore has worked as an Architect in complex system integration projects for Airport systems with high availability. Kishore has avid experience in architecting large scale build and packaging tools for mainframe platform integrated via thin clients and eclipse IDE.
Similar to On CloudStack, Docker, Kubernetes, and Big Data…Oh my ! By Sebastien Goasguen, @sebgoa (20)
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
2. Who am I ?
• Joined Citrix OSS team in July 2012
• Associate professor at Clemson
University prior
• High Performance Computing, Grid
computing
• At CERN summer 2009/2010, built their
first cloud on opennebula
• http://sebgoa.blogspot.com
@sebgoa
3. • Apache CloudStack and licloud committer + PMC
member
• Looking at techs and how they work together
• Half dev, half community manager, + half event planner
What do I do ?
16. Goals
• Utility computing
• Elasticity of the infrastructure
• On-demand
• Pay as you go
• Multi-tenant
• Programmable access
17. So what…
Let’s assume this is solved.
What is not solved:
- Application deployment
- Application scalability
- Application portability
- Application composability
27. CoreOS
• Linux distribution
• Rolling upgrades
• Minimal OS
• Docker support
• etcd and fleet tools
to manage distributed
applications based on
containers.
• Cloud-init support
• Systemd units
36. CoreOS clustering
etcd HA key value store
• Raft election algorithm
• Writes when majority in cluster has committed
update
• e.g 5 nodes, tolerates 2 nodes failure
fleet distributed init system (schedules
systemd units in a cluster)
• Submits systemd units cluster wide
• Affinity, anti-affinity, global “scheduling”
53. Kubernetes on CloudStack
Find a CloudStack cloud that supports
CoreOS
Then use:
https://github.com/runseb/ansible-kubernetes
Based on the Ansible cloudstack module
54. Cloud API
Libcloud startup
scripts
Etcd cluster
5 nodes
Discovery service to
bootstrap
Kubernetes cluster
5 nodes
Start Kube* services via fleet
Run guestbook example
PR welcome:
https://github.com/runseb/
kubernetes-exoscale
OLD WAY
55. Cloud (e.g CloudStack based = exoscale)
coreOS coreOS coreOS
K* K* K*
Docker
container
Docker
container
Docker
container
API calls to
Kubernetes API
66. New Distributed systems for:
Large scale datasets
• From scientific instruments
• From Web apps logs
Complex datasets
• Not necessarily large.
Object stores
• S3 clones
67. BigData and map-reduce
• While BigData is often associated with HDFS,
Map-Reduce is the algorithm used to
parallelize data processing.
• BigData ≠ Map-Reduce ≠ HDFS
• Map-reduce is a way to express
embarrassingly parallel work easily.
• You can do Map-Reduce without HDFS.
• e.g Basho map-reduce on riackCS
80. Clouds and BigData
• Object store + compute IaaS to build EC2+S3
clone
• BigData solutions as storage backends for
image catalogue and large scale instance
storage.
• BigData solutions as workloads to CloudStack
based clouds.
81. EC2, S3 clone
• An open source IaaS with an EC2
wrapper e.g Opennebula, CloudStack
• Deploy a S3 compatible object store –
separately- e.g riakCS
• Two independent distributed systems
deployed
Cloud = EC2 + S3
82. Big Data
as IaaS backend
“Big Data” solutions can be used as secondary
storage in CloudStack
.
83. Example
• Open source IaaS + EC2 wrapper, e.g
CloudStack
• Deploy S3 compatible object store, e.g
riakCS or Ceph or glusterFS
• Use S3 as image store
• Your EC2 service is a customer to your S3
service
+ Logstash + elasticsearch for logs/
monitoring
85. A note on Scheduling
• Core problem of computer science
• knapsack is NP complete
• Central scheduling has been used for a long
time in HPC
• Optimizing the cluster utilization requires
multi-level scheduling (e.g backfill,
preemption etc..)
• Google Omega paper 2013
• Mesos 2009/2011, ASF Dec 2011
87. Food for thought
Mesos Framework for managing VM ?
Workload sharing in your data-center:
• Big Data
• VM
• Services
Cloud and BigData
88. Conclusions
• Big Data is “catching up”
• Tackle the “big three” head on:
• BigData, Cloud and DevOps
• Add a big data backend to your cloud
from the start
• Provide Big Data services on your cloud
90. Get Involved with Apache
CloudStack
Web: http://cloudstack.apache.org/
Mailing Lists: cloudstack.apache.org/mailing-lists.html
IRC: irc.freenode.net: 6667 #cloudstack #cloudstack-dev
Twitter: @cloudstack
LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859
If it didn’t happen on the mailing list, it didn’t happen.
91. The Velocity Conference
Santa Clara, May 27-29
• 2 days of keynotes & sessions
• 1 day of tutorials
• New full-day trainings
• Amazing presenters – Jez Humble,
Patrick Meenan, Mesosphere, Fastly &
more
Use discount code
CLOUDSTACK20 during
registration for 20% off
http://velocityconf.com/velocity2015