Paul Angus (ShapeBlue) - Push infrastructure with Ansible #DOXLONOutlyer
Ansible is one of the new breed of tools that encompasses configuration management, orchestration and software defined infrastructure. Find out how many companies are spinning up entire environments from source code including vm's, networks, dns, firewalls, load balancers etc.
Video: https://www.youtube.com/watch?v=unPVe2pcego
Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London
Follow DOXLON on twitter http://www.twitter.com/doxlon
This talk will compare the major cloud hosting companies and what products and services they offer. Google's App Engine, Amazon's AWS, Rackspace's Cloud Services and Linode will be compared. We will go beyond a mere checklist of features and dive into such topics as Perl support, cost structures, development strategies, underlying architectures, performance and security.
Presentation at Apache CloudStack Collaboration Conference Europe in Amsterdam on 21st November 2013.
http://lanyrd.com/2013/cloudstack-collaboration-conference-europe/scrdzm/
Paul Angus (ShapeBlue) - Push infrastructure with Ansible #DOXLONOutlyer
Ansible is one of the new breed of tools that encompasses configuration management, orchestration and software defined infrastructure. Find out how many companies are spinning up entire environments from source code including vm's, networks, dns, firewalls, load balancers etc.
Video: https://www.youtube.com/watch?v=unPVe2pcego
Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London
Follow DOXLON on twitter http://www.twitter.com/doxlon
This talk will compare the major cloud hosting companies and what products and services they offer. Google's App Engine, Amazon's AWS, Rackspace's Cloud Services and Linode will be compared. We will go beyond a mere checklist of features and dive into such topics as Perl support, cost structures, development strategies, underlying architectures, performance and security.
Presentation at Apache CloudStack Collaboration Conference Europe in Amsterdam on 21st November 2013.
http://lanyrd.com/2013/cloudstack-collaboration-conference-europe/scrdzm/
Google Cloud Platform, Compute Engine, and App EngineCsaba Toth
Introduction to Google Cloud Platform's compute section, Google Compute Engine, Google App Engine. Place these technologies into the cloud service stack, and later show how Google blurs the boundaries of IaaS and PaaS.
This presentation covers best practices for running MongoDB on AWS. We also discuss how to utilize the automation features of MMS to spin up new clusters in minutes on AWS.
June 2015 Berlin Buzzwords Presentation
http://berlinbuzzwords.de/file/bbuzz-2015-szehon-ho-hive-spark
https://berlinbuzzwords.de/session/hive-spark
Speaker Interview:
https://berlinbuzzwords.de/news/speaker-interview-szehon-ho
How does GROU.PS scale to serving 1PB of assets each month. memcache, nginx, gearman, tornado, libevent, kqueue, epoll, mysql, sharding, replication, memcached, tokyo cabinet
Find out more about:
• Techniques and tips to optimize trainingon Apache MXNet
• Infrastructure performance:storage and I/O, GPU throughput, distributed training, CPU-based training, cost
• Model performance:data augmentation, initializers, optimizers, etc.
• Level 666: you should be familiar with Deep Learning and MXNet
This technical update shows the new and noteworthy in GigaSpaces XAP 7.0.
It explains how the combination of a state-of-the-art in-memory data grid, a Jetty web container, and a grid-based business logic execution framework, forms a single, easy-to -use platform on which you can build and run extremely scalable applications, ranging from transactional systems through large-scale web applications to SaaS-delivered services.
Key release highlights covered in this webinar include dramatic performance and scalability improvements, new monitoring and administration tools, and new data grid APIs.
We All Live in a Yellow (Serverless) SubmarineFITC
The term “serverless” is finding it’s way into everyday development, but is it more than just a buzzword? In this talk, Luke will go over the idea of serverless, and how it can be useful to modern day web developers with real examples of where REDspace has managed to be successful with it.
Luke will also give some code examples, talk about some “serverless” libraries and review some of the pitfalls of going serverless, and where we might see it going in the future.
Objective
To understand what the serverless web is all about, and how it can be useful.
Target Audience
Developers who are interested in the serverless revolution.
Assumed Audience Knowledge
Basic JavaScript
Level
Beginner
Five Things Audience Members Will Learn
What is serverless?
The benefits of going serverless
How to deploy a serverless function to AWS
When not to go serverless
Options for migrating apps to serverless
Using Ansible to deploy a 6-node Hortonworks Data Platform (hadoop) cluster on AWS with the ObjectRocket ansible-hadoop playbook.
Presented at the Ansible NOVA MeetUp on February 23, 2017: https://www.meetup.com/Ansible-NOVA/events/236853616/
Save up to 50% Off on Authentic Egyptian Artifacts from our 2016 Extravaganza Sale.
Sadigh Gallery's latest sale flyer features over 600 various forms of Egyptian antiquities that date from the First Intermediate to the Ptolemaic Period.
Save up to 50% Off on items including statues, funerary items, ushabtis, scarabs, plaques, steles, sandstone reliefs, pottery and so much more.
All orders are by phone only. Call us Toll Free at 1(800)426-2007 or 1(212)725-7537 to place your order. For faster service, please have the item and page number ready. Thank you.
Google Cloud Platform, Compute Engine, and App EngineCsaba Toth
Introduction to Google Cloud Platform's compute section, Google Compute Engine, Google App Engine. Place these technologies into the cloud service stack, and later show how Google blurs the boundaries of IaaS and PaaS.
This presentation covers best practices for running MongoDB on AWS. We also discuss how to utilize the automation features of MMS to spin up new clusters in minutes on AWS.
June 2015 Berlin Buzzwords Presentation
http://berlinbuzzwords.de/file/bbuzz-2015-szehon-ho-hive-spark
https://berlinbuzzwords.de/session/hive-spark
Speaker Interview:
https://berlinbuzzwords.de/news/speaker-interview-szehon-ho
How does GROU.PS scale to serving 1PB of assets each month. memcache, nginx, gearman, tornado, libevent, kqueue, epoll, mysql, sharding, replication, memcached, tokyo cabinet
Find out more about:
• Techniques and tips to optimize trainingon Apache MXNet
• Infrastructure performance:storage and I/O, GPU throughput, distributed training, CPU-based training, cost
• Model performance:data augmentation, initializers, optimizers, etc.
• Level 666: you should be familiar with Deep Learning and MXNet
This technical update shows the new and noteworthy in GigaSpaces XAP 7.0.
It explains how the combination of a state-of-the-art in-memory data grid, a Jetty web container, and a grid-based business logic execution framework, forms a single, easy-to -use platform on which you can build and run extremely scalable applications, ranging from transactional systems through large-scale web applications to SaaS-delivered services.
Key release highlights covered in this webinar include dramatic performance and scalability improvements, new monitoring and administration tools, and new data grid APIs.
We All Live in a Yellow (Serverless) SubmarineFITC
The term “serverless” is finding it’s way into everyday development, but is it more than just a buzzword? In this talk, Luke will go over the idea of serverless, and how it can be useful to modern day web developers with real examples of where REDspace has managed to be successful with it.
Luke will also give some code examples, talk about some “serverless” libraries and review some of the pitfalls of going serverless, and where we might see it going in the future.
Objective
To understand what the serverless web is all about, and how it can be useful.
Target Audience
Developers who are interested in the serverless revolution.
Assumed Audience Knowledge
Basic JavaScript
Level
Beginner
Five Things Audience Members Will Learn
What is serverless?
The benefits of going serverless
How to deploy a serverless function to AWS
When not to go serverless
Options for migrating apps to serverless
Using Ansible to deploy a 6-node Hortonworks Data Platform (hadoop) cluster on AWS with the ObjectRocket ansible-hadoop playbook.
Presented at the Ansible NOVA MeetUp on February 23, 2017: https://www.meetup.com/Ansible-NOVA/events/236853616/
Save up to 50% Off on Authentic Egyptian Artifacts from our 2016 Extravaganza Sale.
Sadigh Gallery's latest sale flyer features over 600 various forms of Egyptian antiquities that date from the First Intermediate to the Ptolemaic Period.
Save up to 50% Off on items including statues, funerary items, ushabtis, scarabs, plaques, steles, sandstone reliefs, pottery and so much more.
All orders are by phone only. Call us Toll Free at 1(800)426-2007 or 1(212)725-7537 to place your order. For faster service, please have the item and page number ready. Thank you.
How to Build High Value Relationships Under Theory ZAnne Kohler
As we enter into commercial relationships with our trading partners, we tend to spend much more time working on the pre-nuptial agreement and less time on establishing the kind of relationship that will lead to a successful marriage. Prior to executing a contract, NO Value has been exchanged and we would argue that the process used to get to the contract has actually destroyed value because behaviors displayed during the pre-nuptial phase (usually adversarial) will set the tone for the relationship long-term.
If we believe that our commercial relationships - whether we are on the buy side or sell side are CRITICAL to our success then our approach to Building relationships must change. Establishing strong relationships from the start can be a competitive differentiator for your company and should be THE primary role of the contracting professional. We should be striving to be either a 'Customer of Choice' (buy-side) or a 'Supplier of Choice' (sell-side) because there is significant inherent value in building that kind of relationship.
Those designations can result in getting the best resources, tapping into innovation, help in entering markets, etc. which can create additional value for our organizations. In Part II of this two part series, Anne will explore a unique approach on how to Build (not manage) sustainable relationships with your trading partners. This is a session you won't want to miss. In addition, if you are interested in joining a small working group to address these issues, that opportunity will be available to you.
Great Deals on Authentic Ancient Artifacts, Coins and Jewelry from various cultures. Save up to 50% OFF on ancient Egyptian antiquities such as statues, scarabs, wall fragments, funerary items, ushabtis and more.
Asian, Pre-Columbian, Roman and Holy Land Antiquities can also be found in this flyer.
All items guaranteed authentic and come with a Lifetime Certificate of Authenticity.
Presentación Crowdfunding para proyectos de innovación culturalColaBoraBora
Presentación del informe "Modelos de financiación alternativa. Crowdfunding para proyectos de innovación cultural", editado por ColaBoraBora para Karraskan.
Centro Cultural Montehermoso (Vitoria-Gasteiz), 15 de diciembre 2015.
+ info http://www.karraskan.org/taller-financiacion/
In PostgreSQL kann man sich mit "explain" ansehen, welchen Execution Plan PostgreSQL für eine Query verwendet. Das hilft beim Suchen von Performance Problemen und hilft, den Durchsatz der Database zu steigern.
Crowdfunding para proyectos de innovación cultural. Karraskan/ColaBoraBoraColaBoraBora
Un informe realizado por ColaBoraBora para Karraskan. Bilbao (Bizkaia), Abril 2015.
Este informe es un encargo de KARRASKAN, una asociación profesional para el fomento de la innovación en cultura y la cultura de la innovación en Euskadi. http://www.karraskan.org/. El objetivo de este colectivo por realizar un estudio en torno a modelos de financiación alternativos en cultura, parte de la necesidad detectada desde el sector de abrir vías de financiación en cultura que impliquen a nuevos agentes, y analizar los efectos que esto puede tener sobre la propia producción y difusión cultural, en cuanto a nuevas formas de entender los públicos y comunidades afectadas, los procesos de pre-producción y comunicación, etc.
Managing your Hadoop Clusters with Apache AmbariDataWorks Summit
Deploying, configuring, and managing large Apache Hadoop and HBase clusters can be quite complex. Once you have your clusters, keeping them up and running and making sure that the SLAs are met presents even more challenges and headaches to Hadoop operators. To make matters worse, managing upgrades can be a nightmare. Hadoop users are presented with their own fair share of difficulties such as slow running jobs and not knowing why they are slow. For third-party software vendors interested in incorporating Hadoop management and monitoring capabilities, there does not seem to be an obvious, easy solution. Apache Ambari is aimed at making lives of Hadoop operators, users, and integrators simpler by providing a management interface to do all of that and more. This session presents usages of Ambari`s Web UI for Hadoop operators (deploying, managing, and monitoring) as well as Hadoop users (job analytics). The talk will also touch upon Ambari`s REST API and how it is used in the real world. The session concludes by revealing the future roadmap of Ambari including queue management, upgrade, disaster recovery, high availability, and more.
Apache Ambari provides a 100% open source and intuitive set of tools to monitor, manage and efficiently provision your Apache Hadoop cluster. Ambari simplifies the operation and hides the complexity of Hadoop, making Hadoop appear like a single, cohesive data platform. Hadoop cluster provisioning and ongoing management can be a complicated task, especially when there are hundreds or thousands of nodes involved. Ambari allows you to control Hadoop cluster services from a single point. In this session, we will provide an overview of the Apache Ambari key features, architecture and web service-based APIs.
Speaker: Ravi Mutyala
When we started using Apache Accumulo on large scale, our key concern was on monitoring the health of the cluster. Accumulo exposes metrics through JMX. Ganglia and Nagios are the de-facto metrics and monitoring tools for hadoop clusters. We identified that integration with ganglia, nagios and Apache Ambari will provide ease of use both for monitoring and managing Accumulo clusters. We started with ganglia and nagios integration which helps reuse all the hadoop monitoring infrastructure for Accumulo. Our next target is Apache Ambari integration for Accumulo.
In this talk, we focus on why we need to integrate and how this can be done. We will show a Hands On for ganglia and nagios integration and share the status of ambari integration.
You want to use MySQL in Amazon RDS, Rackspace Cloud, Google Cloud SQL or HP Helion Public Cloud? Check this out, from Percona Live London 2014. (Note that pricing of Google Cloud SQL changed prices on the same day after the presentation)
CloudStack / Saltstack lightning talk at DevOps AmsterdamSebastien Goasguen
This is my lightning talk from DevOps days Amsterdam on June 14th. I present a quick hack I did during LinuxTag berlin. Saltstack is an alternative to Puppet and Chef, written in Python it has an active community and is easy to use. Saltstack has a cloud client called salt-cloud which can use apache libcloud to interact with Cloud providers. I created a coudstack driver in saltstack and patched libcloud to be able to use salt on a CloudStack cloud. This talk lasted 5 minutes as per the rules of DevOps Lightning talk.
Apache Ambari is the only 100% open source management and provisioning tool for Apache Hadoop and Hortonworks Data Platform (HDP). Recent innovations of Apache Ambari have focused on opening Apache Ambari into a pluggable management platform that can automate cluster provisioning, deploy 3rd party software and provide custom operational and developers views to the end user. In this session Hortonworks will cover 3 key integration points of Apache Ambari including Stacks, Views and Blueprints and deliver working examples of each.
With a focus on Amazon AWS RDS MySQL and PostgreSQL, Rackspace cloud, Google Cloud SQL, Microsoft Azure for MySQL and PostgreSQL as well as a hint of the other clouds
Jeff Sposetti of Ambari discusses the Apache project Ambari used to help deploy and provision Hadoop Clusters
- Ambari Overview and the Community
- Ambari Architecture - Provisioning Clusters and Services -Standard Services: HDFS, YARN, MR2, Hive, new Services: Storm, Falcon
- Management and Monitoring Capabilities -Nagios and Ganglia Integration
- Key Innovation Features -Ambari Stacks providing dynamic service lifecycle -Ambari BluePrints powering Savannah OpenStack -Ambari Views enabling custom UI development
Mark Burgess' presentation at #lspe on "Configuration Management" http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/73725682/
Kuyper Hoffmann's presentation from the #lspe "Private Clouds" event: http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/48901162/
Vladimir Vuksan's presentation on Ganglia at the "Not Nagios" episode of The Bay Area Large-Scale Production Engineering meetup: http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/15481164/
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
The Metaverse and AI: how can decision-makers harness the Metaverse for their...Jen Stirrup
The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives?
How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
3. Today, We’ll Go Over…
• Intro
• Features
• Demo
• Architecture
• Futures
• Q & A
Page 3
4. Ambari: Enterprise Hadoop Operations
Ambari is an open source framework for provisioning,
managing and monitoring Apache Hadoop clusters
HADOOP
Storage & Process
at Scale
AMBARI
PROVISION
MANAGE
MONITOR
AMBARI
WEB
Page 4
5. Apache Ambari – 100% Open Source!
• Active community
• 50+ Contributors / 30+ Committers
• 170+ Ambari User Group Members
• Steady progress/release cycle
Page 5
Release
Version
Release Date JIRAs
Resolved
0.9.0 Sep 2012 402
1.2.0 Feb 2013 441
1.2.1 Mar 2013 134
1.2.2 Apr 2013 106
1.2.3 Jun 2013 515
1.2.4 Jul 2013 113
1.2.5 Sep 2013 350
1.4.x Oct 2013 580+
Current Release
Today’s Demo
6. Features Today
Provisioning: Simplified deployment across platforms
Managing: Consistent controls across the Stack
Monitoring: Visibility into key cluster metrics
- Single pane of glass for Hadoop & System status
- Pre-configured metrics & alerts
- Single point for cluster operations
- Customize w/o dealing with Hadoop complexities
- Advanced configurations and host controls
- Wizard-driven cluster install experience
- Deploy 10s,100s or 1000s of Hadoop servers
- Cloud, virtual and physical environments
Page 6
7. New Features
• Add slaves components to hosts
• Stop/Start All Services
• Enable Kerberos Security
• Enable NameNode HA
• Host Status Filtering
Page 7
8. New Features
• Provision, manage and monitoring Hadoop 2.0 Stack
• HDFS2, YARN, MR2
• HBase
–New Heatmaps specific to Region Servers
Page 8
9. New Features
• Ambari supports Postgres, Oracle
• Configure Hive and Oozie to use MySQL or Oracle
• Ambari + Redhat GlusterFS Integration
• Deploy / manage cluster with dist. file system other than HDFS
–HCFS: GlusterFS as first implementation
–Pluggability with other HCFS’s
Page 9
11. Ambari System Architecture
Ambari Server
Host
Agent
gmond
Host
Agent
gmond
Ganglia
Server
Agent
Host
Agent
gmondgmetad
gmond
Ambari
Web
DB
REST
/clusters
Nagios
Server
Agent
Page 11
12. Ambari Architecture
Page 12
DB
Orchestrator SPI
REST API
Request Dispatcher
Ambari
Web
Ambari
Server
Metrics
AuthProvider
/clusters
/services
/hosts
/workflows/jobs
/users, …
User
Store
java
RDBMS
javascript
RDBM
S
AD/
LDAP
REST API for
integration
Auth
Provider
Cluster
Configurations
Web Client
100% REST
Ambari
Agents ganglia nagios
Alerts
Pluggable
Service
Providersfalcon
Data Mgmt
jmx
python
puppet
13. Ambari Futures
Extend core capabilities to
include the critical tasks
associated with provisioning
and operating Hadoop clusters.
Cluster Operations
Enable insight into job
performance and reduce the
burden on specialized Hadoop
skills and knowledge.
Job Diagnostics
Expose integration and
customization points so Hadoop
can interoperate with existing
operational tooling.
Extensible Platform
Make Hadoop easier to manage
Page 13
14. Beyond 1.4
• Ambari Python CLI
• Delete Hosts
• Add Services
• Improved Configuration Mgmt with host-level overrides
• Log Aggregation
• Rack awareness
• Manage Full Stack HA
• Customizable Metric Graphs and Heatmaps
• Capacity Scheduler Usage Reporting
• Job Diagnostics Improvements
• Ambari Server HA
• Finer-grain Ambari user roles
• Cluster Manifest (cluster takeover)
• New Services: Accumulo, Falcon, Flume, Knox, Storm, …
• HDFS Mirroring/Rebalance
• HBase Compaction
Page 14
15. Host Group Configuration Controls
• Set custom configuration properties at the host level
for one or more hosts
• Important for handing “heterogeneous” clusters
• AMBARI-1509 and AMBARI-1370
HEAPSIZE= 1024
HEAPSIZE= 2048
Page 15
16. Ambari + Accumulo Integration
• Using Ambari to deploy / manage cluster with
Accumulo
–Google Summer of Code project
–See AMBARI-1930
MapReduce
Hive
Distributed File System
HBasePig
Page 16
17. Job Diagnostics
• Enhanced swimlane visualizations
• See job DAG with task overlay
• See task scatter plot across jobs
Page 17
18. Stack Definitions
• Design Goals
–Ambari should be able to support choice of Hadoop stacks
–Ambari should enable adding new components to an existing stack
• Define which Services are available (services)
• Define where to get the packages (repos)
S S S SStack B
repos
services
S S S SStack A
repos
services
S S S S
Stack C
extends
Stack B
repos
services
S
S+
Page 18
19. Cluster Blueprints
• Perform “Headless Install”
• Export blueprint from cluster
• Boot wizard with blueprint
• Save wizard as blueprint
BLUEPRINT
<stack>
<host>
<service>
<component>
<config>
Ambari
Server
MANIFEST
<host>
<meta>
<configs>
<configs>
<config>
Blueprint
Page 19
20. APIs, APIs, and More APIs
• Consistent front-end REST API
• Service Provider plugin architecture
“ZERO TOUCH” INSTALLS
“LIGHTS OUT” WORKFLOWS
“BRING MY OWN” SCENARIOS
Blueprint ClusterAmbari
Alarm Decom Inform
Custom Provider
Page 20