In the past years at Ustream we built several systems which had to be scalable and resilient by themselves. This included automatic monitoring and scaling, protection against overloads, and the like. We learned a lot of valuable lessons on the way, and I would like to share those experiences, with practical examples and open source tools.
You know PowerShell and you must have heard of DSC, but 6 years after its creation, where are we at?
Join Gael Colas, a well-known DSC contributor and Microsoft MVP, in this session, he will show what's happening in the DSC community, how to get started, where to find information or help, and some best practices to follow.
He will demo some concepts, practices and use cases, share some code, and insights about who's behind DSC and what they are doing, so you have no excuse for not learning Configuration Management!
- Demo code: https://github.com/gaelcolas/packer-templates
Follow & connect with Gael Colas:
- Twitter: https://twitter.com/gaelcolas
- LinkedIn: https://www.linkedin.com/in/gaelcolas/
- Blog: https://gaelcolas.com/
Thanks to dotdigital Group (https://dotdigital.com / https://twitter.com/dotdigital) for providing the venue, food and drinks. We very much appreciate your continued support of our community of PowerShell & DevOps tech enthusiasts.
Join our next event at https://www.meetup.com/PowerShell-London-UK/. We are running at least one Meetup every month.
#PowerShell #PSDSC
An introduction to the reference architectures content from the Microsoft patterns & practices team.
This covers common IaaS (infrastructure) and PaaS (managed services) scenarios.
http://aka.ms/architecture
Be sure to read the notes!
Cloud computing is a general term for anything that involves delivering hosted services over the Internet. These services are broadly divided into three categories: Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS).
CloudBrew 2016 - Building IoT solution with Service FabricTeemu Tapanila
When building your IoT solution you face different steps. They normally include Device registration, Data Ingestion, Data processing and Data analysis. So come hear how to model this process on micro services architecture and then hosting whole thing on premise or on a cloud with Azure Service Fabric.
You know PowerShell and you must have heard of DSC, but 6 years after its creation, where are we at?
Join Gael Colas, a well-known DSC contributor and Microsoft MVP, in this session, he will show what's happening in the DSC community, how to get started, where to find information or help, and some best practices to follow.
He will demo some concepts, practices and use cases, share some code, and insights about who's behind DSC and what they are doing, so you have no excuse for not learning Configuration Management!
- Demo code: https://github.com/gaelcolas/packer-templates
Follow & connect with Gael Colas:
- Twitter: https://twitter.com/gaelcolas
- LinkedIn: https://www.linkedin.com/in/gaelcolas/
- Blog: https://gaelcolas.com/
Thanks to dotdigital Group (https://dotdigital.com / https://twitter.com/dotdigital) for providing the venue, food and drinks. We very much appreciate your continued support of our community of PowerShell & DevOps tech enthusiasts.
Join our next event at https://www.meetup.com/PowerShell-London-UK/. We are running at least one Meetup every month.
#PowerShell #PSDSC
An introduction to the reference architectures content from the Microsoft patterns & practices team.
This covers common IaaS (infrastructure) and PaaS (managed services) scenarios.
http://aka.ms/architecture
Be sure to read the notes!
Cloud computing is a general term for anything that involves delivering hosted services over the Internet. These services are broadly divided into three categories: Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS).
CloudBrew 2016 - Building IoT solution with Service FabricTeemu Tapanila
When building your IoT solution you face different steps. They normally include Device registration, Data Ingestion, Data processing and Data analysis. So come hear how to model this process on micro services architecture and then hosting whole thing on premise or on a cloud with Azure Service Fabric.
We recently hosted a webinar on Azure SQL Database titled “Building Cloud Apps using Azure SQL Database”, where we discussed:
• Reasons why enterprises should adopt Azure SQL Database
• How to Secure and store data in the cloud from your mobile app on any device
• Deliver a global mobile app with high availability and low latency leveraging Azure SQL database.
In the land of Micro Services the question of analytics, complexity of algorithms, schema reporting gets well defined with a resilient data model. The culture and design principles should embrace failure and faults, similar to anti-fragile systems
Azure AWS real time-interview questions part 9Malleswar Reddy
It covers Azure Admin real-time interview questions which are collected from various company interviews. As part of this video, I covered Azure admin related questions.
https://youtu.be/hHPgAScQIXI
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completaElasticsearch
Las aplicaciones suelen ser la interfaz de cliente principal en las organizaciones modernas, lo que tiene un impacto directo sobre resultados como los ingresos y pérdidas de clientes. Elastic APM puede ayudarte a ofrecer mejores prácticas mediante la rápida detección de los cuellos de botella en materia de rendimiento y de las regresiones en los nuevos despliegues . Descubre cómo obtener una visión completa sobre los servicios en los que se basan tus aplicaciones, del frontend al backend, para optimizar su rendimiento.
Elastic APM: Amplía tus logs y métricas para ver el panorama completoElasticsearch
No importa dónde se encuentre en su viaje a la nube nativa, Elastic APM ayuda a ofrecer mejores experiencias al cliente al detectar los cuellos de botella en el rendimiento e identificar más rápidamente las regresiones de las nuevas implementaciones.
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...MSDEVMTL
11 Janvier 2016
Groupe Azure
Sujet: Développer des microservices avec Service Fabric
Conférienciers: Stephane Lapointe, MVP Azure & Alexandre Brisebois, TSP Azure Microsoft
Une soirée complète sur Service Fabric! Nous débuterons par la base en explorant l'architecture de Service Fabric et les modèles de programmation qu'il propose, les "Reliable Actors" et "Reliable Services" en mettant l'emphase sur le modèle Acteurs.
Comme il est plus agréable d'apprendre avec des exemples, nous verrons le cycle de vie complet d'une application au sein de Service Fabric et les outils offert par la plateforme; développement, packaging, déploiement, tests de stabilités, mises à jour et maintenance sont les sujets que nous explorerons.
À la fin de cette rencontre, vous aurez les connaissances nécessaires pour mener à bien un projet de microservices en utilisant les acteurs dans Service Fabric.
(ENT210) Accelerating Business Innovation with DevOps on AWS | AWS re:Invent ...Amazon Web Services
IT must innovate at the speed of market change and many enterprises are realizing that DevOps and cloud computing are a means to this end. Cloud-based DevOps solutions that enforce fine-grain governance policies and automate software releases across the development tool chain can accelerate application time to market while also improving software quality. In this session, attendees learn the following:
- How cloud and DevOps together can significantly accelerate software release cycles, so you can speed business innovation and gain competitive advantage
- Best practices for leveraging CSC Agility Platform, AWS, and a hybrid IT strategy for DevOps
- How to eliminate software release bottlenecks via policy-based automation, orchestration, and governance of application deployment environments.
Sponsored by CSC.
How a National Transportation Software Provider Migrated a Mission-Critical T...Amazon Web Services
In this webinar, Cascadeo will show you how they helped a national transportation software provider build an AWS architecture that enables them to effectively support more than 3,300 complex integration tests against nightly builds of their Interoperable Train Control Messaging (ITCM) application. You’ll also learn about how this software provider can scale on-demand, has improved governance and cost management, and rapidly supports new projects without increasing IT overhead using AWS.
A guide through the Azure Messaging services - Update ConferenceEldert Grootenboer
https://www.updateconference.net/en/2019/session/a-guide-through-the-azure-messaging-services
A guide through the Azure Messaging services - Update Conference
Enterprise DevOps is different then DevOps in startups and smaller companies. This session how AWS/CSC address this. How AWS IaaS level automation via CloudFormation, UserData, Console, APIS and some PaaS OpsWorks/Beanstalk is complimented by CSC Agility Platform. CSC Agility adds application compliance and security to the AWS infrastructure compliance and security. CSC Agility allows for the creation of architecture blueprints for predefined application offerings.
We recently hosted a webinar on Azure SQL Database titled “Building Cloud Apps using Azure SQL Database”, where we discussed:
• Reasons why enterprises should adopt Azure SQL Database
• How to Secure and store data in the cloud from your mobile app on any device
• Deliver a global mobile app with high availability and low latency leveraging Azure SQL database.
In the land of Micro Services the question of analytics, complexity of algorithms, schema reporting gets well defined with a resilient data model. The culture and design principles should embrace failure and faults, similar to anti-fragile systems
Azure AWS real time-interview questions part 9Malleswar Reddy
It covers Azure Admin real-time interview questions which are collected from various company interviews. As part of this video, I covered Azure admin related questions.
https://youtu.be/hHPgAScQIXI
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completaElasticsearch
Las aplicaciones suelen ser la interfaz de cliente principal en las organizaciones modernas, lo que tiene un impacto directo sobre resultados como los ingresos y pérdidas de clientes. Elastic APM puede ayudarte a ofrecer mejores prácticas mediante la rápida detección de los cuellos de botella en materia de rendimiento y de las regresiones en los nuevos despliegues . Descubre cómo obtener una visión completa sobre los servicios en los que se basan tus aplicaciones, del frontend al backend, para optimizar su rendimiento.
Elastic APM: Amplía tus logs y métricas para ver el panorama completoElasticsearch
No importa dónde se encuentre en su viaje a la nube nativa, Elastic APM ayuda a ofrecer mejores experiencias al cliente al detectar los cuellos de botella en el rendimiento e identificar más rápidamente las regresiones de las nuevas implementaciones.
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...MSDEVMTL
11 Janvier 2016
Groupe Azure
Sujet: Développer des microservices avec Service Fabric
Conférienciers: Stephane Lapointe, MVP Azure & Alexandre Brisebois, TSP Azure Microsoft
Une soirée complète sur Service Fabric! Nous débuterons par la base en explorant l'architecture de Service Fabric et les modèles de programmation qu'il propose, les "Reliable Actors" et "Reliable Services" en mettant l'emphase sur le modèle Acteurs.
Comme il est plus agréable d'apprendre avec des exemples, nous verrons le cycle de vie complet d'une application au sein de Service Fabric et les outils offert par la plateforme; développement, packaging, déploiement, tests de stabilités, mises à jour et maintenance sont les sujets que nous explorerons.
À la fin de cette rencontre, vous aurez les connaissances nécessaires pour mener à bien un projet de microservices en utilisant les acteurs dans Service Fabric.
(ENT210) Accelerating Business Innovation with DevOps on AWS | AWS re:Invent ...Amazon Web Services
IT must innovate at the speed of market change and many enterprises are realizing that DevOps and cloud computing are a means to this end. Cloud-based DevOps solutions that enforce fine-grain governance policies and automate software releases across the development tool chain can accelerate application time to market while also improving software quality. In this session, attendees learn the following:
- How cloud and DevOps together can significantly accelerate software release cycles, so you can speed business innovation and gain competitive advantage
- Best practices for leveraging CSC Agility Platform, AWS, and a hybrid IT strategy for DevOps
- How to eliminate software release bottlenecks via policy-based automation, orchestration, and governance of application deployment environments.
Sponsored by CSC.
How a National Transportation Software Provider Migrated a Mission-Critical T...Amazon Web Services
In this webinar, Cascadeo will show you how they helped a national transportation software provider build an AWS architecture that enables them to effectively support more than 3,300 complex integration tests against nightly builds of their Interoperable Train Control Messaging (ITCM) application. You’ll also learn about how this software provider can scale on-demand, has improved governance and cost management, and rapidly supports new projects without increasing IT overhead using AWS.
A guide through the Azure Messaging services - Update ConferenceEldert Grootenboer
https://www.updateconference.net/en/2019/session/a-guide-through-the-azure-messaging-services
A guide through the Azure Messaging services - Update Conference
Enterprise DevOps is different then DevOps in startups and smaller companies. This session how AWS/CSC address this. How AWS IaaS level automation via CloudFormation, UserData, Console, APIS and some PaaS OpsWorks/Beanstalk is complimented by CSC Agility Platform. CSC Agility adds application compliance and security to the AWS infrastructure compliance and security. CSC Agility allows for the creation of architecture blueprints for predefined application offerings.
Evangelos Kapsalakis, Partner Specialist at Microsoft, provides valuable insights on Microsoft Azure and its flexibility when it comes to migration deployment. From Cloud Migration Through Automation: Next Level Flexibility virtual event, hosted on September 30, 2020
Windows Azure - Uma Plataforma para o Desenvolvimento de AplicaçõesComunidade NetPonto
A plataforma Windows Azure abre espaço a desenvimento de aplicações utilizando o novo paradigma: "A Nuvem". Aplicações escaláveis, redundantes, e mais próximas do utilizador final. Isto tudo utilizando como base os conhecimentos que já tem e o novo Visual Studio 2010.
At Ustream the teams developing the streaming technology stack are also responsible for operating it. This means we have our monitoring and alerting in place (including those based on error logs I mentioned above, but many others too) which alert the engineers themselves. I would like to talk about how we made this transition from the traditional setup where the devs did the coding and the sysops did the operation - what lessons we learned, how we convinced the sysop guys to give us permissions and so on
This 7-second Brain Wave Ritual Attracts Money To You.!nirahealhty
Discover the power of a simple 7-second brain wave ritual that can attract wealth and abundance into your life. By tapping into specific brain frequencies, this technique helps you manifest financial success effortlessly. Ready to transform your financial future? Try this powerful ritual and start attracting money today!
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
ER(Entity Relationship) Diagram for online shopping - TAEHimani415946
https://bit.ly/3KACoyV
The ER diagram for the project is the foundation for the building of the database of the project. The properties, datatypes, and attributes are defined by the ER diagram.
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
Talk presented at Kubernetes Community Day, New York, May 2024.
Technical summary of Multi-Cluster Kubernetes Networking architectures with focus on 4 key topics.
1) Key patterns for Multi-cluster architectures
2) Architectural comparison of several OSS/ CNCF projects to address these patterns
3) Evolution trends for the APIs of these projects
4) Some design recommendations & guidelines for adopting/ deploying these solutions.
6. Defining scalability
Scalability is the ability to handle increased workload
by repeatedly applying a costeffective strategy for
extending a system’s capacity.
(CMU paper, 2006)
How well a solution to some problem will work when
the size of the problem increases. When the size
decreases, the solution must fit. (dictionary.com and
Theo Schlossnagle, 2006)
8. Disposability Maximize robustness with
fast startup and graceful
shutdown
Disposable processes
Graceful shutdown on
SIGTERM
Handling sudden death:
robust queue backend
9. Startup and
Shutdown
Automate all the things
Chef
Docker
Gold image based
deployment
Immutable
Handling tasks before
shutdown
10. Backing Services Treat backing services as
attached resources
No distinction between
local and third party
services
Easily swap out resources
Export services via port
binding
Become the backing
service for another app
11. Processes,
concurrency
Stateless processes (not
even sticky sessions)
Process types by work type
We <3 linux process
Shared-nothing adding
concurrency is safe
Process distribution
spanning machines
12. Statelessness Store everything in a
datastore
Aggregate data
Chandra
Scalable datastores
Redis
Cassandra
Aerospike
Handling user sessions
13. Monitoring Application state and
metrics
Dashboards
Alerting
Health
Remove failing nodes
Capacity
Act on trends
20. Extras Debugging features
Logs
Clojure / JS consoles
Runtime configuration
via env
Scaling API
Integrating several
cloud providers
Automatic start / stop
21. Reading
Scalable Internet Architectures by Theo Schlossnagle
The 12-factor App: http://12factor.net/
Carnegie Mellon Paper: http://www.sei.cmu.edu/reports/06tn012.pdf
Circuit Breaker: http://martinfowler.com/bliki/CircuitBreaker.html
Release It! by Michael T. Nygard
Definition
Requirements coming from 12-factor, and some added by us
Some more detail and tools on selected requirements
30 day viewer graph. Clear peaks -> need for scaling
Quick description of the streaming stack, roles of components, how they require scaling
- Transcontroller/transcoder scaling
- UMS scaling
Quick description of the streaming stack, roles of components, how they require scaling
- Transcontroller/transcoder scaling
- UMS scaling
Carnegie Mellon University paper by Charles B. Weinstock, John B. Goodenough: On System Scalability
LINFO: The Linux Information Project http://www.linfo.org/
Example: calling imagemagick or curl from code – they might be there or might not be
Bundle everything into the app instead
Disposable process: they can be started or stopped at a moment’s notice
For a web process, graceful shutdown is achieved by ceasing to listen on the service port (thereby refusing any new requests), allowing any current requests to finish, and then exiting. Implicit in this model is that HTTP requests are short (no more than a few seconds), or in the case of long polling, the client should seamlessly attempt to reconnect when the connection is lost.
For a worker process, graceful shutdown is achieved by returning the current job to the work queue.
Docker: build images from dockerfile, deploy from repository
Tasks before shutdown: moving jobs, log collection, sleep
A backing service is any service the app consumes over the network as part of its normal operation. Examples include datastores (such as MySQL or CouchDB), messaging/queueing systems (such as RabbitMQ or Beanstalkd), SMTP services for outbound email (such as Postfix), and caching systems (such as Memcached).
Put a resource locator in the config only – environment variables
Example: Easily swap out a local mysql to a remote service
The app does not rely on runtime injection of a webserver into the execution environment to create a web-facing service. The web app exports HTTP as a service by binding to a port, and listening to requests coming in on that port.
One app can become the backing service for another app, by providing the URL to the backing app as a resource handle in the config for the consuming app
Handle diverse workloads by assigning each type of work to a process type. For example, HTTP requests may be handled by a web process, and long-running background tasks handled by a worker process
An individual VM can only grow so large (vertical scale), so the application must also be able to span multiple processes running on multiple physical machines.
Aggregate everything within the app and write it out in bulk – careful about write frequency, must not lose too many data on a crash
Redis: scales reads, write problematic
Cassandra: quick scaling questionable
Aerospike: scales reads and writes, working together with their eng team
User sessions: persistent connection, NIO+
Alerting -> openduty
Two important groups: Health vs capacity
Report everything to graphite, constantly check graph trends automatically
Apps are self-aware, they know their health
App instances report into Zookeeper and thus know about each other
Central logic can request resource based on capacity or graph, app can request based on self-check or zookeeper
Zookeeper, Consul: miért, mik az előnyei
load balancing distributes workloads across multiple computing resources
Flexibility: can increase or decrease its own size, example: Threadpools
Adapting to CPU, RAM, disk, network
App level: transcontroller selects transcoder
App level balance with proxy can be SPOF, careful
Resource policies: even distribution, keep large chunks free for possible large tasks (transcoder use case), group requests together on some attribute (pro, etc)
Failure inevitable because: large numbers, hw issues, independent network
Decoupling: serving one request should not wait on others
Hystrix by Netflix 2011/12
Circuit Breaker: Martin Fowler post from 2014
Service decoupling example: inserting layers between DB and UMS -> RGW. Then another layer between RGW and UMS -> Queue
Logs: logs as stream / stdout (factor #9), collect / transport / process
Scaling API: Other considerations: price, network line to the cloud provider, instance type (spot vs normal)