This document summarizes the creation of a self-managed load balancer on AWS to distribute mail traffic across multiple mail gateway servers. It describes the existing mail system architecture, the need for a load balancer due to traffic volume limitations, and the technical implementation using Linux Virtual Server (LVS) and keepalived for load balancing and iptables for network address translation (SNAT) to support load balancing of SMTP traffic. The results were an increased ability to scale mail gateway servers elastically and observe traffic patterns from email services like Google Apps. A note of caution is provided about network bandwidth limitations based on the EC2 instance type used for the load balancer.
With the increasing maturity of OpenStack Neutron, and increasing support for a large number of services , we'll examine how SDNs can leverage the container technology for supporting "as a Service" components.
Kube-proxy enables access to Kubernetes services (virtual IPs backed by pods) by configuring client-side load-balancing on nodes. The first implementation relied on a userspace proxy which was not very performant. The second implementation used iptables and is still the one used in most Kubernetes clusters. Recently, the community introduced an alternative based on IPVS. This talk will start with a description of the different modes and how they work. It will then focus on the IPVS implementation, the improvements it brings, the issues we encountered and how we fixed them as well as the remaining challenges and how they could be addressed. Finally, the talk will present alternative solutions based on eBPF such as Cilium.
A talk delivered at the Oct 1st Sydney AWS Meetup, on how Domain uses CloudFormation, DSC and Octopus Deploy to support our microservices arcitecture with continuous delivery
I would like to present our CI Provisioning with Openstack solution and how it improved our development. The CI provisioning is meant to replace your static CI env servers to a auto provisioned during your build stage for every commit.
Abstract:
1. CI Flow Quick view
2. Openstack CI integration maven plugin
3. Orchestration using facters
4. Openstack instance queue for faster provisioning
With the increasing maturity of OpenStack Neutron, and increasing support for a large number of services , we'll examine how SDNs can leverage the container technology for supporting "as a Service" components.
Kube-proxy enables access to Kubernetes services (virtual IPs backed by pods) by configuring client-side load-balancing on nodes. The first implementation relied on a userspace proxy which was not very performant. The second implementation used iptables and is still the one used in most Kubernetes clusters. Recently, the community introduced an alternative based on IPVS. This talk will start with a description of the different modes and how they work. It will then focus on the IPVS implementation, the improvements it brings, the issues we encountered and how we fixed them as well as the remaining challenges and how they could be addressed. Finally, the talk will present alternative solutions based on eBPF such as Cilium.
A talk delivered at the Oct 1st Sydney AWS Meetup, on how Domain uses CloudFormation, DSC and Octopus Deploy to support our microservices arcitecture with continuous delivery
I would like to present our CI Provisioning with Openstack solution and how it improved our development. The CI provisioning is meant to replace your static CI env servers to a auto provisioned during your build stage for every commit.
Abstract:
1. CI Flow Quick view
2. Openstack CI integration maven plugin
3. Orchestration using facters
4. Openstack instance queue for faster provisioning
Introducing Exactly Once Semantics To Apache KafkaApurva Mehta
Here are slides from my talk on introducing exactly once semantics to Apache Kafka. The talk was given at the Kafka Summit NYC, 8 May 2017.
The slides dive into the design of transactions in Apache Kafka.
Journey to Microservice architecture via Amazon LambdaAxilis
Microservices are one of the latest trends in architecture design.
Made popular by the introduction of Amazon Lambda, Google Cloud Functions and Azure Functions. They seem to offer a way to structure code as a set of independent services that interact together to work as one, making each part simpler and offering an easy way to scale up. But just as every other technology they bring their own set of challenges.
Join us on lessons we learned while converting simple application to work on Lambda.
(WEB401) Optimizing Your Web Server on AWS | AWS re:Invent 2014Amazon Web Services
Tuning your EC2 web server will help you to improve application server throughput and cost-efficiency as well as reduce request latency. In this session we will walk through tactics to identify bottlenecks using tools such as CloudWatch in order to drive the appropriate allocation of EC2 and EBS resources. In addition, we will also be reviewing some performance optimizations and best practices for popular web servers such as Nginx and Apache in order to take advantage of the latest EC2 capabilities.
Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013Puppet
"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat.
Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously.
Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.
Mit Docker ist es einfach geworden, Applikationen lokal zu starten, ohne zusätzliche Abhängigkeiten installieren zu müssen. Einen Cluster auf seinem eigenen Rechner laufen zu lassen ist kein großes Ding mehr. Mit ECS bietet AWS einen Container-Management-Service für die Cloud an, der verspricht, Container entsprechend ihrem Ressourcenbedarf und Verfügbarkeitserfordernissen automatisch im Cluster zu platzieren.
Aber was passiert dazwischen? Und ist es wirklich so einfach?
In diesem Talk werden wir betrachten, welche existierenden Services von AWS verwendet werden können, um Container automatisch zu deployen, und was zusätzlich alles benötigt wird, um sie im Betrieb laufen zu lassen.
Rust is a relatively new programming language promising performance, reliability and productivity. In order to learn Rust Gerard converted a micro service he already wrote in Clojure and Kotlin, both JVM languages, to Rust. The microservice processes two kinds of events, and uses PostgreSQL to keep state. As part of the conversion he wrote a library to use the Confluent Schema Registry with Rust.
What can be said about the promises of Rust compared to the JVM? Gerard will tell about his experiences in using Rust with Kafka, and presenting some benchmarks comparing the different languages. The focus of the benchmarks will be the end to end latency and resource usage. He will conclude with some remarks why it may or may not be a good idea to add some Rust to your Kafka.
Running Kubernetes at scale is challenging and you can often end up in situations where you have to debug complex and unexpected issues. This requires understanding in detail how the different components work and interact with each other. Over the last 3 years, Datadog migrated most of its workloads to Kubernetes and now manages dozens of clusters consisting of thousands of nodes each. During this journey, engineers have debugged complex issues with root causes that were sometimes very surprising. In this talk Laurent and Tabitha will share some of these stories, including a favorite: how a complex interaction between familiar Kubernetes components allowed an OOM-killer invocation to trigger the deletion of a namespace.
Peter Zaitsev "18 ways to fix MySQL bottlenecks"Fwdays
The presentation will be focused on dealing with an unexpected high traffic event as it is happening. We will address the impact a traffic spike can have on your database – from poor query response time to replication lag issues – and provide actionable tips and tricks. We will talk about how to bring your database under control, earn performance gains, and delight your customers.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipelineKubeAcademy
One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
Solving some of the scalability problems at booking.comIvan Kruglov
Booking.com uses Sereal in many applications. One of the biggest use case though is the events pipeline. It was built to delivery messages (events) from generation point to various processors in near real-time fashion. These days it servers billions of messages per day.
One of our processors recently faced scalability issues due to growth of the volume of delivered events.
In this talk I would like to share what problem we had, how we addressed it and which new features of Sereal helped us.
The complexity of a typical OpenNebula installation brings a special set of challenges on the monitoring side. In this talk, I will show monitoring of a full stack of from the physical servers to storage layer and ONE daemon. Providing an aggregated view of this information allows you see the real impact of a certain failure. I would like to also present a use case for a “closed-loop” setup where new VMs are automatically added to the monitoring without human intervention, allowing for an efficient approach to monitoring the services a OpenNebula setup provides.
Introducing Exactly Once Semantics To Apache KafkaApurva Mehta
Here are slides from my talk on introducing exactly once semantics to Apache Kafka. The talk was given at the Kafka Summit NYC, 8 May 2017.
The slides dive into the design of transactions in Apache Kafka.
Journey to Microservice architecture via Amazon LambdaAxilis
Microservices are one of the latest trends in architecture design.
Made popular by the introduction of Amazon Lambda, Google Cloud Functions and Azure Functions. They seem to offer a way to structure code as a set of independent services that interact together to work as one, making each part simpler and offering an easy way to scale up. But just as every other technology they bring their own set of challenges.
Join us on lessons we learned while converting simple application to work on Lambda.
(WEB401) Optimizing Your Web Server on AWS | AWS re:Invent 2014Amazon Web Services
Tuning your EC2 web server will help you to improve application server throughput and cost-efficiency as well as reduce request latency. In this session we will walk through tactics to identify bottlenecks using tools such as CloudWatch in order to drive the appropriate allocation of EC2 and EBS resources. In addition, we will also be reviewing some performance optimizations and best practices for popular web servers such as Nginx and Apache in order to take advantage of the latest EC2 capabilities.
Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013Puppet
"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat.
Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously.
Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.
Mit Docker ist es einfach geworden, Applikationen lokal zu starten, ohne zusätzliche Abhängigkeiten installieren zu müssen. Einen Cluster auf seinem eigenen Rechner laufen zu lassen ist kein großes Ding mehr. Mit ECS bietet AWS einen Container-Management-Service für die Cloud an, der verspricht, Container entsprechend ihrem Ressourcenbedarf und Verfügbarkeitserfordernissen automatisch im Cluster zu platzieren.
Aber was passiert dazwischen? Und ist es wirklich so einfach?
In diesem Talk werden wir betrachten, welche existierenden Services von AWS verwendet werden können, um Container automatisch zu deployen, und was zusätzlich alles benötigt wird, um sie im Betrieb laufen zu lassen.
Rust is a relatively new programming language promising performance, reliability and productivity. In order to learn Rust Gerard converted a micro service he already wrote in Clojure and Kotlin, both JVM languages, to Rust. The microservice processes two kinds of events, and uses PostgreSQL to keep state. As part of the conversion he wrote a library to use the Confluent Schema Registry with Rust.
What can be said about the promises of Rust compared to the JVM? Gerard will tell about his experiences in using Rust with Kafka, and presenting some benchmarks comparing the different languages. The focus of the benchmarks will be the end to end latency and resource usage. He will conclude with some remarks why it may or may not be a good idea to add some Rust to your Kafka.
Running Kubernetes at scale is challenging and you can often end up in situations where you have to debug complex and unexpected issues. This requires understanding in detail how the different components work and interact with each other. Over the last 3 years, Datadog migrated most of its workloads to Kubernetes and now manages dozens of clusters consisting of thousands of nodes each. During this journey, engineers have debugged complex issues with root causes that were sometimes very surprising. In this talk Laurent and Tabitha will share some of these stories, including a favorite: how a complex interaction between familiar Kubernetes components allowed an OOM-killer invocation to trigger the deletion of a namespace.
Peter Zaitsev "18 ways to fix MySQL bottlenecks"Fwdays
The presentation will be focused on dealing with an unexpected high traffic event as it is happening. We will address the impact a traffic spike can have on your database – from poor query response time to replication lag issues – and provide actionable tips and tricks. We will talk about how to bring your database under control, earn performance gains, and delight your customers.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipelineKubeAcademy
One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
Solving some of the scalability problems at booking.comIvan Kruglov
Booking.com uses Sereal in many applications. One of the biggest use case though is the events pipeline. It was built to delivery messages (events) from generation point to various processors in near real-time fashion. These days it servers billions of messages per day.
One of our processors recently faced scalability issues due to growth of the volume of delivered events.
In this talk I would like to share what problem we had, how we addressed it and which new features of Sereal helped us.
The complexity of a typical OpenNebula installation brings a special set of challenges on the monitoring side. In this talk, I will show monitoring of a full stack of from the physical servers to storage layer and ONE daemon. Providing an aggregated view of this information allows you see the real impact of a certain failure. I would like to also present a use case for a “closed-loop” setup where new VMs are automatically added to the monitoring without human intervention, allowing for an efficient approach to monitoring the services a OpenNebula setup provides.
CentOS 7 was officially released in July, 2014.
There are many significant changes in it. So you have to learn how they are if you want to use it or smoothly migrate settings from the previous version.
I started studying English, my TOEIC score was 400 points or less.I didn’t know how to improve my English skill but I tried some method to study English. I will introduce how to study English.
Anton Moldovan "Building an efficient replication system for thousands of ter...Fwdays
For one of our projects, we needed to improve the current content delivery system for terminals. In this talk, I will share our experience in building an efficient data replication system for thousands of terminals. We will touch on architecture decisions and tradeoffs, technologies that we used, and a bit of load testing.
Spoiler: We didn't use Kafka.
How To Set Up SQL Load Balancing with HAProxy - SlidesSeveralnines
We continuously see great interest in MySQL load balancing and HAProxy, so we thought it was about time we organised a live webinar on the topic! Here is the replay of that webinar!
As most of you will know, database clusters and load balancing go hand in hand.
Once your data is distributed and replicated across multiple database nodes, a load balancing mechanism helps distribute database requests, and gives applications a single database endpoint to connect to.
Instance failures or maintenance operations like node additions/removals, reconfigurations or version upgrades can be masked behind a load balancer. This provides an efficient way of isolating changes in the database layer from the rest of the infrastructure.
In this webinar, we cover the concepts around the popular open-source HAProxy load balancer, and show you how to use it with your SQL-based database clusters. We also discuss HA strategies for HAProxy with Keepalived and Virtual IP.
Agenda:
* What is HAProxy?
* SQL Load balancing for MySQL
* Failure detection using MySQL health checks
* High Availability with Keepalived and Virtual IP
* Use cases: MySQL Cluster, Galera Cluster and MySQL Replication
* Alternative methods: Database drivers with inbuilt cluster support, MySQL proxy, MaxScale, ProxySQL
Load Balancing MySQL with HAProxy - SlidesSeveralnines
Agenda:
* What is HAProxy?
* SQL Load balancing for MySQL
* Failure detection using MySQL health checks
* High Availability with Keepalived and Virtual IP
* Use cases: MySQL Cluster, Galera Cluster and MySQL Replication
* Alternative methods: Database drivers with inbuilt cluster support, MySQL proxy, MaxScale, ProxySQL
Container orchestration from theory to practiceDocker, Inc.
"Join Laura Frank and Stephen Day as they explain and examine technical concepts behind container orchestration systems, like distributed consensus, object models, and node topology. These concepts build the foundation of every modern orchestration system, and each technical explanation will be illustrated using SwarmKit and Kubernetes as a real-world example. Gain a deeper understanding of how orchestration systems work in practice and walk away with more insights into your production applications."
Implementing an IPv6 Enabled Environment for a Public Cloud TenantShixiong Shang
"Implementing an IPv6 Enabled Environment for a Public Cloud Tenant" case study I delivered in OpenStack Vancouver Summit (May, 2015) jointly with Anik and Sharmin from Cisco System.
We all know that load testing is important, but it's all too common that it's left to the very end of a project and it's invariably the first thing that gets dropped when budgets and timeframes get cut. Furthermore, most of us don't know where or how to start implementing effective load tests, let alone how to analyse the results.
Lindsay Holmwood, Software Manager at Bulletproof Networks, will be talking about integrating performance testing into your application development + deploy cycle from the very beginning, using inexpensive and easy to use SaaS tools.
There will be a hands on demonstration of the Blitz load + performance testing tool, coupled with a brief dive into the Blitz API internals to retrieve and analyse advanced reporting information.
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Similar to Self Created Load Balancer for MTA on AWS (20)
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
9. 1) LVS + keepalived
- Command
・Join a child server
$ ipvsadm -a -t [LB IP]:25 -r [Child Server IP]:25 -m
・Drop a child server
$ ipvsadm -d -t [LB IP]:25 -r [LB IP]:25 -m
・Confirm child servers
$ ipvsadm -Ln
===
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn
InActConn
TCP 10.x.x.x:25 lc
-> 10.x.x.x:25 Masq 1 59 17
-> 10.x.x.x:25 Masq 1 55 3
10. 2) SNAT in iptables
- Problem
When we use EC2 instance for Load Balancer
we cannot create a SMTP connection,
since SMTP trafic returns towerd
EC2 default gateway not toword Load Balancer.
11. 2) SNAT in iptables
# iptables -t nat -A POSTROUTING -m ipvs --vaddr 10.X.X.X --
vport 25 -j SNAT --to-source 10.X.X.X
- Solution
We use an SNAT function in iptables to overwrite a sender IP
address. For example, a sender IP address of packet which is sent
to Port 25 in Load Balancer can be overritten by the local IP
address of LB.
12. Tools
・Fabric task to join/drop child server and update keepalived
$ fab join_ozmax_gw:10.x.x.x
$ fab drop_ozmax_gw:10.x.x.x
$ fab update_ozmax_lb_healthcheck
・Scrpt to automatically generate keepalived.conf in accordance
with MX records
・Script to update iptables when LB instance starts
・Script to observe number of connections, network bandwidth
etc.
What we needed
13. Results
・We can increase a child server as much as
and whenever we want !
・We can observe how Google Apps
changes their way to send mails !
・We can also use a Self-Created LB as an
internl LB in EC2!
14. A Note of Causion
There is a limitation of network bandwidth
dependent on instance type.
m1.small -> m1.medium -> c3.xlarge
15. Extended Gateway Server Now
Gateway
Server
Gateway
Server
Gateway
Server
Gateway
Server
Load Balancer
× n
