Your SlideShare is downloading. ×

IP Multicast on ec2

6,849
views

Published on

Published in: Technology

9 Comments
12 Likes
Statistics
Notes
  • i has installed keepalived on two instances at the same vpc subnet,but when i run tcpdump vrrp command ,not found other machine vrrp data package,only found oneself send vrrp data package , Can't VRRP protocol used in VPC?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Great presentation! At Cloudar, we recently implemented multicast traffic between two tomcat application servers on AWS using an n2n tunnel, we documented the setup here: http://www.cloudar.be/multicast-on-aws/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I should have noticed earlier that you have tried with DR configuration. I also tried it and wanted it to work, but unfortunately it does not work at least at the moment of writing. Private->Global IP address mapping fails and the packet returned from the backend server does not go out. I had to fall back to LVN NAT mode too because of that.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @Kenta Yasukawa I have already disabled src/dst check. In my setup I could see the SYN packet reaching the Real Server and Real sends the SYN+ACK and it never reaches the client (client is an external client not inside AWS). I am thinking AWS is blocking the mac spoofing in egress. How can I disable that?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @Kaustabh I missed your comment for long time. Apologies. Yes, since unicast support is added, multicast is not always needed for keepalived. One advantage of using multicast I think is that nodes can be added/removed dynamically. But I've not personally played around with the unicast support, so I leave it for experts to comment on.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
6,849
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
9
Likes
12
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. IP Multicast on EC2 -- Who said VRRP doesn’t work on AWS?-- Kenta Yasukawa Solutions Architect Amazon Data Services Japan
  • 2. What this presentation is about My experiences on low layer networking stuff on EC2 • Around L3 or L2 • No L1 stuff We can’t reach L1 of AWS
  • 3. Do you use VPC? EC2-Classic Amazon Virtual Private Cloud (VPC) Each user gets private sections to deploy instances
  • 4. VPC is not only for private networks What EC2-VPC can do whereas EC2-Classic cannot • Static private IP address allocation • Multiple IP addresses allocation • Multiple network interfaces • Dynamic security group membership configuration • Outbound packet filtering by security group • NACL • …
  • 5. L2 looks different from an EC2 instance EC2-Classic • Same answer for any ARP request EC2-VPC • Familiar output!
  • 6. It means L2 tricks may work in VPC L2 NAT (e.g. ebtables, tc) Software which requires L2 addressing (e.g. LVS) DSR (Direct Server Return) Load balancer Pseudo IP broadcast/multicast Will explain this in more detail
  • 7. How Pseudo L2 Broadcast works VPC Subnet IP Multicast Dst: Ethernet Broadcast address Src: Sender’s MAC address Duplicate Unicast-ify Note: Disable Src/Dest on ENI Fine. Let’s use unicast then. IP Multicast packet is sent over Ethernet broadcast frame which just goes into black hole on EC2 Applications bind to IP address, not MAC address. So most of them just work fine.
  • 8. How to implement Script below is based on packet capturing (Just a Proof of Concept) • Easy to understand • It performs ok for an ENI which does not get much traffic • Daemonizing is necessary (e.g. supervisord) https://gist.github.com/kntyskw/5231182
  • 9. How to implement tc mirred + pedit + csum The sample shell script configures tc to perform the job https://gist.github.com/kntyskw/5633755 Much better both from performance and maintenance perspective A little trickier to understand, though Requires two ENIs for a subnet because tc mirred does not like to send loop for copied packets # ec2_multicast.sh <interface to grab multicast packets from> <interface to send modified packets to> [target MAC address 1] [target MAC address 2] ... # ec2_multicast.sh eth0 eth1 00:11:22:33:44:55 66:77:88:99:aa:bb Example: Usage:
  • 10. Determine neighbors’ MAC addresses Use AWS API • Example: https://gist.github.com/kntyskw/5413698
  • 11. Is it useful? It has quite some overhead since a packet is duplicated and sent multiple times But maybe ok to use for infrequent messaging? Sure. Don’t use it for sreaming HD movies Right. So, what about using it for node discovery and heartbeating? Ask yourself. Yeah, like VRRP?
  • 12. Let’s now try to make VRRP work Virtual Router Redundancy Protocol (VRRP) • Gives a virtual IP address to a group of nodes (VRRP group) which forms a fault tolerant cluster • Nodes in a VRRP group communite with IP multicast
  • 13. Verification with LVS + Keepalived VPC Subnet LVS + keepalived LVS + keepalived WebWebWeb 172.31.0.0/20 Elastic IP Secondary: 172.31.24.1 Primary: 172.31.24.100 Primary: 172.31.24.101 VRRP Normal state Request routing path
  • 14. Verification with LVS + Keepalived VPC Subnet LVS + keepalived LVS + keepalived WebWebWeb 172.31.0.0/20 Elastic IP Secondary: 172.31.24.1 Primary: 172.31.24.100 Primary: 172.31.24.101 VRRP Upon failover Request routing path
  • 15. How to take over an IP address VPC is not using ARP to determine dest MAC address • Needs to provision the mapping b/w IP address and ENI Call AWS API to reassign VIP from Master to backup upon failover https://gist.github.com/kntyskw/5417140
  • 16. Keepalived configuration Execute the script when promoted to the master vrrp_instance VI_1 { state BACKUP interface eth1 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.31.24.1 dev eth0 } notify_master /etc/keepalived/assign_vip.sh }
  • 17. Advantages of using LVS UDP load balancing! • Neither ELB nor HAProxy can do LVS does not need to terminate TCP • Lower overhead No need to involve user land process • Lower latency • Higher throughput
  • 18. Quick benchmarking results sss LVS v.s. HAProxy • Both run on a m1.small instance • Tested to fetch a 4KB HTML file with BeesWithMachineGuns • Each of 8 bees will fire 2000 rounds, 10 at a time. 0 500 1000 1500 2000 2500 3000 3500 4000 4500 LVS HAProxy Requestperseconds Throughput 0 10 20 30 40 50 60 Average tp50 tp90 ResponseTimein[ms] Response Time LVS HAProxy
  • 19. Things to note when you depend on L2 VPC Subnet cannot span across multiple AZs ENI cannot reach a subnet in a different AZ Availability Zone Availability Zone VPC Subnet Availability Zone Availability Zone
  • 20. Building Multi-AZ architecture with LVS VPC Subnet LVS + keepalived LVS + keepalived WebWebWeb 172.31.0.0/20 Elatic IP Secondary: 172.31.24.1 Primary: 172.31.24.100 Primary: 172.31.24.1 01 VRRP If you can use multiple IP addresses • Use DNS RR as well as ELB does • Route53 Health Check can be used to detect failure Request flow VPC Subnet LVS + keepalived LVS + keepalived WebWebWeb 172.31.64.0/20 Secondary: 172.31.78.1 Primary: 172.31.78. 100 Primary: 172.31.78.101 VRRP Availability Zone Availability Zone Amazon Route 53
  • 21. Building Multi-AZ architecture with LVS VPC Subnet LVS + keepalived LVS + keepalived WebWebWeb 172.31.0.0/20 Elatic IP Secondary: 172.31.24.1 Primary: 172.31.24.100 Primary: 172.31.24.1 01 VRRP If you have to stick to one single IP address (e.g. due to NAT traversal issue for UDP based services) • Perform Heartbeating b/w VRRP master nodes and move EIP upon failover Request flow Availability Zone Availability Zone VPC Subnet LVS + keepalived WebWebWeb 172.31.64.0/20 Primary: 172.31.78.101 VRRP LVS + keepalived Secondary: 172.31.78.1 Primary: 172.31.78. 100
  • 22. Concluding remarks VPC gives you freedom on L2 which allows: • L2 NAT • L2 addressing • Pseudo Broadcast/Multicast Pseudo IP Multicast enables most applications which require IP multicast • Confirmed LVS + Keepalived works well LVS has advantages over other LBs depending on use cases • Don’t forget to build Multi-AZ architecture Use VPC and get more options for architecting!

×