Михаил расскажет о внедрении IPv6 в основной части Яндекса — его поиске. Вы узнаете, как это происходило, с каким проблемами пришлось столкнуться, как их решали и какие есть планы на дальнейшее развитие.

2. Implementing
IPv6 in
Yandex.Search
Michael Zelenkov
Yandex.Search

3. IPv6 support inside
Yandex.Search

4. First Yandex IPv6 net
inet6num:
netname:
descr:
country:
org:
admin-c:
tech-c:
status:
mnt-by:
mnt-lower:
mnt-routes:
source:
2a02:6b8::/32
RU-YANDEX-20080815
YANDEX LLC
RU
ORG-YA1-RIPE
YNDX1-RIPE
YNDX1-RIPE
ALLOCATED-BY-RIR
RIPE-NCC-HM-MNT
YANDEX-MNT
YANDEX-MNT
RIPE # Filtered
organisation:
org-name:
org-type:
ORG-YA1-RIPE
YANDEX LLC
LIR

5. 2009
Running IPv6 on internal
technology network - fastbone

6. Yandex uses two physically
separated networks:
backbone and fastbone.

7. IPv4
backbone
802.1q
dual
stack
fastbone
IPv4+IPv6

8. IPv4
backbone
802.1q
dual
stack
fastbone
IPv6

9. IPv4
IPv6
32 bit
128 bit

10. IPv4
IPv6
127.0.0.1
::1

11. §2.2 RFC 4291
3. An alternative form that is sometimes more convenient when dealing
with a mixed environment of IPv4 and IPv6 nodes is
x:x:x:x:x:x:d.d.d.d, where the 'x's are the hexadecimal values of
the six high-order 16-bit pieces of the address, and the 'd's are
the decimal values of the four low-order 8-bit pieces of the
address (standard IPv4 representation).

12. DNS IPv4-mapping
……………
ws5-023.yandex.ru IN
ws5-027.yandex.ru IN
ws5-031.yandex.ru IN
A
A
A
93.158.151.38
93.158.151.39
93.158.151.40
……………
ws5-023.search.yandex.net
ws5-027.search.yandex.net
ws5-031.search.yandex.net
……………
IN
IN
IN
AAAA
AAAA
AAAA
2a02:6b8:0:287::93.158.151.38
2a02:6b8:0:287::93.158.151.39
2a02:6b8:0:287::93.158.151.40

13. Welcome to FreeBSD!
# ping6 2a02:6b8:0:14a1::178.154.167.10
PING
2a02:6b8:0:14a1::178.154.167.10(2a02:6b8:0:14a1::b29a:a70a)
56 data bytes
64 bytes from 2a02:6b8:0:14a1::b29a:a70a: icmp_seq=1 ttl=64
time=0.029 ms
^C
--- 2a02:6b8:0:14a1::178.154.167.10 ping statistics --1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.029/0.029/0.029/0.000 ms

14. IPv6 support on FreeBSD
/etc/rc.conf.d/ip6addrctl
ipv6_enable="YES”
/etc/rc.conf.d/auto_linklocal
ipv6_enable="YES"
sysctl net.inet6.ip6.accept_rtadv=0

15. /etc/ya.subr
IPv4 backbone network/mask
IPv6 backbone network/mask vlan
IPv4 fastbone network/mask vlan
IPv6 fastbone network/mask vlan
myt4 178.154.167.0/25 2a02:6b8:0:14a1::/64
2a02:6b8:0:a05::/64 764
667
178.154.151.0/25
767
# . /etc/ya.subr
# ya_network_info 178.154.167.1
ya_net_num=99 ya_DC=myt4 ya_net=178.154.167.0/25 ya_VLAN=667
ya_ip=178.154.167.1 ya_defrouter=178.154.167.126 ya_porter=178.154.167.1
ya_porter_backup=178.154.164.1 ya_porters="178.154.167.1 178.154.164.1
178.154.165.1 178.154.172.1 178.154.173.1 178.154.174.1 178.154.175.1
178.154.149.1" ya_broadcast=178.154.167.127 ya_netmask=255.255.255.128
ya_net6=2a02:6b8:0:14a1::/64 ya_ip6=2a02:6b8:0:14a1::178.154.167.1
ya_defrouter6=fe80::1 ya_prefix6=64 ya_FB_net=178.154.151.0/25
ya_FB_VLAN=767 ya_FB_ip=178.154.151.1 ya_FB_defrouter=178.154.151.126
ya_FB_netmask=255.255.255.128 ya_FB_ipv6_net=2a02:6b8:0:a05::/64
ya_FB_ipv6_VLAN=764 ya_FB_ipv6_ip=2a02:6b8:0:a05::178.154.167.1
ya_FB_ipv6_defrouter=2a02:6b8:0:a05::1 ya_FB_ipv6_netmask=64
#

16. /etc/rc.d/ya-netif & /etc/rc.d/ya-fastbone
# cat /etc/rc.conf.local
ipaddr="178.154.167.1"
hostname="porter080.yandex.ru"
adapter_options="-lro -tso"
#
# ifconfig | grep inet
inet 178.154.167.1 netmask 0xffffff80 broadcast 178.154.167.127
inet6 fe80::22cf:30ff:fe00:8e8%em0 prefixlen 64 scopeid 0x4
inet6 2a02:6b8:0:14a1::b29a:a701 prefixlen 64
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xc
inet 127.0.0.1 netmask 0xff000000
inet 178.154.151.1 netmask 0xffffff80 broadcast 178.154.151.127
inet6 fe80::22cf:30ff:fe00:8e8%vlan767 prefixlen 64 scopeid 0xd
inet6 2a02:6b8:0:a05::b29a:a701 prefixlen 64
inet6 fe80::22cf:30ff:fe00:8e8%vlan764 prefixlen 64 scopeid 0xe
#

17. /etc/rc.d/ya-routing
# netstat -rn -f inet
Routing tables
Internet:
Destination
default
5.45.194.0/24
5.45.202.0/24
.............
Internet6:
Destination
::/96
default
::1
::ffff:0.0.0.0/96
..............
2a02:6b8:b000::/64
2a02:6b8:f000::/64
fe80::/10
..............
Gateway
178.154.167.126
178.154.167.126
178.154.167.126
Flags
UGS
UGS
UGS
Refs
Use
0 52124803
0
0
0
0
Netif Expire
em0
em0
em0
Gateway
::1
fe80::1%em0
link#12
::1
Flags
UGRS
UGS
UH
UGRS
fe80::1%em0
2a02:6b8:0:a05::1
::1
UGS
UGS
UGRS
Netif Expire
lo0 =>
em0
lo0
lo0
em0
vlan764
lo0

18. IPv4+IPv6
backbone
802.1q
dual
stack
fastbone
IPv6

19. yandex.ru & search.yandex.net
# host ws5-000.yandex.ru
ws5-000.yandex.ru has address 93.158.148.33
ws5-000.yandex.ru mail is handled by 10 outbound-relay.yandex.net.
# host ws5-000.search.yandex.net
ws5-000.search.yandex.net has IPv6 address 2a02:6b8:0:287::5d9e:9421
# host ws5-000.fb.yandex.ru
ws5-000.fb.yandex.ru has IPv6 address 2a02:6b8:0:a02::5d9e:9421
# host fb-ws5-000.search.yandex.net
fb-ws5-000.search.yandex.net has IPv6 address 2a02:6b8:0:a02::5d9e:9421
#

20. named.conf

21. short names & resolv.conf
# hostname –s
porter080
#
#
# cat /etc/resolv.conf
# Generated on 05/07/13 at 14:49:10 by resolvconf for porter080 (inet
178.154.167.1 netmask 255.255.255.128)
domain yandex.ru
nameserver 127.0.0.1
nameserver 141.8.146.1
options timeout:1 attempts:1
#
# host porter080
porter080.yandex.ru has address 178.154.167.1
porter080.yandex.ru mail is handled by 10 outbound-relay.yandex.net.
# host porter080.search
Host porter080.search not found: 3(NXDOMAIN)
# host porter080.search.yandex.net
porter080.search.yandex.net has IPv6 address 2a02:6b8:0:14a1::b29a:a701
#

22. troubles with ipmi

23. http://www.intel.com/support/network/sb/CS-028553.htm
23

24. http://lists.ipxe.org/pipermail/ipxe-devel/2012-August/001781.html
24

25. Some graphs
25

26. Some graphs
26

27. Some graphs
27

28. Some graphs
28

29. To the IPv6-future!

30. Michael Zelenkov
mzelenkov@yandex-team.ru
@mzelenkov