Golang in goagent
Upcoming SlideShare
Loading in...5
×
 

Golang in goagent

on

  • 2,732 views

 

Statistics

Views

Total Views
2,732
Views on SlideShare
2,703
Embed Views
29

Actions

Likes
6
Downloads
36
Comments
0

3 Embeds 29

https://twitter.com 26
https://si0.twimg.com 2
https://duckduckgo.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Golang in goagent Golang in goagent Presentation Transcript

  • GoLang in GoAgent@phuslu2012/11/10
  • GoAgent is a• Google AppEngine Application• HTTP/HTTPS Proxy• Written by Python(2011)
  • GoAgent ABC
  • GoAgent Tricks• HOSTS• DNS resolve• CRLF Injection• HTTPS multiple-connection
  • GoAgent Tricks - HostsD:>ping www.google.cn正在 Ping www.google.cn [203.208.46.161] 具有 32 字节的数据 :来自 203.208.46.161 的回复 : 字节 =32 时间 =32ms TTL=43来自 203.208.46.161 的回复 : 字节 =32 时间 =33ms TTL=42来自 203.208.46.161 的回复 : 字节 =32 时间 =32ms TTL=42来自 203.208.46.161 的回复 : 字节 =32 时间 =33ms TTL=42telent 203.208.46.161 80GET /fetch.py HTTP/1.1Host: goagentd.appspot.comHTTP/1.1 200 OKContent-Length: XXXXHello GoAgent
  • GoAgent Tricks - Hosts>>> socket.gethostbyname(www.google.com)74.125.128.99‘>>> print urllib2.urlopen(urllib2.Request(https://74.125.128.99/fetch.py,headers={Host:goagent-dd.appspot.com})).read().decode(utf8)GoAgent Python Server 2.0.8 已经在工作了,部署时间 2012-09-29 02:03:19
  • GoAgent Tricks – TCP DNS resolveD:>nslookup -vc www.facebook.com 8.8.8.8服务器 : google-public-dns-a.google.com•Address: 8.8.8.8非权威应答 :名称 : www.facebook.comAddresses: 2a03:2880:10:1f02:face:b00c:: 69.171.234.21
  • GoAgent Tricks – UDP DNS resolve1. Send UDP DNS query to 8.8.8.8/8.8.4.4/168.95.1.12. Socket recvform 512 bytes DNS answer3. If DNS answer in blacklist, goto 2 1. blacklist set([4.36.66.178, 8.7.198.45, 37.61.54.158, 46.82.174.68, 59.24.3.173, 64.33.88.161, 64.33.99.47, 64.66.163.251‘, ‘…’]4. If DNS answer not in blacklist, return
  • GoAgent Tricks – CRLF Inectiontelent www.google.com.hk 80rnGET /fetch.py HTTP/1.1Host: goagentd.appspot.comHTTP/1.1 200 OKContent-Length: XXXXHello GoAgent
  • GoAgent Tricks– new CRLF Inectiontelent www.google.com.hk 80GET / HTTP/1.1rnrnrrrnGET /fetch.py HTTP/1.1Host: goagentd.appspot.comHTTP/1.1 304 MoveLocation: http://www.google.com.hk/webhp?hl=zh-CNrnrnHTTP/1.1 200 OKContent-Length: XXXXHello GoAgent
  • GoAgent Tricks – HTTPS Connect
  • Go is a• New• Experimental• Concurrent• Garbage-Collected• Systems Language.
  • Why a new language?• Go fast!• Make programming fun again.• No new major systems language in a decade. But much has changed: - sprawling libraries & dependency chains - dominance of networking - client/server focus - massive clusters - the rise of multi-core CPUs• Major systems languages were not designed with all these factors in mind.
  • Why move to GoLang?• Fast• Native.• More Standard Libs.• GoRoutine• Multi-Core support?
  • Why move to GoLang - Fast• Fast Event Loop – Epoll/IOCP backend in Linux/Windows – Flat net/http package• Static Type – Find out error in compiling time – Run faster in runtime
  • Why move to GoLang - Native• Standalone – No Runtime – No Dependency• Small – 1 M bytes in windows
  • Why move to GoLang - Standard Libs• crypto/x509 VS pyOpenSSL• Same points: – Issue key/certs• Difference: – crypto/x509 is a standard and pure golang lib, no dependency – pyOpenSSL is a 3rd python lib and based on openssl-devel• Conclusion: – pyOpenSSL is hard to integrate – pyOpenSSL must be built in on every platform
  • Why move to GoLang - Standard Libs• index/suffixarray VS str.endswith• Same points: – Judge a domain is a google domain family or not• Python Example: google_domains = [‘.google.com’, ‘.google.com.hk’, ‘.googleapis.com’, ‘.g.cn’] domain = ‘www.facebook.com’ for pattern in google_domains: if domain.endswith(pattern): # direct forward Else: # fetch content from GAE
  • Why move to GoLang - Standard Libs• index/suffixarray VS str.endswith• Same points: – Judge a domain is a google domain family or not• GoLang Example: index := suffixarray.New(google_domains) domain = ‘www.facebook.com’ if index.LookUp(domain) { // direct forward } else { // fetch content from GAE }
  • Why move to GoLang - Standard Libs• index/suffixarray VS str.endswith• Same points: – Judge a domain is a google domain family or not• Conclusion: – index/suffixarray is a killer feature for goagent
  • Why move to GoLang - Standard Libs• What’s more – net/http vs SimpleHTTPServer – log/syslog vs python-logging – crypto/rc4 vs python-rc4 – …• Conclusion: – golang standard libs are effective for networking programing
  • Why move to GoLang - GoRoutine• Python Example
  • Why move to GoLang - GoRoutine• GoLang Example
  • Why move to GoLang – GoRoutine• Conclusion: – GoAgent could connect GAE more faster by using goroutine
  • Why move to GoLang – Multi-core• Python threading vs GoLang goroutine• Python threading is limited by GIL• GoLang runtime.GOMAXPROCS(NCPU) could enable multi-core support on the fly.• Scenarios? – https/openssl – rc4/lzw
  • CONCLUSION• GoLang is a fast, native and effective language• Golang makes networking programming easier.• Goroutine is a killer language feature.• Want to try it?Want to help?Want to build libraries or tools? http://golang.org
  • Any Queries??