This document provides an overview of HTTP including: - HTTP is a stateless protocol that does not require servers to retain user information across requests. - Popular HTTP proxy tools like Fiddler and Burp Suite can be used to inspect and debug HTTP traffic. - Key parts of HTTP include requests methods, response codes, headers for accepting content types, encoding, authentication, and more. - Common players that interact with HTTP include web servers, load balancers, caching servers, CDNs, and security tools.

1. HTTP
All you need to know

2. Who are you?

3. Gökhan Şengün
R&D Business Dev., New Product & Solutions Manager
www.gokhansengun.com
@gokhansengun

4. Aim
• Better understand HTTP basics to debug problems better
• Know HTTP players to see the big picture
• Know useful tools to do things faster

5. HTTP
• HTTP is a stateless protocol.
• How is being stateless like?
• A stateless protocol does not require the server to retain information or
status about each user for the duration of multiple requests.

6. Http Proxy

7. Popular Http Proxies
• Fiddler
• Burp Suite
• Browser Developer Tools (Embedded Proxy)

8. Demo – Bare Metal - Using Telnet

9. Demo – Browser Developer Tool

10. Demo – Fiddler

11. Demo – Burp Suite

12. Http Protocol – Important Parts

13. Methods
Method Used for
GET Retrieve a resource
POST Create / Update a resource [Not Idempotent]
PUT Create / Update a resource [Idempotent]
DELETE Delete a resource
HEAD Retrieve a resource except the body

14. Response Codes
Code Meaning
1xx Informative
2xx Success
3xx Requires Additional Action
4xx Client Error (It is your fault)
5xx Server Error (It is my fault)

15. Accept (Req)
MIME used for media-type. Client gives hint about the types that
it understands well and preference.
Syntax:
• Accept: <MIME_type>/<MIME_subtype>
Examples:
• Accept: application/json, text/xml;q=0.9, */*;q=0.8

16. Content-Type (Req / Resp)
MIME used for media-type
Examples:
• Content-Type: text/html; charset=utf-8
• Content-Type: application/json
• Content-Type: text/xml

17. Demo – Accept and Content-Type

18. Host (Req)
• Hints the web server about the domain name requested
• Optionally includes port, default
• HTTP: 80
• HTTPS: 443
Examples:
• Host: www.gokhansengun.com
• Host: localhost:8090

19. Connection (Req / Resp)
• Hint from both client and the web server about TCP connection
• close: if either party for some reason wants to close
• keep-alive: if either party want to keep open for further requests
• Persistent connection (default in HTTP/1.1
• RFC 2616 limits 2 connection per host, browsers have 6 now.
Examples:
• Connection: close
• Connection: keep-alive

20. BTW: Http Pipelining
• Only Idempotent
requests allowed (GET,
HEAD)
• Guess why?
• Has benefit only on
high latency setups.

21. Accept-Languge (Req)
• Hint from client about its language preference
Examples:
• Accept-Language: en-US,en;q=0.8
• Accept-Language: tr-TR, tr;q=0.9, en;q=0.8, *;q=0.5

22. Demo – Accept-Language

23. Accept-Encoding (Req)
• Hint from client about its encoding preference
Examples:
• Accept-Encoding: Accept-Encoding: gzip, deflate, sdch
• Omit for non-encoding

24. Demo – Accept-Encoding

25. Referer (Req)
• Hint from client about the last page user navigated from.
• Allows analytics, caching, logging
Examples:
• Referer: http://ads.xyz.com

26. User-Agent (Req)
• Hint from client about the type of client
Examples:
• User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/54.0.2840.71 Safari/537.36

27. Cache-Control (Req / Resp)
• Hint from server to all over the world about resource’s cache
eligibility.
• Cache-Control: no-cache
• Cache-Control: public
• Cache-Control: private
• Cache-Control: no-store
• Cache-Control: max-age=300
• Cache-Control: public, max-age=31536000

28. Post / Redirect / Get Pattern (1)
• Problem (Multiple Post requests)

29. Post / Redirect / Get Pattern (2)

30. Post / Redirect / Get Pattern (3)
• Solution

31. Demo – Mix

32. HTTP Players
• Web Servers
• Load Balancers
• DDoS Protection and WAF Systems
• Cache Server
• CDN (Content Delivery Networks)
• Cloudflare

33. Web Servers
• Nginx
• Apache
• IIS

34. Load Balancers
• Balance HTTP load between servers
• Balance statefully (needs your SSL private key)
• Cache responses
• Alters requests and responses
• Blocks, rate-limits requests
• Does SSL-offloading (needs your SSL private key and
beneficial only if you have HW LB)

35. DDoS Protection Systems and WAF
• Observes traffic (needs your SSL private key)
• Detects malicious activity – several attacks
• Blocks IP, IP Range
• Redirects to No CAPTCHA or reCAPTCHA
• Rate-limits requests

36. Cache Servers
• Caches any type of HTTP responses from origion
• Could be static file or reference data
• Like very very simple KV store
• Powerful if scripting allowed
Examples:
• Varnish
• Nginx

37. CDN (Content Delivery Network)
• Caches the content on the edges
• Request does not enter your data center
• Very very efficient

39. Cloudflare
• CDN
• Load Balancing (Cloud – Region Based through DNS)
• DDoS
• WAF
• Rate Limiting
• Website Optimization
• Cache Header Optimization
• AutoMinify
• Aggressive Gzip
• Automatic Content Caching

40. Cookies
• Helps stateless HTTP protocol statefulness when necessary,
• Has restrictions in EU.
Types:
• Session Cookies
• Persistent Cookies

41. Authentication and Tokens
• Basic Authentication
• Forms Authentication
• Token Authentication

42. Session Cookie vs Token Auth

43. HTTP Security
• Use SSL/TLS for transport layer security (HTTPS everything)
• Why?
• Set Cookies with HttpOnly
• Avoid Cross Site Scripting
• Set Cookies with Secure
• Avoid sending cookies in HTTP requests
• Use HSTS (HTTP Strict Transport Security) header
• Instruct browser to comm only with HTTPS for a period of time
• Avoid SSL-stripping attacks

44. HTTP Performance Measurement
• Use Apache ab
• Use Apache JMeter (blogs from www.gokhansengun.com)
• http://loader.io/
• https://www.blazemeter.com/
• Use APM (Application Performance Monitoring) tools
• NewRelic, Dynatrace, Riverbed, App

45. Scaling HTTP
• Use Cache Server
• Use CDN
• Cache Aggressively
• Use DNS load balancing
• Use SPA (Single Page Application) Technique
• Minify and bundle JS / CSS

46. Questions?