curl is a command line tool used to transfer data from or to a server using various protocols. It allows sending HTTP requests and receiving HTTP responses, including reading response headers and content. curl is useful for web developers to test HTTP requests and debug communications between servers and clients. It has options to set request headers, send POST data, handle cookies, perform authentication, and more.

1. cURL
( )
2011-09-12( )

2. curl
- transfer a URL
• curl is a tool to transfer data from
or to a server, using one of the
supported protocols (DICT, FILE,
FTP, FTPS, GOPHER, HTTP,
HTTPS, IMAP, IMAPS, LDAP,
LDAPS, POP3, POP3S, RTMP, RTSP,
SCP, SFTP, SMTP, SMTPS, TELNET
and TFTP).
2011-09-12( )

3. why cURL is useful?
• curl -O http://example.com/
wallpapers/img[01-19].jpg
2011-09-12( )

4. why cURL is useful
for Web Developers
2011-09-12( )

5. HTTP
$ telnet www.google.co.jp 80
Trying 74.125.153.105...
Connected to www.l.google.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.google.co.jp
< >
2011-09-12( )

6. GET / HTTP/1.1
[method] [request-uri] HTTP/1.1
2011-09-12( )

7. 2011-09-12( )

8. Host: www.google.co.jp
Host: < >
2011-09-12( )

9. HTTP
$ telnet www.google.co.jp 80
Trying 74.125.153.105...
Connected to www.l.google.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.google.co.jp
< >
2011-09-12( )

10. curl
$ curl -v http://www.google.co.jp
* About to connect() to www.google.co.jp port 80
(#0)
* Trying 74.125.153.106... connected*
Connected to www.google.co.jp (74.125.153.106)
port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.21.7 ...
> Host: www.google.co.jp
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 08 Sep 2011 06:47:44 GMT
< Expires: -1
< Cache-Control: private, max-age=0
2011-09-12( )

11. curl --head
curl --verbose|-v
(curl --trace-ascii dump)
2011-09-12( )

12. HTTP Header
$ curl -v http://www.google.co.jp
* (ry)
> GET / HTTP/1.1
> User-Agent: curl/7.21.7 ...
> Host: www.google.co.jp
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 08 Sep 2011 06:47:44 GMT
< Expires: -1
< Cache-Control: private, max-age=0
2011-09-12( )

13. HTTP Request Headers
Accept
Accept-Language
Authorization
Host
Referer
User-Agent
2011-09-12( )

14. $ curl -v -H 'Accept-Language: ja'
http://twitter.com |
nokogrep -a content 'meta[name=description]'
> GET / HTTP/1.1
> User-Agent: curl/7.21.7
> Host: twitter.com
> Accept: */*
> Accept-Language: ja
>
< HTTP/1.1 200 OK
...
2011-09-12( )

15. $ curl -v -H 'Accept-Language: en'
http://twitter.com |
nokogrep -a content 'meta[name=description]'
> GET / HTTP/1.1
> User-Agent: curl/7.21.7
> Host: twitter.com
> Accept: */*
> Accept-Language: en
>
< HTTP/1.1 200 OK
...
Instantly connect to what's most important to
you. Follow your friends, experts, favorite
celebrities, and breaking news.
2011-09-12( )

16. $ curl -v
-H 'Accept: application/json'
http://jobs.dev/jobs |
ruby -rpp -rjson -e 'pp JSON.parse(STDIN.read)'
> GET /jobs HTTP/1.1
> User-Agent: curl/7.21.7
> Host: jobs.dev
> Accept: application/json
2011-09-12( )

17. < HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< X-UA-Compatible: IE=Edge
< ETag: "1490b2d47b09a5abb92bd7eb09c4c51e"
< Cache-Control:
max-age=0, private, must-revalidate
< X-Runtime: 0.059816
< Connection: keep-alive
< Transfer-Encoding: chunked
[{"created_at"=>"2011-09-07T05:56:22Z",
"deadline"=>"2011-09-07",
"description"=>" !!!",
"id"=>1,
"public"=>true,
"title"=>"10 ",
"updated_at"=>"2011-09-07T09:59:33Z"}]
2011-09-12( )

18. HTTP Response
HTTP/1.1 200 OK
Date: Mon, 12 Sep 2011 01:27:34 GMT
Server: hi
Status: 200 OK
ETag: "ceedd0ac4f0339411aab820e225afcdf"
Content-Type: text/html; charset=utf-8
Content-Length: 44188
Pragma: no-cache
Expires: Tue, 31 Mar 1981 05:00:00 GMT
Cache-Control: no-cache, no-store, must-
revalidate, pre-check=0, post-check=0
X-XSS-Protection: 1; mode=block
2011-09-12( )

19. HTTP Response
HTTP/1.1 200 OK
Date: Mon, 12 Sep 2011 01:27:34 GMT
Server: hi
Status: 200 OK
ETag: "ceedd0ac4f0339411aab820e225afcdf"
Content-Type: text/html; charset=utf-8
Content-Length: 44188
Pragma: no-cache
Expires: Tue, 31 Mar 1981 05:00:00 GMT
Cache-Control: no-cache, no-store, must-
revalidate, pre-check=0, post-check=0
X-XSS-Protection: 1; mode=block
2011-09-12( )

20. Status line
200
302
400
404
500
503
2011-09-12( )

21. HTTP Status is app UI
200
404 ?
410(Gone)
2011-09-12( )

22. HTTP Response Headers
Location
Server
Content-Type
Content-Length
Expire, ETag, Cache-Control
2011-09-12( )

23. curl --data|-d
2011-09-12( )

24. POST via curl
$ curl -v
-d email=moro@example.jp
-d password=hogehoge
http://example.com/users
POST /users HTTP/1.1
User-Agent: curl/7.21.7
Host: example.com
Accept: */*
Content-Length: 36
Content-Type:
application/x-www-form-urlencoded
2011-09-12( )

25. HTTP/1.1 201 Created
Content-Type: application/json
Date: Mon, 12 Sep 2011 01:43:41 GMT
Server: journey/0.4.0
Content-Length: 71
Connection: keep-alive
{"email":"moro@example.jp","_id":"4e6d63cd981
5710100000005","profiles":[]}
2011-09-12( )

26. curl --user|-u
2011-09-12( )

27. HTTP Authentication
$ curl -v
-u moro@example.jp:hogehoge
http://example.com/account
GET /account HTTP/1.1
Authorization:
Basic bW9yb0BleGFtcGxlLmpwOmhvZ2Vob2dl
User-Agent: curl/7.21.7
Host: example.com
Accept: */*
2011-09-12( )

28. HTTP Authentication
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 12 Sep 2011 01:51:00 GMT
Server: journey/0.4.0
Content-Length: 71
Connection: keep-alive
{"email":"moro@example.jp","_id":"4e6d63cd9
815710100000005","profiles":[]}
2011-09-12( )

29. cookie
&
cURL
http://flic.kr/p/5ekGdX
2011-09-12( )

30. cookie
• HTTP {Cookie}
Web
http://www.studyinghttp.net/cookies
2011-09-12( )

31. [res] Set-Cookie
[req] Cookie
• Netscape
Set-Cookie Cookie 2
HTTP
http://www.studyinghttp.net/cookies#HeadersForCookie
2011-09-12( )

32. 2011-09-12( )

33. 2011-09-12( )

34. curl --cookie|-b
2011-09-12( )

35. without cookie
$ curl -q http://jobs.dev |
nokogrep '#header p.greeting'
Who are you?
Sign in
2011-09-12( )

36. Cookie & cURL
$ curl -v
-b '_jobs_session=BAh7CEkiD3Nl( )'
http://jobs.dev |
nokogrep '#header p.greeting'
GET / HTTP/1.1
User-Agent: curl/7.21.7
Host: jobs.dev
Accept: */*
Cookie: _jobs_session=BAh7CEkiD3Nl( )
2011-09-12( )

37. Cookie & cURL
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-UA-Compatible: IE=Edge
ETag: "8c0c63144925e4cc29b655906c221a3f"
Set-Cookie:_jobs_session=BAh7CEkiD3Nl( )
X-Runtime: 0.156343
Connection: keep-alive
Transfer-Encoding: chunked
Hi moro@example.jp
Sign out
2011-09-12( )

38. 2011-09-12( )

39. $ curl -v
-d 'user[email]=moro@example.jp'
-d 'user[password]=hogehoge'
http://jobs.dev/users/sign_in
POST /users/sign_in HTTP/1.1
User-Agent: curl/7.21.7
Host: jobs.dev
Accept: */*
Content-Length: 51
Content-Type:
application/x-www-form-urlencoded
2011-09-12( )

40. HTTP/1.1 302 Moved Temporarily
Location: http://jobs.dev/
Content-Type: text/html; charset=utf-8
X-UA-Compatible: IE=Edge
Cache-Control: no-cache
Set-Cookie: _jobs_session=BAh7CE( )
X-Runtime: 0.217055
Connection: keep-alive
Transfer-Encoding: chunked
<html><body>You are being <a
href="http://jobs.dev/">redirected</
a>.</body></html>
2011-09-12( )

41. $ curl -v
-b '_jobs_session=BAh7CE( )'
http://jobs.dev/ |
nokogrep '#header p.greeting'
GET / HTTP/1.1
User-Agent: curl/7.21.7
Host: jobs.dev
Accept: */*
Cookie: _jobs_session=BAh7CE( )
2011-09-12( )

42. HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-UA-Compatible: IE=Edge
ETag: "98d16875bdceb8c5451e7706e6071ece"
Cache-Control:
max-age=0, private, must-revalidate
Set-Cookie: _jobs_session=BAh7CUki( )
X-Runtime: 0.193951
Connection: keep-alive
Transfer-Encoding: chunked
Hi moro@example.jp
Sign out
2011-09-12( )

43. 2011-09-12( )

44. see also
• http://www.studyinghttp.net/
• http://www.studyinghttp.net/
rfc_ja/rfc2616
• http://amazon.jp/dp/
4774142042/morodiary05-22/
ref=nosim/
2011-09-12( )