Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Spider Course Day 1


Published on

Published in: Technology, Sports, Education
  • Be the first to comment

Spider Course Day 1

  1. 1. INTRODUCTION TO WEB Spider Web Weaving Course. Day 1 Harishankaran K
  2. 2. HTTP <ul><li>Hypertext Transfer Protocol </li></ul><ul><li>HTTP is a request/response protocol between a client and a server. </li></ul><ul><li>The client making an HTTP request—such as a web browser , spider , or other end-user tool—is referred to as the user agent . </li></ul><ul><li>The responding server —which stores or creates resources such as HTML files and images—is called the origin server . </li></ul><ul><li>In between the user agent and origin server may be several intermediaries, such as proxies , gateways , and tunnels . </li></ul>
  3. 3. REQUEST/RESPONSE PROTOCOL <ul><li>The client sends the request. </li></ul><ul><li>The server sends a response according to the request from the client. </li></ul>
  4. 4. REQUEST/RESPONSE PROTOCOL <ul><li>The client sends a HTTP request . </li></ul><ul><li>The server receives the request. </li></ul><ul><li>Server may do some processing according to the request sent. </li></ul><ul><li>It returns HTTP response . </li></ul>
  6. 6. WEB SERVER <ul><li>Program that accepts HTTP requests from the client, and provides an HTTP response to the client. </li></ul><ul><li>The HTTP response usually consists of an HTML document, but can also be a raw file, an image, or some other type of document. </li></ul><ul><li>Examples are Apache, Microsoft IIS, Google GFE, lighthttpd etc. </li></ul>
  7. 7. WEB BROWSER <ul><li>Web browsers communicate with Web servers primarily using HTTP to fetch web pages. </li></ul><ul><li>Examples are Firefox, Opera, Internet Explorer, Elinks, Safari etc </li></ul><ul><li>Web browsers format HTML information for display, so the appearance of a Web page may differ between browsers. </li></ul>
  8. 8. SPIDER <ul><li>A program or automated script which browses the World Wide Web in a methodical, automated manner. </li></ul><ul><li>Other names are web crawler, ants, automatic indexers, bots, and worms. </li></ul><ul><li>Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. </li></ul>
  9. 9. HTTP REQUEST <ul><li>GET /course/ HTTP/1.1 </li></ul><ul><li>Host: spider </li></ul><ul><li>User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20070603 Fedora/ Firefox/ </li></ul><ul><li>Accept: text/html </li></ul><ul><li>Accept-Language: en-us,en </li></ul><ul><li>Accept-Encoding: gzip,deflate </li></ul><ul><li>Accept-Charset: ISO-8859-1,utf-8 </li></ul><ul><li>Keep-Alive: 300 </li></ul><ul><li>Connection: keep-alive </li></ul>
  10. 10. HTTP RESPONSE <ul><li>HTTP/1.x 200 OK </li></ul><ul><li>Date: Mon, 04 Feb 2008 03:58:24 GMT </li></ul><ul><li>Server: Apache/2.2.0 (Fedora) </li></ul><ul><li>X-Powered-By: PHP/5.2.2 </li></ul><ul><li>Content-Length: 2742 </li></ul><ul><li>Connection: close </li></ul><ul><li>Content-Type: text/html; charset=UTF-8 </li></ul>
  11. 11. HTTP STATUS CODES <ul><li>1** Informational </li></ul><ul><li>2** Success </li></ul><ul><li>3** Redirection </li></ul><ul><li>4** Client Error </li></ul><ul><li>5** Server Error </li></ul><ul><li>200 – OK </li></ul><ul><li>403 – Forbidden </li></ul><ul><li>404 – Not Found </li></ul><ul><li>500 – Internal Server Error </li></ul>
  12. 12. SCRIPTING LANGUAGES <ul><li>Web browser should support the scripting languages. </li></ul><ul><li>Apache supports php, python, and many other languages. </li></ul><ul><li>Web server executes the php file in the server to produce dynamic HTML content. </li></ul><ul><li>Examples are PHP, Python, Ruby, Perl etc… </li></ul>
  13. 13. DATABASE SERVER <ul><li>Data Base Management System is used to store the data. </li></ul><ul><li>Most scripting languages have inbuilt API support to connect to the database server and process the data. </li></ul><ul><li>The database server can be a separate server or can run in the same server. </li></ul><ul><li>Example are MySQL, MSSQL, PostgreSQL </li></ul>
  14. 14. WEB MODEL
  15. 15. LAMP <ul><li>WAMP </li></ul>Role Name Operating System Linux Web server Apache Database MySQL Scripting Language PHP Role Name Operating System Windows Web server Apache Database MySQL Scripting Language PHP
  16. 16. LAMP
  17. 17. HTTP SESSION STATE <ul><li>HTTP is a stateless protocol. </li></ul><ul><li>The advantage of a stateless protocol is that hosts do not need to retain information about users between requests, but this forces web developers to use alternative methods for maintaining users' states. </li></ul><ul><li>A common method for solving this problem involves sending and requesting cookies. </li></ul>
  19. 19. COOKIE
  20. 20. INTRODUCTION TO UNIFORM SERVER Spider Web Weaving Course. Day 1 Harishankaran K
  21. 21. STEPS <ul><li>Download Uniserver from the URL mentioned. </li></ul><ul><li>Extract the file. </li></ul><ul><li>Start the server using Server_start file in the Uniserver folder. </li></ul><ul><li>Add the files under W:/www folder. </li></ul><ul><li>Stop the server using Stop file </li></ul>
  22. 22. SAMPLE FILES <ul><li>Index.html </li></ul><ul><li><html> </li></ul><ul><li><head><title>Hello</title></head> </li></ul><ul><li><body> </li></ul><ul><li>Ha ha ha. He he he. Hoo hoo hoo. My first web page is ready. :). </li></ul><ul><li></body> </li></ul><ul><li></html> </li></ul>
  23. 23. SAMPLE FILES <ul><li>index.php </li></ul><ul><li><?php </li></ul><ul><li>echo phpinfo(); </li></ul><ul><li>?> </li></ul>
  24. 24. <ul><li>Thank you </li></ul>