Selenium Webcrawler
PRODUCED BY:
RABIA KHALID
Download and install java
 First of all you need to install JDK(java development kit)
Second you have to install JRE(java runtime environment)
http://www.oracle.com/technetwork/java/javase/downloads/index.html
Download JDK and JRE
Install Firefox
 Now install mozila firefox
 https://www.mozilla.org/en-US/firefox/new/
Install Firebug and FireXpath
 Now install firebug and fire path into mozila firefox
https://addons.mozilla.org/en-us/firefox/addon/firebug/
 Now install FIreXpath
https://addons.mozilla.org/fr/firefox/addon/firepath/
Firebug
 Now click on firebug
On right-top of browser
To get firebug on.
 After installing it will be
looked like this.
Download Eclipse
 Download Eclipse..
 http://www.eclipse.org/downloads/packages/eclipse-ide-java-and-dsl-developers/marsr
 Download 64-bit or 32-bit according to your operating system..
Install selenium web Driver
 selenium web driver
 http://docs.seleniumhq.org/download/
Start Eclipse
 Extract all files after downloading and install them..
 Now Start eclipse..
Conti…
 First time when you start eclipse it will ask for workspace where your work
will be stored.
 After selecting workspace folder , eclipse will be opened.
 Now create new project in eclipse and import all libraries of selenium.
Create New Project
 Create project
 File -> New -> java project
Conti..
 Give name to your project
Create class
 New right click on your project.
 create new class.
Conti..
 Now give name to your class.
 Select public static void option.
 finish
Add libraries of selenium
 Now right click on your project
 select properties.
Conti…
 Now select java build path.
Conti..
 Now select libraries
 Add external JARs..
Conti…
 Now import all selenium JARs..
Conti..
 It will be looked like this.
Code
 Now selenium libs are added to your project.
 Now write your code to test.
…………….. Start…………….
 WebDriver driver = new FirefoxDriver();
 driver.get("URL"); // write URL here of any website to open it in browser
 driver.close();
Test webdriver
Now write code and test by
Click on run button.
This code will run google on
mozilafirefox.IT means test is passed.
If any error occurs then check your
libraries of selenium. all libs should
be imported.
Commands
Clicking on any element or button of webpage
driver.findElement(By.id("submitButton")).click();
Store text of targeted element in variable
String dropdown = driver.findElement(By.tagName("select")).getText();
Typing text in text box or text area.
driver.findElement(By.name("fname")).sendKeys("My First Name");
 Applying Implicit wait in webdriver
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
 Get page title in selenium webdriver
driver.getTitle();
 Get Current Page URL In Selenium WebDriver
driver.getCurrentUrl();
 Navigate to URL or Back or Forward in Selenium Webdriver
driver.navigate().to("http://www.google.com"); // you can add any
website
driver.navigate().back(); // navigate to back webpage
driver.navigate().forward(); //navigate to forward webpage
 Here is my code which extract links from a website and show data by
getting elements of website..
 https://github.com/RabiaMughal001/crawler/blob/master/webcrawler.jav
a
 Thanks….
 If you have any kind of problem then email me at
rabiakhalid945@gmail.com

Selenium webcrawler

  • 1.
  • 2.
    Download and installjava  First of all you need to install JDK(java development kit) Second you have to install JRE(java runtime environment) http://www.oracle.com/technetwork/java/javase/downloads/index.html
  • 3.
  • 4.
    Install Firefox  Nowinstall mozila firefox  https://www.mozilla.org/en-US/firefox/new/
  • 5.
    Install Firebug andFireXpath  Now install firebug and fire path into mozila firefox https://addons.mozilla.org/en-us/firefox/addon/firebug/  Now install FIreXpath https://addons.mozilla.org/fr/firefox/addon/firepath/
  • 6.
    Firebug  Now clickon firebug On right-top of browser To get firebug on.  After installing it will be looked like this.
  • 7.
    Download Eclipse  DownloadEclipse..  http://www.eclipse.org/downloads/packages/eclipse-ide-java-and-dsl-developers/marsr  Download 64-bit or 32-bit according to your operating system..
  • 8.
    Install selenium webDriver  selenium web driver  http://docs.seleniumhq.org/download/
  • 9.
    Start Eclipse  Extractall files after downloading and install them..  Now Start eclipse..
  • 10.
    Conti…  First timewhen you start eclipse it will ask for workspace where your work will be stored.  After selecting workspace folder , eclipse will be opened.  Now create new project in eclipse and import all libraries of selenium.
  • 11.
    Create New Project Create project  File -> New -> java project
  • 12.
    Conti..  Give nameto your project
  • 13.
    Create class  Newright click on your project.  create new class.
  • 14.
    Conti..  Now givename to your class.  Select public static void option.  finish
  • 15.
    Add libraries ofselenium  Now right click on your project  select properties.
  • 16.
    Conti…  Now selectjava build path.
  • 17.
    Conti..  Now selectlibraries  Add external JARs..
  • 18.
    Conti…  Now importall selenium JARs..
  • 19.
    Conti..  It willbe looked like this.
  • 20.
    Code  Now seleniumlibs are added to your project.  Now write your code to test. …………….. Start…………….  WebDriver driver = new FirefoxDriver();  driver.get("URL"); // write URL here of any website to open it in browser  driver.close();
  • 21.
    Test webdriver Now writecode and test by Click on run button. This code will run google on mozilafirefox.IT means test is passed. If any error occurs then check your libraries of selenium. all libs should be imported.
  • 22.
    Commands Clicking on anyelement or button of webpage driver.findElement(By.id("submitButton")).click(); Store text of targeted element in variable String dropdown = driver.findElement(By.tagName("select")).getText(); Typing text in text box or text area. driver.findElement(By.name("fname")).sendKeys("My First Name");
  • 23.
     Applying Implicitwait in webdriver driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);  Get page title in selenium webdriver driver.getTitle();  Get Current Page URL In Selenium WebDriver driver.getCurrentUrl();  Navigate to URL or Back or Forward in Selenium Webdriver driver.navigate().to("http://www.google.com"); // you can add any website driver.navigate().back(); // navigate to back webpage driver.navigate().forward(); //navigate to forward webpage
  • 24.
     Here ismy code which extract links from a website and show data by getting elements of website..  https://github.com/RabiaMughal001/crawler/blob/master/webcrawler.jav a  Thanks….  If you have any kind of problem then email me at rabiakhalid945@gmail.com

Editor's Notes

  • #6 FireBug is used to extract elements from website which we need.. There is other slide in which I have shown how to used firebug.