course: Data mining Language:java Platform: Eclipse This note is about web crawler or spider which is created with selenium web driver and using java.. this will help you to access any element on website to check validity of your website. In this i have used firebug and fireXpath add-ons of mozila firefox which make easier to access any element. JDK and JRE are used for java. You can access links,elements,current link etc with help of these notes.

1. Selenium Webcrawler
PRODUCED BY:
RABIA KHALID

2. Download and install java
 First of all you need to install JDK(java development kit)
Second you have to install JRE(java runtime environment)
http://www.oracle.com/technetwork/java/javase/downloads/index.html

3. Download JDK and JRE

4. Install Firefox
 Now install mozila firefox
 https://www.mozilla.org/en-US/firefox/new/

5. Install Firebug and FireXpath
 Now install firebug and fire path into mozila firefox
https://addons.mozilla.org/en-us/firefox/addon/firebug/
 Now install FIreXpath
https://addons.mozilla.org/fr/firefox/addon/firepath/

6. Firebug
 Now click on firebug
On right-top of browser
To get firebug on.
 After installing it will be
looked like this.

7. Download Eclipse
 Download Eclipse..
 http://www.eclipse.org/downloads/packages/eclipse-ide-java-and-dsl-developers/marsr
 Download 64-bit or 32-bit according to your operating system..

8. Install selenium web Driver
 selenium web driver
 http://docs.seleniumhq.org/download/

9. Start Eclipse
 Extract all files after downloading and install them..
 Now Start eclipse..

10. Conti…
 First time when you start eclipse it will ask for workspace where your work
will be stored.
 After selecting workspace folder , eclipse will be opened.
 Now create new project in eclipse and import all libraries of selenium.

11. Create New Project
 Create project
 File -> New -> java project

12. Conti..
 Give name to your project

13. Create class
 New right click on your project.
 create new class.

14. Conti..
 Now give name to your class.
 Select public static void option.
 finish

15. Add libraries of selenium
 Now right click on your project
 select properties.

16. Conti…
 Now select java build path.

17. Conti..
 Now select libraries
 Add external JARs..

18. Conti…
 Now import all selenium JARs..

19. Conti..
 It will be looked like this.

20. Code
 Now selenium libs are added to your project.
 Now write your code to test.
…………….. Start…………….
 WebDriver driver = new FirefoxDriver();
 driver.get("URL"); // write URL here of any website to open it in browser
 driver.close();

21. Test webdriver
Now write code and test by
Click on run button.
This code will run google on
mozilafirefox.IT means test is passed.
If any error occurs then check your
libraries of selenium. all libs should
be imported.

22. Commands
Clicking on any element or button of webpage
driver.findElement(By.id("submitButton")).click();
Store text of targeted element in variable
String dropdown = driver.findElement(By.tagName("select")).getText();
Typing text in text box or text area.
driver.findElement(By.name("fname")).sendKeys("My First Name");

23.  Applying Implicit wait in webdriver
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
 Get page title in selenium webdriver
driver.getTitle();
 Get Current Page URL In Selenium WebDriver
driver.getCurrentUrl();
 Navigate to URL or Back or Forward in Selenium Webdriver
driver.navigate().to("http://www.google.com"); // you can add any
website
driver.navigate().back(); // navigate to back webpage
driver.navigate().forward(); //navigate to forward webpage

24.  Here is my code which extract links from a website and show data by
getting elements of website..
 https://github.com/RabiaMughal001/crawler/blob/master/webcrawler.jav
a
 Thanks….
 If you have any kind of problem then email me at
rabiakhalid945@gmail.com