Download presentation

  • 3,692 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
3,692
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
37
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Det är en Ruby bibliotek

Transcript

  • 1. Davor Building test automation frameworks with Ruby (Watir, FireWatir, Selenium RC)
  • 2. Why…
      • Why test?
      • Why test automation?
      • Why test automation framework?
      • Why ruby?
  • 3. Why is software testing necessary?
      • Software is likely to have defects
        • People make errors (mistakes)
        • Errors can cause faults (bugs, defects)
        • Faults can cause failures.
        • Failures can cost a lot of money
      • find defects
      • gain confidence about level of quality
      • prevent future defects
      • evaluate and mitigate risk
  • 4. What are benefits of test automation?
      • Consistent and repeatable testing process
      • Improved regression tests - decrease cost of change
      • Smoke testing
      • More tests can be run in less time
      • 24/7 operation - better use of resources
      • Configuration testing
      • Data driven
      • Human resources are free to perform advanced manual tests
      • Execution of tests that can't be done manually
      • Tests can access system parameters that are not visible to tester
      • Simple reproduction of found defects
      • Testing costs decrease with reuse
  • 5. Test automation limitations
      • Automated tests just complement and not replace manual tests. All test should not be automated.
      • Automated tests cover only a part of possible defects
      • Probability to find defects is lower then for manual tests
      • Automation of test case decrease cost of execution but does not increase ability to find defects
      • Tools can be very complex and require significant training
      • Maintenance of the tests can be expensive
      • Automated test are sensitive to software changes
      • Handling of unexpected behaviour is not so good
  • 6. Software Frameworks
    • Definition (Wikipedia):
    • “ A software framework is a re-usable design for a software system (or subsystem). A software framework may include support programs, code libraries, a scripting language, or other software to help develop and glue together the different components of a software project.”
    In test automation framework describes the collection of components that support your tests. Those components can also be described as frameworks.
  • 7. Why do we need test automation frameworks?
      • Separate routine calls from designed tests
      • Modularity
      • Reuse of components (code)
      • Object oriented concepts
      • Encapsulation
      • Hide complexity
      • Test sripts can be written by testers with little or no programming skills
      • Less maintenance
      • Test script more reliable
      • Test script have longer life
      • etc.
  • 8. Why should we use Ruby?
      • Totally free: use, copy, modify and distribute
      • Fully object-oriented, dynamic scripting language
      • Interpreted language
      • Untyped language
      • Flexible language
      • Rich core library
      • Highly portable
      • Scalable (reuse modules)
      • Extensible (write extensions in C/C++)
      • Exception handling, garbage collector …
      • Clean and easy syntax, easy to use - high productivity
      • Ruby puts the fun back into programming.
  • 9. Start with Ruby
    • Install Ruby :
    • http://www.ruby-lang.org/en/downloads/
    • http://rubyforge.org/frs/?group_id=167
    • Ruby Programming book:
    • http://phrogz.net/ProgrammingRuby/
    • Ruby User Guides:
    • http://www.ruby-doc.org/docs/UsersGuide/rg/
    • http://www.math.umd.edu/~dcarrera/ruby/0.3/
    • http://www.ruby-doc.org/gettingstarted/
    • http://rubylearning.com/
    • http://www.zenspider.com/Languages/Ruby/QuickRef.html
  • 10. Ruby Editors / IDE
      • Free
      • Eclipse with RDT or DLTK
      • Aptana IDE (Aptana RadRails)
      • RDE (Ruby IDE)
      • NetBeans IDE
      • SciTE
      • Komodo Editor
      • Crimson Editor and similar
    • Comercial
      • Arachno Ruby IDE
      • InteliJ IDEA
      • Komodo IDE
  • 11. WATiR
      • Watir – Web Application Testing in Ruby
      • open-source library for automating web browsers
      • works with Internet Explorer on Windows
      • Curently beeing ported to support Firefox (FireWatir) and Safari (SafariWatir)
      • Uses IE Component Object Model (COM) interface to directly access the objects on the page
      • Install Watir:
      • > gem install watir
  • 12. FireWatir
      • Allows Watir scripts written for IE to work with the Firefox.
      • Some changes are required
      • Uses JSSH (JavaScript Shell) to manipulate Firefox
      • Install FireWatir and JSSH
      • http://code.google.com/p/firewatir/downloads/list
  • 13. Selenium RC
      • Selenium Remote Control provides a Selenium Server, which can automatically control any supported browser. It works by using Selenium Core, a pure-HTML+JS library that performs automated tasks in JavaScript.
      • Several programming languages are supported except Ruby (Java, C#, Pyton, PHP, Perl)
      • Download Selenium
      • http://www.openqa.org/selenium-rc/download.action
      • You can also download gem by using command:
      • > gem install selenium
  • 14. Using The Ruby Interactive Shell (IRB)
      • Type commands directly into the interpreter. You don’t need to
      • save a file and run it. Start it from command line by typing ’ irb ’.
      • irb> require ’watir’
      • => true
      • irb> include Watir
      • => Object
      • irb> br = IE.start(www.google.se)
    > irb irb> 2 + 2 => 4
      • irb> br = IE.attach(:title, ”Google”)
      • irb> br.button(:name, ”btnG”).flash
    You can save a lot of time when attaching to page you want to test and investigating it.
  • 15. Building automated test framework approach
  • 16. Simple test case
    • Test scenario:
    • Start Internet Explorer and go to www.google.se
    • Search for ”Testway”
      • The Google result page should contain link www.testway.se
    • Click the link
      • Testway home page should open
  • 17.
    • The created code can be helpful but needs refactoring
    require 'watir' include Watir require 'test/unit' class TC_recorded < Test::Unit::TestCase def test_recorded ie = IE.new ie.goto('http://www.google.se/') ie.text_field(:name, 'q').set('testway') ie.button(:name, 'btnG').click ie.link(:url, 'http://www.testway.se/').click end end Recorded by WatirRecorder++
  • 18. Selenium IDE
  • 19. Unit Test Framework (Watir & FireWatir) require 'test/unit'   class TestSuiteSimple < Test :: Unit :: TestCase def setup @browser = df_setup end def teardown df_teardown end   def test_01_simple_search @browser . text_field( :name , &quot;q&quot; ) . set( &quot;Testway&quot; ) @browser . button( :name , &quot;btnG&quot; ) . click assert( @browser . link( :href , /www.testway.se/ ) . exists?) @browser . link( :href , /www.testway.se/ ) . click assert_equal( &quot;Testway - Certified Test Specialists&quot; , @browser . title) end end
  • 20. (Fire)Watir - setup & teardown def df_setup puts &quot; >>>>> #{ to_s () } <<<<< &quot; # output to console puts Time . now . strftime( &quot;%c&quot; ) browser = nil browser = start_browser(“www.google.se”) #(ENV['TEST_URL']) assert_match( /Google/ , browser . title, “The page does not exists&quot; ) return browser end def df_teardown # In case of the failure save a snapshot if (passed? == false) puts &quot; - Test case failed&quot; p = get_project_folder + &quot;/Test_logs/Screenshots/#{to_s()}.jpg&quot; take_screenshot(p) end @browser . close if @browser end
  • 21. (Fire)Watir– start_browser def start_browser (url) browser = nil count = 0 if ENV [ 'browser' ] == &quot;IE“ require 'watir‘ begin Timeout . timeout 30 do browser = Watir :: IE . start_process(url) browser . wait end rescue Timeout :: Error kill_ie count += 1 puts &quot;Timeout error: #{ count } &quot; sleep 2 retry unless count == 3 end elsif ENV [ 'browser' ] == &quot;FF&quot; # require 'firewatir‘ # browser = FireWatir::Firefox.start(url) end return browser end
  • 22. How does it look like in Selenium?
    • require &quot;selenium“
    • require &quot;test/unit&quot;  
    • class TestSuiteSimple < Test :: Unit :: TestCase
    • def setup
    • @ selenium = Selenium :: SeleniumDriver . new( &quot;localhost&quot; , 4444 , &quot;*iehta&quot; , &quot;http://www.google.se&quot; , 10000 ) # *iexplore (*iehta is experimental)
    • @ selenium . start
    • end
    • def teardown
    • @ selenium . stop
    • end
    • def test01_simple_search
      • @ selenium . open &quot;/“
      • @ selenium . type &quot;q&quot; , &quot;testway“
      • @ selenium . click &quot;btnG“
      • @ selenium . wait_for_page_to_load &quot;30000&quot; assert ( @selenium . is_element_present( &quot;link=Testway - Certified Test Specialists&quot; ))
      • @ selenium . click &quot;link=Testway - Certified Test Specialists&quot;
      • @ selenium . wait_for_page_to_load &quot;30000“
      • assert_equal ( &quot;Testway - Certified Test Specialists&quot; , @ selenium . get_title )
    • end
    • end
  • 23. (Fire)Watir - Help functions # Better readability with help functions def test_01_simple_search @browser . text_field( :name , &quot;q&quot; ) . should_exist @browser . text_field( :name , &quot;q&quot; ) . set( &quot;Testway&quot; ) @browser . button( :name , &quot;btnG&quot; ) . should_exist @browser . button( :name , &quot;btnG&quot; ) . click @browser . link( :text , &quot;Testway - Certified Test Specialists&quot; ) . should_exist @browser . link( :text , &quot;Testway - Certified Test Specialists&quot; ) . click @browser . should_have_title( &quot;Testway - Certified Test Specialists&quot; ) end module Watir class IE def should_have_title (value) assert(self.respond_to?( &quot;title&quot; ), &quot;This method can not be used&quot; ) assert_equal(value, self.title, &quot;The title is not correct. It is #{ self.title } - it should be #{ value } &quot; ) end def should_exist assert(self.respond_to?( &quot;exists?&quot; ), &quot;This method can not be used&quot; ) assert(self.exists?, &quot;The object does not exist&quot; ) end end end For FireWatir: module FireWatir class Firefox
  • 24. Selenium - Help functions # Help functions Selenium – some examples module Selenium class SeleniumDriver def should_have_title (value , extra_info = &quot;-&quot; ) assert( self . respond_to?( &quot;get_title&quot; ), &quot;'This method can not be used #{ __FILE__ } : #{ __LINE__ } &quot; ) assert_equal(value, self . get_title, &quot;The title is not correct. It is: #{ self . get_title } - it should be: #{ value } #{ __FILE__ } : #{ __LINE__ } Extra info: #{ extra_info } &quot; ) end def should_not_have_title (value , extra_info = &quot;-&quot; ) assert( self . respond_to?( &quot;get_title&quot; ), &quot;This method can not be used #{ __FILE__ } : #{ __LINE__ } &quot; ) assert_not_equal(value, self . get_title, &quot;The title is not correct. It is: #{ self . get_title } - it should NOT be: #{ value } #{ __FILE__ } : #{ __LINE__ } Extra info: #{ extra_info } &quot; ) end def should_contain_text (value , extra_info = &quot;-&quot; ) assert( self . respond_to?( &quot;is_text_present&quot; ), &quot;This method can not be used #{ __FILE__ } : #{ __LINE__ } &quot; ) assert( self . is_text_present(value), &quot;The page does NOT contain text: - #{ value } #{ __FILE__ } : #{ __LINE__ } Extra info: #{ extra_info } &quot; ) end end end
  • 25. (Fire)Watir – Win Map def test_01_simple_search google = Google . new( @browser ) google . text_field_search . should_exist google . text_field_search . set( &quot;Testway&quot; ) google . button_search . should_exist google . button_search . click @browser . link( :text , &quot;Testway - Certified Test Specialists&quot; ) . should_exist @browser . link( :text , &quot;Testway - Certified Test Specialists&quot; ) . click @browser . should_have_title( &quot;Testway - Certified Test Specialists&quot; ) end class Google def initialize (browser); @browser = browser ; end def text_field_search ; @browser . text_field( :name , &quot;q&quot; ) ; end def button_search ; @browser . button( :name , &quot;btnG&quot; ) ; end end
  • 26. Selenium – Win Map
    • def test_01_simple_search
      • @selenium . open &quot;/“
      • @selenium . type Google :: text_field_search, &quot;testway”
      • @selenium . click Google :: button_search
      • @selenium . wait_for_page_to_load &quot;30000“
      • assert( @selenium . is_element_present( &quot;link=Testway - Certified Test Specialists&quot; ))
      • @selenium . click &quot;link=Testway - Certified Test Specialists&quot;
      • @selenium . wait_for_page_to_load &quot;30000&quot;
      • assert_equal( &quot;Testway - Certified Test Specialists&quot; , @selenium . get_title)
    • end
    class Google def self . text_field_search ; &quot;//input[@name='q']“ ; end def self . button_search ; &quot;//input[@type='submit' and @name='btnG']“ ; end end
  • 27. (Fire)Watir - encapsulating
    • def test_01_simple_search
      • google = Google . new( @browser )
      • text_field_set(google . text_field_search, &quot;Testway&quot; )
      • button_click(google . button_search)
      • link_click( @browser . link( :text , &quot;Testway - Certified Test Specialists&quot; ))
      • @browser . should_have_title( &quot;Testway - Certified Test Specialists&quot; )
    • end
  • 28. (Fire)Watir – encapsulating 2
    • module Actions
    • def text_field_set (obj_ctrl, txt)
      • assert(obj_ctrl . exists?, &quot;Object does not exist - #{ caller } &quot; )
      • assert(obj_ctrl . enabled?, &quot;Object is not enabled - #{ caller
      • assert( ! obj_ctrl . readOnly?, &quot;Object is read only - #{ caller } &quot; ) assert(obj_ctrl . respond_to?( &quot;set&quot; ), &quot;Object does not respond to 'set' - #{ caller } &quot; )
      • obj_ctrl . set( &quot; #{ txt } &quot; )
      • assert_equal(obj_ctrl . value, &quot; #{ txt } &quot; , &quot; #{ obj_ctrl . value } is not equal #{ txt } - #{ caller } &quot; )
    • end
    • def button_click (obj_ctrl, w = true )
      • assert(obj_ctrl . exists?, &quot;Object does not exist - #{ caller } &quot; )
      • assert(obj_ctrl . enabled?, &quot;Object is not enabled - #{ caller } &quot; )
      • assert(obj_ctrl . respond_to?( &quot;click&quot; ), &quot;Object does not respond to 'click' - #{ caller } &quot; )
      • begin
    • if w == true obj_ctrl . click
    • else
    • obj_ctrl . click_no_wait
    • end
    • rescue => e
    • puts &quot;obj_ctrl.click failure: #{ to_s () } #{ __FILE__ } : #{ __LINE__ } &quot;
    • end
    • end
    • end
  • 29. Selenium - encapsulating def test_01_simple_search @selenium . open &quot;/&quot; @selenium . text_field_set( Google :: text_field_search, &quot;testway&quot; ) @selenium . button_click( Google :: button_search) @selenium . should_contain_text( &quot;Testway - Certified Test Specialists&quot; ) @selenium . link_click( &quot;link=Testway - Certified Test Specialists&quot; ) @selenium . should_have_title( &quot;Testway - Certified Test Specialists&quot; ) end
  • 30. Selenium – encapsulating 2 module Selenium class SeleniumDriver require 'test/unit/assertions' include Test :: Unit :: Assertions def text_field_set (locator, txt) assert( self . is_element_present(locator), &quot;Object does not exist&quot; ) assert( self . is_editable(locator), &quot;Object is not enabled &quot; ) assert( self . is_visible(locator), &quot;Object is not visible&quot; ) assert( self . respond_to?( &quot;type&quot; ), &quot;Object does not respond to 'type'&quot; ) assert( self . respond_to?( &quot;get_value&quot; ), &quot;Object does not respond to 'get_value'&quot; ) self . type(locator, txt) end def button_click (locator) assert( self . is_element_present(locator), &quot;Object does not exist&quot; ) assert( self . is_visible(locator), &quot;Object is not visible&quot; ) assert( self . respond_to?( &quot;click&quot; ), &quot;Object does not respond to 'click'&quot; ) self . click(locator) self . wait_for_page_to_load( 20000 ) end end end
  • 31. (Fire)Watir - domain functions # If you are using often the same set of actions, you could group them def test_02_simple_search_refactored google_search_and_verify( &quot;Testway&quot; , &quot;Testway - Certified Test Specialists&quot; ) google_click_link_and_verify( &quot;Testway - Certified Test Specialists&quot; ) end def google_search_and_verify (srch_text, text_to_find = nil ) text_to_find = srch_text unless text_to_find google = GooglePage . new( @browser ) text_field_set(google . text_field_search, srch_text) button_click(google . button_search) @browser . should_contain_text(text_to_find) end def google_click_link_and_verify (lnk_text, page_title = nil ) page_title = lnk_text unless page_title link_click( @browser . link( :text , lnk_text)) @browser . should_have_title(page_title) end  
  • 32. Data - driven testing def test_03_data_driven #################################################### # Local variables #################################################### data_array = Array . new path = &quot; #{ ENV [ 'my_project' ] } /test_data/google_search.xls&quot; #################################################### data_array = get_data_array(path) for i in 0. . . data_array . length @browser.goto( www.google.se ) google_search_and_verify(data_array [ i ]. search_text, data_array [ i ]. text_to_find) google_click_link_and_verify(data_array [ i ]. text_to_find, data_array [ i ]. page_title) end end
  • 33. Test Reports
    • Direct output to a text file
    • Download and unpack Test::Unit::Reporter from:
    • http ://files.rubyforge.vm.bytemark.co.uk/test-report/
    • Issue the following commands:
    > ruby simple_test1.rb > simple_test_out.txt > ruby setup.rb config > ruby setup.rb install
  • 34. Test Reporter Usage require 'test/unit/testsuite' require 'test/unit/ui/reporter' require 'simple_test1_unit_ref' require 'fileutils' # Create a new test suite Watir_Presentation suite = Test :: Unit :: TestSuite . new ( &quot;Watir_Presentation&quot; ) # Add TestSuite_Simple to Watir_Presentation test suite suite << TestSuiteSimple . suite # Create/Open the directory for test results dir = File . join( Dir . pwd, '/Reports/‘ + Time . now . strftime( &quot;%d-%b-%y&quot; )) FileUtils . mkdir_p dir # Run the test suite Test :: Unit :: UI :: Reporter . run(suite, dir, :html)
  • 35. Test Reporter
  • 36. Something to think about…
    • Test automation is software development – treat it like that
    • Coding standards, naming conventions, source control, documentation, error handling, code review, etc.
    • Invest in maintainability from the beginning - it will save your time later on
    • Framework, error handling, win mapping, wrapper function etc.
    • The aim of the test automation is not to create code but good tests
    • The best written code can represent a lousy test – it is worthless then
    • Automated tests complement and not replace manual tests - all tests will not be automated
    • Automate only tests suitable for automation
  • 37. Questions ? [email_address]