0
<Insert Picture Here>




How the Oracle JDeveloper Team Test JDeveloper
Gerard Davison : Senior Principle Software Engine...
Presentation Agenda


•   Introduction      <Insert Picture Here>

•   Automation
•   Tools
•   Abbot
•   Selenium
•   Con...
<Insert Picture Here>



Introduction
Introduction


• Admission
  – We are not testing saints
  – We do not have 100% code coverage
  – We probably never will
...
Introduction


• JDeveloper
  – We have everything from Swing to Excel
     • Our focus is on the Java parts today
  – We ...
<Insert Picture Here>

                                                 Building
                                     Sir ...
Introduction -Terminology


• Unit testing
  – The testing the Java Classes directly
• Integration testing
  – Testing it ...
<Insert Picture Here>



Automation
Automation




             What to automate?
                 The big question
What to automate?


• What is it for?
  – Cost saving
     • Humans - expensive, quite squishy – not entirely reliable
   ...
What to automate?


• Where to begin?
  – Fit to test / DFT
     • Earlier rather than later
     • Handover automation fr...
What to automate?


• What type of Automation?
  – Unit testing – can only take you so far
     • Verifies that what the d...
How do we automate?


• Calling Java APIs
  – Unit testing - in complicated situations it can be hard to set up
    an env...
What to automate?


• How much Automation?
  – Build verification is limited by the time it takes between builds

  – Use ...
<Insert Picture Here>



Tools
Build tooling


• Build tools
  – Needs to
     • run automatically
        – Generally as a result of SCM operation
     ...
Build tooling


• Build tools
  – We have lots of custom code built up over years
  – Get an answer as quickly as possible...
Build tooling


• Cruise Control
  – Ant wrapper
  – Email notification
• Hudson
  – Lots of web 2.x UI
  – Using Webstart...
<Insert Picture Here>

                                               Consistency
                                        ...
Automation Environment


• Work on a Dev machine the same as on the
  Automated test machine
• Make sure your test environ...
Automation Environment


• Quick set ups
   – Don’t want to test the setup 100 times
      • We use zips to store project ...
Testing Tools Environment


• Integration testing gets asynchronous real quick,
   – Harder than nice linear JUnit testing...
Testing Tools Environment
Other tools worth mentioning


• Free
   – FEST
   – JFCUnit
   – HtmlUnit
   – WebTest
• Expensive
   – WinRunner
   – Si...
<Insert Picture Here>



Abbot
Abbot


• Automation of Swing using java.awt.Robot
  – Also SWT version, not used it though
• Provides a Java API
• We use...
Costello


• Watches that you are doing
• Record actions steps
• Creates references to components
  – Not brittle, will su...
Integration into JDeveloper


• Invoke tests / Costello inside of JDeveloper
  – For faster turnaround
• Can invoke tests ...
<Insert Picture Here>



Abbot Demo
Abbot tips


• Use the “Name” property to distinguish similar
  components
• Wait don’t delay
  – Ignore progress bars, te...
Abbot tips


• Call
  – Invoke static Java methods
• Script
  – Run BeanShell script
Abbot in JDeveloper facts and figures


• An example
  – DB team (Our best team – test coverage speaking)
     • Connectio...
<Insert Picture Here>



Selenium
Selenium


• Suite of tools

  – Remote Control
     • Integrates tests with your build

  – IDE
     • Exclusive to Firef...
Selenium Remote Control


• Comes in two parts

  – Server
     • Starts and stops the target browser
     • Acts as an ht...
Selenium IDE


• Creates tests
  – Extension to Firefox

  – Simple to use
     • Record and playback
     • Insertion for...
<Insert Picture Here>



Selenium Demo
Selenium tips


• Maximize your browser

• Has to be verification for any action performed
  – If you clicked a button, wh...
Selenium tips


• Limited to the DOM tree
  – Stylesheet or Style classes are not inherently testable

• Use clear ID’s
  ...
<Insert Picture Here>



Conclusion
Conclusion


• Automated tests

  – Have some – it helps

  – Not too much though – it takes time

  – Build and Test – ha...
For More Information




• Abbot
  – http://abbot.sourceforge.net/
• Selenium
  – http://selenium.seleniumhq.org/


• Your...
How the JDeveloper team test JDeveloper at UKOUG'08
Upcoming SlideShare
Loading in...5
×

How the JDeveloper team test JDeveloper at UKOUG'08

2,981

Published on

Presentation give at UKOUG'08 on how the JDeveloper team test JDeveloper using Abbot and Selenium

Published in: Technology
1 Comment
1 Like
Statistics
Notes
  • Now embedded at www.ukocn.com, the largest on-line Community Network for Oracle in the UK:

    http://www.ukocn.com/forums/developer-tools/jdeveloper/how-jdeveloper-team-test-jdeveloper

    www.ukocn.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,981
On Slideshare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
68
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "How the JDeveloper team test JDeveloper at UKOUG'08"

  1. 1. <Insert Picture Here> How the Oracle JDeveloper Team Test JDeveloper Gerard Davison : Senior Principle Software Engineer Geoff Waymark : Principal QA Engineer
  2. 2. Presentation Agenda • Introduction <Insert Picture Here> • Automation • Tools • Abbot • Selenium • Conclusion
  3. 3. <Insert Picture Here> Introduction
  4. 4. Introduction • Admission – We are not testing saints – We do not have 100% code coverage – We probably never will • But – We have tried and succeeded in testing the ‘hard bits’ – We have learned some lessons – Turns out a little can go a long way
  5. 5. Introduction • JDeveloper – We have everything from Swing to Excel • Our focus is on the Java parts today – We tend to be using technologies before you • We create some of the technology • This means we are ahead of test tools – JDeveloper 8mloc in 35k source files over 650 modules • 4 continents, at least 6 versions of English – Two builds • Sanity – verify no broken code • Nightly - full release build ready for a customer
  6. 6. <Insert Picture Here> Building Sir Winston Churchill “To build may have to be the slow and laborious task of years. To destroy can be the thoughtless act of a single day ”
  7. 7. Introduction -Terminology • Unit testing – The testing the Java Classes directly • Integration testing – Testing it all working together • Automation – Making it all work whilst you are in bed / lunch / pub • Poking tooling – Pretending to be you whilst you are in bed / lunch / pub
  8. 8. <Insert Picture Here> Automation
  9. 9. Automation What to automate? The big question
  10. 10. What to automate? • What is it for? – Cost saving • Humans - expensive, quite squishy – not entirely reliable • Machines - comparatively cheap, don’t get bored – Build verification • Is today's build worse than yesterdays? – Ready to use • Most worn path works • What would have to work to demo to your boss works – Dull things – Performance
  11. 11. What to automate? • Where to begin? – Fit to test / DFT • Earlier rather than later • Handover automation from Dev to QA – Unit testing – Smoke testing with poking tools • When to begin? – Unit testing, straight away, if not sooner – Integration testing takes place later in the cycle • Less code churn • Fewer UI quakes
  12. 12. What to automate? • What type of Automation? – Unit testing – can only take you so far • Verifies that what the developer thinks should happen happens – Integration testing – better, but you need more complicated tools • Verifies that what the rest of the tool thinks should happen happens
  13. 13. How do we automate? • Calling Java APIs – Unit testing - in complicated situations it can be hard to set up an environment. – Often requires mock objects as things rarely work in isolation • Using a Poking tool – Control swing using a java.awt.Robot based tool – Running a HTML based service from a web browser – Likely native to the OS – Better representation of what the user will actually see
  14. 14. What to automate? • How much Automation? – Build verification is limited by the time it takes between builds – Use code coverage to meet a set value • Lines in the sand are easy to measure – Less value than you’d think early on. – Less is more • Don’t have more results than you can analyze • Can you fix the tests In good time if they break
  15. 15. <Insert Picture Here> Tools
  16. 16. Build tooling • Build tools – Needs to • run automatically – Generally as a result of SCM operation • notify you when something goes wrong – SMS alerts and the like* • provide nice reporting, rss feeds and the like • be beneficial for the developer * Important for “Testing whilst at the pub”
  17. 17. Build tooling • Build tools – We have lots of custom code built up over years – Get an answer as quickly as possible • Sanity builds • Run tests in parallel Build 1S Build 1N Build 2S SRG 1S SRG 1N SRG 2S LRG 1N
  18. 18. Build tooling • Cruise Control – Ant wrapper – Email notification • Hudson – Lots of web 2.x UI – Using Webstart you can add machine to build environment • Useful if you have many time zones • cron or Windows scheduler – Not really a build tool but can assist
  19. 19. <Insert Picture Here> Consistency A. Huxley “Consistency is contrary to nature, contrary to life. The only completely consistent people are the dead..”
  20. 20. Automation Environment • Work on a Dev machine the same as on the Automated test machine • Make sure your test environment runs on all platforms – We have to hand test on a Mac still • Just like Java - write once, debug everywhere – Platform specifics like file separators, install locations • Select which browsers you want to run in – Firefox, IE, Chrome? What version? • Size is important - what display size should you run your tests in? – Do you have a minimum display size for your app
  21. 21. Automation Environment • Quick set ups – Don’t want to test the setup 100 times • We use zips to store project context • Start tool and run inside rather than once for each test • Collect forensics – Make a copy of the logs - ideally for each test – If doing UI testing, take a picture when it fails • Keep developers happy – It you don’t make it easy tests won’t get run – Assign someone smart to set this up.
  22. 22. Testing Tools Environment • Integration testing gets asynchronous real quick, – Harder than nice linear JUnit testing – “Wait don’t delay” – Make sure your testing tool understands this otherwise find a new one
  23. 23. Testing Tools Environment
  24. 24. Other tools worth mentioning • Free – FEST – JFCUnit – HtmlUnit – WebTest • Expensive – WinRunner – SilkTest – QTP
  25. 25. <Insert Picture Here> Abbot
  26. 26. Abbot • Automation of Swing using java.awt.Robot – Also SWT version, not used it though • Provides a Java API • We use the ability to record and playback xml scripts – Reduces the skill level required to create a test • Is extensible so can handle custom components • Gerard is a committer so we can fix and extended where necessary
  27. 27. Costello • Watches that you are doing • Record actions steps • Creates references to components – Not brittle, will survive UI code changing • UI a little bit special • Does the job, can be a lot quicker than writing code • You have to do some tidying up later though
  28. 28. Integration into JDeveloper • Invoke tests / Costello inside of JDeveloper – For faster turnaround • Can invoke tests from command line • Some small tweaks to make Costello work better – Integration with our source control system – Better default scripts
  29. 29. <Insert Picture Here> Abbot Demo
  30. 30. Abbot tips • Use the “Name” property to distinguish similar components • Wait don’t delay – Ignore progress bars, test for something more reliable • Run tests in “read-only” displays – VNC useful for this on Unix – Fast user switching for the same effect on Macs – Windows users just have to go for a cup of tea • Disable all popup notification – Nothing more annoying than an IM ruining your test run – Screensavers are great but won’t help the tests • Don’t record mouse movements, wont run again
  31. 31. Abbot tips • Call – Invoke static Java methods • Script – Run BeanShell script
  32. 32. Abbot in JDeveloper facts and figures • An example – DB team (Our best team – test coverage speaking) • Connections = 70% code coverage (block) • Modeller = 68% code coverage (block) • 86 Abbot tests • 889 JUnit tests • 40% coverage is JUnit on it’s own • Abbot on its own would also be 40% showing the intersection of the tests
  33. 33. <Insert Picture Here> Selenium
  34. 34. Selenium • Suite of tools – Remote Control • Integrates tests with your build – IDE • Exclusive to Firefox – Grid • Not used by Oracle – We have an internal tool that predates Grid
  35. 35. Selenium Remote Control • Comes in two parts – Server • Starts and stops the target browser • Acts as an http proxy for requests • Also includes the selenium core – Client libraries • Client libraries for your favourite language • Flexible – Integrates nicely with existing reports JUnit,etc…
  36. 36. Selenium IDE • Creates tests – Extension to Firefox – Simple to use • Record and playback • Insertion for hand coding • Looks like a nicer Costello – Save tests in your preferred language • HTML – {Default} • Java – Good for us! • C# ,Ruby ,PHP ,Python ,Perl – Good for everybody
  37. 37. <Insert Picture Here> Selenium Demo
  38. 38. Selenium tips • Maximize your browser • Has to be verification for any action performed – If you clicked a button, what did it do? • Assertions should have meaningful messages • Document the test – Clearer for everybody – Test spec – JavaDoc for Java
  39. 39. Selenium tips • Limited to the DOM tree – Stylesheet or Style classes are not inherently testable • Use clear ID’s – ‘table1:0:openPopup’ is better than ‘table1:0:link1’ • Improve your test to get more from it – Refactor IDE recorded tests • Reliability • Internationalization • Data driven tests
  40. 40. <Insert Picture Here> Conclusion
  41. 41. Conclusion • Automated tests – Have some – it helps – Not too much though – it takes time – Build and Test – have to be linked – Development and QA in partnership
  42. 42. For More Information • Abbot – http://abbot.sourceforge.net/ • Selenium – http://selenium.seleniumhq.org/ • Your speakers – gerard.davison@oracle.com • http://kingsfleet.blogspot.com/ – geoff.waymark@oracle.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×