Collaborative Results, Resource Repository,                      UI Enhancements   TestMaker 6.5 PlanFrank Cohen, CEO(408)...
Open Source Test Automation                                                                                             In...
3                     Open Source Test Automation                                                    Intellectual Property...
4                      Open Source Test Automation                                                   Intellectual Property...
5                       Open Source Test Automation                                                     Intellectual Prope...
6                                             Open Source Test Automation                                                 ...
7                    Open Source Test Automation                                                Intellectual Property of P...
8                      Open Source Test Automation                                                    Intellectual Propert...
9                                                   Open Source Test Automation                                           ...
10                                                       Open Source Test Automation                                      ...
Open Source Test Automation                                           Intellectual Property of PushToTest. (c) 2011 All ri...
12                      Open Source Test Automation                                                    Intellectual Proper...
13                     Open Source Test Automation                                                   Intellectual Property...
14                       Open Source Test Automation                                                      Intellectual Pro...
15                                            Open Source Test Automation                                                 ...
16                                Open Source Test Automation                                                             ...
17                                Open Source Test Automation                                                             ...
18                                                Open Source Test Automation                                             ...
19                       Open Source Test Automation                                                   Intellectual Proper...
20                     Open Source Test Automation                                                  Intellectual Property ...
21                                  Open Source Test Automation                                                           ...
22         Open Source Test Automation                         Intellectual Property of PushToTest. (c) 2011 All rights re...
23                 Open Source Test Automation                                                                            ...
24           Open Source Test Automation                             Intellectual Property of PushToTest. (c) 2011 All rig...
25                Open Source Test Automation                                                                            I...
26          Open Source Test Automation                            Intellectual Property of PushToTest. (c) 2011 All right...
27                  Open Source Test Automation                                                                           ...
28                                                 Open Source Test Automation                                            ...
29                 Open Source Test Automation                                                                  Intellectu...
30                 Open Source Test Automation                                                                  Intellectu...
31                 Open Source Test Automation                                                                  Intellectu...
32                                                     Open Source Test Automation                                        ...
33                  Open Source Test Automation                                             Intellectual Property of PushT...
34                                   Open Source Test Automation                                                          ...
35                                   Open Source Test Automation                                                          ...
36                                  Open Source Test Automation                                                           ...
37                               Open Source Test Automation                                                              ...
38                                               Open Source Test Automation                                              ...
Open Source Test Automation                                           Intellectual Property of PushToTest. (c) 2011 All ri...
40        Open Source Test Automation                      Intellectual Property of PushToTest. (c) 2011 All rights reserv...
41                                      Open Source Test Automation                                                       ...
42                                       Open Source Test Automation                                                      ...
43                                      Open Source Test Automation                                                       ...
44                      Open Source Test Automation                                                    Intellectual Proper...
Upcoming SlideShare
Loading in...5
×

PushToTest TestMaker 6.5 Open Source Test Design Document

1,797

Published on

PushToTest TestMaker version 6.5 product design document for a major feature enhancement. Contains user interface definitions, product roadmap, and feature requirements. Please comment on this to improve TestMaker.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,797
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

PushToTest TestMaker 6.5 Open Source Test Design Document

  1. 1. Collaborative Results, Resource Repository, UI Enhancements TestMaker 6.5 PlanFrank Cohen, CEO(408) 871-0122fcohen@pushtotest.comNovember 8, 2011 Intellectual Property of PushToTest. (c) 2011 All rights reserved. Note: All trademarks mentioned in this document are the trademarks of their respective holders.
  2. 2. Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Product RoadmapTestMaker 6.0 TestMaker 6.1 TestMaker 6.5 TestMaker 72011 2011 2011 2012 • Improved Test • Tabular Results Charts • Easier Test Controller • TestCards Phase 2 Operation Speed • Flex 4 In-Browser • Reports • Web Reports • Jasper Business Recording • Collaborative Charts • Web Controller Intelligence Reporting • TestMaker JAT for • Live Charts for Drill • Test Management • Test Object Designer Oracle Applications Downs • Rally • Record/Playback • Citrix Applications • Test Object Repository • TestLink • Firefox, IE, Safari, • TestCards Phase 1 • Share Test Chrome, Opera • HP QC • Rally Integration Resources • Open Source • Zephyr • Web Reports • Search Alternative to • QMetry HP QTP • Web Controller • Access Control • Adds Sahi, Flex • Bug Fixes • LDAP/Rally • TestCards 1 • Data and Object Enabled • HtmlUnit Viewer
  3. 3. 3 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.TestMaker 6.5‣ Resource Storage in the Repository ‣ Access Control for users and LDAP integration ‣ Search‣ Reports ‣ Live Reports ‣ Collaborative Reports‣ Easier Test Controller ‣ Streaming Results using Snapshot Protocol
  4. 4. 4 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Needs Research for 6.5‣ TestMaker to restore Windows and IE proxy on quit‣ Console Connects To TestNodes (Cloud) Through Proxy‣ URL to Web-based TestMaker Test Results Automatically Saved To Rally Test Case‣ L7 and HPOpenView reports gathered automatically
  5. 5. 5 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Not In TestMaker 6.5‣ Full Web Interface for TestMaker‣ Test ManagementFailed Previous Test Operations ‣ Rerun Tests of Zephyr, Quality Center Integration ‣ TestLink, QMetry,Unique DPL v2)‣ Colossus“Push” Bundle Deployment, Use Resources Instead DPL (aka‣ DynamicTestNode Sahi Proxy SSL Certificate Registration in‣ Remote Community‣ Open Designer,view “How Are We Doing” see Editor too‣ No dashboard has a custom status display (icon and text) settable‣ Each Controller step context by API from a test‣ Graphical view to edit and manage RDBMS for DPL content
  6. 6. 6 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource Repository‣ File System or Repository‣ Editor, TestMaker Object Designer, Console‣ Open, Save, Save As Open Test Resource Save Test Resource Repositories Repositories Local Repository Local Repository QA Unit 1 Repository QA Unit 1 Repository / Last Modified Creator / Last Modified Creator Project X June 12, 2011 6:05 pm mcohen 3rd Party Connector Project June 12, 2011 6:12 pm mcohen Project X June 12, 2011 6:05 pm mcohen TestScenarios June 12, 2011 12:05 pm mcohen 3rd Party Connector Project June 12, 2011 6:12 pm mcohen Load Test 103 Scenario June 12, 2011 1:05 pm mcohen TestScenarios June 12, 2011 12:05 pm mcohen Project Y June 12, 2011 3:05 pm mcohen Load Test 103 Scenario June 12, 2011 1:05 pm mcohen Project Z June 12, 2011 5:05 pm mcohen Project Y June 12, 2011 3:05 pm mcohen Project Z June 12, 2011 5:05 pm mcohen Load Test 133 Scenario Cancel Open Show Versions Cancel Save Show Versions
  7. 7. 7 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource Repository Search‣ Search File Names‣ Search TestMaker Object Designer Meta Data ‣ Name, Creator, Links, Version, Types, Operations, Goals ‣ Object Repository objects: Story, Form, Object ‣ Object name, Type, Accessor, Comment‣ Search Editor Meta Data ‣ Author Name, Creator, Links, Versions, Categories
  8. 8. 8 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Access Control‣ TestCard Viewable by Anonymous, Role, User Name‣ TestScenarioRole, User Name Stop, Pause, Create, Delete by Anonymous, View, Change, Run,‣ Repository Read, Change, Create DB, Create Contents, Delete by Anonymous, Role, User Name‣ Role Create, Change, Delete by Anonymous, Role, User Name‣ Reports Create, Change, Delete by Anonymous, Role, User Name
  9. 9. 9 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.SSO Option with Rally‣ Optionally Uses Rally LDAP for User Authentication PushToTest TestCards - Best Buy help Welcome To TestMaker TestCards TestMaker Enterprise 6.0.1 User ID fcohen Licensed for 50 virtual users and 10 service monitors Password ••••••••• Sign-in Options Sign-in Using Rally
  10. 10. 10 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Enhanced Controller‣ Web (for TCS 1) and SWING‣ Reports Button -> New Reports Ajax Load Test‣ Tabs Expand To New Window 513 40 Seconds Remaining Executing iteration: 1 Users * 1 513 Success 12.67 % Failure 65 Failed Instant Results TestNode Status Local TestNode Log Local TestNode Error Log Real Time Scalability Index Chart 6 5 4 3 2 1 2.5 TPS 4.6 TPS 1 TPS 0 1 Users * 1 2 Users * 1 4 Users * 1 213 Success 475 Success 513 Success 5 Failed 35 Failed 65 Failed .02% Failure 7.36% Failure 12.67% Failure Reports Configuration Logs
  11. 11. Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved. Next let’s talk about Reports Design ‣ Resource Plan ‣ User Interface Design ‣ Required Changes to TestMaker 6.011
  12. 12. 12 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Reports Summary‣ User Interface Design (Tomas) ‣ Reports - A report is a collection of charts ‣ Charts - Athe Transaction of the test results repository. For example, graphical view Distribution chart‣ Live Charts - A user interface to see custom step logging data‣ ScriptRunner (EOP) Enhancements (Luis Carlos) ‣ DesignerScript -commandtoinstore screenshots, DOM after executing each Options a .ds file ‣ soapUI - Options to save each TestCase steps request & response ‣ Java - Options to save request, response, stack trace
  13. 13. 13 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Required Changes‣ PTTStepListener and Repository Schema ‣ StepRequest (String, may contain CR&LF characters) ‣ StepResponse (Blob, may contain CR&LF characters) ‣ StepStackTrace (String, up to 10K characters) ‣ StepExceptionName (String) ‣ Screenshot (Blob)‣ TestScenario Schema Changes ‣ Define Report, Charts, Drill Downs
  14. 14. 14 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Optional Enhancements‣ Caching of Jasper generated charts to avoid reprocessing time
  15. 15. 15 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Run A Report PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Run Report Edit Report Delete Report New Report TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors Reports s Weekly Scalability Report img TSH Working Team 2 s Load Test for Mods To 3rd Party Gov img Performance after changes s Functional Test for Geek Squad Reg img Testing new registration process
  16. 16. 16 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Report Appears In Window PushToTest TestMaker 6 - Weekly Scalability Report - Created on September 10, 2011 Weekly Scalability Report Created on September 10, 2011 Transaction Distribution Chart
  17. 17. 17 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.New/Edit A Report PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository TestMaker Enterprise 6.0.1 New Report Licensed for 50 virtual users and 10 service monitors Report Name: Cancel Next
  18. 18. 18 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Add Charts To The Report PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Report Settings TestMaker Enterprise 6.0.1 Licensed for 50 virtual users Report Name: Weekly Scalability Report and 10 service monitors Header: Weekly Scalability Report Footer: M Cohen - mcohen@bestbuy.com x3817 New Chart Edit Chart Delete Chart Charts Transaction Distribution Chart ! Scalability Index ! Transactions by Avg Duration Report Details Cancel Save
  19. 19. 19 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Chart Types‣ Scalability Chart ‣ Bar Chart ‣ Grouped Data Points‣ Transaction Chart ‣ Line and Data Point Chart ‣ Shows Individual Points of Data‣ Resource Chart ‣ Bar Chart ‣ Resource (CPU, Net, Memory) Usage Percentage‣ Data Chart ‣ Grid Chart ‣ Transaction and Step Data
  20. 20. 20 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Scalability Chart Settings‣ Vertical ‣ Transactions Per Second ‣ Count of All Transactions ‣ Count of Failed Transactions ‣ Count of Successful Transactions ‣ Avg, Min, Max Transaction Time‣ Horizontal: VU Iteration‣ Secondary Axis (%) ‣ Failure Percentage ‣ Success Percentage ‣ Resource Monitoring (Choose from CPU, Net, Memory)‣ Option: Display List of Step Names
  21. 21. 21 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Scalability Chart Settings PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Scalability Chart Settings TestMaker Enterprise 6.0.1 Licensed for 50 virtual users Transactions Per Second (TPS) and 10 service monitors Count of All Transactions Count of Successful Transactions Count of Failed Transactions Avg, Min, Max Transaction Time Average Secondary Axis Failure Percentage Success Percentage Resource Monitoring CPU Option: Display List of Step Names Cancel Save
  22. 22. 22 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Transaction Chart Settings‣ Vertical ‣ Transaction Duration Time‣ Vertical Options ‣ Successful ‣ Failed ‣ Duration: Avg, Min, Max‣ Horizontal ‣ Time Offset Into Test ‣ Specific Start/End Time
  23. 23. 23 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Transaction Chart Settings PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Transaction Chart Settings TestMaker Enterprise 6.0.1 Licensed for 50 virtual users Transaction Duration Time and 10 service monitors Successful Failed Duration Average Horizontal Axis Show standard date/time Cancel Save
  24. 24. 24 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource Chart Settings‣ Vertical ‣ Percentage used ‣ Grouped by CPU, Net, Memory‣ Horizontal ‣ Time Offset Into Test ‣ Specific Start/End Time
  25. 25. 25 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource Chart Settings PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Resource Chart Settings TestMaker Enterprise 6.0.1 Licensed for 50 virtual users Percentage Used and 10 service monitors Grouped by CPU Horizontal Axis Show standard date/time Cancel Save
  26. 26. 26 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Data Chart Settings‣ Successful Transactions‣ Failed Transactions‣ Option: Show Steps
  27. 27. 27 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Data Chart PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Data Chart Settings TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors Successful Transactions Failed Transactions Show Steps Cancel Save
  28. 28. 28 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Live Charts for Drill Downs PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository Live Charts TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors New Chart View Chart Edit Chart Delete Chart Charts Transaction Distribution Chart Scalability Index Transactions by Avg Duration
  29. 29. 29 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Live Charts for Drill Downs PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors
  30. 30. 30 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Live Charts for Drill Downs PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors
  31. 31. 31 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Live Charts for Drill Downs PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors
  32. 32. 32 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Click To See Steps PushToTest TestMaker 6 - Best Buy help TestMaker Results Repository TestMaker Enterprise 6.0.1 Licensed for 50 virtual users and 10 service monitors Steps In Selected Transaction - soapUI TestSuite Request 1 175 Request 2 3,201 Groovy Script - Validate 643
  33. 33. 33 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Additional Requirements‣ Keyboard left-right arrows move selected point‣ User entered Notes‣ Steps need labeling
  34. 34. 34 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Click Step To See Detail Steps In Selected Transaction - soapUI TestSuite Request 1 175 Request 2 3,201 Groovy Script - Validate 643 Request Response Error Custom <soapenv:Envelope xmlns:soapenv="http:// <soapenv:Header/> <soapenv:Body> <exam:Change_Price> <ProductNumber>1033</ProductNumber> <Type>add_product</Type> <Quantity>55</Quantity> <Location>2</Location>
  35. 35. 35 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.For Java, DS Steps Steps In Selected Transaction - DesignerScript Log-in To Application 175 Product Catalog Search 3,201 Buy Products 675 Request Response Error Screenshot Command: _click Locator: textbox("company") Value: "PushToTest"
  36. 36. 36 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.For Java, DS Steps Request Response Error Screenshot Row1_ColumnB: 38178 Row2_ColumnB: 98123 Row3_ColumnB: 18177 Request Response Error Screenshot Unable To Find Flight Number Exception java.lang.Throwable at boo.hoo.StackTrace$FirstNested$SecondNested. at boo.hoo.StackTrace$FirstNested.<init>(St at boo.hoo.StackTrace.<init>(StackTrace.j
  37. 37. 37 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Add A Step As A Chart Steps In Selected Transaction - DesignerScript Log-in To Application 175 Product Catalog Search 3,201 Buy Products 675 Add This Transaction To Report As New Chart Add This Step To Report As New Chart
  38. 38. 38 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Add More To The Report PushToTest TestMaker 6 - Weekly Scalability Report - Created on September 10, 2011 Weekly Scalability Report Created on September 10, 2011 Transaction Distribution Chart Step Detail Chart - Transaction 18372 - Step Log-in To Application Request Command: _click Locator: textbox("company") Value: "PushToTest" Response Row1_ColumnB: 38178 Row2_ColumnB: 98123 Row3_ColumnB: 18177 Error Unable To Find Flight Number Exception java.lang.Throwable at boo.hoo.StackTrace$FirstNested$SecondNested. at boo.hoo.StackTrace$FirstNested.<init>(St at boo.hoo.StackTrace.<init>(StackTrace.j Screenshot
  39. 39. Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved. Next let’s talk about Resource Repository Design ‣ Resource Plan ‣ User Interface Design ‣ Required Changes to TestMaker 6.039
  40. 40. 40 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Repository Client Access‣ Designer‣ Editor‣ Controller‣ EOPs
  41. 41. 41 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource Repository - Open Open Test Resource Repositories Local Repository QA Unit 1 Repository / Last Modified Creator Project X June 12, 2011 6:05 pm mcohen 3rd Party Connector Project June 12, 2011 6:12 pm mcohen TestScenarios June 12, 2011 12:05 pm mcohen Load Test 103 Scenario June 12, 2011 1:05 pm mcohen Project Y June 12, 2011 3:05 pm mcohen Project Z June 12, 2011 5:05 pm mcohen Show Versions Cancel Open
  42. 42. 42 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource Repository - Save Save Test Resource Repositories Local Repository QA Unit 1 Repository / Last Modified Creator Project X June 12, 2011 6:05 pm mcohen 3rd Party Connector Project June 12, 2011 6:12 pm mcohen TestScenarios June 12, 2011 12:05 pm mcohen Load Test 103 Scenario June 12, 2011 1:05 pm mcohen Project Y June 12, 2011 3:05 pm mcohen Project Z June 12, 2011 5:05 pm mcohen Load Test 133 Scenario Show Versions Cancel Save
  43. 43. 43 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Resource RepositoryPreferences Repositories TestCard Server Notifications Repositories Local Repository Local Repository Type MySQL QA Unit 1 Repository Location jdbc:mysql//localhost:1527/testmaker User testmaker Add Remove Password testmaker Save
  44. 44. 44 Open Source Test Automation Intellectual Property of PushToTest. (c) 2011 All rights reserved.Learn More‣ http://www.pushtotest.com‣ Post your feedback, comments, ideas, designs to PushToTest Forums‣ Attend the Open Source Test Workshop ‣ http://workshop.pushtotest.com‣ Send Feedback to fcohen@pushtotest.com
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×