Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Appium & Selenium Alone vs Appium & Selenium with Perfecto
Similar to Appium & Selenium Alone vs Appium & Selenium with Perfecto (20)
More from Lizzy Guido (she/her)
More from Lizzy Guido (she/her) (14)
Recently uploaded
Recently uploaded (20)
Appium & Selenium Alone vs Appium & Selenium with Perfecto
- 1. Appium & Selenium Alone vs. Appium & Selenium Together with Perfecto Tech Talk for Partners David Broerman Technology Alliance Director North America & LATAM Email: davidb@perfectomobile.com
- 2. Agenda • Intro to Perfecto • Automation Testing with Selenium/Appium and Perfecto • High Level Overview of Selenium/Appium & how Perfecto fits in • Mobile Application Types • Understanding the CQ Lab - Architecture/Setup and PlugIns • Quick Demo • iOS, Appium and Perfecto • As of iOS 10, Apple has announced deprecation of its legacy UI Automation framework. Official Apple testing framework for iOS 11 is XCUITest framework. • What is Perfecto doing and additional information • Q&A
- 3. Intro to Perfecto Perfecto Offers: A Cloud Based Continuous Quality Lab enabling testing and monitoring of all digital platforms on real devices and real end user conditions, for enterprises adopting agile delivery processes.8/11/2017 3
- 4. Intro to Perfecto Dominant leader & visionary Strong Ecosystem Perfecto @ a glance Working with Enterprises and Partners to deliver exceptional digital experiences. 4 Setting the bar for market success • Founded in late 2006, Business HQ in Boston • Over 300 employees globally • Fortune 500 customer base 8/11/2017 “Perfecto Mobile marks shift in mobile apps quality with Continuous Quality Lab”
- 5. Cloud Based Quality Platform For The Digital Era 5© 2017, Perfecto Mobile Ltd. All Rights Reserved. Open & Integrated into Dev/Test/Ops Any test tool Any CI server Any test framework Any IDE Any Location Code/Debug Functional Test Performance Test Production Insight One Lab for all Digital Platforms Reporting & Analytics Wind Tunnel Layer Customer Support Browsers Desktops Mobile Devices IoT
- 6. Open and Integrated 6 Open & Integrated Xcode Selenium 8/11/2017
- 7. Automation Testing, Mobile Application Types and Objects
- 8. Automation Testing • Selenium RemoteWebDriver • Open Source Automation Framework for automating browser testing • Perfecto Mobile extends Selenium to support mobile device browsers and apps • Multi-language and IDE support • Appium • “Aims to automate any mobile app from any language and any test framework” • Open Source - Extends Selenium for native (also hybrid applications) • Supports iOS and Android • Multi-language and IDE support • Perfecto • Testing on real devices in the cloud • Real End User Conditions • Perfecto Mobile proprietary functionalities (not available in Selenium/Appium) • GPS Location, Device Vitals, Network Virtualization, Objects (visual)
- 9. Mobile Application Types Standard testing tool sets Use Selenium RWD for web applications Use Appium for native appsUse Selenium RWD or Appium for hybrid apps
- 10. Perfecto Mobile – Enterprise Grade Complete SDLC (Manual, Automation, Perf, Monitoring) Customer Support (24x7), NOC, SLA’s CQL, Continuously up to date Enterprise Security, SSO, Governance, Private Clouds Perfecto Mobile – Environmental Capabilities Network Virtualization/ Profiles Large scale lab DaaS Hybrid Cloud solution Single user performance Device and App Vitals GPS Injection Device logs Global device coverage, real carriers Perfecto Mobile - Automation Both visual & Native objects Testing connects to a CQL – Always On Rich Media reports Parallel Execution CI Ready Unattended Test Perfecto Mobile – Appium Test Framework Coverage Appium Test Framework Local Devices Only – Not a Test Lab Native Objects Only, No Visual Analysis In App Testing, No System Level – Breaks CI Only Automation Solution (No Manual, No Perf/Monitoring) Not enterprise grade Security (local testing) Community Driven Limited Support No end-user environments Very basic test reports No support model Customers manage device issues
- 11. Website & Apps – Object Trees/Identifying Objects • Web Objects • DOM • Same Tree (PM or Appium) • Native App Objects • PM or Appium framework • Separate naming for Android and iOS • Hybrid Application • PM automation framework • Mix of Native and DOM objects DOM Perfecto Mobile Appium
- 14. Global Data Centers Network 148/11/2017
- 15. Architected for Digital Quality at High Velocity 8/11/2017 15© 2017, Perfecto Mobile Ltd. All Rights Reserved. Maximize Digital Test Coverage Maximize % of Automation 1. Cloud-first architecture total reliability, unlimited scale, secure, global and available 24/7 2. Platform connectivity layer™ (PCL) architecture • Making open source enterprise grade • Enables high test automation coverage 3. One Script, One Lab for all digital platforms 4. Embedded into your toolchain to streamline activities 5. WindTunnel™ Layer persona based user environment testing for true digital coverage 6. DigitalZoom™ Reporting provides cross platform visibility and fast drill down Quality Across Dev- QA-Ops Fast Feedback Loop FOUR PILLARS OF SUCCESS
- 16. Perfecto PCL™ - Architectural Workflow 8/11/2017 16© 2017, Perfecto Mobile Ltd. All Rights Reserved. Standard Open Source Platform Under Test Standard Appium Server/ Selenium Grid Standard OSS Test Code In Dev IDE’s Platform Under Test Standard OSS Test Code In Dev IDE’s Standar d OSS Standard Appium Server/ Selenium Grid Perfecto PCL™ Benefits Standard Open Source With Perfecto PCL™ Test across platforms Same day support for any platform using any test framework Enhanced test coverage • Full system level control • Handle interruptions: text msg, system msg • Test audio (e.g. voice commands) • Test image (e.g. check deposit) • Test environment conditions (Wind Tunnel™) Visual validation of screen layout Always On lab
- 17. Pre-requisites (for using IDE PlugIn) • 1) Example: IntelliJ or Eclipse working environment • 2) MobileCloud for IntelliJ or Eclipse plugin (download provided as part of package by Perfecto) Go to IntelliJ or Eclipse website to download and install. Note: For IntelliJ, verify Gradle build automation system is installed on your computer. 1 2
- 18. • 3) Set up Perfecto Lab Connection • 4) Start Creating Projects 3 4 Pre-requisites (using IntelliJ)
- 19. Example Project/Script (IntelliJ) • Create a new or modify existing Perfecto Lab Project in IntelliJ • (we’re using IntelliJ today – can also build projects in other IDE’s/Tools like UFT, Visual Studio, Android Studio, Eclipse) • Add sample code to your project (using whatever programming language) using the Perfecto plugin recorder or code editor • Run the test • Review the results © 2016, Perfecto Mobile Ltd. All Rights Reserved. 19
- 20. • Go to Help > Install New Software. Example (Eclipse Install) 1
- 21. Example (Eclipse Install) • Confirm details and click Next. 2
- 22. Exlipse - Perfecto Lab Plugin and Credentials • Window-> Preferences-> Perfecto Lab • Enter your cloud host • Enter your username • Enter your password • Click OK 3
- 23. Eclipse - Available Templates • The Perfecto Plugin loads two Templates into the Eclipse Project Creation Wizard • Access from File->New->Project->Perfecto • Appium Template Project • Selenium Template Project • These templates are meant to be examples which can be used to quickly get started working with Perfecto. • These are not meant to be final test cases as you need to add additional code for that. 4
- 24. Reporting/Results - Usage and Flow Link Execution Capability Download PDF: PerfectoLabUtils.downloadReport(driver, "pdf", "C:testreport"); Link to STR - String reportURL = (String)(driver.getCapabilities().getCapability(WindTunnelUtils.SINGLE_TEST_REPORT_URL_CAPABILITY Reporting Tab Capability – Get Grid URL & create testName Reporting Tab Grid View/Dashboard Legacy Test Report Single Test Report A B C D E
- 25. iOS, Appium and Perfecto • As of iOS 10, Apple has announced deprecation of its legacy UI Automation framework. Official Apple testing framework for iOS 11 is XCUITest framework (iOS 11 still not officially released to public but is in beta). • Benefits: Enhanced script execution performance, support of new features (3D touch, Siri injection) • What is Perfecto doing and Why? • Upgrading its iOS automation infrastructure and Clouds to rely on the XCUITest framework • The XCUITest framework exposes a different object tree which will require adaptations in existing scripts. • Perfecto will maintain support for the deprecated UI automation infrastructure, but it is Perfecto’s recommendation to our partners/customers to begin the transition to the new automation infrastructure now (or soon). • Perfecto clouds will support Appium 1.6 server side, so Appium scripts can run as-is in Perfecto. • Continue running existing scripts while adapting scripts and testing them by setting the relevant Appium capability "automationName" to "XCUITest © 2016, Perfecto Mobile Ltd. All Rights Reserved. 25
- 26. Quick Demo
- 27. Q&A
- 28. Thank You David Broerman Technology Alliance Director North America & LATAM Email: davidb@perfectomobile.com
Editor's Notes
- Hello Everyone – welcome to Perfecto’s tech talk webinar. My name is ….
- Before we get started with the Agenda, just a quick housekeeping item…at the end we should have a few mins for questions and answers. There is a panel in the webinar named Q&A to submit questions during the webinar, so if you think of something, please ask. If we are unable to get to finish questions for any reason, then we will send out the info to all attendees. Here’s the agenda…
- The Perfecto CQ (Continuous Quality) Lab platform is a cloud architected quality platform. One lab for all digital platforms, from browsers, desktops to mobile to IoT We have capabilities to mimic the real user environment with our Wind Tunnel Including Location, orientation, network conditions like 3g 4g, and everything you need to reproduce your users environment While providing Quality visibility and trending with our reporting and analytics for fast feedback loops Completely accessible/embeddable in your SDLC process and tools Accessible from coding to customer support To integrated in Any test tool, CI server, test framework, IDE and location
- Perfecto is committed to helping enterprises leverage existing practices, assets and expertise Perfecto Mobile strives integrate with popular tools used by enterprises. Customers can leverage existing assets and expertise . Our support for various tools is constantly expanding. Some customers also take advantage of our published API to create custom extensions to different frameworks.
- We will focus on two testing environments in today’s webinar that are supported by Perfecto Mobile: Selenium RemoteWebDriver and Appium. (For you more advanced users), already familiar with the old WebDriver: RemoteWebDriver is an implementation class of the WebDriver interface that a test script developer can use to execute their test scripts via the RemoteWebDriver server on a remote machine.)
- There are three types of mobile applications – Web (regular desktop/mobile browsers), Hybrid and Native. The standard Selenium RemoteWebDriver allows us to test desktop browsers; and Perfecto Mobile has extended this to include browsers running on mobile devices. Perfecto Mobile recommends using: Selenium RemoteWebDriver for testing web applications (regular desktop and mobile browsers) Appium for native applications, and Selenium RemoteWebDriver or Appium for hybrid applications
- Identifying objects on the different types of websites and mobile applications can be done in multiple ways. We will just focus on using the following, according to Perfecto Mobile best practices: the DOM objects for testing web applications Appium framework for testing Native applications, and the Perfecto Mobile framework for testing hybrid applications The Framework and use of the Object Spy will enable us to find, view properties, and perform interactions on the application using objects (such as a clicking on something on the device screen, selecting an image or text element, and setting text in an edit field). The Object Tree for Web applications uses a standard set of UI Element types to describe how to draw the User Interface. This is the DOM tree and the UI Elements in the standard can be used to describe any website or web application.) The Perfecto Mobile and Appium Frameworks: When looking at the world of mobile applications, it is not as simple as just Desktop Browsers. Apple has its own proprietary set of UI Elements and this is different from the proprietary set used by Android. Perfecto Mobile created a framework that allows us to translate the competing UI Elements of the different application UI sets into a “more generic” set of UI Elements and describe the UI structure of an application using these generic UI Elements. This is the Perfecto Mobile framework Object Tree. The Perfecto Mobile framework identifies the objects in the application with the same standard. This means that the object tree representation of the elements will be the same on both iOS and Android applications. So this allow us to use the same objects for both Android and iOS applications, also referred to as ScriptOnce.) This will save time with not only creating automation scripts, but will provide easier and less time consuming maintenance of scripts. Appium, that we use to test native apps, references the UI Elements using the names provided by the proprietary description files. This is the Appium framework Object Tree. The Appium framework uses two different object tree representations – UI Automator for Android and UI Automation for iOS. This means that when we develop our automation test, according to this mode of work, we will need to create two separate tests (one for Android and another for iOS).
- How does it work?
- The first picture is inside one of Perfecto’s Secure Datacenters, with racks of servers and devices locked in cabinets. Notice in the second picture, that we have real devices connected, via a USB cable, to the Perfecto Handset Server (or HSS). The HSS, which is responsible for managing all device connectivity, gets its commands from the MobileCloud manager (MCM) – which is a server. The MCM is the component responsible for executing all of the scripts as well as storing within its repository and reports. Through this USB cable we get a live streaming from the device and are able to control the device (what we call system level control); including any device real user commands/gestures (such as tap, swipe, click etc.). We don’t jailbreak/root or take the devices apart. We purchase them directly from the manufacturer, or we can take devices from you or a customer and also put them in the cloud.
- Perfecto operates 14 global data centers, including North America, South America, Canada, Israel, Europe, India (Gujarat - Guu Ju Rat) , Australia, and China. Photos: Top Left & Right – WiFi paddles/repeaters enabling increased device density | Top Right – Another Picture inside one of our Datacenters Bottom Left, FCC approved cellular tower repeater | Bottom Right, GPS signal repeater Center: Global NOC located in Wakefield, MA
- How is our perfecto hosted cq lab different? There are six key capabilities that make our lab uniquely capable to satisfy all four pillars of success They are: Cloud is not hosting devices. It’s much more than that. We connect differently to platforms than anyone else. This removes instability and barriers to high automation. One lab for responsive web is an example, multiple screen sizes can be tested using the same script across both mobile and desktop browsers. Our solution is architected to allows us to detach our lab from the interface – allowing the lab to be embedded across tools We are the only vendor with the ability to perform persona based testing, defined as a parameter in you test scripts. We decouple the persona conditions form the test case, allowing the same test to be run across different personas (with different network, location, background apps, etc). It is unique in that no reporting tool on the market provides a comprehensive digital view with the flexibility to see digital platform in one view, and zoom in to the point of failure, by platform/and step in the flow.
- Our platform connectivity layer provides a many benefits over standard open source. Same day support was a big pain encountered in 2017 for those not using Perfecto. lack of support for latest iOS platform (iOS 10, iPhone7) - It took 4 months for Appium to support iOS 10 in a GA release. iOS10 adoptions was at 54% of the market within the first 6 months. With Perfecto, our customer saw 0 interruption. You can even test on beta platforms the day they are released. Open source has no beta platform support. This can be the difference between testing on new platforms a month or more before they are released to the public. The Perfecto hosted cloud has a built in implementation of Remote Web Driver. So you are able to use Selenium and Appium through Perfecto’s Server. So for example, instead of having to set up an Appium Server locally, and then tether devices (1 iOS device per mac or a couple of Android devices to a windows machine), you can just connect to Perfecto’s server instance through the cloud and interact with the devices like you had them in your hand. So the benefit there is you or your team not having to deal with a local server instance, plus not having to manage physical devices or plugging them into machines locally. A few other benefits with using Perfecto’s instance, is being able to interact with objects using visual, setting end user conditions (like GPS location, phone calls and sms messages), network virtualization (3G 4G), and capturing device vitals (CPU, Memory and Bytes in/Bytes out). The bottom line is that it’s the same Selenium/Appium that users are used to working with, plus the additional Perfecto functionalities that we bring to the table. This setup allows you and your teams to be more productive, test on real devices with those all important real end user conditions PCL (platform connectivity layer) provides control over how we interact with the device, without modifying the interface or the jailbreaking or rooting the device itself, while providing system level control to the device. It is this capability, the PCL, that enables complex flows to be automated, enabling you to get to 80- 90% automation.
- Show Perfecto PlugIns page
- Step by Step of the plugin installation Community: https://community.perfectomobile.com/series/18679-perfecto-lab-eclipse-plugin-setup-configuration
- Configure your Perfecto plugin environment Eclipse go to Window > Preferences > Perfecto Lab. Cloud Example: partners.perfectomobile.com, Username Example: user@perfectomobile.co, Password Example: test1234, connect using HTTPS is likely desired. ---------- These are your default credentials, and can be overridden from within your script.
- The templates available through the Perfecto Plugin are provided as a means of getting setup and started with the Perfecto Continuous Quality Lab quickly. They supply means of setup, test, tear down as well as things such as implicit waits to hand hold you as you get used to using Selenium with Perfecto. There are also utility classes supplied to assist in performing Perfecto specific functions (wind tunnel, report download, etc)
- Here is some additional information on Perfecto Reporting, A –Execution of tests from your favorite IDE, language, testing framework B – Can see reports in the Reporting Tab – In Perfecto Cloud C - Legacy Test Report - Can be Exported out and saved in different formats (example html, pdf). Can also be coded for in script, so maybe you are using Selenium/Appium for your testing and want to save script as a pdf format. D- Single Test Report – Link in Legacy Report or can be coded for in script (URL) E - Grid View/Dashboard – New Reporting Area – can see tests in Dashboard format, filter and drill down Just a quick note, Perfecto doesn’t limit you to just these reports/dashboards. You could also use an Open Source reporting framework like Allure or a dashboard/business intelligence tool like Splunk.
- Other Notes: Improved Open Source Alignment As part of the cloud update to support the new XCUITest-based automation infrastructure, Perfecto clouds will support Appium 1.6 server side, so Appium scripts can run as-is in Perfecto In parallel, we are removing few proprietary Perfecto functions in favor of their Appium equivalents We plan to keep and increase functions that extend test coverage beyond what Appium provides. Documentation: Beyond the general communication on this topic, please see the detailed Perfecto guide located at http://developers.perfectomobile.com/display/TT/Migrating+iOS+test+scripts+to+XCUITest-based+automation+infrastructure Code Samples: Code samples are available, showcasing an XCUITest-based automation infrastructure scripts written in Appium, Quantum etc. Tools: As we anticipate significant effort in identifying object identifiers and functions that need to change as part of this transition, Perfecto will provide an analyzer that will reflect the changes needed in individual identifiers, test scripts or according to test executions in the cloud. Professional Services: In case the transition seems daunting or customers would like advice or guidance, please contact our customer success managers to coordinate relevant activity.
- Run example Project in an IDE Eclipse/Java/Selenium/Maven Project with TestNG Project Name: DigitalWebScenarioParallel This script will highlight our script once technology, running 2 desktop browsers and 2 mobile devices (an iOS and Android) –in parallel using testng Open RJReynolds.xml – Open RJReynolds.java - This test will open Camel.com – and register To run: Right click DigitalWebScenarioParallel Go to run configurations Make sure DigitalWebScenarioParallel_RJReynolds.xml selected under testng Make sure suite radio button selected and path of C:\Users\davidb\workspace1\DigitalWebScenarioParallel\RJReynolds.xml Click run Once complete – show the report URL’s
- Let’s take a few minutes for Q&A. Remember, there is a panel in the webinar named Q&A to submit questions (if you have already done that, great) if not, please take a minute or so now to type out your questions, We’ll go on mute for a minute, and then come back and answer the questions.
- Thanks everyone for attending today. Have a good one and hopefully we will see you next time on another Perfecto Tech Talk.