Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Mobile Monitoring Best Practices
Similar to Mobile Monitoring Best Practices (20)
More from Lizzy Guido (she/her)
More from Lizzy Guido (she/her) (14)
Recently uploaded
Recently uploaded (20)
Mobile Monitoring Best Practices
- 1. Perfecto Mobile Monitoring Introduction Ensuring Ongoing Mobile App Quality Partners Tech Talk Webinar David Broerman Partner Enablement Manager Email: davidb@perfectomobile.com
- 2. Agenda • Intro to Perfecto • Who is Perfecto, Basic CQ Lab Setup and Configuration • Discovery Questions around Monitoring • Monitoring Key Requirements • Real User Monitoring vs. Synthetic Monitoring • Steps to set up Monitoring • Basic demo Perfecto/Smart Bear showing various monitoring pieces • Success Story • Q&A and Wrap Up © 2016, Perfecto Mobile Ltd. All Rights Reserved. 2
- 3. Intro to Perfecto Perfecto Offers: A Cloud Based Continuous Quality Lab enabling testing and monitoring of all digital platforms on real devices and real end user conditions, for enterprises adopting agile delivery processes.3
- 4. Intro to Perfecto Dominant leader & visionary Strong Ecosystem Perfecto @ a glance Working with Enterprises and Partners to deliver exceptional digital experiences. “Perfecto Mobile marks shift in mobile apps QA with Continuous Quality Lab” 4 Setting the bar for market success • Founded in late 2006, Business HQ in Boston • Over 300 employees globally • Fortune 500 customer base
- 5. Cloud Based Quality Platform For The Digital Era 06/30/17 5© 2017, Perfecto Mobile Ltd. All Rights Reserved. Open & Integrated into Dev/Test/Ops Any test tool Any CI server Any test framework Any IDE Any Location Code/Debug Functional Test Performance Test Production Insight One Lab for all Digital Platforms Reporting & Analytics Wind Tunnel Layer Customer Support Browsers Desktops Mobile Devices IoT
- 6. Architected for Digital Quality at High Velocity 06/30/17 6© 2017, Perfecto Mobile Ltd. All Rights Reserved. Maximize Digital Test Coverage Maximize % of Automation 1. Cloud-first architecture total reliability, unlimited scale, secure, global and available 24/7 2. Platform connectivity layer™ (PCL) architecture • Making open source enterprise grade • Enables high test automation coverage 3. One Script, One Lab for all digital platforms 4. Embedded into your toolchain to streamline activities 5. WindTunnel™ Layer persona based user environment testing for true digital coverage 6. DigitalZoom™ Reporting provides cross platform visibility and fast drill down Quality Across Dev-QA-Ops Fast Feedback Loop FOUR PILLARS OF SUCCESS
- 7. Perfecto PCL™ - Architectural Workflow 06/30/17 7© 2017, Perfecto Mobile Ltd. All Rights Reserved. Standard Open Source Platform Under Test Standard Appium Server/ Selenium Grid Standard OSS Test Code In Dev IDE’s Platform Under Test Standard OSS Test Code In Dev IDE’s Standard OSS Benefits Standard Open Source With Perfecto PCL™ Test across platforms Same day support for any platform using any test framework Enhanced test coverage •Full system level control •Handle interruptions: text msg, system msg •Test audio (e.g. voice commands) •Test image (e.g. check deposit) •Test environment conditions (Wind Tunnel™) Visual validation of screen layout Always On lab
- 9. Global Data Centers Network 906/30/17
- 10. Discovery 1. Are you monitoring your mobile app UX today? 2. Do you need to modify your application/device for monitoring? 3. How long (users impacted) before you’re aware there’s an outage/degradation? 4. Do you get alerts and are those actionable? 5. Walk me through your triage process for mobile 6. Do you have visibility into 3rd party impact? 7. Describe the business impact of lengthy remediation process?
- 11. Monitor using real devices Measure the metrics mobile users care about Gain insight into the real user experience Ensure the monitoring solution SLA is better than your service target Enable mobile monitoring integration into proven enterprise-grade solutions Drive mutual language between Ops and Dev Common tools and workflows are key to reduce MTTR (mean time to resolution) 5 Key Requirements for Mobile Monitoring
- 12. Monitoring: Different Perspectives More 3rd parties Server Client SyntheticSynthetic SyntheticSynthetic Real User Monitoring and Synthetic Monitoring are complementary Monitoring Methodologies. Synthetic is proactively collecting availability and response times. Used to troubleshoot and isolate. Monitoring Approach Benefit Pros Cons Applicability Synthetic Real Device Availability Early warning Benchmark Triage SaaS- Easy to deploy (IT independence) Expand existing tools End user performance Analytics High Real User Monitoring (RUM) Performance Real user experience (Performance) Crash reports Analytics (some) Performance impact (of the tag) Application security End user privacy Triage process applicability Medium Synthetic Emulation Availability Leverage existing tools Not mobile Low
- 13. Perfecto Synthetic Monitoring Alerting Drives Early Awareness (MTTK) & correction (MTTR) Real Devices Coverage Geos/Scenarios/Carriers
- 14. MobileCloud Monitoring Solution Architecture
- 15. 1- Describe the end-user scenario • Access Perfecto Mobile’s real device(s) to create the script • Record scripts from the end-user perspective • Leverage Perfecto’s ScriptOnceTM technology for cross-device compliancy User ProcessUser Process
- 16. 2- Set your monitoring schedule • Select devices, networks and locations • Set the monitoring schedule & maintenance windows • Configure alerts (Email, SMS, Post, SNMP etc.); Maintenance Windows User ProcessUser Process
- 17. 3- Access the Monitoring Dashboard User ProcessUser Process
- 18. 4- View the Perfecto User Experience Report, Every Execution Enter Username Enter Password Click Sign In Failure to Login User ProcessUser Process
- 19. Demo
- 20. Question/Answer & Wrap-Up Thank You for attending! David Broerman Partner Enablement Manager Email: davidb@perfectomobile.com
Editor's Notes
- Welcome everyone and thanks for attending today’s session on Wind Tunnel, Personas and Real User Experience Testing My name is ….
- The Perfecto CQ (Continuous Quality) Lab platform is a cloud architected quality platform. One lab for all digital platforms, from browsers, desktops to mobile to IoT We have capabilities to mimic the real user environment with our Wind Tunnel Including Location, orientation, network conditions like 3g 4g, and everything you need to reproduce your users environment While providing Quality visibility and trending with our reporting and analytics for fast feedback loops Completely accessible/embeddable in your SDLC process and tools Accessible from coding to customer support To integrated in Any test tool, CI server, test framework, IDE and location
- How is our perfecto hosted cq lab different? There are six key capabilities that make our lab uniquely capable to satisfy all four pillars of success They are: Cloud is not hosting devices. It’s much more than that. We connect differently to platforms than anyone else. This removes instability and barriers to high automation. One lab for responsive web is an example, multiple screen sizes can be tested using the same script across both mobile and desktop browsers. Our solution is architected to allows us to detach our lab from the interface – allowing the lab to be embedded across tools We are the only vendor with the ability to perform persona based testing, defined as a parameter in you test scripts. We decouple the persona conditions form the test case, allowing the same test to be run across different personas (with different network, location, background apps, etc). It is unique in that no reporting tool on the market provides a comprehensive digital view with the flexibility to see digital platform in one view, and zoom in to the point of failure, by platform/and step in the flow.
- Our platform connectivity layer provides a many benefits over standard open source. Same day support was a big pain encountered in 2017 for those not using Perfecto. lack of support for latest iOS platform (iOS 10, iPhone7) - It took 4 months for Appium to support iOS 10 in a GA release. iOS10 adoptions was at 54% of the market within the first 6 months. With Perfecto, our customer saw 0 interruption. You can even test on beta platforms the day they are released. Open source has no beta platform support. This can be the difference between testing on new platforms a month or more before they are released to the public. The Perfecto hosted cloud has a built in implementation of Remote Web Driver. So you are able to use Selenium and Appium through Perfecto’s Server. So for example, instead of having to set up an Appium Server locally, and then tether devices (1 iOS device per mac or a couple of Android devices to a windows machine), you can just connect to Perfecto’s server instance through the cloud and interact with the devices like you had them in your hand. So the benefit there is you or your team not having to deal with a local server instance, plus not having to manage physical devices or plugging them into machines locally. A few other benefits with using Perfecto’s instance, is being able to interact with objects using visual, setting end user conditions (like GPS location, phone calls and sms messages), network virtualization (3G 4G), and capturing device vitals (CPU, Memory and Bytes in/Bytes out). The bottom line is that it’s the same Selenium/Appium that users are used to working with, plus the additional Perfecto functionalities that we bring to the table. This setup allows you and your teams to be more productive, test on real devices with those all important real end user conditions PCL (platform connectivity layer) provides control over how we interact with the device, without modifying the interface or the jailbreaking or rooting the device itself, while providing system level control to the device. It is this capability, the PCL, that enables complex flows to be automated, enabling you to get to 80- 90% automation.
- The first picture is inside one of Perfecto’s Secure Datacenters, with racks of servers and devices locked in cabinets. Notice in the second picture, that we have real devices connected, via a USB cable, to the Perfecto Handset Server (or HSS). The HSS, which is responsible for managing all device connectivity, gets its commands from the MobileCloud manager (MCM) – which is a server. The MCM is the component responsible for executing all of the scripts as well as storing within its repository and reports. Through this USB cable we get a live streaming from the device and are able to control the device (what we call system level control); including any device real user commands/gestures (such as tap, swipe, click etc.). We don’t jailbreak/root or take the devices apart. We purchase them directly from the manufacturer, or we can take devices from you or a customer and also put them in the cloud.
- Perfecto operates 14 global data centers, including North America, South America, Canada, Israel, Europe, India (Gujarat - Guu Ju Rat) , Australia, and China. Photos: Top Left & Right – WiFi paddles/repeaters enabling increased device density | Top Right – Another Picture inside one of our Datacenters Bottom Left, FCC approved cellular tower repeater | Bottom Right, GPS signal repeater Center: Global NOC located in Wakefield, MA
- Variety of real devices
- RUM: Passive tags (JS, SDK, agents) resident or dynamically injected into code executed on the target device and reporting end user experience (in the case of client RUM) and server metrics (in the case of server RUM), as well as vitals (CPU, battery etc.). Synthetic: Proactive monitoring using dedicated test equipment in sampled form. Collecting availability and response time. Used to troubleshoot and isolate.
- ----- Meeting Notes (1/8/13 19:02) ----- Closing the loop between DevOps members
- Let’s take a few minutes to see if we have any questions, and answer them. Thanks everyone for attending today…Please let us know if we can help with any additional information on these topics. Have a good day, evening or night wherever you may be.