Downloaded 10 times
Uploaded byQualitest
IoT Quality Challenges - Testing & Engineering
The document discusses the challenges and methodologies in IoT testing and engineering, emphasizing the critical need for interoperability and quality assurance as the IoT landscape rapidly evolves. It highlights the significant economic impact of IoT, projected to add $1.7 trillion in value, and outlines the capabilities of IBM's IoT portfolio, including solutions for various industries. The content also addresses testing challenges such as device complexity, architecture, and the importance of integrating testing as part of DevOps to optimize time to market.
More Related Content
![IoT and Cybersecurity: What can be done? by Gerald Faulhaber at Inform[ED] Io...](https://cdn.slidesharecdn.com/ss_thumbnails/3-170509161241-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to IoT Quality Challenges - Testing & Engineering
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
IoT Quality Challenges - Testing & Engineering
- 1. IoT Quality Challenges Testing& Engineering
- 2. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 3. Webinar Presenters Benny Sand VPKnowledge and Innovation Joel Goodman CTP Watson IoT Platform
- 4. | World’s 2ndLargest Independent Testing Company | QualiTest was recognized as the only visionary pure play QA and testing Service Company in the “ Magic Quadrant” About QualiTest Group
- 5.
- 6. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 7. What once seemed like hyperbole, nowseems like understatement. IoT has taken hold USD 1.7 trillion in value added by the IoT to the global economy in 2019.1 82 percent of enterprise decision makers say IoT is strategic or transformative to their enterprise.3 From 13 billion to 29 billion Projected growth in connected devices by 2020 2 1 Business Insider, "The 'Internet of Things' will be the world's most massive device market and save companies billions of dollars," John Greenough, April 14, 2015. 2 IDC, Worldwide Internet of Things Forecast, 2015-2020, Doc #256397, May 2015. 3 IDC, “Internet of Things: New Worldwide Demand Side Research on Perceptions and Plans for Adoption 2015,” September 10, 2015,
- 8. Proven value, immediateopportunity Leaders are putting the physical world to work and achieving results now Transforming the customer experience Boosting Operational efficiency Disrupting their industries
- 9. The IBM portfoliofor cognitive IoT A comprehensive, integrated set of capabilities Solutions Enabling new business models with integrated solutions for industry Applications Optimizing operations for business impact Platform Everything you need to innovate with IoT Powered by IBM Watson Local Deployment Enabled by IBM Cloud Connecting the data that matters Business Transformation
- 10. A closer lookat IBM’s IoT portfolio Platform Everything you need to innovate with IoT Product Family Watson IoT Platform Continuous Engineering Connect Attach: MQTT, HTTPS Visualize Collect & Organize Device Management Users Information Management Parsing Unstructured Data Storage & Archiving Metadata Management Reporting Streaming Information Risk Management Proactive Protection Analytics Predictive Cognitive Realtime Performance Contextual Bluemix Open Standards Based Services Full Development Lifecycle DevOps Services IBM Security
- 11. A closer lookat IBM’s IoT portfolio - Real-Time Informed Decisions - Increased Asset Performance - Optimized Cost of Maintenance - Manage Risk to Operations - Maintain Operative Compliance Product Family Maximo Asset Management TRIRIGA Facilities and Real Estate Management Asset Management Performance Management Connected Products Facilities Management Space Optimization Energy Management Applications Optimizing operations for business impact
- 12. A closer lookat IBM’s IoT portfolio Solutions Enabling new business models with integrated solutions for industry Product Family IoT for Manufacturing IoT for Automotive IoT for Electronics IoT for Retail IoT for Insurance
- 13. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 14. Create a Bluemixaccount Navigate to https://console.ng.bluemix.net/
- 15. Create an WatsonIoT Platform Instance
- 16. Create an WatsonIoT Platform Instance
- 17. Create an WatsonIoT Platform Instance
- 18. Create anWatson IoTPlatform Instance
- 19. Creating a simulateddevice
- 20. The IBM WatsonIoT Platform
- 21. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 22. IoT Challenges | Sensing acomplex environment | Connectivity | Power is critical | Security is vital | Complexity | Evolving architectures, protocol wars and competing standards | Lack of a Shared Infrastructure | Data Control & Sharing | Cloud is important
- 23. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 24. IoT – Testing Challenges | Architectureof the hardware and of software, means more configurations must be tested, e.g. Android fragmentation | Complexity of the software and system | The devices may have many bugs | The hardware constraint - avoid recalls | Development time considerations, such as time to market pressure, which exist in IT and Mobile, will continue with IoT | Resource considerations such as limitations in: memory, processing power, bandwidth, battery life, etc.
- 25. IoT – Testing Challenges | Embeddedfunctionality | Web provided functionality | Performance both of the network communication and internal computation | Conformance and Robustness | Safety / Regulation testing | Security including privacy, autonomy and control | Smartness of the device and the user interface or of the software in some devices (may hide bugs)
- 26. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 27. Constraints | IoT isagile but… | IoT is multi disciplinary | Understanding the Com protocols | E2E testing is required | Regression end | Verity of tools
- 28. Testing Evaluation Criteria | Test Coverage |Functionality & Performance | Conformance | Interoperability | Means of Testing | Test Beds | Test Houses | Certifications programs | Testing events / Plug Set | Method | Manual, Automation | Cost of testing | Direct | Time | Tools, labor means | ROI | The cost of recall
- 29. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 30. Interoperability Challenges | Technical Interoperability |Syntactical Interoperability | Semantic Interoperability | Organizational Interoperability | Field test versus Lab test
- 31. DevOps Principles (C.A.L.M.S.) | C –Culture | A – Automation | L – Lean | M – Measurement | S – Sharing
- 32. Where does QA fitin DevOps? | Addressing a misconception | QA is not a bottleneck | DevOps is about closing the distance between development and operations through: | Meaningful collaboration | Early engagement | Continuous processes
- 33. Test Automation andTime to Market | DevOpsis largely about improving time to market | Test Automation is a must-have, no longer just a nice- to-have | No longer justify test automation with ROI | Now viewed as one piece of the overall DevOps approach | Key metric is always time to market
- 34. Agenda Platform - Demo IoTTesting Challenges Summary & Questions Interoperability & DevOpsIoT Testing Methodologies Introduction IoT ChallengesThe IBM Portfolio
- 35. Summary | Quality isa BIG issue in the IoT marketplace | Interoperability testing should be addressed via processes and technologies | Testing in the IoT testing is multi-disciplinary evolves HW, Cloud SW , Usability , Power and more. | Cognitive IoT is a game changer | Through IBM’s platform it is quick and easy to create and test an IoT solution
- 36.
- 37.
Editor's Notes
- #3 Read the Agenda
- #4 Read the Agenda
- #5 The world's second largest Independent pure play testing specialist. we have subsidiaries in the US Europe and Asia Testing is all we do. We have a real passion for testing, ensuring things work! And we have recognized by Gartner as the only visionary pure play QA and testing services company
- #6 IoT
- #7 Read the Agenda
- #10 Notes for presenter: This slide can be inserted after or in place of slide 9 (The IBM portfolio for cognitive IoT) Here is how this depiction of the portfolio relates to the three areas on slide 9: “Platform” on this slide – relates to Cloud-based platform for development and production on slide 9 “Applications” and “Solutions” on this slide – relate to Software and services tailored to specific industry needs on slide 9 “Powered by Watson” on this slide – relates to IBM Watson cognitive computing application programming interfaces (APIs), software and services on slide 9 Slides 19–21 drill down on specific offerings and capabilities within platform, applications, and solutions This graphic enables you to tell a fuller story, adding the following elements: Connecting the data that matters IBM makes connecting devices easy We have a vibrant, ever-expanding ecosystem of ecosystem partners, including silicon and device manufacturers We are working with AT&T, National Instruments, ARM, Semtech, The Weather Company and more to ensure the secure and seamless integration of data services and solutions on IBM’s open platform We actively contribute to open source and participate in standards bodies to drive ecosystem-wide interoperability, security, scalability. We are working with Industrial Internet Consortium, LoRa Alliance, Open InterConnect, and Allseen Alliance Enabled by IBM Cloud IBM operates more than 41 cloud data centers in the Americas, Asia, Australia and Europe This enables customers to provision cloud resources exactly where and when they need them to minimize network latency, improve application performance, and comply with local data regulations Local Deployment IBM also supports local on-premise deployment and hybrid cloud Business transformation All of these portfolio elements work together to achieve the end state of business transformation, which can be driven by any or all of the three key IoT outcomes: boosting operational efficiency, transforming the customer experience/relationship, or driving industry disruption
- #14 Read the Agenda
- #22 Read the Agenda
- #23 Sensing a complex environment Innovative ways to sense and deliver information from the physical world to the cloud Connectivity Variety of wired and wireless connectivity standards are required to enable different application needs Power is critical Many IoT applications need to run for years over batteries and reduce the overall energy consumption Security is vital Protecting users' privacy and manufacturers' IP; detecting and blocking malicious activity IoT is complex IoT application development needs to be easy for all developers, not just to experts Cloud is important IoT applications require end-to-end solutions including cloud services Evolving architectures, protocol wars and competing standards. With so many players involved with the IoT, there are bound to be ongoing turf wars as legacy companies seek to protect their proprietary systems advantages and open systems proponents try to set new standards. There may be multiple standards that evolve based on different requirements determined by device class, power requirements, capabilities and uses. This presents opportunities for platform vendors and open source advocates to contribute and influence future standards. “So, what really is a step?” Kalmar asked, referring to measurement unit included in most fitness trackers. Definition of that simple measurement unit can confuse users, given the range of activities that are included in its definition. The problem may sound as if it is related to linguistics but it is, in fact, technical one. As Kalmar clarified later, measurement of a “step” depends to a large extent on components, such as accelerometers, within that device. This is a problem due to the lack of open standards. The lack of open standards manifests itself at the Institutional level within the IoT space. For example, the Internet Engineering Task Force or IETF incorporates inputs from multiple stakeholders, which include a broad mix of policy makers to engineers to formulate the framework for future development on the Internet. No such organization exists for the Internet of Things, even though the technology borrows liberally from both hardware and web development practices. Establishing such standards will help standardize and clearly frame the answer to Kalmar’s question about a step by breaking it down, as she already has, in terms of constituent technology. According to her, well-documented APIs could be a start by providers could be a start towards open standards. Concrete use cases and compelling value propositions. Lack of clear use cases or strong ROI examples will slow down adoption of the IoT. Although technical specifications, theoretical uses and future concepts may suffice for some early adopters, mainstream adoption of IoT will require well-grounded, customer-oriented communications and messaging around “what’s in it for me.” Detailed explanations of a specific device or technical details of a component won’t cut it when buyers are looking for a “whole solution” or complete value-added service. IoT providers will have to explain the key benefits of their services or face the proverbial “so what.” Lack of a Shared Infrastructure: In its simplest form, the Internet of Things is a complex interconnection of hardware, such as sensors and actuators, and software that works at the assembly level. Together, these constitute a platform for developers and companies. The vertical nature of this platform has contributed to fragmentation of its infrastructure. “We have the Apple AAPL +1.74% Internet of things and the Google GOOG +1.53% Internet of Things,” explained Kalmar during her talk. In turn, this has contributed to a race between these companies to own the vertical stack rather than develop products or services that benefit consumers and developers. There are few open source platforms in the Internet of Things. As has been shown in the case of software, open source platforms reduce product development costs and encourage creativity and collaboration. Each IoT device currently is installed in its platform and ecosystem. Data Control: From the user perspective, this is one of the more significant barriers to large-scale adoption of the technology. Data control is commonly mistaken for data ownership. “It is no longer about who owns the data,” clarifies Kalmar. “It is about control and about deciding who gets access to my data” Enabling access to private data is a slippery slope. There are net positives such as Google Flu Trends, which accurately predicted Flu outbreak cases, faster than the Centers for Disease Control. However, as recent developments have shown, sharing personal data can be a two-way street. Data Sharing: In the Internet of Things paradigm, data is gold. However, data provisioning builds off a social contract between large corporations and customers. Corporations provide a free or nominally-priced service in exchange for a consumer’s personal data. This data is either sold to advertisers or used to develop further products or services useful to consumers. Third-party applications, which build off the core service, poach customers (and related customer data) from such applications. For established networks and large corporations, this can be detrimental practice because such applications eventually poach their customers. In such a scenario, large corporations need to balance their approach to open source with commercial considerations.
- #24 Read the Agenda
- #27 Read the Agenda
- #30 Read the Agenda
- #31 Interoperability testing involves testing whether a given software program or technology is compatible with others and promotes cross-use functionality. This kind of testing is now important as many different kinds of technology are being built into architectures made up of many diverse parts, where seamless operation is critical for developing a user base. Field test refers to the testing done on a commercial network or pilot network to verify mobile or network behaviours. There are different reasons field tests are carried out around different art of the world mainly: To verify that UE or mobile can operate on real commercial network with other UEs To debug certain issues in UEs like call drops, throughput issues, handover issues, etc To debug some operator issues like loss of coverage, performance issue, so operators can tweak or upgrade network. To test a newly deployed per-commercial network For benchmarking against other mobile phones. Technical Interoperability is usually associated with hardware/ software components, systems and platforms that enable machine-to-machine communication to take place. This kind of interoperability is often centred on (communication) protocols and the infrastructure needed for those protocols to operate. Syntactical Interoperability is usually associated with data formats. Certainly, the messages transferred by communication protocols need to have a well-defined syntax and encoding, even if it is only in the form of bit-tables. However, many protocols carry data or content, and this can be represented using high-level transfer syntaxes such as HTML, XML or ASN.1. Semantic Interoperability is usually associated with the meaning of content and concerns the human rather than machine interpretation of the content. Thus, interoperability on this level means that there is a common understanding between people of the meaning of the content (information) being exchanged. Organizational Interoperability is the ability of organizations to effectively communicate and transfer (meaningful) data (information) even though they may be using a variety of different information systems over widely different infrastructures, possibly across different geographic regions and cultures. The factors in interoperability testing include syntax and data format compatibility, sufficient physical and logical connection methods, and ease of use features. Software programs need to be able to route data back and forth without causing operational issues, losing data, or otherwise losing functionality. In order to facilitate this, each software component needs to recognize incoming data from other programs, handle the stresses of its role in an architecture, and provide accessible, useful results. One example of an industry where interoperability testing is important is in the medical field. Digital medical record technologies need to be interoperable on many levels in order to ensure that various providers will be able to transfer patient records from one office to another. Many other industries have similar needs, which is why interoperability testing is such an emerging part of software production.
- #32 Technical Interoperability is usually associated with hardware/ software components, systems and platforms that enable machine-to-machine communication to take place. This kind of interoperability is often centred on (communication) protocols and the infrastructure needed for those protocols to operate. Syntactical Interoperability is usually associated with data formats. Certainly, the messages transferred by communication protocols need to have a well-defined syntax and encoding, even if it is only in the form of bit-tables. However, many protocols carry data or content, and this can be represented using high-level transfer syntaxes such as HTML, XML or ASN.1. Semantic Interoperability is usually associated with the meaning of content and concerns the human rather than machine interpretation of the content. Thus, interoperability on this level means that there is a common understanding between people of the meaning of the content (information) being exchanged. Organizational Interoperability is the ability of organizations to effectively communicate and transfer (meaningful) data (information) even though they may be using a variety of different information systems over widely different infrastructures, possibly across different geographic regions and cultures. The factors in interoperability testing include syntax and data format compatibility, sufficient physical and logical connection methods, and ease of use features. Software programs need to be able to route data back and forth without causing operational issues, losing data, or otherwise losing functionality. In order to facilitate this, each software component needs to recognize incoming data from other programs, handle the stresses of its role in an architecture, and provide accessible, useful results. One example of an industry where interoperability testing is important is in the medical field. Digital medical record technologies need to be interoperable on many levels in order to ensure that various providers will be able to transfer patient records from one office to another. Many other industries have similar needs, which is why interoperability testing is such an emerging part of software production.
- #33 Under DevOps, QA needs to have a strong and close relationship with both Development and Operations. So what is involved in establishing and operating this collaboration? On the Development side it is imperative to engage as early in the SDLC as possible. Even testing as low as Unit Testing can no longer be owned entirely by Development. The focus of QA needs to be brought down to these lower levels of testing with much more emphasis than has been true in the past. The result should be a reduction in the overall demand for manual testing and testing at the GUI level, which is great because higher level tests inherently have higher maintenance and execution costs. When it comes to Operations, testing an monitoring needs to be happening in all environments. If Operations is engaged late in the process, developing monitoring capability for production environments and linking the feedback to QA processes can lag behind the speed at which product is delivered. This means that pre production environments need to be as production like as possible. To ensure accurate and complete monitoring, this additionally requires that the environments be more accessible to QA, both from the perspective of existing environments and the creation of additional environments. While in this sense, Operations supports QA, QA also needs to support Operations in strategizing its monitoring capability to appropriate report on meaningful product features and availability.
- #34 Development Engage as early as possible in the life cycle Generate test automation at lower levels within the product Reduce the overall demand for manual testing and testing at the GUI level
- #35 Read the Agenda
- #36 Quality is a BIG issue in the IoT marketplace due to the combination of HW ,SW and services Interoperability testing should be addressed via processes and technologies Testing is a significant factor to the IoT arena Continuous testing helps extend agile and lean practices across the development and delivery life cycle Remove traditional testing bottlenecks such as unavailable test environments to increase efficiency Provide project teams with continuous feedback on service quality to reduce business risks Continuous delivery , Continuous Integration and Continues Testing provide Contentious feedback which hopefully lead to Continuous success Testing in the IoT world is an agent of a process per a project / process
- #37 Quality is a BIG issue in the IoT marketplace due to the combination of HW ,SW and services Interoperability testing should be addressed via processes and technologies Testing is a significant factor to the IoT arena Continuous testing helps extend agile and lean practices across the development and delivery life cycle Remove traditional testing bottlenecks such as unavailable test environments to increase efficiency Provide project teams with continuous feedback on service quality to reduce business risks Continuous delivery , Continuous Integration and Continues Testing provide Contentious feedback which hopefully lead to Continuous success Testing in the IoT world is an agent of a process per a project / process