The document discusses testing for Internet of Things (IoT) software. It begins with an introduction to IoT and describes emerging IoT applications and the typical IoT technology stack. It then discusses challenges in testing IoT software and how the role of quality assurance is changing. The document outlines various areas of IoT testing including connectivity, security, performance, functionality and more. It provides examples of test cases for each area. Finally, it proposes a strategy for effective IoT software testing that emphasizes automation, virtualization, robust backends, and testing at the design stage.

1. IoT Testing and Quality
Assurance
Girish Kolapkar
SAS Research and Development (India)

2. Agenda
• Introduction to Internet of Things (IoT)
• Industries approaching the opportunity of IoT
• Emerging IoT Applications
• Typical IoT Technology Stack
• Challenges testing software’s supporting IoT
• Software QA’s Role Redefined
• IoT Testing Areas
• IoT Testing Types
• Strategy for effective IoT software testing
• Q&A

3. Introduction
• The Internet of Things (IoT) is the network of physical
objects or “things” embedded with electronics, software,
sensors, and network connectivity, which enables these
objects to collect and exchange data.
• Software industry is blending the physical and logical cyber
worlds with the Internet of Things.
• IoT isn’t merely a buzzword. It’s where we’re headed.
• Offers potential for businesses worldwide to unlock value
and benefits from devices, systems and services
communicating with each other, using the internet as a
backbone.

4. Industries approaching the opportunity
of IoT
Internet
Of Things
Smart Cities and Homes
Connected Customer
RetailHealthcare
Energy
Connected Car/
Transportation
Communications
Manufacturing

5. Emerging IoT Applications
• Information & Analysis
• Tracking Behaviour
• Examples- SCM, Trucks with sensors for predictive
maintenance
• Enhanced Situational Awareness
• Examples- Weather Warning Systems
• Sensor-driven Decision Analytics
• Examples- Oil and gas company optimizing oilfield
production

6. Emerging IoT Applications (Contd.)
• Automation & Control
• Process Optimization
• Examples-Manufacturing Assembly Line
• Optimized Resource Consumption
• Examples-Energy Distribution Networks
• Complex Autonomous Systems
• Examples- Collision Avoidance Systems

7. Typical IoT Technology Stack
Data Streams at the EdgeData Streams at the Edge
ThingDeviceSensorActuator
Controller, Agent,
Short HaulLong haul Communication
InfrastructureInfrastructure
Network, Compute ,Data Storage
Application PlatformApplication Platform
Event Management, Intelligent Filter Transform,
Steaming Model Execution
ApplicationApplication
WebsitesMobile AppsIndustry Specific
appliances & consoles,
AlertsReports Decision Dashboards
Device Hub Gateway, Device Management
API DesignBuild

8. Challenges testing software’s supporting
IoT
Big Data-Edges generates astronomical volume of data, speed & variety

9. Challenges testing software’s supporting
IoT (Contd.)
• Complex mix of modern IT & Web services with embedded
systems
• Security
• Expensive test environment setup
• Too Many- networks, protocols, gateways, device hubs

10. Challenges testing software’s supporting
IoT (Contd.)
• Multi-user Applications & distributed environment
deployment
• Applications require decision support in real-time, batch,
hybrid modes
• Debugging requires understanding of multiple technologies
• Difficult to establish point of failures
• Limited User Interfaceconsole

11. Software QA’s Role Redefined
• More focus on user centric testing
• Just not find bugs,prevent them
• Quality advocates, influencing both development and
operational processes
• Software testers who do not have experience testing
hardware or embedded systems may need to develop
some new skills
• Testers with domain knowledge

12. IoT Testing Areas
• Connectivity
• Security
• Performance
• Functionality
• Compatibility
• Localization
• User Experience

13. IoT Testing Areas-Connectivity
• Seamless connectivity
• Data restore when no connectivity
• Sample Test cases
• Verify that all the gadgets involved in the Internet of
things testing are able to register to the network.
• Verify that application ensure data security and storage
so that whenever the connection is restored, data is
back to shape as it was.

14. IoT Testing Areas-Security
• More possibilities and avenues for cyber attacks
• More threats with financial data. E.g. RFID based highway
toll collection systems
• IoT ecosystem provides multiple entry point for attacks
• Lost or stolen devices
• Sample Test cases
• Verify no unauthorized access to device or information.
• Verify data on compromised IoT devices can be remote
wiped out.

15. IoT Testing Areas-Performance
• Device- to-Device communication
• Device to Server communication
• Server To Server communication
• Network bandwidth, latency, packet loss
• Sample Test cases
• Verify response time against benchmarked time with
defined connectivity conditions.

16. IoT Testing Areas-Functionality
• WebMobile applications
• Functionalbusiness requirements
• Access control
• Data Storage
• IdentityRoles management
• Sample Test cases
• Verify medical monitoring system returning correct
outputs.

17. IoT Testing Areas-Compatibility
• Device-to-device communication protocols are not
compatible, forcing the user to select a family of devices
that employ a common protocol
• Lack of standardization leads to incompatibility issues
• Compatibility needs to be validated for combinations of
devices, protocols, operating systems
• Sample Test cases
• Verify IoT software supports defined set of decives.

18. IoT Testing Areas-User Experience
• Refers to a person's total experience using a particular
product, system or service
• Familiar
• Clear & Concise
• Feedback
• Consistent
• Attractive
• Efficient
• Forgiving

19. IoT Testing Types
• Compatibility Testing
• Functional Testing
• Performance Testing
• Security Testing
• Exploratory Testing
• Load Testing

20. Strategy for effective IoT software
testing
• Automation rules-Develop automation systems, through
code, that can ensure quality standards at each level and
which assures -
• Response Time
• Data Validation
• Real Time Data Accuracy
• Secure Access control
• Early detection of issues at design time
• Needs more detailed Acceptance Criteria

21. • Robust Backend – NoSQL ?
• Minimal manual end to end testing
• Virtualization
• Simulators for various industries
• Remote Testing
• Mock up the API
Strategy for effective IoT software testing (Contd.)

22. • Elevate a test approach rather than simply executing it
• Analyse Test Results and Test effectiveness
• Measure quality of Test
• Timely review testing tool set as per user feedback,
remote debugging and quick turnaround
Strategy for effective IoT software testing (Contd.)

23. • Generate Realistic Test data
• Data Generator Tool
• Test Analytics
• Understand signals in data to derive new patterns
based test cases
Strategy for effective IoT software testing (Contd.)

24. • Effective Test Data Management Tool
• Data on Demand
• Data Integration
• Data Security
• Data validation
Strategy for effective IoT software testing (Contd.)

25. • Test suite of source adapters
• Predefined adapters e.g. Hadoop (+YARN),
RabbitMQ
• Test for publish(read) only- e.g. Sniffer,Twitter,
SYSLOG, HTTP RESTful Interfaces
• Test for subscribe(write) only- e.g. SOAP,SMTP
• Customizable publishsubscribe API’s
Example- Test Event Stream Processing Engine (Contd.)

26. • Test visual interface which definerefine
event stream models
Example- Test Event Stream Processing Engine (Contd.)

27. • Deploy & Test On Workstation
• E.g. Test for iPhone sensor logs
• Deploy & Test on Edge Gateway
• E.g. Test for physical raspberry-pi
Example- Test Event Stream Processing Engine (Contd.)

28. • Connect iPhone and Laptop to wireless network
Example- Test Event Stream Processing Engine (Contd.)

29. • Step 1: Install and check SensorLog app on iPhone, get ip
address.
• Step2: Start ESP server and put ESP model in test mode
• Step3:Run SensorLog and start streaming data through
socket
• Step4: Run adapter script to read iPhone sensor data
• Step5: Open visualization to see streaming data
• Choose line graph to show sensor data and moving
average
• Shake your iPhone to see graph changes
Example- Test Event Stream Processing Engine (Contd.)

30. QUESTIONS?

31. Thank You!
girish.kolapkar@gmail.com
https://www.linkedin.com/in/kolapkar
https://www.facebook.com/girish.kolapkar