The Internet of Things is an exciting and scary place, full of new protocols, completely lacking standards, and never as simple as it should be. There are plenty of people telling you what to build, how to build it, and how to make money on it…but what about making sure that your IoT solution works, that it’s fast, and that it’s safe? Learn how you can test IoT devices using REST, MQTT, or an SDK to make sure the IoT of tomorrow actually lives up to all the hype. Examples will be based off of real-world and virtualized environments.
If you are in this audience, congratulations, you’re responsible for the future of mankind, no pressure
This is the goal I want to impress on you today
Problem: How do you do that in a world
that’s constantly changing faster than we can keep up, and
where you know less about the way people will use your products than ever before?
We have arrived at the life of a guerilla tester, because…
Testing isn’t about what you know, it’s about how you explore what you don’t.
Both IoT and Microservices break the world up into smaller bits, and for good reason
The physical world is complicated, requires lots of detangling / parsing or instrumentation
The information space, data and code, can get out of hand surprisingly fast (spaghetti code)
Lots of things. Traditional: laptops, mobile phones, tablets, POS stations, scanners, ATMs
Non-consumer stuff: medical and scientific instruments, automotive, robotics
IT: Servers, Gateways, Switches, Balancers, Firewalls, Filters, UPC, HVAC, Lighting, Entertainment
Infra: Buildings, bridges, tunnels, dams
There are lots of “not things”
Your website, a hub of information you can’t life without, but it’s not a “thingy”
Your API is a purely digital product. There is no real-life counterpart to your API. The concept of APIs is a purely information-age concept.
AWS Lambda, IronWorker, StackStorm, IFTTT
What’s different in a world inundated by connected bits?
“bits” demand composability, your “things” MUST be recomposed in ways you didn’t intend
Data on context (locality, identity, capability) will be even more important than ever before
Metadata is often more important than data; social connections, not status updates
Failures before more real, tangible, tactile, and personal (J I N D O)
HVAC malfunction in the middle of winter, alarms, medical misinformation, car brakes
Jindo Haenam bridge outfitted with structural sensors, when’s the best time to take advantage of that system?
Our testing principals will have to adapt to the shifting emphasis on “orchestration” of things and security
What I’ve been calling “orchestration testing”, includes unit/service/workflow testing, but also concepts like consumer-driven contract testing, API and device virtualization, and necessarily asynchronous testing processes to simulate how the real world situation plays out. Testing beyond the “happy path” scenarios becomes just as necessary as getting the obvious business cases out of the way. In fact, letting testers identify SPOFs and poorly performing 3rd parties will become vital to rapid-release dev cycles.
Why do I call this a jungle?
A densely packed ecosystem with lots of unrestrained growth
A lack of standards, MQTT and CoAP are working on this
SDKs run rampant, ultra-proprietary technology impedes innovation and growth
There is a food chain, and big players look to devour
The good news: some things are easier to test than others; what makes something easy to test?
How simple is it? How many moving parts?
How many interaction points does it have?
Does it require physical interaction?
Is it well documented (for people AND machines)?
Toolchain / ecosystem support
Interoperability, standard protocols
What does a mixed-environment test look like in tooling? How does MQTT fit alongside our existing knowledge?
Why does this matter to testers?
If it can’t be tested (easily), it needs to go back to the drawing board; get up, stand up
Testing processes will move from narrow constraints to few constraints with lots of things
“Orchestration testing” will require architect-like thinking over automation process
Things like Docker will be invented to solve the “lots of bits” problem, and affect our workflow
How can we make sure the IoT doesn’t run off the rails?
Iterate, iterate, iterate; you’ll never know if you don’t try; small, anticipated failure teaches
Earlier testing; reduces downstream cost, proves feasibility
Contribute to open standards, don’t just use them; devs and testers alike
Think about the whole picture; how will my devices and services be used and misused?
And that’s all I had. Thank you for letting me come here and speak!