The Internet of Things (IoT) is omnipresent. More and more hardware devices get connected and will collect and share huge amounts of data in the near future. This progress will lead to a digital and hyper-connected world. Though, in such growing networks of interconnected things, quality assurance (QA) will become a continuous challenge. Especially aspects like conformance, interoperability and security but also performance and robustness will require an increased attention from QA perspective.
Test Execution Infrastructure for IoT Quality analysisAxel Rennoch
Recently IoT testing becomes a popular topic in the industry and academic context. New challenges have been identified and existing test methods and techniques need to be collected, optimized and applied. Furthermore, innovative software development approaches are under consideration and partly implemented. However automated test execution still need powerful means and infrastructure. Open source projects like the Eclipse IoT-Testware project can provide valuable tools for advanced testing in IoT. The presentation gives an overview and first results with our IoT test Infrastructure.
Towards a certification scheme for IoT security evaluationAxel Rennoch
Many European and international standardization bodies and industrial organizations do provide more or less detailed specification catalogues addressing IoT product security requirements, test cases and evaluation methods. In this contribution, a dedicated set of relevant standards, guides and recommendations which recently have been recognized by the European Union Agency for Cybersecurity (ENISA) will be introduced. Special attention is given to their contribution for the security evaluation process and the product quality itself, including the level of details regarding their suitability for test definition and execution.
On making standards organizations & open source communities work hand in handBenjamin Cabé
Did you know that the Eclipse Foundation is home to many open source implementations of standards from a dozen of standards defining organizations: IETF, ISO, oneM2M, OASIS, etc.
We do believe that open source is key to standards' adoption, and this presentation shares some thoughts on what makes a standard successful, and how Eclipse has proved with recent success stories that open source and open communities are a key factor.
VERIGRAPH: A pre-deployment verification service for Virtualized Network Functions (aka NFV virtual appliances) running on OpenStack with OPNFV.
Formal Methods: rigorous mathematical methods based on mathematical models for analysing (computer-based) systems
Formal Verification: applied to SDN/NFV-based networks
Verify a formal network model satisfies some invariants or network policies (e.g., absence of loops and black holes, reachability, security policies, etc.)
Summit 16: Automated Platform for Testing VNF Performance and Interoperabili...OPNFV
VNF sizing is one of the big issues for users, integrators and also venders/providers of VNF. In legacy system, network functions are integrated in the systems with carefully sizing by estimated performance metrics, but in NFV, performance and metrics of VNF are depends on flavors. To decide which flavor is suitable for requirements, we have to know the basic data of each metrics with variable flavors. But there are many parameters, so automatic testing is necessary. We had developed the test automation system of VNF that can automatically conducting the sequence of tests with variable parameters such as number of core, memory size, interface speed, etc. We already created the test cases that evaluate BGP interoperability of virtual router and performance test of Cisco, Juniper, Brocade, and so on. We think the activity is similar to Yardstick, so we would like to collaborate with them. In this talk, we summarize our activities and development result, and introducing our latest plan to collaborate with OPNFV.
Test Execution Infrastructure for IoT Quality analysisAxel Rennoch
Recently IoT testing becomes a popular topic in the industry and academic context. New challenges have been identified and existing test methods and techniques need to be collected, optimized and applied. Furthermore, innovative software development approaches are under consideration and partly implemented. However automated test execution still need powerful means and infrastructure. Open source projects like the Eclipse IoT-Testware project can provide valuable tools for advanced testing in IoT. The presentation gives an overview and first results with our IoT test Infrastructure.
Towards a certification scheme for IoT security evaluationAxel Rennoch
Many European and international standardization bodies and industrial organizations do provide more or less detailed specification catalogues addressing IoT product security requirements, test cases and evaluation methods. In this contribution, a dedicated set of relevant standards, guides and recommendations which recently have been recognized by the European Union Agency for Cybersecurity (ENISA) will be introduced. Special attention is given to their contribution for the security evaluation process and the product quality itself, including the level of details regarding their suitability for test definition and execution.
On making standards organizations & open source communities work hand in handBenjamin Cabé
Did you know that the Eclipse Foundation is home to many open source implementations of standards from a dozen of standards defining organizations: IETF, ISO, oneM2M, OASIS, etc.
We do believe that open source is key to standards' adoption, and this presentation shares some thoughts on what makes a standard successful, and how Eclipse has proved with recent success stories that open source and open communities are a key factor.
VERIGRAPH: A pre-deployment verification service for Virtualized Network Functions (aka NFV virtual appliances) running on OpenStack with OPNFV.
Formal Methods: rigorous mathematical methods based on mathematical models for analysing (computer-based) systems
Formal Verification: applied to SDN/NFV-based networks
Verify a formal network model satisfies some invariants or network policies (e.g., absence of loops and black holes, reachability, security policies, etc.)
Summit 16: Automated Platform for Testing VNF Performance and Interoperabili...OPNFV
VNF sizing is one of the big issues for users, integrators and also venders/providers of VNF. In legacy system, network functions are integrated in the systems with carefully sizing by estimated performance metrics, but in NFV, performance and metrics of VNF are depends on flavors. To decide which flavor is suitable for requirements, we have to know the basic data of each metrics with variable flavors. But there are many parameters, so automatic testing is necessary. We had developed the test automation system of VNF that can automatically conducting the sequence of tests with variable parameters such as number of core, memory size, interface speed, etc. We already created the test cases that evaluate BGP interoperability of virtual router and performance test of Cisco, Juniper, Brocade, and so on. We think the activity is similar to Yardstick, so we would like to collaborate with them. In this talk, we summarize our activities and development result, and introducing our latest plan to collaborate with OPNFV.
Cotopaxi - IoT testing toolkit (Black Hat Asia 2019 Arsenal)Jakub Botwicz
Presentation about Cotopaxi toolkit from Black Hat Asia 2019 Arsenal session. Author: Jakub Botwicz
https://www.blackhat.com/asia-19/arsenal/schedule/index.html#cotopaxi-iot-protocols-security-testing-toolkit-14325
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2qoUklo.
Mark Price talks about techniques for making performance testing a first-class citizen in a Continuous Delivery pipeline. He covers a number of war stories experienced by the team building one of the world's most advanced trading exchanges. Filmed at qconlondon.com.
Mark Price is a Senior Performance Engineer at Improbable.io, working on optimizing and scaling reality-scale simulations. Previously, he worked as Lead Performance Engineer at LMAX Exchange, where he helped to optimize the platform to become one of the world's fastest FX exchanges.
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...GetInData
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The talk is focused on administration, development and monitoring platform with Apache Spark, Apache Flink and Kubeflow in which the monitoring stack is based on Prometheus stack.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Dmitri Zimine's slides on Design Summit at OpenStack Barcelona 2016. Talked about the history of two projects, technical differences, discussed an overlap and drafted a path forward.
https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/16999/mistral-mistral-and-stackstorm
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...Amir Zmora
On September 19-23 there was the W3C TPAC meeting in Lisbon. Dan will cover some of the highlights of the recent Lisbon WebRTC meeting, including what items are the sticking points, where work is focusing, progress estimates, and thoughts on what might go into the next version of WebRTC after 1.0 is finished.
Alex will cover the W3C testing platform: "Test The Web Forward". W3C, unlike IETF, is developing and maintaining a complete test suite for all its JS APIs. No specification is actually accepted by W3C and final without the corresponding test suite. Topics that will be addressed include what this testing platform implements, its status with respect to WebRTC and now it is used by different browser vendors as an indication of their compliance with the standards.
As always, we encourage you to submit your general WebRTC related questions beforehand in the Questions & Topics section to make sure we answer them during the session.
Event sponsored by WebRTC.Ventures & Blacc Spot Media
Timing verification of real-time automotive Ethernet networks: what can we ex...RealTime-at-Work (RTaW)
Switched Ethernet is a technology that is profoundly reshaping automotive communication architectures as it did in other application domains such as avionics with the use of AFDX backbones. Early stage timing verification of critical embedded networks typically relies on simulation and worst-case schedulability analysis. When the modeling power of schedulability analysis is not sufficient, there are typically two options: either make pessimistic assumptions or ignore what cannot be modeled. Both options are unsatisfactory because they are either inefficient in terms of resource usage or potentially unsafe. To overcome those issues, we believe it is a good practice to use simulation models, which can be more realistic, along with schedulability analysis. The two basic questions that we aim to study here is what can we expect from simulation, and how to use it properly? This empirical study explores these questions on realistic case-studies and provides methodological guidelines for the use of simulation in the design of switched Ethernet networks. A broader objective of the study is to compare the outcomes of schedulability analyses and simulation, and conclude about the scope of usability of simulation in the desi gn of critical Ethernet networks
Monitoring in Big Data Platform - Albert Lewandowski, GetInDataGetInData
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The webinar was organized by GetinData on 2020. During the webinar we explaned the concept of monitoring and observability with focus on data analytics platforms.
Watch more here: https://www.youtube.com/watch?v=qSOlEN5XBQc
Whitepaper - Monitoring ang Observability for Data Platform: https://getindata.com/blog/white-paper-big-data-monitoring-observability-data-platform/
Speaker: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Cotopaxi - IoT testing toolkit (Black Hat Asia 2019 Arsenal)Jakub Botwicz
Presentation about Cotopaxi toolkit from Black Hat Asia 2019 Arsenal session. Author: Jakub Botwicz
https://www.blackhat.com/asia-19/arsenal/schedule/index.html#cotopaxi-iot-protocols-security-testing-toolkit-14325
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2qoUklo.
Mark Price talks about techniques for making performance testing a first-class citizen in a Continuous Delivery pipeline. He covers a number of war stories experienced by the team building one of the world's most advanced trading exchanges. Filmed at qconlondon.com.
Mark Price is a Senior Performance Engineer at Improbable.io, working on optimizing and scaling reality-scale simulations. Previously, he worked as Lead Performance Engineer at LMAX Exchange, where he helped to optimize the platform to become one of the world's fastest FX exchanges.
Functioning incessantly of Data Science Platform with Kubeflow - Albert Lewan...GetInData
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The talk is focused on administration, development and monitoring platform with Apache Spark, Apache Flink and Kubeflow in which the monitoring stack is based on Prometheus stack.
Author: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Dmitri Zimine's slides on Design Summit at OpenStack Barcelona 2016. Talked about the history of two projects, technical differences, discussed an overlap and drafted a path forward.
https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/16999/mistral-mistral-and-stackstorm
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...Amir Zmora
On September 19-23 there was the W3C TPAC meeting in Lisbon. Dan will cover some of the highlights of the recent Lisbon WebRTC meeting, including what items are the sticking points, where work is focusing, progress estimates, and thoughts on what might go into the next version of WebRTC after 1.0 is finished.
Alex will cover the W3C testing platform: "Test The Web Forward". W3C, unlike IETF, is developing and maintaining a complete test suite for all its JS APIs. No specification is actually accepted by W3C and final without the corresponding test suite. Topics that will be addressed include what this testing platform implements, its status with respect to WebRTC and now it is used by different browser vendors as an indication of their compliance with the standards.
As always, we encourage you to submit your general WebRTC related questions beforehand in the Questions & Topics section to make sure we answer them during the session.
Event sponsored by WebRTC.Ventures & Blacc Spot Media
Timing verification of real-time automotive Ethernet networks: what can we ex...RealTime-at-Work (RTaW)
Switched Ethernet is a technology that is profoundly reshaping automotive communication architectures as it did in other application domains such as avionics with the use of AFDX backbones. Early stage timing verification of critical embedded networks typically relies on simulation and worst-case schedulability analysis. When the modeling power of schedulability analysis is not sufficient, there are typically two options: either make pessimistic assumptions or ignore what cannot be modeled. Both options are unsatisfactory because they are either inefficient in terms of resource usage or potentially unsafe. To overcome those issues, we believe it is a good practice to use simulation models, which can be more realistic, along with schedulability analysis. The two basic questions that we aim to study here is what can we expect from simulation, and how to use it properly? This empirical study explores these questions on realistic case-studies and provides methodological guidelines for the use of simulation in the design of switched Ethernet networks. A broader objective of the study is to compare the outcomes of schedulability analyses and simulation, and conclude about the scope of usability of simulation in the desi gn of critical Ethernet networks
Monitoring in Big Data Platform - Albert Lewandowski, GetInDataGetInData
Did you like it? Check out our blog to stay up to date: https://getindata.com/blog
The webinar was organized by GetinData on 2020. During the webinar we explaned the concept of monitoring and observability with focus on data analytics platforms.
Watch more here: https://www.youtube.com/watch?v=qSOlEN5XBQc
Whitepaper - Monitoring ang Observability for Data Platform: https://getindata.com/blog/white-paper-big-data-monitoring-observability-data-platform/
Speaker: Albert Lewandowski
Linkedin: https://www.linkedin.com/in/albert-lewandowski/
___
Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets.
Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries.
https://getindata.com
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Monitoring Java Application Security with JDK Tools and JFR Events
Functional and non-functional testing with IoT-Testware
1. Axel Rennoch, Alexander Kaiser, Sascha Hackel
Software Quality Days 2019
16. January 2019, Vienna, Austria
FUNCTIONAL AND NON-FUNCTIONAL TESTING WITH
IOT-TESTWARE
2. 2
• IoT Testing
− Challenges and scope
− IoT test language: TTCN-3
• Project IoT-T
− Eclipse IoT-Testware
− Standardization & Certification
• Summary and outlook
AGENDA
3. 3
• Mirai botnet, October 2016:
− botnet using insecure configured IoT-devices (~100.000)
− attack causes blackout and disruption
(e.g. Amazon, Netflix, Twitter, Github)
• Wannacry, May 2017
− ransomeware affecting the whole world (e.g. hospitals in the U.K.)
• KRACK: Key Reinstallation Attack, October 2017
− Replay attack on Wi-Fi Protected Access protocol
• Spectre and Meltdown, January 2018
− Spectre: vulnerability that allows observable side effects from mispredicted
speculative executions
− Meltdown: hardware vulnerability that allows to read all memory
MOTIVATION FOR QUALITY
5. 5
IOT ARCHITECTURE
The Three Software Stacks Required for IoT Architectures, Eclipse IoT Working Group, September 2016
telemetry
commands
telemetry
commands
8. 8
After the acceptance and system tests there will be
a long operation phase => new test phase „operation“
LONG OPERATION LIFETIME
9. 9
INTEGRATION OF SEVERAL TESTING APPROACHES
IoT
Testing
Software
Testing
System
Testing
Security
Testing
Test
Automation
Protocol
Testing
10. 10
• Less resources needed (time and money)
• Avoid human mistakes due to manually testing
• During test development and execution
• Speed-up of regression tests and product time-to-market
TEST AUTOMATION
12. 12
• Toolset (selection of available means)
Protocol tester/monitor (Eclipse Titan, Wireshark)
Test devices (RFID kit, Bluetooth test device)
GUI tester (Selenium, SikuliX, Chrome headless)
Web services tester (soapUI)
…
• Public Testsuites (in development)
Application of a standardized notation
Abstract and platform-independent
TESTWARE
14. 14
• TTCN-3 is the Testing and Test Control Notation
• Internationally standardized testing language for formally defining test
scenarios.
• Designed purely for testing
CHALLENGE TEST AUTOMATION
testcase Hello_Bob () {
p.send(“How do you do?“);
alt {
[]p.receive(“Fine!“);
{setverdict( pass )};
[else]
{setverdict( inconc )} //Bob asleep!
}
}
15. 15
• One test technology for different tests
Distributed, platform-independent testing
Integrated graphical test development, documentation and analysis
Adaptable, open test environment
• Areas of Testing
Conformance and functional testing
Interoperability and integration testing
Real-time, performance, load and stress testing
Security testing
Regression testing
• Used for system and product qualification and certification
DESIGN PRINCIPLES OF TTCN-3
18. 18
Take available software and tools …
… and adding public testuites as a result of insights from IoT testing:
IOT-TESTWARE
…
https://projects.eclipse.org/projects/technology.iottestware
19. 19
• Supplement to running and active Eclipse projects
Paho, OM2M, Titan
• New project at Eclipse Foundation:
https://projects.eclipse.org/projects/technology.iottestware
TTCN-3 test suites for CoAP, MQTT, OPC-UA, LoRa?
• Assured licenses for users
• Currently in cooperation with
relayr GmbH, Ericsson, LAAS/CNRS, itemis AG, Spirent Communications,
Easy Global Market, Iskratel/Sintesio, …
THE ECLIPSE PROJECT
20. 20
SAMPLE TESTSUITE STRUCTURE: MQTT
Broker as SUT
All mandatory message data fields
Regular and illegal data
(Fixed/variable header, payload)
Protocol features
General
Connect/disconnect (session)
Subscribe/unsubscribe
Immediate publish
Last will and Testament (LWT)
Heartbeats keepAlive values
Topic
Error handling
Client as SUT
…
21. 21
TEST DEVELOPMENT SAMPLE: MQTT
TESTZIEL-KATALOG
Test configurations
Test Suite Structure
Test purpose (catalogue)
Test implementation (TTCN-3)
26. 26
Results for CoAP:
- Initially, 4421 fuzzed test data for CoAP were generated
- After sending the data to a (local) CoAP server, it crashed after date “1107”
https://www.fokus.fraunhofer.de/de/sqc/security_testing
https://github.com/fraunhoferfokus/Fuzzino/blob/master/doc/Fuzzino_XML_Description.pdf
FUZZINO RESULTS AND RESOURCES
36. 36
• New Working Group (TST) will develop
IoT test catalogues and specifications (not covered elsewhere)
• The types of testing include
conformance, interoperability, security and performance testing
• The initial technical focus will be:
− IoT network layer
(communication protocols, node connectivity, edge computing etc.),
− Basic security of IoT devices
ETSI TC MTS
38. 38
BASE SECURITY CERTIFICATION SCOPE
The Three Software Stacks Required for IoT Architectures, Eclipse IoT Working Group, September 2016
IoT-Testlab Scope
(basic security level
certification)
telemetry
commands
telemetry
commands
41. 41
Advanced testing technology:
Open source IoT-Testware (code):
External (open source) SW
Standardized IoT test purposes:
SUMMARY
42. 42
• Adding more protocols to IoT-Testware
AMQP, LWM2M, 6LoWPAN, LPWAN
• Increased security level for certification
• Cooperation/liaisons (in preparation) with
ETSI TC Cyber/SmartM2M, oneM2M, OPC Foundation ...
OUTLOOK