Session 4 - How to debug IoT Agents
Fernando López, Cloud & Platform Senior Expert
fernando.lopez@fiware.org
@flopezaguilar
FIWARE Foundation, e.V.
Learning Goals
§ How to diagnose common problems
§ How to test services in isolation
§ How to set-up and interpret debug
§ Configuring IoT Agents:
• Where are settings described and defined?
• How can they be overridden?
§ Adding diagnostic debug
§ Where to look for help and advice?
1
FIWARE Catalogue
22
Data/APIManagement
PublicationMonetization
Core Context Management
(Context Broker)
Context
Processing, Analysis, Visualization
Interface to
IoT, Robotics and third party systems
Deploymenttools
2
Development of
Context-aware applications
(Orion, STH-Comet,
Cygnus, Quantum Leap, Draco)
Connection to the
Internet of Things
(IDAS, OpenMTC)
Real-time
processing of
context events
(Perseo)
Handling authorization
and access control to
APIs
(Keyrock, Wilma,
AuthZForce, APInf )
Publication and
Monetization of Context
Information
(CKAN extensions, Data/API
Biz Framework, IDRA)
Creation of
Application Dashboards
(Wirecloud)
Real-time
Processing of media
streams
(Kurento)
Business Intelligence
(Knowage)
Connection to robots
(Fast RTPS,Micro XRCE-DDS)
Big Data
Context Analysis
(Cosmos)
Cloud Edge
(FogFlow)
Documents exchange
(Domibus)
Generic Scenario: IoT Sensors to FIWARE Context Data
3
§ We are using an IoT Agent to receive data
from a series of devices using a common
protocol
§ The IoT Agent must interpret the data and create
and modify NGSI data entities in the Context Broker
§ The system must react on changes of state. In this
example, changes of state of data entities must
somehow be persisted into a database using Cygnus,
but we could equally be doing some processing or
visualization.
§ In a working system is possible we would want amend
the transport or add security.
What can possibly go wrong?
§ Listening on the wrong port/resource-path/protocol
§ Check data is received on each part of the chain
§ Not paying attention to the FIWARE service headers
§ Misinterpreting received measures
§ Reading Numbers as Strings
§ Misusing localhost in a containerized environment
4
What can possibly go wrong?
§ Setting up subscriptions too narrowly
§ Provisioning services and devices incorrectly
§ Using deprecated interfaces (NGSI v1)
§ Using obsolete or incompatible Generic Enablers
5
Configuration and Provisioning
§ config.js
§ Base configuration file found in root of each enabler
§ Annotated
§ Most options switched off
§ Using Docker, you can:
§ Inject your own config.js volume
§ Override common ENVs
§ Override specific ENVs
§ Provisioning Groups and Devices
§ Further override options
6
Setting Debug
Debug options vary considerably between components:
§ Orion: -logLevel DEBUG -noCache -logForHumans
§ IoT Agents: IOTA_LOG_LEVEL=DEBUG
§ Cygnus: CYGNUS_LOG_LEVEL=DEBUG
§ Keyrock: DEBUG=idm:*
§ PEP Proxy: Amend the log_config.json file directly
7
Debugging
§ Check Services are running docker ps
Follow the logs docker logs -f fiware-iot-agent
§ PM2 Monitoring docker exec -it fiware-iot-agent pm2 monit
§ Listening to MQTT traffic docker run -it --rm --name mqtt-subscriber
--network fiware_default efrecon/mqtt-client
sub -h mosquitto -t "/#"
§ Sending dummy MQTT data docker run -it --rm --name mqtt-publisher
--network fiware_default efrecon/mqtt-client
pub -h mosquitto -m "c|1” –t
"/TEF/motion001/attrs"
8
Sanity Checks
§ Always stick to a specified version of each component
§ Check by using /version endpoints
§ Check component versions against releases
§ Read debug to see where data is/is not being retrieved
§ Initially use "idPattern": ".*" in subscriptions
§ Check the notification details such as timesSent
§ Ensure IOTA_CB_NGSI_VERSION and IOTA_AUTOCAST are set
§ Check your mapped attributes on provisioned devices
9
Add diagnostic debug by amending the source
§ git clone - grep the source code for your error message
§ git clone - amend source code - add debug - create image direct from source
- instructions in Dockerfile
docker build -t iot-agent .
§ Fork source code and amend - create image from fork
docker build -t iot-agent . 
--build-arg GITHUB_ACCOUNT=<your account> 
--build-arg GITHUB_REPOSITORY=<your repo> 
--build-arg SOURCE_BRANCH=<your branch>
§ All FIWARE projects are open-source - PRs accepted.
10
Where to get help
§ Read the Documentation
• https://www.fiware.org/developers/catalogue
§ Find the Source Code on GitHub
• https://github.com/FIWARE/catalogue
• Raise issues with individual enablers
§ Ask development questions on Stack Overflow
• https://stackoverflow.com/questions/tagged/fiware
• Always tag question fiware plus relevant tags
• Stack Overflow is a third-party site. Follow their rules of engagement:
• Development questions only
• Provide logs/evidence of your efforts
§ Use the FIWARE Q+A
• https://ask.fiware.org/questions
• Usable for non-development questions
11
Summary
§ Avoid simple problems by running basic sanity checks first
§ Test each link in the chain to diagnose your real issue
§ Setting up debug differs for each component - always check the
documentation for details
§ IoT Agents can be set up and overridden many different ways:
• Base config.js, injected config.js, Docker ENV overrides
• Device specific provisioning
§ Read the documentation on ReadTheDocs
§ If you’re still having problems, add diagnostic debug by
amending the source code and running a local image
§ Further help can be found on Ask, Stack Overflow or raising
issues on GitHub
12
Thank you!
http://fiware.org
Follow @FIWARE on Twitter
13

How to debug IoT Agents

  • 1.
    Session 4 -How to debug IoT Agents Fernando López, Cloud & Platform Senior Expert fernando.lopez@fiware.org @flopezaguilar FIWARE Foundation, e.V.
  • 2.
    Learning Goals § Howto diagnose common problems § How to test services in isolation § How to set-up and interpret debug § Configuring IoT Agents: • Where are settings described and defined? • How can they be overridden? § Adding diagnostic debug § Where to look for help and advice? 1
  • 3.
    FIWARE Catalogue 22 Data/APIManagement PublicationMonetization Core ContextManagement (Context Broker) Context Processing, Analysis, Visualization Interface to IoT, Robotics and third party systems Deploymenttools 2 Development of Context-aware applications (Orion, STH-Comet, Cygnus, Quantum Leap, Draco) Connection to the Internet of Things (IDAS, OpenMTC) Real-time processing of context events (Perseo) Handling authorization and access control to APIs (Keyrock, Wilma, AuthZForce, APInf ) Publication and Monetization of Context Information (CKAN extensions, Data/API Biz Framework, IDRA) Creation of Application Dashboards (Wirecloud) Real-time Processing of media streams (Kurento) Business Intelligence (Knowage) Connection to robots (Fast RTPS,Micro XRCE-DDS) Big Data Context Analysis (Cosmos) Cloud Edge (FogFlow) Documents exchange (Domibus)
  • 4.
    Generic Scenario: IoTSensors to FIWARE Context Data 3 § We are using an IoT Agent to receive data from a series of devices using a common protocol § The IoT Agent must interpret the data and create and modify NGSI data entities in the Context Broker § The system must react on changes of state. In this example, changes of state of data entities must somehow be persisted into a database using Cygnus, but we could equally be doing some processing or visualization. § In a working system is possible we would want amend the transport or add security.
  • 5.
    What can possiblygo wrong? § Listening on the wrong port/resource-path/protocol § Check data is received on each part of the chain § Not paying attention to the FIWARE service headers § Misinterpreting received measures § Reading Numbers as Strings § Misusing localhost in a containerized environment 4
  • 6.
    What can possiblygo wrong? § Setting up subscriptions too narrowly § Provisioning services and devices incorrectly § Using deprecated interfaces (NGSI v1) § Using obsolete or incompatible Generic Enablers 5
  • 7.
    Configuration and Provisioning §config.js § Base configuration file found in root of each enabler § Annotated § Most options switched off § Using Docker, you can: § Inject your own config.js volume § Override common ENVs § Override specific ENVs § Provisioning Groups and Devices § Further override options 6
  • 8.
    Setting Debug Debug optionsvary considerably between components: § Orion: -logLevel DEBUG -noCache -logForHumans § IoT Agents: IOTA_LOG_LEVEL=DEBUG § Cygnus: CYGNUS_LOG_LEVEL=DEBUG § Keyrock: DEBUG=idm:* § PEP Proxy: Amend the log_config.json file directly 7
  • 9.
    Debugging § Check Servicesare running docker ps Follow the logs docker logs -f fiware-iot-agent § PM2 Monitoring docker exec -it fiware-iot-agent pm2 monit § Listening to MQTT traffic docker run -it --rm --name mqtt-subscriber --network fiware_default efrecon/mqtt-client sub -h mosquitto -t "/#" § Sending dummy MQTT data docker run -it --rm --name mqtt-publisher --network fiware_default efrecon/mqtt-client pub -h mosquitto -m "c|1” –t "/TEF/motion001/attrs" 8
  • 10.
    Sanity Checks § Alwaysstick to a specified version of each component § Check by using /version endpoints § Check component versions against releases § Read debug to see where data is/is not being retrieved § Initially use "idPattern": ".*" in subscriptions § Check the notification details such as timesSent § Ensure IOTA_CB_NGSI_VERSION and IOTA_AUTOCAST are set § Check your mapped attributes on provisioned devices 9
  • 11.
    Add diagnostic debugby amending the source § git clone - grep the source code for your error message § git clone - amend source code - add debug - create image direct from source - instructions in Dockerfile docker build -t iot-agent . § Fork source code and amend - create image from fork docker build -t iot-agent . --build-arg GITHUB_ACCOUNT=<your account> --build-arg GITHUB_REPOSITORY=<your repo> --build-arg SOURCE_BRANCH=<your branch> § All FIWARE projects are open-source - PRs accepted. 10
  • 12.
    Where to gethelp § Read the Documentation • https://www.fiware.org/developers/catalogue § Find the Source Code on GitHub • https://github.com/FIWARE/catalogue • Raise issues with individual enablers § Ask development questions on Stack Overflow • https://stackoverflow.com/questions/tagged/fiware • Always tag question fiware plus relevant tags • Stack Overflow is a third-party site. Follow their rules of engagement: • Development questions only • Provide logs/evidence of your efforts § Use the FIWARE Q+A • https://ask.fiware.org/questions • Usable for non-development questions 11
  • 13.
    Summary § Avoid simpleproblems by running basic sanity checks first § Test each link in the chain to diagnose your real issue § Setting up debug differs for each component - always check the documentation for details § IoT Agents can be set up and overridden many different ways: • Base config.js, injected config.js, Docker ENV overrides • Device specific provisioning § Read the documentation on ReadTheDocs § If you’re still having problems, add diagnostic debug by amending the source code and running a local image § Further help can be found on Ask, Stack Overflow or raising issues on GitHub 12
  • 14.