Successfully reported this slideshow.
Your SlideShare is downloading. ×

A Successful Architecture for Qt Embedded Systems

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 34 Ad

A Successful Architecture for Qt Embedded Systems

Download to read offline

Shall you use one computer with multiple touch displays or multiple display computers? Do you need a window manager? Shall you build a custom Linux system with Yocto or use a container-based operating system? Which system-on-chip (SoC) is the right choice to handle three video streams, sensor data from a dozen electronic control units (ECUs) and the cloud connection? How do strong machine vibrations affect HDMI, CAN, Ethernet and USB connectors?

I’ll answer these and many more questions for the architecture of driver terminals as you find them in agricultural and construction machines. The answers are based on my own experience, as I architected and helped the infotainment systems of a car OEM and the driver terminals of the Krone forage harvesters and the ROPA sugar beet harvesters. The questions for industrial machines, medical devices and consumer products are the same, but the answers differ because of different constraints.

Shall you use one computer with multiple touch displays or multiple display computers? Do you need a window manager? Shall you build a custom Linux system with Yocto or use a container-based operating system? Which system-on-chip (SoC) is the right choice to handle three video streams, sensor data from a dozen electronic control units (ECUs) and the cloud connection? How do strong machine vibrations affect HDMI, CAN, Ethernet and USB connectors?

I’ll answer these and many more questions for the architecture of driver terminals as you find them in agricultural and construction machines. The answers are based on my own experience, as I architected and helped the infotainment systems of a car OEM and the driver terminals of the Krone forage harvesters and the ROPA sugar beet harvesters. The questions for industrial machines, medical devices and consumer products are the same, but the answers differ because of different constraints.

Advertisement
Advertisement

More Related Content

Recently uploaded (20)

Advertisement

A Successful Architecture for Qt Embedded Systems

  1. 1. A Successful Architecture for Qt Embedded Systems Burkhard Stubert
  2. 2. How does the right architecture look? Architecture shaped by its surrounding ecosystem Running Example: Harvester Terminal 2012 2021? 2021/07/08 (C) Burkhard Stubert 2
  3. 3. Regulations Machine Org Machine Part of Larger Ecosystem Qt Embedded System Farmers 2021/07/08 (C) Burkhard Stubert 3
  4. 4. Good, Right and Successful Architectures All architectures Good architectures Technically sound Right architectures Meeting stakeholder needs Successful architectures Delivering value Ports & Adapters Architecture Start with de-facto standard architecture and adapt it! 2021/07/08 (C) Burkhard Stubert 4
  5. 5. Agenda • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection Qt Embedded System App 1 ... App N App 2 Window Manager Operating System System-on-Chip Cloud Machine Operating Conditions Cloud 2021/07/08 (C) Burkhard Stubert 5
  6. 6. Ports-And-Adapters Architecture: Idea USB CAN RS232 UART (W)LAN HDMI PC Port Adapters • Standardised USB Port • Many USB-to-X adapters • Different companies can build different adapters 2021/07/08 (C) Burkhard Stubert 6
  7. 7. Ports-And-Adapters Architecture: Pattern • Port = Interface between BL (inside) and Adapters (outside) • Adapter = Uses or implements Port • 1 Port has N Adapters • Port same for all adapters • Code from inside must not leak to outside and vice versa • Inside: models, connections • Outside: Qt SerialBus, SQL, IPC Business Logic (BL) ModelAPI MachineAPI GUI Test Machine Simulator Mock 2021/07/08 (C) Burkhard Stubert 7
  8. 8. Ports-And-Adapters Architecture: Unfolded GUI Machine CAN ModelAPI MachineAPI 1 GUI Machine Ethernet ModelAPI MachineAPI 2 GUI Simulator IPC ModelAPI MachineAPI 3 Test Mock ModelAPI MachineAPI 4 Product v1 Product v2 PC Simulator System Test • Unfolded into configurations of 3-Layer Architecture • Different people/teams can work on different parts in parallel • High modularity, testability, maintainability, modifiability 2021/07/08 (C) Burkhard Stubert 8
  9. 9. Agenda Machine Operating Conditions Cloud Qt Embedded System App 1 ... App N App 2 Window Manager Operating System System-on-Chip • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection 2021/07/08 (C) Burkhard Stubert 9
  10. 10. Machine Port with CAN Adapter • Port Responsibilities • BL must not depend on QtCanBus library and classes • BL only knows ECU Twins, Quantities and Errors • Port exchanges Quantities and Errors with other Ports over signals, slots and properties • Adapter Responsibilites • Decode & encode CAN frames • Avoid GUI freezes & stuttering Business Logic MachineAPI CAN Adapter Qt CanBus Abstraction Level High Low QCanBusFrame BodyTwin Error Quantity + + ECU1 ECU2 Body ECU4 ECU5 can1 can0 2021/07/08 (C) Burkhard Stubert 10
  11. 11. Running Adapter in Thread, Process, MCU Business Logic MachineAPI Qt CanBus can0 can1 Business Logic MachineAPI Qt CanBus can0 can1 Thread CAN0 Thread CAN1 Qt Signal-Slot Conn. Business Logic MachineAPI CAN Bus can0 can1 Thread CAN0 Thread CAN1 Qt Remote Objects Task CAN0 Task CAN1 RPMsg Process 1 on A53 Process 1 on A53 Process 2 on A53 Process 1 on A53 2 Tasks on M4 2021/07/08 (C) Burkhard Stubert 11
  12. 12. Replacing CAN by UDP Adapter Business Logic MachineAPI CAN Adapter Qt CanBus QCanBusFrame BodyTwin Error Quantity + + Body ECU4 ECU5 can1 Business Logic MachineAPI UDP Adapter Qt Network QNetworkDiagram BodyTwin Error Quantity + + ECU1 ECU2 can0 ECU5 ECU4 Body eth0 ECU1 ECU2 2021/07/08 (C) Burkhard Stubert 12
  13. 13. Agenda Machine Operating Conditions Cloud Qt Embedded System App 1 ... App N App 2 Window Manager Operating System System-on-Chip • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection 2021/07/08 (C) Burkhard Stubert 13
  14. 14. Single Application: No Window Manager • SoC • Without GPU: LinuxFB • With GPU: EGLFS • EGLFS: Renders single Qt app on single fullscreen window • With OpenGL acceleration • EGLFS: Good enough for many Qt embedded systems Qt Embedded System Application 1 EGLFS Operating System System-on-Chip 2021/07/08 (C) Burkhard Stubert 14
  15. 15. When to Use a Window Manager? Use • System divided into multiple apps naturally • Higher reliability • Multi-team development • Multiple apps sneak in: • VNC server for remote support and diagnostics • VNC client for Apple CarPlay and Android Auto Don’t Use • SoC lacks GPU • BSP lacks Wayland support • More complexity • Configure or build WM • IPC between multiple apps 2021/07/08 (C) Burkhard Stubert 15
  16. 16. Multiple Applications: Custom Window Manager • Requirements • Most work done on full-screen main window • Few switches to other windows • Show additional info (split view) • Suitable for touch and hard keys • Not an option: X11 • Nearly impossible to place windows correctly • Best option: Wayland compositor Main ECUs Fields Steer Cam Main Main Turbine Cam Main ECUs Fields Cam Steer 2021/07/08 (C) Burkhard Stubert 16
  17. 17. Multiple Applications: Wayland Compositor • QML Wayland compositor • Just a QML program (with graphical effects and animations) • Fully customisable • Licensing • From Qt 5.14: No LGPL • Weston • Default Wayland compositor • Freely configurable • Under MIT license • E.g., GENIVI layer manager Qt Embedded System App 1 ... App N App 2 Wayland Compositor Operating System System-on-Chip 2021/07/08 (C) Burkhard Stubert 17
  18. 18. Agenda Cloud Machine Operating Conditions Cloud Qt Embedded System App 1 ... App N App 2 Window Manager Operating System System-on-Chip • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection 2021/07/08 (C) Burkhard Stubert 18
  19. 19. Linux, RTOS, Android or Windows? • Linux • Most used in Embedded • No license fees • BSP always available • Many free high-quality tools • Fits many use cases • Known by many developers • RTOS • Best for safety-critical and real- time systems (e.g., medical, trains, ECUs) • High license fees: QNX, Integrity • Free: FreeRTOS, Zephyr • Well-suited for Cortex-M core on hybrid SoCs (e.g., iMX8M) • Windows • Embedded: end of life in 2023 • IoT: connect edge with cloud • High license fees • Android • Growing in consumer products (TVs, STBs, cars) • Huge app ecosystem (e.g., games) 2021/07/08 (C) Burkhard Stubert 19
  20. 20. Custom, Desktop or Containerised Linux? • Custom Linux • Built mostly with Yocto • Steep learning curve • Use 1000s of ready-made packages • Device drivers, OTA update, VNC, machine & cloud communication, deep learning • Adapt BSP to your needs • Containerised Linux • Torizon OS from Toradex • Start with application development right away • No time spent building packages • Just add package to container • Easy to update • Pull containers from repo • Host Linux with OSTree • Problem: • Dependency management for many containers • Desktop Linux (e.g., Ubuntu) • Bad UX 2021/07/08 (C) Burkhard Stubert 20
  21. 21. Agenda • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection 2021/07/08 (C) Burkhard Stubert 21
  22. 22. Switch Multiple Systems with One Display Each • 4 terminals • with possibly different SoC, OS, window manager, HMI • each connected to CAN and LAN • Cons • Lots of duplicate hardware & software • High material & development costs • Pros • No PCB modifications needed Qt EmSys Qt EmSys Qt EmSys Qt EmSys CAN LAN 2021/07/08 (C) Burkhard Stubert 22
  23. 23. One System with Multiple Displays • 1 system with 4 displays • Terminals turned into apps • Single SoC, OS, window manager, HMI • Single LAN and CAN connection • System becomes Super-ECU • Pros • No duplicate hardware & software • Low material & development costs • Easier system update • Mitigate chip shortage • Cons • PCB modifications needed Qt Embedded System App 1 App 3 App 4 App 2 CAN 2021/07/08 (C) Burkhard Stubert 23 Super-ECU LAN
  24. 24. Agenda Cloud Machine Operating Conditions Cloud Qt Embedded System App 1 ... App N App 2 Window Manager Operating System System-on-Chip • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection 2021/07/08 (C) Burkhard Stubert 24
  25. 25. Operating Conditions Qt Embedded System Dust & Water Temperature Light Vibration Industrial temperature range: −40 °C to 85 °C Terminals: >= IP53 Dust protected Withstand water spills 2021/07/08 (C) Burkhard Stubert 25
  26. 26. Temperature • Temperature ranges • Military: −55 °C to 125 °C • Automotive: −40 °C to 125 °C • Industrial: −40 °C to 85 °C • Commercial: 0 °C to 75 °C • Consumer: 0 °C to 40 °C • Operate in extremes only for 15 min until AC kicks in • Wider ranges imply higher prices Level Solid Particles Water 3 Tools, thick wires Spraying water 4 Wires, crews, ants Splashing water 5 Dust protected Water jets 6 Dust tight Powerful jets 7 X Immersion <= 1m 8 X Immersion <= 3m 9K X High-pressure and high-temperature jets Dust & Water • IP rating: IP<dust><water> • Harvester terminals: >= IP53 • Food scales: IP69K • Watch out for thermal problems 2021/07/08 (C) Burkhard Stubert 26
  27. 27. Light • Displays must be readable • in sun, day and night light • from different positions • Good: • High brightness • High contrast • Wide viewing angles • Anti-reflection • Bad: High brightness alone causes overheating 2021/07/08 (C) Burkhard Stubert 27
  28. 28. Vibration • Standard RJ45, DSUB-9, USB or HDMI connectors become loose • Use special Deutsch or M12 connectors • Deutsch not suited for high data rates (e.g., HDMI, DP) • Problems with resistive and capacitive touch screens 2021/07/08 (C) Burkhard Stubert 28
  29. 29. Agenda Cloud Machine Operating Conditions Cloud Qt Embedded System App 1 ... App N App 2 Window Manager Operating System System-on-Chip • Software Architecture • Applications • Machine Communication • Window Manager • Operating System • Hardware Architecture • System with Multiple Displays • Operating Conditions • SoC Selection 2021/07/08 (C) Burkhard Stubert 29
  30. 30. Significant Requirements for SoC Selection Requirement 2016 2021 App Cores needed: multipe apps/displays, machine comms, video streams 2-4 >= 4 GPU needed: multiple apps, dynamic contents, animations, 3D Yes Yes MCU needed: real-time or safety-critical functionality No Yes Displays (LVDS, HDMI, DP or DSI connectors) 1x buitin >= 2x ext. Temperature range Automotive Automotive CAN 2x 2x Ethernet 1x 10Mbps 1x 1Gbps Wifi/Bluetooth No Yes USB Host Yes Yes UART (RS232) 4x 4x Lifetime 10-15 years 10-15 years Neural engine (e.g., image recognition) No Yes 2021/07/08 (C) Burkhard Stubert 30
  31. 31. Selecting a Computer-on-Module (CoM) Toradex Module CPU GPU MCU Disp °C CAN Eth Wifi BT USB UART Life DNN Colibri i.MX7 2 0 1 1 Y 2 N N 2 7 6 N Apalis i.MX6 4 1 0 2 Y 2 Y N 5 5 7 N Apalis i.MX8 Max 2+4 2 2 4 Y 3 Y Y 4 7 9 Y Verdin i.MX8M Mini 4 1 1 1 Y 2 Y Y 2 4 14 N Verdin i.MX8M Plus 4 1 1 3 Y 2 Y Y 2 4 15 Y Toradex Module < 100 < 250 < 500 < 1000 Colibri i.MX7 €74.90 €71.90 €67.40 €64.40 Apalis i.MX6 €143.15 €137.40 €128.85 €123.10 Apalis i.MX8 Max €258.20 €247.90 €232.40 €222.05 Verdin i.MX8M Mini €117.95 €113.25 €106.15 €101.45 Verdin i.MX8M Plus €144.05 €138.30 €129.65 €123.90 2021/07/08 (C) Burkhard Stubert 31
  32. 32. Off-The-Shelf Terminals • With builtin display (“terminal”) • Only few iMX8’s available • No second display • Without display (“Box PC”) • Connectors not vibration-resistant • Rarely support for 2nd display EPC-S201 CCPilot VS 2021/07/08 (C) Burkhard Stubert 32
  33. 33. Custom-Made Terminals • Assemble box PC or terminal from its main parts • Full control over CoM selection, IP level and connectors • Modify PCB to reflect connectivity requirements • Additional external displays • Additional CANs • Wifi/BT, LTE • RFID card reader 2021/07/08 (C) Burkhard Stubert 33
  34. 34. 🙏 Thank You 🙏 Mail: burkhard.stubert@embeddeduse.com Web: https://embeddeduse.com Newsletter: http://eepurl.com/gOMQoX

Editor's Notes

  • Lead with questions:
    Question are the same for QtEmSys, but answers differ and have differerent priorities
    Questions from different perspectives unearth requirements (= answers)
  • TODO: Start with inner circle and display next circles by click (animation)
  • You see my learning process over the last 25 years.

    Start with hexagonal architecture as de-facto standard architecture (presumptive architecture).
    => Reduces the search space!
    Fairbanks: “Presumptive architectures are usually successful.”

    Good = efficient
    Right = effective
  • The ports-and-adapters architecture is the base for all our GUI applications

    The idea for it comes from hardware ports and adapters …
  • Lets transfer the hardware idea into software
  • The PC with its USB ports becomes the business logic with several APIs or ports

    If we unfold the adapter-BL-adapter paths of the hexagon, we get multiple instances of 3-layer architecture
  • High modularity, testability, maintainability, modifiability
  • On harvester: terminal talks over CAN with machine; 2 different embedded systems
    On TV: “machine” == TV tuner, App talks to TV tuner over IPC, app and machine on same embedded system
  • Next slide: You must move CAN processing out of the main GUI thread.
  • Motivation: Avoid GUI freezes & stuttering, avoid buffer overflows

    How and where CAN adapters run does not matter!
    MachineAPI is always the same!
  • In the next section, we’ll answer two questions:
    Does our system need a window manager?
    Which window manager is best suited for our system?
  • First top-level window must exist for entire lifetime of app
    Further top-level windows rendered onto first one


    EGL is an interface between Khronos rendering APIs such as OpenGL ES or OpenVG and the underlying native platform window system. 

    From Qt docs: “This behavior is necessary because with EGLFS there is always exactly one native window and one EGL window surface; these belong to the widget or window that is created first. This approach works well when there is a main window that exists for the application's lifetime and all other widgets are either non top-levels or are created afterwards, once the main window is shown.”

    Good enough for KRONE and ROPA terminals
  • Full-screen and split view: Place window into area with fixed position and size

    Similar to phone
  • Third option: KWin = window manager for KDE Plasma 5
  • The choice of the OS is simple: Use Linux!
  • Qt runs on all of them!
  • Desktop Linux: (general purpos)
    Customisation: hide desktop window manager (X11)
    Security: free access, installation of any s/w

    Torizon: well-suited for single application systems (yields 1 container)
    TODO: Does Torizon with multiple containers have the same problems as microservices?

  • Now that we have a clear picture of sw architecture.
    But the nicest software architecture doesn’t help, if not supported by hardware

    How do we design systems with multi displays?
    The traditional way of the photo
  • No PCB modifications needed: adapters for DisplayPort, WLAN chip (not standard)
  • Switch becomes part of system: X/IP turned into IPC

    System becomes Super-ECU: Integrate functionality of many ECUs into system (e.g. on M4s or litte As)
    Example: Tesla

    PCB modifications needed: may go away when SoCs support HDS
  • Strong influence on hardware selection
  • Wider temperature range implies higher costs
  • Electronics only works in specified temperature ranges

    Price: Automotive >= 1.25 * Industrial

    The casing must protect electronics against dust & water
  • Finale Grande:
  • Black: run-off-the-mill: rock-solid but boring
    Red: differentiate between terminal and super-ECU

    Adapt requirements for other Qt embedded systems!!!

    Wifi/BT: Integrate phones and save cables
  • The terminal chooses the SoC and the connectors– not you!

    If we want to build our Super-ECU, we must custom-build it


    Closest: Crosscontrol CCPilot V700
    Closest: Advantech EPC-R3720

×